vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #462
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.q3fhGFJVl3', fmt=raw size=107374182414worker2: QEMU running (pid 52)15worker2 # mke2fs 1.47.4 (6-Mar-2025)16lb # Disk image does not exist, creating the virtualisation disk image...17worker1 # Disk image does not exist, creating the virtualisation disk image...18worker2 # Discarding device blocks: 0/262144 done19client # Disk image does not exist, creating the virtualisation disk image...20client: QEMU running (pid 49)21worker1 # Formatting '/build/vm-state-worker1/tmp.paigeu2W52', fmt=raw size=107374182422worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes23client # Formatting '/build/vm-state-client/tmp.ftrJTuonkl', fmt=raw size=107374182424worker1: QEMU running (pid 51)25lb # Formatting '/build/vm-state-lb/tmp.GsQzVzR5of', fmt=raw size=107374182426lb: QEMU running (pid 50)27worker1 # mke2fs 1.47.4 (6-Mar-2025)28lb # mke2fs 1.47.4 (6-Mar-2025)29worker2 # Filesystem UUID: 3cc2e6f2-ddbc-4aac-bfc2-f217b04afba330client # mke2fs 1.47.4 (6-Mar-2025)31worker1 # Discarding device blocks: 0/262144 done32(finished: start all VMs, in 0.41 seconds)33client # Discarding device blocks: 0/262144 done34worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes35worker1 # Filesystem UUID: d4ab78fc-1ea3-4871-97d8-08e0f53ebb3b36client: waiting for unit network-addresses-eth1.service37lb # Discarding device blocks: 0/262144 done38worker2 # Superblock backups stored on blocks:39client: waiting for the VM to finish booting40lb # Creating filesystem with 262144 4k blocks and 65536 inodes41worker2 # 32768, 98304, 163840, 22937642client # Creating filesystem with 262144 4k blocks and 65536 inodes43worker2 # 44client # Filesystem UUID: 26db757d-852e-4173-94cc-d4cac9be0c2e45lb # Filesystem UUID: 1256a82c-057b-4f74-946b-682636ad8fb846worker1 # Superblock backups stored on blocks:47worker2 # Allocating group tables: 0/8 done48client # Superblock backups stored on blocks:49lb # Superblock backups stored on blocks:50worker1 # 32768, 98304, 163840, 22937651worker2 # Writing inode tables: 0/8 done52client # 32768, 98304, 163840, 22937653lb # 32768, 98304, 163840, 22937654worker1 # 55worker2 # Creating journal (8192 blocks): done56worker2 # Writing superblocks and filesystem accounting information: 0/8 done57client # 58worker1 # Allocating group tables: 0/8 done59worker1 # Writing inode tables: 0/8 done60lb # 61client # Allocating group tables: 0/8 done62client # Writing inode tables: 0/8 done63lb # Allocating group tables: 0/8 done64worker1 # Creating journal (8192 blocks): done65lb # Writing inode tables: 0/8 done66worker1 # Writing superblocks and filesystem accounting information: 0/8 done67lb # Creating journal (8192 blocks): done68worker1 # 69worker2 # 70client # Creating journal (8192 blocks): done71lb # Writing superblocks and filesystem accounting information: 0/8 done72worker1 # Virtualisation disk image created.73worker2 # Virtualisation disk image created.74client # Writing superblocks and filesystem accounting information: 0/8 done75lb # 76client # 77lb # Virtualisation disk image created.78client # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85worker2 # 86worker2 # 87worker1 # 88worker1 # 89lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0090worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0091worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0092lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...93lb # 94lb # 95lb # 96lb # 97worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...98worker1 # 99worker1 # 100worker1 # 101worker1 # 102worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...103worker2 # 104worker2 # 105worker2 # 106worker2 # 107lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00108worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00109worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00110client # 111client # 112lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...113lb # 114lb # 115worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...116worker1 # 117worker1 # 118worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...119worker2 # 120worker2 # 121client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123client # 124client # 125client # 126client # 127client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129client # 130client # 131lb # Booting from ROM...132worker1 # Booting from ROM...133worker2 # Booting from ROM...134client # Booting from ROM...135lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7fpzfs641a0zaw1gqkgj7q7yq9772av3-nixos-system-lb-test/init regInfo=/nix/store/wsp5fr8ydpxcpcd15422khca9k57z9qk-closure-info/registration console=ttyS0,115200n8 console=tty0137worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026138lb # [ 0.000000] BIOS-provided physical RAM map:139lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable140lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved141lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026143lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable144lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved145worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3ixnkyap7vs59q8in2x1qxyxsm4j4ika-nixos-system-worker2-test/init regInfo=/nix/store/1x49x2z5h5a0is1il4ldcxsvaqgrcyh0-closure-info/registration console=ttyS0,115200n8 console=tty0146worker2 # [ 0.000000] BIOS-provided physical RAM map:147lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved148lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved149worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable150worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m4d7z5x94sq68abkv4fy104i2p3svdr2-nixos-system-worker1-test/init regInfo=/nix/store/wg83isw2fhcly3v7zbw92kc6p96qkg87-closure-info/registration console=ttyS0,115200n8 console=tty0151lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved152worker1 # [ 0.000000] BIOS-provided physical RAM map:153worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved154lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved155worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable156worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved157lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved158worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved159worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable160lb # [ 0.000000] NX (Execute Disable) protection: active161worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved162lb # [ 0.000000] APIC: Static calls initialized163worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved164lb # [ 0.000000] SMBIOS 2.8 present.165worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable166worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved167worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved168lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014169lb # [ 0.000000] DMI: Memory slots populated: 1/1170worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved171worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved172lb # [ 0.000000] Hypervisor detected: KVM173worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved174lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000175worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved176lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00177worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved178worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved179lb # [ 0.000000] kvm-clock: using sched offset of 498649617 cycles180worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved181worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved182worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved183lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns184worker2 # [ 0.000000] NX (Execute Disable) protection: active185worker1 # [ 0.000000] NX (Execute Disable) protection: active186lb # [ 0.000005] tsc: Detected 2400.012 MHz processor187worker2 # [ 0.000000] APIC: Static calls initialized188worker1 # [ 0.000000] APIC: Static calls initialized189worker2 # [ 0.000000] SMBIOS 2.8 present.190lb # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000191worker1 # [ 0.000000] SMBIOS 2.8 present.192lb # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs193worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014194worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014195worker1 # [ 0.000000] DMI: Memory slots populated: 1/1196lb # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT197worker2 # [ 0.000000] DMI: Memory slots populated: 1/1198worker1 # [ 0.000000] Hypervisor detected: KVM199worker2 # [ 0.000000] Hypervisor detected: KVM200lb # [ 0.002765] found SMP MP-table at [mem 0x000f5450-0x000f545f]201worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000202lb # [ 0.002776] Using GB pages for direct mapping203worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000204worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00205lb # [ 0.002890] RAMDISK: [mem 0x5e367000-0x5ffcffff]206worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00207worker1 # [ 0.000001] kvm-clock: using sched offset of 505101524 cycles208lb # [ 0.002899] ACPI: Early table checksum verification disabled209client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026210worker2 # [ 0.000000] kvm-clock: using sched offset of 503886859 cycles211lb # [ 0.002902] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )212worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns213worker1 # [ 0.000004] tsc: Detected 2400.012 MHz processor214lb # [ 0.002906] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)215worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns216worker1 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000217worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor218lb # [ 0.002910] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)219worker1 # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs220worker2 # [ 0.000807] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000221client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test/init regInfo=/nix/store/y35njjqlp6ymfwdgsads3fxdi4zcknfy-closure-info/registration console=ttyS0,115200n8 console=tty0222lb # [ 0.002916] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)223client # [ 0.000000] BIOS-provided physical RAM map:224worker1 # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT225lb # [ 0.002918] ACPI: FACS 0x000000005FFE0000 000040226worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs227worker1 # [ 0.002739] found SMP MP-table at [mem 0x000f5450-0x000f545f]228client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable229lb # [ 0.002920] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)230worker1 # [ 0.002751] Using GB pages for direct mapping231worker2 # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT232client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved233worker1 # [ 0.002878] RAMDISK: [mem 0x5e367000-0x5ffcffff]234lb # [ 0.002921] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)235worker2 # [ 0.002756] found SMP MP-table at [mem 0x000f5450-0x000f545f]236client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved237worker1 # [ 0.002886] ACPI: Early table checksum verification disabled238worker2 # [ 0.002768] Using GB pages for direct mapping239worker1 # [ 0.002889] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )240lb # [ 0.002923] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)241client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable242worker2 # [ 0.002922] RAMDISK: [mem 0x5e367000-0x5ffcffff]243client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved244worker2 # [ 0.002931] ACPI: Early table checksum verification disabled245worker1 # [ 0.002893] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)246lb # [ 0.002924] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)247client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved248worker2 # [ 0.002934] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )249worker1 # [ 0.002898] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)250lb # [ 0.002926] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]251client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved252worker1 # [ 0.002905] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)253lb # [ 0.002927] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]254worker2 # [ 0.002938] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)255client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved256worker1 # [ 0.002907] ACPI: FACS 0x000000005FFE0000 000040257lb # [ 0.002927] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]258client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved259worker2 # [ 0.002942] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)260lb # [ 0.002928] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]261worker1 # [ 0.002908] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)262client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved263worker2 # [ 0.002949] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)264lb # [ 0.002928] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]265worker1 # [ 0.002910] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)266client # [ 0.000000] NX (Execute Disable) protection: active267worker2 # [ 0.002951] ACPI: FACS 0x000000005FFE0000 000040268client # [ 0.000000] APIC: Static calls initialized269lb # [ 0.002929] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]270client # [ 0.000000] SMBIOS 2.8 present.271worker1 # [ 0.002911] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)272lb # [ 0.002929] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]273worker2 # [ 0.002952] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)274lb # [ 0.003152] No NUMA configuration found275worker1 # [ 0.002913] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)276client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014277client # [ 0.000000] DMI: Memory slots populated: 1/1278lb # [ 0.003153] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]279worker1 # [ 0.002914] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]280worker2 # [ 0.002954] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)281client # [ 0.000000] Hypervisor detected: KVM282lb # [ 0.003155] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]283worker1 # [ 0.002915] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]284client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000285worker2 # [ 0.002956] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)286lb # [ 0.005247] Zone ranges:287client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00288worker1 # [ 0.002916] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]289lb # [ 0.005248] DMA [mem 0x0000000000001000-0x0000000000ffffff]290client # [ 0.000000] kvm-clock: using sched offset of 518729227 cycles291worker2 # [ 0.002957] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)292lb # [ 0.005249] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]293worker1 # [ 0.002916] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]294lb # [ 0.005250] Normal empty295lb # [ 0.005251] Device empty296worker1 # [ 0.002917] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]297worker2 # [ 0.002958] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]298client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns299lb # [ 0.005252] Movable zone start for each node300lb # [ 0.005253] Early memory node ranges301client # [ 0.000006] tsc: Detected 2400.012 MHz processor302worker2 # [ 0.002959] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]303worker1 # [ 0.002917] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]304client # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000305lb # [ 0.005253] node 0: [mem 0x0000000000001000-0x000000000009efff]306worker1 # [ 0.002918] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]307worker2 # [ 0.002960] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]308worker1 # [ 0.003146] No NUMA configuration found309lb # [ 0.005254] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]310client # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs311worker2 # [ 0.002960] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]312worker1 # [ 0.003147] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]313lb # [ 0.005255] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]314client # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT315worker1 # [ 0.003149] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]316lb # [ 0.005275] On node 0, zone DMA: 1 pages in unavailable ranges317worker2 # [ 0.002961] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]318client # [ 0.002748] found SMP MP-table at [mem 0x000f5450-0x000f545f]319worker1 # [ 0.005321] Zone ranges:320client # [ 0.002761] Using GB pages for direct mapping321lb # [ 0.005521] On node 0, zone DMA: 97 pages in unavailable ranges322worker1 # [ 0.005322] DMA [mem 0x0000000000001000-0x0000000000ffffff]323worker2 # [ 0.002961] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]324client # [ 0.002842] RAMDISK: [mem 0x5e366000-0x5ffcffff]325lb # [ 0.030556] On node 0, zone DMA32: 40 pages in unavailable ranges326worker1 # [ 0.005323] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]327lb # [ 0.031007] ACPI: PM-Timer IO Port: 0x608328worker1 # [ 0.005324] Normal empty329client # [ 0.002852] ACPI: Early table checksum verification disabled330worker2 # [ 0.002962] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]331worker1 # [ 0.005325] Device empty332worker2 # [ 0.003186] No NUMA configuration found333lb # [ 0.031018] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])334client # [ 0.002856] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )335worker1 # [ 0.005325] Movable zone start for each node336worker1 # [ 0.005326] Early memory node ranges337lb # [ 0.031044] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23338client # [ 0.002860] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)339worker2 # [ 0.003187] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]340worker1 # [ 0.005326] node 0: [mem 0x0000000000001000-0x000000000009efff]341lb # [ 0.031047] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)342worker2 # [ 0.003190] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]343client # [ 0.002865] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)344worker2 # [ 0.005289] Zone ranges:345worker1 # [ 0.005327] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]346lb # [ 0.031048] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)347client # [ 0.002873] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)348worker2 # [ 0.005290] DMA [mem 0x0000000000001000-0x0000000000ffffff]349worker1 # [ 0.005328] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]350lb # [ 0.031049] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)351client # [ 0.002876] ACPI: FACS 0x000000005FFE0000 000040352worker1 # [ 0.005347] On node 0, zone DMA: 1 pages in unavailable ranges353worker2 # [ 0.005292] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]354lb # [ 0.031050] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)355worker2 # [ 0.005293] Normal empty356worker1 # [ 0.005606] On node 0, zone DMA: 97 pages in unavailable ranges357worker2 # [ 0.005293] Device empty358client # [ 0.002877] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)359lb # [ 0.031051] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)360worker2 # [ 0.005294] Movable zone start for each node361worker1 # [ 0.031074] On node 0, zone DMA32: 40 pages in unavailable ranges362client # [ 0.002879] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)363lb # [ 0.031053] ACPI: Using ACPI (MADT) for SMP configuration information364worker1 # [ 0.031544] ACPI: PM-Timer IO Port: 0x608365worker2 # [ 0.005295] Early memory node ranges366lb # [ 0.031054] ACPI: HPET id: 0x8086a201 base: 0xfed00000367worker1 # [ 0.031555] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])368worker2 # [ 0.005295] node 0: [mem 0x0000000000001000-0x000000000009efff]369client # [ 0.002880] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)370lb # [ 0.031058] TSC deadline timer available371worker1 # [ 0.031584] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23372lb # [ 0.031062] CPU topo: Max. logical packages: 1373worker2 # [ 0.005296] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]374client # [ 0.002882] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)375lb # [ 0.031063] CPU topo: Max. logical dies: 1376worker1 # [ 0.031586] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)377worker2 # [ 0.005297] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]378lb # [ 0.031063] CPU topo: Max. dies per package: 1379client # [ 0.002883] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]380worker1 # [ 0.031588] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)381lb # [ 0.031067] CPU topo: Max. threads per core: 1382worker2 # [ 0.005315] On node 0, zone DMA: 1 pages in unavailable ranges383client # [ 0.002884] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]384worker1 # [ 0.031589] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)385lb # [ 0.031068] CPU topo: Num. cores per package: 1386worker2 # [ 0.005587] On node 0, zone DMA: 97 pages in unavailable ranges387client # [ 0.002885] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]388lb # [ 0.031068] CPU topo: Num. threads per package: 1389worker1 # [ 0.031590] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)390lb # [ 0.031068] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs391worker2 # [ 0.030557] On node 0, zone DMA32: 40 pages in unavailable ranges392client # [ 0.002885] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]393worker1 # [ 0.031591] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)394worker2 # [ 0.031022] ACPI: PM-Timer IO Port: 0x608395lb # [ 0.031086] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()396client # [ 0.002886] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]397worker1 # [ 0.031593] ACPI: Using ACPI (MADT) for SMP configuration information398worker2 # [ 0.031032] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])399lb # [ 0.031116] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]400client # [ 0.002886] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]401worker1 # [ 0.031594] ACPI: HPET id: 0x8086a201 base: 0xfed00000402worker1 # [ 0.031597] TSC deadline timer available403worker2 # [ 0.031058] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23404client # [ 0.002887] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]405lb # [ 0.031118] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]406worker1 # [ 0.031601] CPU topo: Max. logical packages: 1407client # [ 0.003119] No NUMA configuration found408worker2 # [ 0.031061] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)409worker1 # [ 0.031601] CPU topo: Max. logical dies: 1410lb # [ 0.031119] [mem 0x60000000-0xafffffff] available for PCI devices411worker1 # [ 0.031602] CPU topo: Max. dies per package: 1412client # [ 0.003120] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]413lb # [ 0.031121] Booting paravirtualized kernel on KVM414worker2 # [ 0.031062] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)415worker1 # [ 0.031605] CPU topo: Max. threads per core: 1416client # [ 0.003122] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]417client # [ 0.005325] Zone ranges:418worker1 # [ 0.031606] CPU topo: Num. cores per package: 1419worker2 # [ 0.031063] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)420lb # [ 0.031123] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns421worker1 # [ 0.031606] CPU topo: Num. threads per package: 1422client # [ 0.005326] DMA [mem 0x0000000000001000-0x0000000000ffffff]423lb # [ 0.035576] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1424worker2 # [ 0.031064] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)425client # [ 0.005328] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]426worker1 # [ 0.031607] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs427client # [ 0.005329] Normal empty428lb # [ 0.037590] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152429worker2 # [ 0.031065] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)430client # [ 0.005329] Device empty431worker1 # [ 0.031624] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()432lb # [ 0.037627] kvm-guest: PV spinlocks disabled, single CPU433client # [ 0.005330] Movable zone start for each node434worker2 # [ 0.031067] ACPI: Using ACPI (MADT) for SMP configuration information435worker1 # [ 0.031657] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]436client # [ 0.005331] Early memory node ranges437worker2 # [ 0.031068] ACPI: HPET id: 0x8086a201 base: 0xfed00000438client # [ 0.005331] node 0: [mem 0x0000000000001000-0x000000000009efff]439worker1 # [ 0.031658] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]440worker2 # [ 0.031071] TSC deadline timer available441client # [ 0.005332] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]442worker1 # [ 0.031659] [mem 0x60000000-0xafffffff] available for PCI devices443worker2 # [ 0.031075] CPU topo: Max. logical packages: 1444worker1 # [ 0.031661] Booting paravirtualized kernel on KVM445client # [ 0.005333] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]446worker2 # [ 0.031076] CPU topo: Max. logical dies: 1447client # [ 0.005354] On node 0, zone DMA: 1 pages in unavailable ranges448worker2 # [ 0.031076] CPU topo: Max. dies per package: 1449lb # [ 0.037628] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7fpzfs641a0zaw1gqkgj7q7yq9772av3-nixos-system-lb-test/init regInfo=/nix/store/wsp5fr8ydpxcpcd15422khca9k57z9qk-closure-info/registration console=ttyS0,115200n8 console=tty0450worker1 # [ 0.031664] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns451client # [ 0.005609] On node 0, zone DMA: 97 pages in unavailable ranges452worker2 # [ 0.031080] CPU topo: Max. threads per core: 1453worker1 # [ 0.036164] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1454client # [ 0.031504] On node 0, zone DMA32: 40 pages in unavailable ranges455worker2 # [ 0.031080] CPU topo: Num. cores per package: 1456lb # [ 0.037719] Unknown kernel command line parameters "regInfo=/nix/store/wsp5fr8ydpxcpcd15422khca9k57z9qk-closure-info/registration", will be passed to user space.457client # [ 0.031969] ACPI: PM-Timer IO Port: 0x608458worker1 # [ 0.038434] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152459worker2 # [ 0.031081] CPU topo: Num. threads per package: 1460lb # [ 0.037732] random: crng init done461client # [ 0.031984] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])462worker1 # [ 0.038478] kvm-guest: PV spinlocks disabled, single CPU463worker2 # [ 0.031081] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs464lb # [ 0.037733] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes465client # [ 0.032012] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23466worker2 # [ 0.031097] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()467lb # [ 0.039851] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)468client # [ 0.032015] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)469client # [ 0.032017] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)470worker2 # [ 0.031126] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]471lb # [ 0.040892] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)472lb # [ 0.040923] Fallback order for Node 0: 0473client # [ 0.032018] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)474worker2 # [ 0.031128] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]475lb # [ 0.040926] Built 1 zonelists, mobility grouping on. Total pages: 393078476worker1 # [ 0.038480] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m4d7z5x94sq68abkv4fy104i2p3svdr2-nixos-system-worker1-test/init regInfo=/nix/store/wg83isw2fhcly3v7zbw92kc6p96qkg87-closure-info/registration console=ttyS0,115200n8 console=tty0477client # [ 0.032019] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)478lb # [ 0.040927] Policy zone: DMA32479worker2 # [ 0.031129] [mem 0x60000000-0xafffffff] available for PCI devices480client # [ 0.032020] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)481lb # [ 0.043628] mem auto-init: stack:all(zero), heap alloc:on, heap free:off482worker2 # [ 0.031131] Booting paravirtualized kernel on KVM483client # [ 0.032022] ACPI: Using ACPI (MADT) for SMP configuration information484worker1 # [ 0.038583] Unknown kernel command line parameters "regInfo=/nix/store/wg83isw2fhcly3v7zbw92kc6p96qkg87-closure-info/registration", will be passed to user space.485lb # [ 0.046836] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1486client # [ 0.032023] ACPI: HPET id: 0x8086a201 base: 0xfed00000487worker1 # [ 0.038595] random: crng init done488lb # [ 0.049950] allocated 3145728 bytes of page_ext489worker2 # [ 0.031133] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns490client # [ 0.032028] TSC deadline timer available491lb # [ 0.059695] ftrace: allocating 48731 entries in 192 pages492worker1 # [ 0.038596] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes493client # [ 0.032032] CPU topo: Max. logical packages: 1494worker2 # [ 0.035576] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1495lb # [ 0.059697] ftrace: allocated 192 pages with 2 groups496client # [ 0.032033] CPU topo: Max. logical dies: 1497lb # [ 0.060771] Dynamic Preempt: lazy498worker1 # [ 0.040667] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)499client # [ 0.032033] CPU topo: Max. dies per package: 1500worker2 # [ 0.037650] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152501lb # [ 0.060907] rcu: Preemptible hierarchical RCU implementation.502worker1 # [ 0.041703] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)503client # [ 0.032037] CPU topo: Max. threads per core: 1504lb # [ 0.060907] rcu: RCU event tracing is enabled.505worker2 # [ 0.037692] kvm-guest: PV spinlocks disabled, single CPU506worker1 # [ 0.041734] Fallback order for Node 0: 0507client # [ 0.032038] CPU topo: Num. cores per package: 1508lb # [ 0.060908] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.509client # [ 0.032038] CPU topo: Num. threads per package: 1510worker1 # [ 0.041737] Built 1 zonelists, mobility grouping on. Total pages: 393078511worker1 # [ 0.041738] Policy zone: DMA32512lb # [ 0.060909] Trampoline variant of Tasks RCU enabled.513client # [ 0.032038] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs514lb # [ 0.060910] Rude variant of Tasks RCU enabled.515worker1 # [ 0.044500] mem auto-init: stack:all(zero), heap alloc:on, heap free:off516client # [ 0.032062] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()517lb # [ 0.060910] Tracing variant of Tasks RCU enabled.518worker1 # [ 0.047498] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1519client # [ 0.032099] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]520lb # [ 0.060911] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.521worker1 # [ 0.050897] allocated 3145728 bytes of page_ext522worker1 # [ 0.060801] ftrace: allocating 48731 entries in 192 pages523lb # [ 0.060912] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1524client # [ 0.032100] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]525worker2 # [ 0.037693] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3ixnkyap7vs59q8in2x1qxyxsm4j4ika-nixos-system-worker2-test/init regInfo=/nix/store/1x49x2z5h5a0is1il4ldcxsvaqgrcyh0-closure-info/registration console=ttyS0,115200n8 console=tty0526worker1 # [ 0.060803] ftrace: allocated 192 pages with 2 groups527client # [ 0.032102] [mem 0x60000000-0xafffffff] available for PCI devices528worker1 # [ 0.061616] Dynamic Preempt: lazy529lb # [ 0.060964] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.530client # [ 0.032103] Booting paravirtualized kernel on KVM531worker1 # [ 0.061748] rcu: Preemptible hierarchical RCU implementation.532worker1 # [ 0.061748] rcu: RCU event tracing is enabled.533lb # [ 0.060966] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.534worker2 # [ 0.037783] Unknown kernel command line parameters "regInfo=/nix/store/1x49x2z5h5a0is1il4ldcxsvaqgrcyh0-closure-info/registration", will be passed to user space.535client # [ 0.032107] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns536worker2 # [ 0.037795] random: crng init done537worker1 # [ 0.061749] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.538lb # [ 0.060967] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.539client # [ 0.036562] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1540worker1 # [ 0.061750] Trampoline variant of Tasks RCU enabled.541worker2 # [ 0.037796] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes542lb # [ 0.065260] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16543worker1 # [ 0.061751] Rude variant of Tasks RCU enabled.544client # [ 0.038686] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152545worker1 # [ 0.061751] Tracing variant of Tasks RCU enabled.546lb # [ 0.065532] rcu: srcu_init: Setting srcu_struct sizes based on contention.547worker2 # [ 0.039812] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)548client # [ 0.038729] kvm-guest: PV spinlocks disabled, single CPU549worker1 # [ 0.061752] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.550lb # [ 0.065538] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns551worker2 # [ 0.040981] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)552worker2 # [ 0.041013] Fallback order for Node 0: 0553worker1 # [ 0.061752] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1554lb # [ 0.065644] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)555worker2 # [ 0.041015] Built 1 zonelists, mobility grouping on. Total pages: 393078556worker1 # [ 0.061805] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.557lb # [ 0.069297] Console: colour VGA+ 80x25558worker2 # [ 0.041017] Policy zone: DMA32559lb # [ 0.069301] printk: legacy console [tty0] enabled560worker1 # [ 0.061807] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.561lb # [ 0.098726] printk: legacy console [ttyS0] enabled562client # [ 0.038731] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test/init regInfo=/nix/store/y35njjqlp6ymfwdgsads3fxdi4zcknfy-closure-info/registration console=ttyS0,115200n8 console=tty0563worker2 # [ 0.043459] mem auto-init: stack:all(zero), heap alloc:on, heap free:off564lb # [ 0.194895] ACPI: Core revision 20250807565worker1 # [ 0.061808] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.566worker2 # [ 0.046352] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1567worker2 # [ 0.049538] allocated 3145728 bytes of page_ext568worker1 # [ 0.066142] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16569client # [ 0.038828] Unknown kernel command line parameters "regInfo=/nix/store/y35njjqlp6ymfwdgsads3fxdi4zcknfy-closure-info/registration", will be passed to user space.570client # [ 0.038841] random: crng init done571worker2 # [ 0.059241] ftrace: allocating 48731 entries in 192 pages572worker1 # [ 0.066403] rcu: srcu_init: Setting srcu_struct sizes based on contention.573lb # [ 0.195743] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns574client # [ 0.038841] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes575worker2 # [ 0.059243] ftrace: allocated 192 pages with 2 groups576lb # [ 0.197270] APIC: Switch to symmetric I/O mode setup577worker2 # [ 0.060046] Dynamic Preempt: lazy578worker1 # [ 0.066410] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns579lb # [ 0.198196] x2apic enabled580client # [ 0.041125] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)581worker2 # [ 0.060207] rcu: Preemptible hierarchical RCU implementation.582worker1 # [ 0.066522] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)583client # [ 0.042155] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)584worker2 # [ 0.060208] rcu: RCU event tracing is enabled.585lb # [ 0.198923] APIC: Switched APIC routing to: physical x2apic586worker1 # [ 0.070145] Console: colour VGA+ 80x25587client # [ 0.042190] Fallback order for Node 0: 0588worker1 # [ 0.070148] printk: legacy console [tty0] enabled589worker2 # [ 0.060209] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.590client # [ 0.042193] Built 1 zonelists, mobility grouping on. Total pages: 393078591worker1 # [ 0.099645] printk: legacy console [ttyS0] enabled592client # [ 0.042195] Policy zone: DMA32593worker2 # [ 0.060210] Trampoline variant of Tasks RCU enabled.594lb # [ 0.200710] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1595worker1 # [ 0.195500] ACPI: Core revision 20250807596worker2 # [ 0.060211] Rude variant of Tasks RCU enabled.597client # [ 0.044625] mem auto-init: stack:all(zero), heap alloc:on, heap free:off598worker2 # [ 0.060211] Tracing variant of Tasks RCU enabled.599client # [ 0.047831] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1600lb # [ 0.201640] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns601client # [ 0.051075] allocated 3145728 bytes of page_ext602worker1 # [ 0.196385] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns603worker2 # [ 0.060212] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.604client # [ 0.061243] ftrace: allocating 48731 entries in 192 pages605worker1 # [ 0.197967] APIC: Switch to symmetric I/O mode setup606client # [ 0.061246] ftrace: allocated 192 pages with 2 groups607worker2 # [ 0.060212] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1608lb # [ 0.203210] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)609client # [ 0.062106] Dynamic Preempt: lazy610worker1 # [ 0.198908] x2apic enabled611client # [ 0.062268] rcu: Preemptible hierarchical RCU implementation.612lb # [ 0.205292] x86/cpu: User Mode Instruction Prevention (UMIP) activated613worker2 # [ 0.060226] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.614client # [ 0.062269] rcu: RCU event tracing is enabled.615worker1 # [ 0.199657] APIC: Switched APIC routing to: physical x2apic616lb # [ 0.206329] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127617client # [ 0.062269] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.618worker2 # [ 0.060228] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.619client # [ 0.062271] Trampoline variant of Tasks RCU enabled.620lb # [ 0.207208] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0621client # [ 0.062271] Rude variant of Tasks RCU enabled.622worker2 # [ 0.060228] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623worker1 # [ 0.201512] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1624client # [ 0.062272] Tracing variant of Tasks RCU enabled.625worker2 # [ 0.064632] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16626lb # [ 0.208212] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto627client # [ 0.062273] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.628worker1 # [ 0.202498] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns629worker2 # [ 0.064897] rcu: srcu_init: Setting srcu_struct sizes based on contention.630client # [ 0.062273] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1631lb # [ 0.209207] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl632worker1 # [ 0.204112] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)633worker2 # [ 0.064903] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns634client # [ 0.062327] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.635lb # [ 0.211207] Transient Scheduler Attacks: Vulnerable: No microcode636worker1 # [ 0.205197] x86/cpu: User Mode Instruction Prevention (UMIP) activated637client # [ 0.062329] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.638lb # [ 0.212207] Spectre V2 : Mitigation: Enhanced / Automatic IBRS639worker2 # [ 0.065001] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)640worker2 # [ 0.068633] Console: colour VGA+ 80x25641worker1 # [ 0.206239] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127642client # [ 0.062330] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.643lb # [ 0.213207] Speculative Return Stack Overflow: Mitigation: Safe RET644worker2 # [ 0.068636] printk: legacy console [tty0] enabled645client # [ 0.066707] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16646worker1 # [ 0.207110] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0647worker2 # [ 0.098079] printk: legacy console [ttyS0] enabled648client # [ 0.066992] rcu: srcu_init: Setting srcu_struct sizes based on contention.649worker2 # [ 0.210236] ACPI: Core revision 20250807650worker1 # [ 0.208113] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto651client # [ 0.066999] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns652lb # [ 0.214207] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization653worker1 # [ 0.209110] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl654client # [ 0.067112] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)655worker2 # [ 0.211213] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns656client # [ 0.070665] Console: colour VGA+ 80x25657lb # [ 0.216214] Spectre V2 : Enabling IBPB for BPF658client # [ 0.070670] printk: legacy console [tty0] enabled659worker1 # [ 0.210110] Transient Scheduler Attacks: Vulnerable: No microcode660worker2 # [ 0.213008] APIC: Switch to symmetric I/O mode setup661client # [ 0.100157] printk: legacy console [ttyS0] enabled662worker2 # [ 0.214113] x2apic enabled663lb # [ 0.217715] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier664worker1 # [ 0.211109] Spectre V2 : Mitigation: Enhanced / Automatic IBRS665client # [ 0.195833] ACPI: Core revision 20250807666lb # [ 0.218208] active return thunk: srso_alias_return_thunk667worker1 # [ 0.212110] Speculative Return Stack Overflow: Mitigation: Safe RET668worker2 # [ 0.214965] APIC: Switched APIC routing to: physical x2apic669lb # [ 0.219836] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'670client # [ 0.196744] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns671lb # [ 0.221096] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'672client # [ 0.198326] APIC: Switch to symmetric I/O mode setup673worker2 # [ 0.216937] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1674client # [ 0.199299] x2apic enabled675lb # [ 0.221937] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'676client # [ 0.200067] APIC: Switched APIC routing to: physical x2apic677worker1 # [ 0.213109] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization678lb # [ 0.222943] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'679worker2 # [ 0.218073] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns680worker1 # [ 0.215116] Spectre V2 : Enabling IBPB for BPF681lb # [ 0.223925] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'682client # [ 0.201856] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1683lb # [ 0.224208] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'684worker1 # [ 0.216111] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier685worker2 # [ 0.219934] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)686worker1 # [ 0.217110] active return thunk: srso_alias_return_thunk687lb # [ 0.225207] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'688client # [ 0.202799] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns689worker2 # [ 0.222139] x86/cpu: User Mode Instruction Prevention (UMIP) activated690lb # [ 0.227094] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256691worker1 # [ 0.218131] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'692client # [ 0.204411] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)693lb # [ 0.227900] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64694worker1 # [ 0.219110] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'695worker2 # [ 0.223070] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127696client # [ 0.205497] x86/cpu: User Mode Instruction Prevention (UMIP) activated697worker2 # [ 0.224592] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0698lb # [ 0.228904] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512699worker1 # [ 0.220109] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'700client # [ 0.206534] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127701lb # [ 0.229906] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024702worker1 # [ 0.221109] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'703worker2 # [ 0.224934] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto704client # [ 0.207409] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0705lb # [ 0.230208] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8706worker1 # [ 0.222109] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'707worker2 # [ 0.226931] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl708worker1 # [ 0.223109] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'709lb # [ 0.231208] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.710client # [ 0.208412] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto711worker2 # [ 0.227931] Transient Scheduler Attacks: Vulnerable: No microcode712worker1 # [ 0.224109] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'713client # [ 0.209409] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl714worker2 # [ 0.228930] Spectre V2 : Mitigation: Enhanced / Automatic IBRS715worker1 # [ 0.225110] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256716client # [ 0.210409] Transient Scheduler Attacks: Vulnerable: No microcode717worker2 # [ 0.229931] Speculative Return Stack Overflow: Mitigation: Safe RET718worker1 # [ 0.226109] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64719client # [ 0.211408] Spectre V2 : Mitigation: Enhanced / Automatic IBRS720worker1 # [ 0.227109] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512721client # [ 0.212408] Speculative Return Stack Overflow: Mitigation: Safe RET722worker1 # [ 0.228110] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024723worker1 # [ 0.229110] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8724worker2 # [ 0.230930] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization725worker2 # [ 0.232938] Spectre V2 : Enabling IBPB for BPF726worker1 # [ 0.230110] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.727client # [ 0.213408] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization728worker2 # [ 0.233932] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier729client # [ 0.215417] Spectre V2 : Enabling IBPB for BPF730worker2 # [ 0.234931] active return thunk: srso_alias_return_thunk731client # [ 0.216409] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier732worker2 # [ 0.235951] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'733client # [ 0.217409] active return thunk: srso_alias_return_thunk734worker2 # [ 0.236931] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'735worker2 # [ 0.237930] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'736client # [ 0.218430] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'737worker2 # [ 0.238930] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'738client # [ 0.219409] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'739worker2 # [ 0.239930] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'740client # [ 0.220408] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'741worker2 # [ 0.240931] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'742client # [ 0.221408] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'743client # [ 0.222408] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'744worker2 # [ 0.241931] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'745client # [ 0.223408] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'746worker2 # [ 0.242931] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256747worker2 # [ 0.243930] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64748client # [ 0.224408] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'749worker2 # [ 0.244930] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512750client # [ 0.226408] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256751worker2 # [ 0.245931] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024752client # [ 0.227408] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64753worker2 # [ 0.246930] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8754client # [ 0.228408] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512755worker2 # [ 0.247930] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.756client # [ 0.229408] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757client # [ 0.230408] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758client # [ 0.231407] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759lb # [ 0.266657] Freeing SMP alternatives memory: 44K760lb # [ 0.267210] pid_max: default: 32768 minimum: 301761lb # [ 0.268298] LSM: initializing lsm=capability,landlock,yama,bpf,ima762lb # [ 0.269314] landlock: Up and running.763lb # [ 0.270208] Yama: becoming mindful.764lb # [ 0.271031] LSM support for eBPF active765lb # [ 0.271815] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766lb # [ 0.273144] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker1 # [ 0.265254] Freeing SMP alternatives memory: 44K768worker1 # [ 0.266033] pid_max: default: 32768 minimum: 301769worker1 # [ 0.266790] LSM: initializing lsm=capability,landlock,yama,bpf,ima770lb # [ 0.275465] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)771worker1 # [ 0.267961] landlock: Up and running.772worker1 # [ 0.268562] Yama: becoming mindful.773lb # [ 0.276749] Performance Events: Fam17h+ core perfctr, AMD PMU driver.774worker1 # [ 0.269320] LSM support for eBPF active775lb # [ 0.277216] ... version: 2776worker1 # [ 0.270217] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)777lb # [ 0.277948] ... bit width: 48778lb # [ 0.278209] ... generic counters: 6779worker1 # [ 0.271153] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)780lb # [ 0.278934] ... generic bitmap: 000000000000003f781lb # [ 0.279210] ... fixed-purpose counters: 0782lb # [ 0.279960] ... fixed-purpose bitmap: 0000000000000000783worker1 # [ 0.273705] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)784lb # [ 0.280209] ... value mask: 0000ffffffffffff785lb # [ 0.281144] ... max period: 00007fffffffffff786worker1 # [ 0.274652] Performance Events: Fam17h+ core perfctr, AMD PMU driver.787worker1 # [ 0.275114] ... version: 2788lb # [ 0.281923] ... global_ctrl mask: 000000000000003f789worker1 # [ 0.275843] ... bit width: 48790lb # [ 0.282305] signal: max sigframe size: 3376791worker1 # [ 0.276112] ... generic counters: 6792lb # [ 0.283089] rcu: Hierarchical SRCU implementation.793worker1 # [ 0.276836] ... generic bitmap: 000000000000003f794lb # [ 0.283802] rcu: Max phase no-delay instances is 400.795worker1 # [ 0.277122] ... fixed-purpose counters: 0796worker2 # [ 0.281182] Freeing SMP alternatives memory: 44K797worker2 # [ 0.281934] pid_max: default: 32768 minimum: 301798worker1 # [ 0.277826] ... fixed-purpose bitmap: 0000000000000000799worker1 # [ 0.278112] ... value mask: 0000ffffffffffff800worker2 # [ 0.282726] LSM: initializing lsm=capability,landlock,yama,bpf,ima801worker2 # [ 0.283727] landlock: Up and running.802worker1 # [ 0.279003] ... max period: 00007fffffffffff803worker2 # [ 0.284354] Yama: becoming mindful.804worker1 # [ 0.279813] ... global_ctrl mask: 000000000000003f805client # [ 0.266390] Freeing SMP alternatives memory: 44K806worker2 # [ 0.285139] LSM support for eBPF active807worker1 # [ 0.280221] signal: max sigframe size: 3376808client # [ 0.266959] pid_max: default: 32768 minimum: 301809lb # [ 0.288666] smp: Bringing up secondary CPUs ...810worker2 # [ 0.285963] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)811worker1 # [ 0.281014] rcu: Hierarchical SRCU implementation.812lb # [ 0.289223] smp: Brought up 1 node, 1 CPU813client # [ 0.267514] LSM: initializing lsm=capability,landlock,yama,bpf,ima814worker1 # [ 0.281660] rcu: Max phase no-delay instances is 400.815client # [ 0.268517] landlock: Up and running.816lb # [ 0.289883] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)817worker2 # [ 0.286977] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818client # [ 0.269409] Yama: becoming mindful.819client # [ 0.270552] LSM support for eBPF active820lb # [ 0.290361] Memory: 1454584K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109340K reserved, 0K cma-reserved)821worker2 # [ 0.289311] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)822client # [ 0.271293] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)823lb # [ 0.291476] devtmpfs: initialized824worker2 # [ 0.290471] Performance Events: Fam17h+ core perfctr, AMD PMU driver.825client # [ 0.272444] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)826lb # [ 0.292349] x86/mm: Memory block size: 128MB827worker2 # [ 0.290939] ... version: 2828worker1 # [ 0.286566] smp: Bringing up secondary CPUs ...829worker2 # [ 0.291625] ... bit width: 48830worker1 # [ 0.287125] smp: Brought up 1 node, 1 CPU831worker2 # [ 0.291933] ... generic counters: 6832client # [ 0.274737] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)833lb # [ 0.294048] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)834worker1 # [ 0.287778] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)835worker2 # [ 0.292609] ... generic bitmap: 000000000000003f836worker2 # [ 0.292933] ... fixed-purpose counters: 0837lb # [ 0.295089] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).838client # [ 0.275962] Performance Events: Fam17h+ core perfctr, AMD PMU driver.839worker2 # [ 0.293598] ... fixed-purpose bitmap: 0000000000000000840client # [ 0.276417] ... version: 2841lb # [ 0.296295] pinctrl core: initialized pinctrl subsystem842worker1 # [ 0.288293] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)843worker2 # [ 0.293933] ... value mask: 0000ffffffffffff844client # [ 0.277085] ... bit width: 48845worker1 # [ 0.289361] devtmpfs: initialized846lb # [ 0.297352] PM: RTC time: 04:23:11, date: 2026-09-21847client # [ 0.277410] ... generic counters: 6848worker2 # [ 0.294778] ... max period: 00007fffffffffff849worker1 # [ 0.290164] x86/mm: Memory block size: 128MB850client # [ 0.278113] ... generic bitmap: 000000000000003f851worker2 # [ 0.295577] ... global_ctrl mask: 000000000000003f852client # [ 0.278411] ... fixed-purpose counters: 0853worker2 # [ 0.296040] signal: max sigframe size: 3376854client # [ 0.279117] ... fixed-purpose bitmap: 0000000000000000855worker2 # [ 0.296813] rcu: Hierarchical SRCU implementation.856worker1 # [ 0.291935] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)857lb # [ 0.300748] NET: Registered PF_NETLINK/PF_ROUTE protocol family858client # [ 0.279410] ... value mask: 0000ffffffffffff859worker2 # [ 0.297479] rcu: Max phase no-delay instances is 400.860worker1 # [ 0.293001] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).861client # [ 0.280276] ... max period: 00007fffffffffff862lb # [ 0.301684] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations863worker1 # [ 0.294204] pinctrl core: initialized pinctrl subsystem864client # [ 0.281087] ... global_ctrl mask: 000000000000003f865client # [ 0.281512] signal: max sigframe size: 3376866lb # [ 0.302229] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations867worker1 # [ 0.295320] PM: RTC time: 04:23:11, date: 2026-09-21868client # [ 0.282337] rcu: Hierarchical SRCU implementation.869lb # [ 0.303470] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations870client # [ 0.283099] rcu: Max phase no-delay instances is 400.871lb # [ 0.304218] audit: initializing netlink subsys (disabled)872worker2 # [ 0.302535] smp: Bringing up secondary CPUs ...873worker2 # [ 0.302947] smp: Brought up 1 node, 1 CPU874lb # [ 0.305384] thermal_sys: Registered thermal governor 'fair_share'875worker1 # [ 0.298947] NET: Registered PF_NETLINK/PF_ROUTE protocol family876worker2 # [ 0.303588] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)877lb # [ 0.305387] thermal_sys: Registered thermal governor 'bang_bang'878worker1 # [ 0.300358] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations879lb # [ 0.306211] thermal_sys: Registered thermal governor 'step_wise'880worker2 # [ 0.304103] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)881worker1 # [ 0.301132] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations882lb # [ 0.307213] audit: type=2000 audit(1789964591.858:1): state=initialized audit_enabled=0 res=1883worker2 # [ 0.305171] devtmpfs: initialized884client # [ 0.288047] smp: Bringing up secondary CPUs ...885client # [ 0.288427] smp: Brought up 1 node, 1 CPU886lb # [ 0.309214] thermal_sys: Registered thermal governor 'user_space'887worker2 # [ 0.305991] x86/mm: Memory block size: 128MB888worker1 # [ 0.302373] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations889client # [ 0.289126] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)890lb # [ 0.309215] thermal_sys: Registered thermal governor 'power_allocator'891worker1 # [ 0.303121] audit: initializing netlink subsys (disabled)892lb # [ 0.310160] cpuidle: using governor menu893worker2 # [ 0.307688] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)894worker1 # [ 0.304231] thermal_sys: Registered thermal governor 'fair_share'895lb # [ 0.311748] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5896client # [ 0.289670] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)897worker1 # [ 0.304233] thermal_sys: Registered thermal governor 'bang_bang'898worker2 # [ 0.308782] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).899client # [ 0.290693] devtmpfs: initialized900worker1 # [ 0.305113] thermal_sys: Registered thermal governor 'step_wise'901worker2 # [ 0.310006] pinctrl core: initialized pinctrl subsystem902lb # [ 0.312436] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]903client # [ 0.291490] x86/mm: Memory block size: 128MB904worker1 # [ 0.306029] thermal_sys: Registered thermal governor 'user_space'905worker2 # [ 0.311076] PM: RTC time: 04:23:11, date: 2026-09-21906lb # [ 0.313212] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry907worker1 # [ 0.306832] audit: type=2000 audit(1789964591.863:1): state=initialized audit_enabled=0 res=1908lb # [ 0.314221] PCI: Using configuration type 1 for base access909client # [ 0.293285] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)910worker1 # [ 0.308115] thermal_sys: Registered thermal governor 'power_allocator'911lb # [ 0.315325] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.912worker1 # [ 0.308127] cpuidle: using governor menu913client # [ 0.294339] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).914worker2 # [ 0.314460] NET: Registered PF_NETLINK/PF_ROUTE protocol family915client # [ 0.295519] pinctrl core: initialized pinctrl subsystem916worker1 # [ 0.310288] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5917worker2 # [ 0.315494] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations918client # [ 0.296670] PM: RTC time: 04:23:11, date: 2026-09-21919worker1 # [ 0.311349] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]920worker2 # [ 0.315952] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations921worker1 # [ 0.312115] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry922worker2 # [ 0.317187] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations923lb # [ 0.320302] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages924worker1 # [ 0.313124] PCI: Using configuration type 1 for base access925worker2 # [ 0.317943] audit: initializing netlink subsys (disabled)926lb # [ 0.321211] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page927client # [ 0.300156] NET: Registered PF_NETLINK/PF_ROUTE protocol family928worker1 # [ 0.314190] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.929worker2 # [ 0.319033] thermal_sys: Registered thermal governor 'fair_share'930lb # [ 0.322202] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages931client # [ 0.300921] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations932worker2 # [ 0.319035] thermal_sys: Registered thermal governor 'bang_bang'933client # [ 0.301433] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations934worker2 # [ 0.319934] thermal_sys: Registered thermal governor 'step_wise'935worker2 # [ 0.320834] thermal_sys: Registered thermal governor 'user_space'936client # [ 0.302684] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations937worker2 # [ 0.321629] audit: type=2000 audit(1789964591.860:1): state=initialized audit_enabled=0 res=1938client # [ 0.303424] audit: initializing netlink subsys (disabled)939worker2 # [ 0.322936] thermal_sys: Registered thermal governor 'power_allocator'940lb # [ 0.327211] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page941client # [ 0.304413] audit: type=2000 audit(1789964591.881:1): state=initialized audit_enabled=0 res=1942worker2 # [ 0.322949] cpuidle: using governor menu943client # [ 0.305569] thermal_sys: Registered thermal governor 'fair_share'944worker1 # [ 0.321354] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages945worker2 # [ 0.325135] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5946client # [ 0.305571] thermal_sys: Registered thermal governor 'bang_bang'947worker1 # [ 0.322113] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page948client # [ 0.306412] thermal_sys: Registered thermal governor 'step_wise'949worker2 # [ 0.326173] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]950client # [ 0.307307] thermal_sys: Registered thermal governor 'user_space'951worker2 # [ 0.326936] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry952client # [ 0.308098] thermal_sys: Registered thermal governor 'power_allocator'953worker2 # [ 0.327944] PCI: Using configuration type 1 for base access954lb # [ 0.333230] ACPI: Added _OSI(Module Device)955client # [ 0.308439] cpuidle: using governor menu956lb # [ 0.333991] ACPI: Added _OSI(Processor Device)957worker2 # [ 0.329003] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.958lb # [ 0.334211] ACPI: Added _OSI(Processor Aggregator Device)959client # [ 0.310643] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5960worker1 # [ 0.327113] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages961worker1 # [ 0.328113] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page962client # [ 0.311654] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]963client # [ 0.312415] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry964client # [ 0.313435] PCI: Using configuration type 1 for base access965worker2 # [ 0.334189] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages966client # [ 0.314608] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.967worker2 # [ 0.334934] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page968worker2 # [ 0.335922] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages969worker1 # [ 0.336697] ACPI: Added _OSI(Module Device)970worker1 # [ 0.337114] ACPI: Added _OSI(Processor Device)971client # [ 0.319549] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages972worker1 # [ 0.337844] ACPI: Added _OSI(Processor Aggregator Device)973client # [ 0.320413] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page974worker2 # [ 0.340934] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page975lb # [ 0.347599] ACPI: 1 ACPI AML tables successfully acquired and loaded976lb # [ 0.349092] ACPI: Interpreter enabled977lb # [ 0.349706] ACPI: PM: (supports S0 S3 S4 S5)978lb # [ 0.350210] ACPI: Using IOAPIC for interrupt routing979client # [ 0.325413] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages980client # [ 0.326410] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page981lb # [ 0.351100] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug982worker1 # [ 0.344641] ACPI: 1 ACPI AML tables successfully acquired and loaded983worker2 # [ 0.351187] ACPI: Added _OSI(Module Device)984worker1 # [ 0.348030] ACPI: Interpreter enabled985worker2 # [ 0.351893] ACPI: Added _OSI(Processor Device)986lb # [ 0.356210] PCI: Using E820 reservations for host bridge windows987worker1 # [ 0.348603] ACPI: PM: (supports S0 S3 S4 S5)988worker2 # [ 0.352435] ACPI: Added _OSI(Processor Aggregator Device)989worker1 # [ 0.349113] ACPI: Using IOAPIC for interrupt routing990lb # [ 0.357336] ACPI: Enabled 2 GPEs in block 00 to 3F991worker1 # [ 0.349970] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug992client # [ 0.336885] ACPI: Added _OSI(Module Device)993client # [ 0.337412] ACPI: Added _OSI(Processor Device)994client # [ 0.338181] ACPI: Added _OSI(Processor Aggregator Device)995worker1 # [ 0.355113] PCI: Using E820 reservations for host bridge windows996worker2 # [ 0.359456] ACPI: 1 ACPI AML tables successfully acquired and loaded997worker1 # [ 0.356250] ACPI: Enabled 2 GPEs in block 00 to 3F998lb # [ 0.365932] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])999worker2 # [ 0.363350] ACPI: Interpreter enabled1000lb # [ 0.367491] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1001worker2 # [ 0.363949] ACPI: PM: (supports S0 S3 S4 S5)1002worker2 # [ 0.364630] ACPI: Using IOAPIC for interrupt routing1003lb # [ 0.368289] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1004lb # [ 0.369335] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1005lb # [ 0.370550] PCI host bridge to bus 0000:001006client # [ 0.347133] ACPI: 1 ACPI AML tables successfully acquired and loaded1007worker2 # [ 0.366997] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1008lb # [ 0.371216] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1009worker1 # [ 0.364890] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1010worker2 # [ 0.367934] PCI: Using E820 reservations for host bridge windows1011lb # [ 0.372210] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1012lb # [ 0.373210] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1013worker1 # [ 0.366648] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1014client # [ 0.350838] ACPI: Interpreter enabled1015lb # [ 0.374210] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1016worker1 # [ 0.367191] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1017worker2 # [ 0.370993] ACPI: Enabled 2 GPEs in block 00 to 3F1018client # [ 0.351428] ACPI: PM: (supports S0 S3 S4 S5)1019client # [ 0.352136] ACPI: Using IOAPIC for interrupt routing1020lb # [ 0.375210] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1021worker1 # [ 0.368236] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1022worker1 # [ 0.369484] PCI host bridge to bus 0000:001023lb # [ 0.376211] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1024client # [ 0.352466] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1025lb # [ 0.377211] pci_bus 0000:00: root bus resource [bus 00-ff]1026worker1 # [ 0.370117] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1027client # [ 0.353411] PCI: Using E820 reservations for host bridge windows1028worker1 # [ 0.371113] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1029lb # [ 0.378146] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1030worker1 # [ 0.372112] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1031client # [ 0.356485] ACPI: Enabled 2 GPEs in block 00 to 3F1032lb # [ 0.379687] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1033worker1 # [ 0.373113] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1034worker1 # [ 0.374113] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1035worker2 # [ 0.379562] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1036worker1 # [ 0.375112] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1037worker2 # [ 0.379940] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1038worker1 # [ 0.376113] pci_bus 0000:00: root bus resource [bus 00-ff]1039worker2 # [ 0.381191] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1040worker1 # [ 0.377182] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1041worker2 # [ 0.382055] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1042lb # [ 0.382240] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1043worker1 # [ 0.378573] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1044worker2 # [ 0.383287] PCI host bridge to bus 0000:001045lb # [ 0.383337] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1046worker2 # [ 0.383937] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1047client # [ 0.365180] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1048lb # [ 0.384232] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1049worker2 # [ 0.384933] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1050client # [ 0.366171] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1051worker2 # [ 0.385933] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1052lb # [ 0.385595] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1053worker2 # [ 0.386933] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1054client # [ 0.368366] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1055worker2 # [ 0.387933] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1056lb # [ 0.386942] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1057client # [ 0.369362] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1058worker2 # [ 0.388933] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1059worker1 # [ 0.381199] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1060client # [ 0.370602] PCI host bridge to bus 0000:001061worker2 # [ 0.389933] pci_bus 0000:00: root bus resource [bus 00-ff]1062worker1 # [ 0.382125] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1063client # [ 0.371308] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1064lb # [ 0.389235] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1065worker2 # [ 0.390853] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1066worker1 # [ 0.383097] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1067lb # [ 0.390068] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1068client # [ 0.372207] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1069lb # [ 0.390925] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1070client # [ 0.373196] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1071worker2 # [ 0.392377] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1072worker1 # [ 0.384090] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1073lb # [ 0.391216] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1074client # [ 0.374281] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1075client # [ 0.375359] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1076lb # [ 0.392745] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1077worker1 # [ 0.386057] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1078client # [ 0.376300] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1079client # [ 0.377396] pci_bus 0000:00: root bus resource [bus 00-ff]1080lb # [ 0.395234] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1081worker1 # [ 0.389140] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1082client # [ 0.378339] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1083lb # [ 0.396094] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1084worker2 # [ 0.394982] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1085worker1 # [ 0.389984] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1086lb # [ 0.396931] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1087worker2 # [ 0.395945] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1088client # [ 0.379871] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1089worker1 # [ 0.390852] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1090worker2 # [ 0.396883] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1091worker1 # [ 0.391118] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1092lb # [ 0.397764] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1093worker2 # [ 0.397922] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1094worker1 # [ 0.392657] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1095lb # [ 0.399217] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1096worker2 # [ 0.399685] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1097lb # [ 0.400050] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1098worker1 # [ 0.395030] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1099lb # [ 0.401003] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1100worker1 # [ 0.395770] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1101client # [ 0.382481] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1102worker2 # [ 0.403022] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1103worker1 # [ 0.396135] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1104client # [ 0.383423] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1105lb # [ 0.402760] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1106worker2 # [ 0.403821] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1107client # [ 0.384355] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1108worker1 # [ 0.397679] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1109worker2 # [ 0.404629] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1110lb # [ 0.404239] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1111worker2 # [ 0.404939] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1112client # [ 0.385413] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1113lb # [ 0.405068] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1114lb # [ 0.405973] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1115worker2 # [ 0.406458] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1116client # [ 0.387142] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1117lb # [ 0.406813] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118worker1 # [ 0.400140] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1119worker1 # [ 0.401127] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1120client # [ 0.389430] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1121worker2 # [ 0.409015] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1122worker1 # [ 0.402135] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1123client # [ 0.390295] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1124worker2 # [ 0.409939] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1125client # [ 0.391114] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1126lb # [ 0.408420] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1127worker1 # [ 0.403690] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1128worker2 # [ 0.410881] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1129client # [ 0.391418] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1130lb # [ 0.409217] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1131lb # [ 0.410153] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1132worker2 # [ 0.412314] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1133client # [ 0.392934] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1134worker1 # [ 0.406089] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1135lb # [ 0.411691] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1136worker1 # [ 0.406783] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1137worker2 # [ 0.413965] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1138worker1 # [ 0.407134] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1139client # [ 0.395349] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1140lb # [ 0.413236] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1141worker2 # [ 0.414881] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1142client # [ 0.396097] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1143lb # [ 0.414090] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1144worker2 # [ 0.415664] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1145client # [ 0.396433] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1146lb # [ 0.414955] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1147worker1 # [ 0.409438] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1148client # [ 0.397978] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1149worker2 # [ 0.416466] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150lb # [ 0.415738] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1151worker1 # [ 0.411899] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1152worker2 # [ 0.418964] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1153worker1 # [ 0.412120] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1154lb # [ 0.417219] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1155worker2 # [ 0.419788] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1156client # [ 0.400422] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1157worker1 # [ 0.413117] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1158lb # [ 0.418088] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1159worker2 # [ 0.420617] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1160client # [ 0.401340] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1161lb # [ 0.418940] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1162worker1 # [ 0.414722] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1163lb # [ 0.419216] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1164client # [ 0.402433] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1165worker2 # [ 0.421551] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1166client # [ 0.403978] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1167lb # [ 0.420714] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1168worker2 # [ 0.422973] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1169worker1 # [ 0.417435] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1170lb # [ 0.422222] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1171client # [ 0.405418] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1172worker2 # [ 0.423845] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1173worker1 # [ 0.418783] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1174client # [ 0.406265] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1175lb # [ 0.423161] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1176worker2 # [ 0.424955] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1177worker1 # [ 0.419172] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1178client # [ 0.407133] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1179worker2 # [ 0.426523] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1180lb # [ 0.424577] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1181worker1 # [ 0.420705] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1182client # [ 0.407968] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1183lb # [ 0.426252] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1184worker2 # [ 0.427966] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1185worker1 # [ 0.422909] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1186lb # [ 0.427141] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1187client # [ 0.410377] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1188worker2 # [ 0.428812] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1189worker1 # [ 0.423753] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1190client # [ 0.411043] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1191lb # [ 0.427928] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1192worker2 # [ 0.429648] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1193worker1 # [ 0.424136] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1194client # [ 0.411434] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1195lb # [ 0.428752] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1196worker2 # [ 0.430501] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1197worker1 # [ 0.425118] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1198client # [ 0.413041] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1199worker1 # [ 0.426656] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1200worker2 # [ 0.432699] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1201lb # [ 0.430922] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1202client # [ 0.414422] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1203worker2 # [ 0.432940] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1204lb # [ 0.431217] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1205client # [ 0.415260] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1206worker1 # [ 0.428125] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1207worker2 # [ 0.433870] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1208lb # [ 0.432165] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1209client # [ 0.416128] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1210worker2 # [ 0.434732] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1211worker1 # [ 0.429054] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1212lb # [ 0.433730] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1213worker2 # [ 0.436188] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1214worker1 # [ 0.430486] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1215client # [ 0.417614] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1216lb # [ 0.434733] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1217worker2 # [ 0.437966] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1218lb # [ 0.435421] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1219client # [ 0.419418] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1220worker1 # [ 0.432144] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1221client # [ 0.420243] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1222worker2 # [ 0.438907] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1223lb # [ 0.436730] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1224worker1 # [ 0.432970] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1225client # [ 0.421269] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1226worker2 # [ 0.440492] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1227worker1 # [ 0.433897] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1228client # [ 0.422287] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1229worker1 # [ 0.435499] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1230client # [ 0.423712] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1231worker2 # [ 0.441941] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1232worker2 # [ 0.442747] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1233worker2 # [ 0.443658] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1234client # [ 0.425423] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1235worker1 # [ 0.437944] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1236worker1 # [ 0.438757] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1237client # [ 0.426434] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1238worker2 # [ 0.444501] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1239worker1 # [ 0.439134] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1240client # [ 0.427950] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1241worker2 # [ 0.445959] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1242worker1 # [ 0.440691] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1243client # [ 0.429442] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1244worker2 # [ 0.446940] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1245client # [ 0.430288] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1246worker2 # [ 0.447874] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1247worker1 # [ 0.441926] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1248lb # [ 0.437438] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1249client # [ 0.431178] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1250worker2 # [ 0.449313] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1251worker1 # [ 0.442937] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1252client # [ 0.431961] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1253worker2 # [ 0.450622] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1254worker1 # [ 0.444790] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1255lb # [ 0.440781] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1256worker2 # [ 0.451144] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1257client # [ 0.433990] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1258lb # [ 0.441447] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1259client # [ 0.434417] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1260worker2 # [ 0.452527] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1261lb # [ 0.443134] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1262client # [ 0.435397] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1263lb # [ 0.444165] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1264client # [ 0.436836] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1265lb # [ 0.445491] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1266client # [ 0.437959] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1267lb # [ 0.446459] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1268client # [ 0.438618] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1269client # [ 0.439970] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1270lb # [ 0.448874] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1271lb # [ 0.449218] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1272worker1 # [ 0.445317] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1273lb # [ 0.450522] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1274worker1 # [ 0.447910] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1275lb # [ 0.451889] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1276worker2 # [ 0.453137] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1277worker1 # [ 0.449029] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1278worker1 # [ 0.450789] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1279worker2 # [ 0.456515] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1280lb # [ 0.455195] ACPI: PCI: Interrupt link LNKA configured for IRQ 101281worker1 # [ 0.451363] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1282worker2 # [ 0.457200] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1283lb # [ 0.456098] ACPI: PCI: Interrupt link LNKB configured for IRQ 101284worker1 # [ 0.452405] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1285worker2 # [ 0.458861] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1286lb # [ 0.457011] ACPI: PCI: Interrupt link LNKC configured for IRQ 111287worker1 # [ 0.453363] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1288lb # [ 0.458153] ACPI: PCI: Interrupt link LNKD configured for IRQ 111289client # [ 0.440609] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1290worker2 # [ 0.459854] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1291lb # [ 0.459019] ACPI: PCI: Interrupt link LNKE configured for IRQ 101292worker2 # [ 0.461175] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1293worker1 # [ 0.455140] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1294lb # [ 0.460088] ACPI: PCI: Interrupt link LNKF configured for IRQ 101295lb # [ 0.461032] ACPI: PCI: Interrupt link LNKG configured for IRQ 111296worker2 # [ 0.462183] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1297worker1 # [ 0.456007] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1298client # [ 0.443966] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1299lb # [ 0.462037] ACPI: PCI: Interrupt link LNKH configured for IRQ 111300worker1 # [ 0.457424] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1301client # [ 0.444664] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1302lb # [ 0.463005] ACPI: PCI: Interrupt link GSIA configured for IRQ 161303worker2 # [ 0.464557] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1304lb # [ 0.463937] ACPI: PCI: Interrupt link GSIB configured for IRQ 171305worker1 # [ 0.458959] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1306worker2 # [ 0.464940] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1307client # [ 0.446317] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1308lb # [ 0.464224] ACPI: PCI: Interrupt link GSIC configured for IRQ 181309lb # [ 0.465222] ACPI: PCI: Interrupt link GSID configured for IRQ 191310client # [ 0.447363] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1311lb # [ 0.466223] ACPI: PCI: Interrupt link GSIE configured for IRQ 201312worker2 # [ 0.466160] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1313client # [ 0.448677] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1314lb # [ 0.467230] ACPI: PCI: Interrupt link GSIF configured for IRQ 211315worker1 # [ 0.462688] ACPI: PCI: Interrupt link LNKA configured for IRQ 101316client # [ 0.449658] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1317worker1 # [ 0.463239] ACPI: PCI: Interrupt link LNKB configured for IRQ 101318worker2 # [ 0.467982] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1319lb # [ 0.468224] ACPI: PCI: Interrupt link GSIG configured for IRQ 221320worker1 # [ 0.464217] ACPI: PCI: Interrupt link LNKC configured for IRQ 111321lb # [ 0.469242] ACPI: PCI: Interrupt link GSIH configured for IRQ 231322worker1 # [ 0.465209] ACPI: PCI: Interrupt link LNKD configured for IRQ 111323worker1 # [ 0.466217] ACPI: PCI: Interrupt link LNKE configured for IRQ 101324lb # [ 0.471237] iommu: Default domain type: Translated1325client # [ 0.451969] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1326lb # [ 0.472031] iommu: DMA domain TLB invalidation policy: lazy mode1327worker1 # [ 0.467211] ACPI: PCI: Interrupt link LNKF configured for IRQ 101328client # [ 0.452433] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1329worker2 # [ 0.472068] ACPI: PCI: Interrupt link LNKA configured for IRQ 101330lb # [ 0.473189] ACPI: bus type USB registered1331worker1 # [ 0.468210] ACPI: PCI: Interrupt link LNKG configured for IRQ 111332worker2 # [ 0.473037] ACPI: PCI: Interrupt link LNKB configured for IRQ 101333client # [ 0.453782] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1334lb # [ 0.473783] usbcore: registered new interface driver usbfs1335worker1 # [ 0.469211] ACPI: PCI: Interrupt link LNKH configured for IRQ 111336worker2 # [ 0.474043] ACPI: PCI: Interrupt link LNKC configured for IRQ 111337lb # [ 0.474225] usbcore: registered new interface driver hub1338worker1 # [ 0.470150] ACPI: PCI: Interrupt link GSIA configured for IRQ 161339worker2 # [ 0.475034] ACPI: PCI: Interrupt link LNKD configured for IRQ 111340lb # [ 0.475103] usbcore: registered new device driver usb1341client # [ 0.455440] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1342worker2 # [ 0.476029] ACPI: PCI: Interrupt link LNKE configured for IRQ 101343worker2 # [ 0.477030] ACPI: PCI: Interrupt link LNKF configured for IRQ 101344lb # [ 0.476977] NetLabel: Initializing1345worker1 # [ 0.471133] ACPI: PCI: Interrupt link GSIB configured for IRQ 171346worker2 # [ 0.478043] ACPI: PCI: Interrupt link LNKG configured for IRQ 111347lb # [ 0.477212] NetLabel: domain hash size = 1281348worker2 # [ 0.479039] ACPI: PCI: Interrupt link LNKH configured for IRQ 111349lb # [ 0.478044] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1350lb # [ 0.478919] NetLabel: unlabeled traffic allowed by default1351worker2 # [ 0.479967] ACPI: PCI: Interrupt link GSIA configured for IRQ 161352client # [ 0.459951] ACPI: PCI: Interrupt link LNKA configured for IRQ 101353lb # [ 0.479223] PCI: Using ACPI for IRQ routing1354worker2 # [ 0.480905] ACPI: PCI: Interrupt link GSIB configured for IRQ 171355worker1 # [ 0.472927] ACPI: PCI: Interrupt link GSIC configured for IRQ 181356client # [ 0.460515] ACPI: PCI: Interrupt link LNKB configured for IRQ 101357worker1 # [ 0.474131] ACPI: PCI: Interrupt link GSID configured for IRQ 191358worker2 # [ 0.481628] ACPI: PCI: Interrupt link GSIC configured for IRQ 181359client # [ 0.461513] ACPI: PCI: Interrupt link LNKC configured for IRQ 111360worker1 # [ 0.475129] ACPI: PCI: Interrupt link GSIE configured for IRQ 201361worker2 # [ 0.482734] ACPI: PCI: Interrupt link GSID configured for IRQ 191362client # [ 0.462517] ACPI: PCI: Interrupt link LNKD configured for IRQ 111363worker1 # [ 0.476093] ACPI: PCI: Interrupt link GSIF configured for IRQ 211364worker2 # [ 0.483729] ACPI: PCI: Interrupt link GSIE configured for IRQ 201365client # [ 0.463511] ACPI: PCI: Interrupt link LNKE configured for IRQ 101366worker1 # [ 0.476861] ACPI: PCI: Interrupt link GSIG configured for IRQ 221367worker2 # [ 0.484703] ACPI: PCI: Interrupt link GSIF configured for IRQ 211368client # [ 0.464508] ACPI: PCI: Interrupt link LNKF configured for IRQ 101369worker1 # [ 0.477124] ACPI: PCI: Interrupt link GSIH configured for IRQ 231370worker2 # [ 0.484949] ACPI: PCI: Interrupt link GSIG configured for IRQ 221371client # [ 0.465513] ACPI: PCI: Interrupt link LNKG configured for IRQ 111372worker2 # [ 0.485881] ACPI: PCI: Interrupt link GSIH configured for IRQ 231373client # [ 0.466511] ACPI: PCI: Interrupt link LNKH configured for IRQ 111374worker1 # [ 0.479189] iommu: Default domain type: Translated1375client # [ 0.467450] ACPI: PCI: Interrupt link GSIA configured for IRQ 161376worker1 # [ 0.480018] iommu: DMA domain TLB invalidation policy: lazy mode1377worker2 # [ 0.487654] iommu: Default domain type: Translated1378client # [ 0.468426] ACPI: PCI: Interrupt link GSIB configured for IRQ 171379worker1 # [ 0.481112] ACPI: bus type USB registered1380worker2 # [ 0.487946] iommu: DMA domain TLB invalidation policy: lazy mode1381worker2 # [ 0.489152] ACPI: bus type USB registered1382worker1 # [ 0.481854] usbcore: registered new interface driver usbfs1383client # [ 0.469424] ACPI: PCI: Interrupt link GSIC configured for IRQ 181384worker2 # [ 0.489848] usbcore: registered new interface driver usbfs1385client # [ 0.470401] ACPI: PCI: Interrupt link GSID configured for IRQ 191386worker1 # [ 0.482129] usbcore: registered new interface driver hub1387worker2 # [ 0.490596] usbcore: registered new interface driver hub1388client # [ 0.471193] ACPI: PCI: Interrupt link GSIE configured for IRQ 201389worker1 # [ 0.483124] usbcore: registered new device driver usb1390worker2 # [ 0.490944] usbcore: registered new device driver usb1391client # [ 0.472165] ACPI: PCI: Interrupt link GSIF configured for IRQ 211392client # [ 0.472423] ACPI: PCI: Interrupt link GSIG configured for IRQ 221393worker1 # [ 0.484854] NetLabel: Initializing1394worker2 # [ 0.492616] NetLabel: Initializing1395worker1 # [ 0.485120] NetLabel: domain hash size = 1281396worker2 # [ 0.492934] NetLabel: domain hash size = 1281397client # [ 0.473445] ACPI: PCI: Interrupt link GSIH configured for IRQ 231398worker2 # [ 0.493650] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1399worker1 # [ 0.485848] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1400worker2 # [ 0.493969] NetLabel: unlabeled traffic allowed by default1401worker1 # [ 0.486171] NetLabel: unlabeled traffic allowed by default1402client # [ 0.475479] iommu: Default domain type: Translated1403worker2 # [ 0.494852] PCI: Using ACPI for IRQ routing1404worker1 # [ 0.487077] PCI: Using ACPI for IRQ routing1405client # [ 0.476284] iommu: DMA domain TLB invalidation policy: lazy mode1406client # [ 0.477431] ACPI: bus type USB registered1407client # [ 0.478246] usbcore: registered new interface driver usbfs1408client # [ 0.479156] usbcore: registered new interface driver hub1409client # [ 0.479422] usbcore: registered new device driver usb1410client # [ 0.481308] NetLabel: Initializing1411client # [ 0.481856] NetLabel: domain hash size = 1281412client # [ 0.482410] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413client # [ 0.483399] NetLabel: unlabeled traffic allowed by default1414client # [ 0.484131] PCI: Using ACPI for IRQ routing1415lb # [ 0.524282] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416lb # [ 0.525206] pci 0000:00:01.0: vgaarb: bridge control possible1417lb # [ 0.525206] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418lb # [ 0.525215] vgaarb: loaded1419lb # [ 0.525968] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420lb # [ 0.526212] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421lb # [ 0.531298] clocksource: Switched to clocksource kvm-clock1422lb # [ 0.534772] VFS: Disk quotas dquot_6.6.01423lb # [ 0.535512] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1424lb # [ 0.536909] pnp: PnP ACPI init1425worker2 # [ 0.538309] pci 0000:00:01.0: vgaarb: setting as boot VGA device1426worker1 # [ 0.530671] pci 0000:00:01.0: vgaarb: setting as boot VGA device1427lb # [ 0.537713] ACPI: IRQ 4 override to edge(!), high(!)1428worker2 # [ 0.538929] pci 0000:00:01.0: vgaarb: bridge control possible1429worker1 # [ 0.531108] pci 0000:00:01.0: vgaarb: bridge control possible1430lb # [ 0.538660] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1431worker2 # [ 0.538929] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1432worker1 # [ 0.531108] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1433worker2 # [ 0.538937] vgaarb: loaded1434worker1 # [ 0.531117] vgaarb: loaded1435lb # [ 0.540190] pnp: PnP ACPI: found 6 devices1436worker2 # [ 0.539639] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01437worker1 # [ 0.531837] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01438worker2 # [ 0.539934] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1439worker1 # [ 0.532114] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1440client # [ 0.527044] pci 0000:00:01.0: vgaarb: setting as boot VGA device1441worker2 # [ 0.544021] clocksource: Switched to clocksource kvm-clock1442client # [ 0.527407] pci 0000:00:01.0: vgaarb: bridge control possible1443worker1 # [ 0.537200] clocksource: Switched to clocksource kvm-clock1444lb # [ 0.547753] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1445client # [ 0.527407] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1446client # [ 0.527416] vgaarb: loaded1447lb # [ 0.549171] clocksource: Switched to clocksource acpi_pm1448client # [ 0.528166] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01449worker2 # [ 0.547521] VFS: Disk quotas dquot_6.6.01450lb # [ 0.550186] NET: Registered PF_INET protocol family1451client # [ 0.528413] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1452worker2 # [ 0.548238] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1453worker1 # [ 0.540574] VFS: Disk quotas dquot_6.6.01454lb # [ 0.551324] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1455worker2 # [ 0.549546] pnp: PnP ACPI init1456worker1 # [ 0.541253] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1457worker2 # [ 0.550316] ACPI: IRQ 4 override to edge(!), high(!)1458worker1 # [ 0.542561] pnp: PnP ACPI init1459worker1 # [ 0.543345] ACPI: IRQ 4 override to edge(!), high(!)1460worker2 # [ 0.551284] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1461client # [ 0.531490] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.552825] pnp: PnP ACPI: found 6 devices1463worker1 # [ 0.544356] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1464worker1 # [ 0.545912] pnp: PnP ACPI: found 6 devices1465client # [ 0.533307] VFS: Disk quotas dquot_6.6.01466client # [ 0.535803] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1467client # [ 0.537167] pnp: PnP ACPI init1468client # [ 0.537972] ACPI: IRQ 4 override to edge(!), high(!)1469client # [ 0.538965] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1470worker2 # [ 0.560457] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1471client # [ 0.540528] pnp: PnP ACPI: found 6 devices1472worker2 # [ 0.561915] clocksource: Switched to clocksource acpi_pm1473worker1 # [ 0.553527] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1474worker2 # [ 0.562892] NET: Registered PF_INET protocol family1475worker1 # [ 0.554956] clocksource: Switched to clocksource acpi_pm1476worker1 # [ 0.555947] NET: Registered PF_INET protocol family1477worker2 # [ 0.564125] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1478lb # [ 0.567202] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1479worker1 # [ 0.557165] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1480lb # [ 0.568721] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1481lb # [ 0.570100] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1482lb # [ 0.571516] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1483client # [ 0.548110] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1484lb # [ 0.572803] TCP: Hash tables configured (established 16384 bind 16384)1485client # [ 0.549543] clocksource: Switched to clocksource acpi_pm1486client # [ 0.550535] NET: Registered PF_INET protocol family1487lb # [ 0.573908] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1488lb # [ 0.575177] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1489client # [ 0.551732] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1490lb # [ 0.576257] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1491lb # [ 0.577501] NET: Registered PF_UNIX/PF_LOCAL protocol family1492lb # [ 0.578497] NET: Registered PF_XDP protocol family1493lb # [ 0.579397] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1494lb # [ 0.580454] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1495lb # [ 0.581507] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1496lb # [ 0.582748] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1497worker2 # [ 0.580400] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1498lb # [ 0.583865] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1499worker1 # [ 0.573567] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1500worker2 # [ 0.581913] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1501lb # [ 0.584992] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1502worker2 # [ 0.583196] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1503worker1 # [ 0.575072] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1504lb # [ 0.586618] ACPI: \_SB_.GSIA: Enabled at IRQ 161505worker2 # [ 0.584483] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1506worker1 # [ 0.576370] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1507worker2 # [ 0.585686] TCP: Hash tables configured (established 16384 bind 16384)1508lb # [ 0.588792] ACPI: \_SB_.GSIB: Enabled at IRQ 171509worker1 # [ 0.577694] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1510worker2 # [ 0.586767] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1511worker1 # [ 0.578968] TCP: Hash tables configured (established 16384 bind 16384)1512lb # [ 0.590782] ACPI: \_SB_.GSIC: Enabled at IRQ 181513worker2 # [ 0.587976] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1514worker1 # [ 0.580076] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1515client # [ 0.567850] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1516worker2 # [ 0.588989] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1517worker1 # [ 0.581306] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1518lb # [ 0.592720] ACPI: \_SB_.GSID: Enabled at IRQ 191519worker2 # [ 0.590156] NET: Registered PF_UNIX/PF_LOCAL protocol family1520client # [ 0.569384] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1521worker1 # [ 0.582396] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1522worker2 # [ 0.591053] NET: Registered PF_XDP protocol family1523lb # [ 0.594283] PCI: CLS 0 bytes, default 641524worker1 # [ 0.583576] NET: Registered PF_UNIX/PF_LOCAL protocol family1525client # [ 0.570713] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1526worker2 # [ 0.591810] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1527worker1 # [ 0.584517] NET: Registered PF_XDP protocol family1528worker2 # [ 0.592772] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1529lb # [ 0.595155] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1530client # [ 0.571990] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1531worker1 # [ 0.585316] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1532worker2 # [ 0.593725] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1533lb # [ 0.596792] Trying to unpack rootfs image as initramfs...1534client # [ 0.573179] TCP: Hash tables configured (established 16384 bind 16384)1535worker1 # [ 0.586288] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1536worker2 # [ 0.594768] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1537client # [ 0.574224] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1538worker1 # [ 0.587245] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1539worker2 # [ 0.595828] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1540client # [ 0.575476] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1541worker1 # [ 0.588307] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1542worker2 # [ 0.596878] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1543client # [ 0.576547] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1544worker1 # [ 0.589420] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1545worker2 # [ 0.598502] ACPI: \_SB_.GSIA: Enabled at IRQ 161546client # [ 0.577762] NET: Registered PF_UNIX/PF_LOCAL protocol family1547worker1 # [ 0.590538] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1548client # [ 0.578751] NET: Registered PF_XDP protocol family1549worker1 # [ 0.592289] ACPI: \_SB_.GSIA: Enabled at IRQ 161550worker2 # [ 0.600547] ACPI: \_SB_.GSIB: Enabled at IRQ 171551client # [ 0.579634] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1552client # [ 0.580635] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1553worker2 # [ 0.602520] ACPI: \_SB_.GSIC: Enabled at IRQ 181554worker1 # [ 0.594569] ACPI: \_SB_.GSIB: Enabled at IRQ 171555client # [ 0.581687] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1556client # [ 0.582771] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1557worker2 # [ 0.604436] ACPI: \_SB_.GSID: Enabled at IRQ 191558worker1 # [ 0.596437] ACPI: \_SB_.GSIC: Enabled at IRQ 181559client # [ 0.583923] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1560worker2 # [ 0.606115] PCI: CLS 0 bytes, default 641561client # [ 0.585049] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1562worker1 # [ 0.598302] ACPI: \_SB_.GSID: Enabled at IRQ 191563client # [ 0.586853] ACPI: \_SB_.GSIA: Enabled at IRQ 161564worker2 # [ 0.607014] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1565worker1 # [ 0.599931] PCI: CLS 0 bytes, default 641566worker2 # [ 0.608768] Trying to unpack rootfs image as initramfs...1567worker1 # [ 0.600833] Trying to unpack rootfs image as initramfs...1568client # [ 0.589073] ACPI: \_SB_.GSIB: Enabled at IRQ 171569client # [ 0.590991] ACPI: \_SB_.GSIC: Enabled at IRQ 181570client # [ 0.592935] ACPI: \_SB_.GSID: Enabled at IRQ 191571worker1 # [ 0.605059] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1572client # [ 0.594614] PCI: CLS 0 bytes, default 641573client # [ 0.595543] Trying to unpack rootfs image as initramfs...1574client # [ 0.599762] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1575lb # [ 0.643098] Initialise system trusted keyrings1576lb # [ 0.644033] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.649194] Initialise system trusted keyrings1578worker2 # [ 0.652090] workingset: timestamp_bits=40 max_order=19 bucket_order=01579worker1 # [ 0.646122] Initialise system trusted keyrings1580worker1 # [ 0.647093] workingset: timestamp_bits=40 max_order=19 bucket_order=01581client # [ 0.639466] Initialise system trusted keyrings1582client # [ 0.642473] workingset: timestamp_bits=40 max_order=19 bucket_order=01583lb # [ 0.667156] Key type asymmetric registered1584lb # [ 0.667893] Asymmetric key parser 'x509' registered1585lb # [ 0.670753] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1586lb # [ 0.673730] io scheduler mq-deadline registered1587lb # [ 0.674477] io scheduler kyber registered1588worker2 # [ 0.673230] Key type asymmetric registered1589worker2 # [ 0.673942] Asymmetric key parser 'x509' registered1590lb # [ 0.678243] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1591lb # [ 0.679526] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1592worker2 # [ 0.677111] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1593worker1 # [ 0.670783] Key type asymmetric registered1594worker1 # [ 0.671519] Asymmetric key parser 'x509' registered1595worker2 # [ 0.680104] io scheduler mq-deadline registered1596lb # [ 0.683456] Linux agpgart interface v0.1031597worker2 # [ 0.680839] io scheduler kyber registered1598worker1 # [ 0.672372] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1599lb # [ 0.684248] ACPI: bus type drm_connector registered1600worker2 # [ 0.683584] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1601lb # [ 0.687115] usbcore: registered new interface driver usbserial_generic1602client # [ 0.664528] Key type asymmetric registered1603worker2 # [ 0.684871] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1604lb # [ 0.688209] usbserial: USB Serial support registered for generic1605worker1 # [ 0.677870] io scheduler mq-deadline registered1606client # [ 0.665211] Asymmetric key parser 'x509' registered1607worker1 # [ 0.678611] io scheduler kyber registered1608client # [ 0.666044] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1609lb # [ 0.691704] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1610lb # [ 0.691704] Please enable it if your BIOS has the CPPC option.1611worker2 # [ 0.690468] Linux agpgart interface v0.1031612worker1 # [ 0.682230] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1613worker2 # [ 0.691248] ACPI: bus type drm_connector registered1614lb # [ 0.693962] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1615worker1 # [ 0.683453] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1616client # [ 0.671594] io scheduler mq-deadline registered1617client # [ 0.672360] io scheduler kyber registered1618worker2 # [ 0.694521] usbcore: registered new interface driver usbserial_generic1619lb # [ 0.697877] drop_monitor: Initializing network drop monitor service1620worker1 # [ 0.687564] Linux agpgart interface v0.1031621worker2 # [ 0.695586] usbserial: USB Serial support registered for generic1622lb # [ 0.699063] NET: Registered PF_INET6 protocol family1623worker1 # [ 0.688319] ACPI: bus type drm_connector registered1624client # [ 0.676443] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1625client # [ 0.677774] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626worker2 # [ 0.699067] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1627worker2 # [ 0.699067] Please enable it if your BIOS has the CPPC option.1628lb # [ 0.703198] Segment Routing with IPv61629worker1 # [ 0.693131] usbcore: registered new interface driver usbserial_generic1630worker2 # [ 0.701281] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1631worker1 # [ 0.694202] usbserial: USB Serial support registered for generic1632lb # [ 0.706710] In-situ OAM (IOAM) with IPv61633client # [ 0.683231] Linux agpgart interface v0.1031634lb # [ 0.707745] IPI shorthand broadcast: enabled1635client # [ 0.683999] ACPI: bus type drm_connector registered1636worker2 # [ 0.705186] drop_monitor: Initializing network drop monitor service1637worker2 # [ 0.706348] NET: Registered PF_INET6 protocol family1638worker1 # [ 0.697789] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1639worker1 # [ 0.697789] Please enable it if your BIOS has the CPPC option.1640worker1 # [ 0.700093] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1641client # [ 0.688606] usbcore: registered new interface driver usbserial_generic1642worker1 # [ 0.701523] drop_monitor: Initializing network drop monitor service1643worker2 # [ 0.710370] Segment Routing with IPv61644client # [ 0.689728] usbserial: USB Serial support registered for generic1645worker2 # [ 0.711018] In-situ OAM (IOAM) with IPv61646worker1 # [ 0.702737] NET: Registered PF_INET6 protocol family1647worker2 # [ 0.712056] IPI shorthand broadcast: enabled1648worker1 # [ 0.706013] Segment Routing with IPv61649client # [ 0.692443] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1650lb # [ 0.716734] sched_clock: Marking stable (583015378, 133668802)->(789053834, -72369654)1651worker1 # [ 0.706656] In-situ OAM (IOAM) with IPv61652client # [ 0.692443] Please enable it if your BIOS has the CPPC option.1653client # [ 0.694697] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1654worker1 # [ 0.709078] IPI shorthand broadcast: enabled1655lb # [ 0.720770] registered taskstats version 11656client # [ 0.697562] drop_monitor: Initializing network drop monitor service1657lb # [ 0.721848] Loading compiled-in X.509 certificates1658client # [ 0.698719] NET: Registered PF_INET6 protocol family1659worker2 # [ 0.720258] sched_clock: Marking stable (570016979, 150026705)->(790874745, -70831061)1660client # [ 0.703734] Segment Routing with IPv61661worker2 # [ 0.725229] registered taskstats version 11662client # [ 0.704493] In-situ OAM (IOAM) with IPv61663worker1 # [ 0.717252] sched_clock: Marking stable (583014773, 133699071)->(794322628, -77608784)1664worker2 # [ 0.726145] Loading compiled-in X.509 certificates1665client # [ 0.707692] IPI shorthand broadcast: enabled1666worker1 # [ 0.722830] registered taskstats version 11667worker1 # [ 0.723831] Loading compiled-in X.509 certificates1668lb # [ 0.737698] Demotion targets for Node 0: null1669lb # [ 0.739737] Key type .fscrypt registered1670client # [ 0.715776] sched_clock: Marking stable (582016448, 133408512)->(788442867, -73017907)1671lb # [ 0.740389] Key type fscrypt-provisioning registered1672client # [ 0.717352] registered taskstats version 11673lb # [ 0.741364] ima: No TPM chip found, activating TPM-bypass!1674client # [ 0.718321] Loading compiled-in X.509 certificates1675lb # [ 0.744701] ima: Allocated hash algorithm: sha11676lb # [ 0.745512] ima: No architecture policies found1677worker2 # [ 0.744059] Demotion targets for Node 0: null1678worker2 # [ 0.744903] Key type .fscrypt registered1679worker2 # [ 0.745647] Key type fscrypt-provisioning registered1680worker2 # [ 0.746623] ima: No TPM chip found, activating TPM-bypass!1681lb # [ 0.749701] PM: Magic number: 10:840:3631682lb # [ 0.751302] RAS: Correctable Errors collector initialized.1683worker1 # [ 0.740729] Demotion targets for Node 0: null1684worker1 # [ 0.741588] Key type .fscrypt registered1685worker2 # [ 0.751063] ima: Allocated hash algorithm: sha11686worker2 # [ 0.751849] ima: No architecture policies found1687worker1 # [ 0.743730] Key type fscrypt-provisioning registered1688worker2 # [ 0.752809] PM: Magic number: 10:840:3631689worker1 # [ 0.744695] ima: No TPM chip found, activating TPM-bypass!1690worker1 # [ 0.745685] ima: Allocated hash algorithm: sha11691worker2 # [ 0.756456] RAS: Correctable Errors collector initialized.1692lb # [ 0.760351] clk: Disabling unused clocks1693worker1 # [ 0.749756] ima: No architecture policies found1694client # [ 0.737435] Demotion targets for Node 0: null1695worker1 # [ 0.750742] PM: Magic number: 10:840:3631696client # [ 0.738334] Key type .fscrypt registered1697lb # [ 0.762695] PM: genpd: Disabling unused power domains1698client # [ 0.739078] Key type fscrypt-provisioning registered1699worker1 # [ 0.752358] RAS: Correctable Errors collector initialized.1700client # [ 0.740054] ima: No TPM chip found, activating TPM-bypass!1701worker2 # [ 0.765169] clk: Disabling unused clocks1702client # [ 0.744447] ima: Allocated hash algorithm: sha11703worker2 # [ 0.765862] PM: genpd: Disabling unused power domains1704client # [ 0.745204] ima: No architecture policies found1705client # [ 0.746170] PM: Magic number: 10:840:3631706worker1 # [ 0.762455] clk: Disabling unused clocks1707client # [ 0.749943] RAS: Correctable Errors collector initialized.1708worker1 # [ 0.764728] PM: genpd: Disabling unused power domains1709client # [ 0.758688] clk: Disabling unused clocks1710client # [ 0.761448] PM: genpd: Disabling unused power domains1711lb # [ 0.916987] Freeing initrd memory: 29092K1712lb # [ 0.920077] Freeing unused decrypted memory: 2028K1713lb # [ 0.922547] Freeing unused kernel image (initmem) memory: 3644K1714lb # [ 0.923633] Write protecting the kernel read-only data: 32768k1715lb # [ 0.925450] Freeing unused kernel image (text/rodata gap) memory: 1208K1716lb # [ 0.926925] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker2 # [ 0.927944] Freeing initrd memory: 29092K1718worker2 # [ 0.931079] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.925303] Freeing initrd memory: 29092K1720worker2 # [ 0.933597] Freeing unused kernel image (initmem) memory: 3644K1721worker2 # [ 0.934674] Write protecting the kernel read-only data: 32768k1722worker1 # [ 0.928431] Freeing unused decrypted memory: 2028K1723worker2 # [ 0.936550] Freeing unused kernel image (text/rodata gap) memory: 1208K1724worker2 # [ 0.938079] Freeing unused kernel image (rodata/data gap) memory: 752K1725worker1 # [ 0.930950] Freeing unused kernel image (initmem) memory: 3644K1726worker1 # [ 0.932065] Write protecting the kernel read-only data: 32768k1727worker1 # [ 0.933982] Freeing unused kernel image (text/rodata gap) memory: 1208K1728client # [ 0.921767] Freeing initrd memory: 29096K1729worker1 # [ 0.935489] Freeing unused kernel image (rodata/data gap) memory: 752K1730client # [ 0.924933] Freeing unused decrypted memory: 2028K1731client # [ 0.927574] Freeing unused kernel image (initmem) memory: 3644K1732client # [ 0.928692] Write protecting the kernel read-only data: 32768k1733client # [ 0.930610] Freeing unused kernel image (text/rodata gap) memory: 1208K1734client # [ 0.932150] Freeing unused kernel image (rodata/data gap) memory: 752K1735lb # [ 0.977947] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736lb # [ 0.979011] Run /init as init process1737lb # [ 0.988856] systemd[1]: Inserted module 'autofs4'1738worker2 # [ 0.989366] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739worker2 # [ 0.990460] Run /init as init process1740worker1 # [ 0.986516] x86/mm: Checked W+X mappings: passed, no W+X pages found.1741worker1 # [ 0.987621] Run /init as init process1742worker2 # [ 1.000329] systemd[1]: Inserted module 'autofs4'1743lb # [ 1.005308] fuse: init (API version 7.45)1744client # [ 0.983478] x86/mm: Checked W+X mappings: passed, no W+X pages found.1745client # [ 0.984603] Run /init as init process1746worker1 # [ 0.997745] systemd[1]: Inserted module 'autofs4'1747lb # [ 1.011175] ACPI: \_SB_.GSIG: Enabled at IRQ 221748lb # [ 1.013387] ACPI: \_SB_.GSIH: Enabled at IRQ 231749lb # [ 1.016329] ACPI: \_SB_.GSIE: Enabled at IRQ 201750client # [ 0.994721] systemd[1]: Inserted module 'autofs4'1751lb # [ 1.018411] ACPI: \_SB_.GSIF: Enabled at IRQ 211752worker2 # [ 1.016466] fuse: init (API version 7.45)1753worker1 # [ 1.013924] fuse: init (API version 7.45)1754worker2 # [ 1.022363] ACPI: \_SB_.GSIG: Enabled at IRQ 221755worker2 # [ 1.024756] ACPI: \_SB_.GSIH: Enabled at IRQ 231756worker2 # [ 1.027671] ACPI: \_SB_.GSIE: Enabled at IRQ 201757worker1 # [ 1.019770] ACPI: \_SB_.GSIG: Enabled at IRQ 221758worker2 # [ 1.029849] ACPI: \_SB_.GSIF: Enabled at IRQ 211759worker1 # [ 1.022443] ACPI: \_SB_.GSIH: Enabled at IRQ 231760client # [ 1.011192] fuse: init (API version 7.45)1761worker1 # [ 1.026437] ACPI: \_SB_.GSIE: Enabled at IRQ 201762worker1 # [ 1.029006] ACPI: \_SB_.GSIF: Enabled at IRQ 211763client # [ 1.017108] ACPI: \_SB_.GSIG: Enabled at IRQ 221764client # [ 1.019588] ACPI: \_SB_.GSIH: Enabled at IRQ 231765client # [ 1.023520] ACPI: \_SB_.GSIE: Enabled at IRQ 201766lb # [ 1.046971] systemd[1]: Successfully made /usr/ read-only.1767client # [ 1.025927] ACPI: \_SB_.GSIF: Enabled at IRQ 211768worker2 # [ 1.056706] systemd[1]: Successfully made /usr/ read-only.1769worker1 # [ 1.057549] systemd[1]: Successfully made /usr/ read-only.1770client # [ 1.057215] systemd[1]: Successfully made /usr/ read-only.1771lb # [ 1.382947] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772lb # [ 1.387689] systemd[1]: Detected virtualization kvm.1773lb # [ 1.388510] systemd[1]: Detected architecture x86-64.1774lb # [ 1.389814] systemd[1]: Running in initrd.1775lb # [ 1.391783] systemd[1]: Initializing machine ID from random generator.1776lb # [ 1.394059] systemd[1]: Hostname set to <lb>.1777worker2 # [ 1.391781] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker2 # [ 1.396905] systemd[1]: Detected virtualization kvm.1779worker2 # [ 1.397807] systemd[1]: Detected architecture x86-64.1780worker2 # [ 1.398715] systemd[1]: Running in initrd.1781worker2 # [ 1.399822] systemd[1]: Initializing machine ID from random generator.1782worker2 # [ 1.401098] systemd[1]: Hostname set to <worker2>.1783worker1 # [ 1.393061] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784worker1 # [ 1.398172] systemd[1]: Detected virtualization kvm.1785worker1 # [ 1.399061] systemd[1]: Detected architecture x86-64.1786worker1 # [ 1.399971] systemd[1]: Running in initrd.1787worker1 # [ 1.401117] systemd[1]: Initializing machine ID from random generator.1788worker1 # [ 1.402351] systemd[1]: Hostname set to <worker1>.1789client # [ 1.393734] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790client # [ 1.405827] systemd[1]: Detected virtualization kvm.1791client # [ 1.407923] systemd[1]: Detected architecture x86-64.1792client # [ 1.410039] systemd[1]: Running in initrd.1793client # [ 1.412558] systemd[1]: Initializing machine ID from random generator.1794client # [ 1.415359] systemd[1]: Hostname set to <client>.1795lb # [ 1.595170] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker2 # [ 1.593923] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797worker1 # [ 1.594988] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798lb # [ 1.636128] systemd[1]: Queued start job for default target Initrd Default Target.1799worker2 # [ 1.634497] systemd[1]: Queued start job for default target Initrd Default Target.1800lb # [ 1.640134] systemd[1]: Created slice Slice /system/modprobe.1801worker2 # [ 1.638497] systemd[1]: Created slice Slice /system/modprobe.1802lb # [ 1.641316] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1803lb # [ 1.642698] systemd[1]: Expecting device /dev/disk/by-label/nixos...1804worker2 # [ 1.639648] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1805lb # [ 1.643737] systemd[1]: Reached target Path Units.1806worker2 # [ 1.641060] systemd[1]: Expecting device /dev/disk/by-label/nixos...1807lb # [ 1.644546] systemd[1]: Reached target Slice Units.1808worker2 # [ 1.642150] systemd[1]: Reached target Path Units.1809worker1 # [ 1.633898] systemd[1]: Queued start job for default target Initrd Default Target.1810lb # [ 1.645469] systemd[1]: Reached target Swaps.1811worker2 # [ 1.642963] systemd[1]: Reached target Slice Units.1812lb # [ 1.646258] systemd[1]: Reached target Timer Units.1813worker2 # [ 1.643819] systemd[1]: Reached target Swaps.1814worker2 # [ 1.644657] systemd[1]: Reached target Timer Units.1815lb # [ 1.647298] systemd[1]: Listening on D-Bus System Message Bus Socket.1816worker1 # [ 1.637053] systemd[1]: Created slice Slice /system/modprobe.1817lb # [ 1.648479] systemd[1]: Listening on Journal Socket (/dev/log).1818worker2 # [ 1.645661] systemd[1]: Listening on D-Bus System Message Bus Socket.1819worker1 # [ 1.638228] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1820lb # [ 1.649606] systemd[1]: Listening on Journal Sockets.1821worker2 # [ 1.646904] systemd[1]: Listening on Journal Socket (/dev/log).1822lb # [ 1.650600] systemd[1]: Listening on udev Control Socket.1823worker1 # [ 1.639609] systemd[1]: Expecting device /dev/disk/by-label/nixos...1824worker2 # [ 1.648123] systemd[1]: Listening on Journal Sockets.1825lb # [ 1.651597] systemd[1]: Listening on udev Kernel Socket.1826worker1 # [ 1.640763] systemd[1]: Reached target Path Units.1827worker2 # [ 1.649164] systemd[1]: Listening on udev Control Socket.1828worker1 # [ 1.641644] systemd[1]: Reached target Slice Units.1829lb # [ 1.652598] systemd[1]: Reached target Socket Units.1830worker2 # [ 1.650159] systemd[1]: Listening on udev Kernel Socket.1831worker1 # [ 1.642544] systemd[1]: Reached target Swaps.1832worker2 # [ 1.651109] systemd[1]: Reached target Socket Units.1833worker1 # [ 1.643347] systemd[1]: Reached target Timer Units.1834lb # [ 1.654468] systemd[1]: Starting Create List of Static Device Nodes...1835worker1 # [ 1.644311] systemd[1]: Listening on D-Bus System Message Bus Socket.1836worker2 # [ 1.653088] systemd[1]: Starting Create List of Static Device Nodes...1837worker1 # [ 1.645600] systemd[1]: Listening on Journal Socket (/dev/log).1838worker1 # [ 1.646799] systemd[1]: Listening on Journal Sockets.1839worker1 # [ 1.647809] systemd[1]: Listening on udev Control Socket.1840worker1 # [ 1.648860] systemd[1]: Listening on udev Kernel Socket.1841worker2 # [ 1.656865] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1842lb # [ 1.659766] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1843worker1 # [ 1.649806] systemd[1]: Reached target Socket Units.1844worker1 # [ 1.651601] systemd[1]: Starting Create List of Static Device Nodes...1845client # [ 1.640245] systemd[1]: bpf-restrict-fs: LSM BPF program attached1846lb # [ 1.665409] systemd[1]: Starting Load Kernel Module configfs...1847worker1 # [ 1.655601] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1848worker2 # [ 1.664123] systemd[1]: Starting Load Kernel Module configfs...1849worker1 # [ 1.662397] systemd[1]: Starting Load Kernel Module configfs...1850lb # [ 1.679880] systemd[1]: Starting Journal Service...1851worker2 # [ 1.677517] systemd[1]: Starting Journal Service...1852worker1 # [ 1.676298] systemd[1]: Starting Journal Service...1853worker2 # [ 1.687224] systemd[1]: Starting Load Kernel Modules...1854lb # [ 1.693788] systemd[1]: Starting Load Kernel Modules...1855worker1 # [ 1.689176] systemd[1]: Starting Load Kernel Modules...1856worker2 # [ 1.695132] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1857lb # [ 1.700746] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1858lb # [ 1.717390] netfs: FS-Cache loaded1859worker2 # [ 1.715280] netfs: FS-Cache loaded1860client # [ 1.694714] systemd[1]: Queued start job for default target Initrd Default Target.1861worker1 # [ 1.696791] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1862client # [ 1.698765] systemd[1]: Created slice Slice /system/modprobe.1863worker2 # [ 1.720045] 9pnet: Installing 9P2000 support1864worker1 # [ 1.712923] netfs: FS-Cache loaded1865client # [ 1.699974] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866lb # [ 1.724975] 9pnet: Installing 9P2000 support1867client # [ 1.701437] systemd[1]: Expecting device /dev/disk/by-label/nixos...1868client # [ 1.702548] systemd[1]: Reached target Path Units.1869client # [ 1.703463] systemd[1]: Reached target Slice Units.1870client # [ 1.704380] systemd[1]: Reached target Swaps.1871worker1 # [ 1.717641] 9pnet: Installing 9P2000 support1872client # [ 1.705195] systemd[1]: Reached target Timer Units.1873client # [ 1.706141] systemd[1]: Listening on D-Bus System Message Bus Socket.1874client # [ 1.707431] systemd[1]: Listening on Journal Socket (/dev/log).1875client # [ 1.708622] systemd[1]: Listening on Journal Sockets.1876client # [ 1.709708] systemd[1]: Listening on udev Control Socket.1877worker2 # [ 1.731315] systemd[1]: Starting Coldplug All udev Devices...1878client # [ 1.710774] systemd[1]: Listening on udev Kernel Socket.1879client # [ 1.711751] systemd[1]: Reached target Socket Units.1880lb # [ 1.735232] systemd[1]: Starting Coldplug All udev Devices...1881client # [ 1.714501] systemd[1]: Starting Create List of Static Device Nodes...1882worker1 # [ 1.727912] systemd[1]: Starting Coldplug All udev Devices...1883client # [ 1.717489] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1884client # [ 1.724497] systemd[1]: Starting Load Kernel Module configfs...1885lb # [ 1.749373] systemd-journald[67]: Collecting audit messages is disabled.1886worker2 # [ 1.748137] systemd[1]: Finished Create List of Static Device Nodes.1887lb # [ 1.752126] systemd[1]: Finished Create List of Static Device Nodes.1888worker1 # [ 1.741309] systemd[1]: Finished Create List of Static Device Nodes.1889worker2 # [ 1.751794] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1890worker1 # [ 1.746303] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1891lb # [ 1.757563] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1892client # [ 1.737497] systemd[1]: Starting Journal Service...1893worker2 # [ 1.761126] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1894lb # [ 1.766130] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1895worker1 # [ 1.755768] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1896worker2 # [ 1.764624] systemd-journald[67]: Collecting audit messages is disabled.1897worker1 # [ 1.759461] systemd-journald[67]: Collecting audit messages is disabled.1898worker2 # [ 1.768466] systemd[1]: modprobe@configfs.service: Deactivated successfully.1899lb # [ 1.772070] systemd[1]: modprobe@configfs.service: Deactivated successfully.1900worker1 # [ 1.762139] systemd[1]: modprobe@configfs.service: Deactivated successfully.1901client # [ 1.751517] systemd[1]: Starting Load Kernel Modules...1902worker2 # [ 1.774417] systemd[1]: Finished Load Kernel Module configfs.1903worker1 # [ 1.768895] systemd[1]: Finished Load Kernel Module configfs.1904lb # [ 1.780070] systemd[1]: Finished Load Kernel Module configfs.1905worker2 # [ 1.779500] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1906worker1 # [ 1.772560] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1907lb # [ 1.784181] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1908worker2 # [ 1.781485] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1909lb # [ 1.786772] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1910worker1 # [ 1.776230] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1911worker2 # [ 1.788060] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1912client # [ 1.758505] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1913worker2 # [ 1.793122] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1914lb # [ 1.795929] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1915worker1 # [ 1.785800] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1916lb # [ 1.797285] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1917client # [ 1.774731] netfs: FS-Cache loaded1918worker1 # [ 1.787357] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1919client # [ 1.779477] 9pnet: Installing 9P2000 support1920client # [ 1.787697] systemd[1]: Starting Coldplug All udev Devices...1921worker2 # [ 1.819643] systemd[1]: Finished Load Kernel Modules.1922client # [ 1.801524] systemd[1]: Finished Create List of Static Device Nodes.1923lb # [ 1.827928] systemd[1]: Finished Load Kernel Modules.1924worker1 # [ 1.817224] systemd[1]: Finished Load Kernel Modules.1925worker2 # [ 1.827238] systemd[1]: Starting Apply Kernel Variables...1926client # [ 1.808008] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1927worker1 # [ 1.823996] systemd[1]: Starting Apply Kernel Variables...1928lb # [ 1.835914] systemd[1]: Starting Apply Kernel Variables...1929client # [ 1.815078] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1930worker2 # [ 1.839132] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1931worker1 # [ 1.835105] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1932client # [ 1.822848] systemd[1]: modprobe@configfs.service: Deactivated successfully.1933client # [ 1.824671] systemd-journald[67]: Collecting audit messages is disabled.1934lb # [ 1.849783] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1935client # [ 1.827795] systemd[1]: Finished Load Kernel Module configfs.1936worker2 # [ 1.850267] systemd[1]: Starting Create Static Device Nodes in /dev...1937worker1 # [ 1.844922] systemd[1]: Starting Create Static Device Nodes in /dev...1938lb # [ 1.858097] systemd[1]: Starting Create Static Device Nodes in /dev...1939client # [ 1.833927] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1940client # [ 1.836850] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1941worker2 # [ 1.861194] systemd[1]: Finished Apply Kernel Variables.1942worker1 # [ 1.858831] systemd[1]: Finished Apply Kernel Variables.1943client # [ 1.846503] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1944client # [ 1.847819] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1945lb # [ 1.736648] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.872722] systemd[1]: Started Journal Service.1946lb # 1947lb # [ 1.741313] systemd-modules-load[68]: Inserted module 'virtio_balloon'1948lb # [ 1.742392] systemd-modules-load[68]: Inserted module 'virtio_gpu'1949worker2 # [ 1.877081] systemd[1]: Started Journal Service.1950lb # [ 1.749273] systemd[1]: Finished Apply Kernel Variables.1951worker2 # [ 1.730209] systemd-modules-load[68]: Inserted module 'dm_mod'1952worker2 # [ 1.732487] systemd-modules-load[68]: Inserted module 'virtio_balloon'1953worker2 # [ 1.735305] systemd-modules-load[68]: Inserted module 'virtio_gpu'1954worker1 # [ 1.743315] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.878081] systemd[1]: Started Journal Service.1955worker2 # [ 1.736292] systemd[1]: Finished Create Static Device Nodes in /dev.1956worker1 # 1957worker1 # [ 1.746214] systemd-modules-load[68]: Inserted module 'virtio_balloon'1958worker2 # [ 1.738348] systemd[1]: Reached target Preparation for Local File Systems.1959worker1 # [ 1.748236] systemd-modules-load[68]: Inserted module 'virtio_gpu'1960worker2 # [ 1.741295] systemd[1]: Reached target Local File Systems.1961worker1 # [ 1.751275] systemd[1]: Finished Create Static Device Nodes in /dev.1962worker2 # [ 1.744069] systemd[1]: Starting Create System Files and Directories...1963worker1 # [ 1.752423] systemd[1]: Reached target Preparation for Local File Systems.1964worker1 # [ 1.753537] systemd[1]: Reached target Local File Systems.1965lb # [ 1.765135] systemd[1]: Finished Create Static Device Nodes in /dev.1966lb # [ 1.766162] systemd[1]: Reached target Preparation for Local File Systems.1967lb # [ 1.767097] systemd[1]: Reached target Local File Systems.1968client # [ 1.876968] systemd[1]: Finished Load Kernel Modules.1969worker2 # [ 1.747883] systemd[1]: Starting Rule-based Manager for Device Events and Files...1970worker1 # [ 1.758094] systemd[1]: Starting Create System Files and Directories...1971lb # [ 1.769825] systemd[1]: Starting Create System Files and Directories...1972worker1 # [ 1.763154] systemd[1]: Starting Rule-based Manager for Device Events and Files...1973client # [ 1.885497] systemd[1]: Starting Apply Kernel Variables...1974lb # [ 1.777078] systemd[1]: Starting Rule-based Manager for Device Events and Files...1975client # [ 1.895546] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1976worker2 # [ 1.773573] systemd[1]: Finished Create System Files and Directories.1977client # [ 1.906519] systemd[1]: Starting Create Static Device Nodes in /dev...1978worker1 # [ 1.790476] systemd[1]: Finished Create System Files and Directories.1979lb # [ 1.802212] systemd[1]: Finished Create System Files and Directories.1980client # [ 1.926026] systemd[1]: Finished Apply Kernel Variables.1981worker2 # [ 1.803246] systemd-udevd[77]: Using default interface naming scheme 'v261'.1982lb # [ 1.829410] systemd-udevd[83]: Using default interface naming scheme 'v261'.1983worker1 # [ 1.819301] systemd-udevd[77]: Using default interface naming scheme 'v261'.1984client # [ 1.807648] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.942627] systemd[1]: Started Journal Service.1985client # 1986client # [ 1.811312] systemd-modules-load[68]: Inserted module 'virtio_balloon'1987client # [ 1.812813] systemd-modules-load[68]: Inserted module 'virtio_gpu'1988client # [ 1.814335] systemd[1]: Finished Create Static Device Nodes in /dev.1989worker2 # [ 1.827069] systemd[1]: Started Rule-based Manager for Device Events and Files.1990client # [ 1.824089] systemd[1]: Reached target Preparation for Local File Systems.1991client # [ 1.826096] systemd[1]: Reached target Local File Systems.1992client # [ 1.829054] systemd[1]: Starting Create System Files and Directories...1993lb # [ 1.853308] systemd[1]: Started Rule-based Manager for Device Events and Files.1994worker1 # [ 1.842635] systemd[1]: Started Rule-based Manager for Device Events and Files.1995client # [ 1.836070] systemd[1]: Starting Rule-based Manager for Device Events and Files...1996client # [ 1.860384] systemd[1]: Finished Create System Files and Directories.1997worker2 # [ 1.877678] systemd[1]: Finished Coldplug All udev Devices.1998worker2 # [ 1.878930] systemd[1]: Reached target System Initialization.1999worker1 # [ 1.888699] systemd[1]: Finished Coldplug All udev Devices.2000worker2 # [ 1.881316] systemd[1]: Reached target Basic System.
loading 2,054 hidden lines…
4055lb # [ 17.589986] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4056lb # [ 17.592904] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4057lb # [ 17.596711] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4058lb # [ 17.601078] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4059lb # [ 17.611560] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4060lb # [ 17.614316] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4061lb # [ 17.619599] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4062lb # [ 17.621696] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4063lb # [ 17.625263] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4064lb # [ 17.628061] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4065lb # [ 17.633739] envoy[910]: [2026-09-21 04:23:29.288][910][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4066lb # [ 17.637469] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4067lb # [ 17.640569] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4068lb # [ 17.644094] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4069lb # [ 17.646187] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4070lb # [ 17.649137] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4071lb # [ 17.651161] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4072lb # [ 17.653507] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4073lb # [ 17.656142] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4074lb # [ 17.657968] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4075lb # [ 17.663470] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4076lb # [ 17.667181] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4077lb # [ 17.670039] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4078lb # [ 17.672849] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4079lb # [ 17.676083] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4080lb # [ 17.678383] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4081lb # [ 17.681408] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4082lb # [ 17.683274] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4083lb # [ 17.687830] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4084lb # [ 17.689493] envoy[910]: [2026-09-21 04:23:29.289][910][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4085lb # [ 17.692100] envoy[910]: [2026-09-21 04:23:29.444][910][info][main] [source/server/server.cc:531] HTTP header map info:4086lb # [ 17.694678] envoy[910]: [2026-09-21 04:23:29.514][910][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4087lb # [ 17.710491] envoy[910]: [2026-09-21 04:23:29.514][910][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4088lb # [ 17.712157] envoy[910]: [2026-09-21 04:23:29.514][910][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4089lb # [ 17.720808] envoy[910]: [2026-09-21 04:23:29.514][910][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4090lb # [ 17.772534] envoy[910]: [2026-09-21 04:23:29.720][910][info][main] [source/server/server.cc:968] runtime: {}4091lb # [ 17.781910] envoy[910]: [2026-09-21 04:23:29.729][910][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4092lb # [ 17.784570] envoy[910]: [2026-09-21 04:23:29.732][910][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014093lb # [ 17.787778] envoy[910]: [2026-09-21 04:23:29.735][910][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014094lb # [ 17.791617] envoy[910]: [2026-09-21 04:23:29.739][910][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4095lb # [ 17.794720] envoy[910]: [2026-09-21 04:23:29.742][910][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4096lb # [ 17.796557] envoy[910]: [2026-09-21 04:23:29.742][910][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4097lb # [ 17.855724] envoy[910]: [2026-09-21 04:23:29.803][910][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4098worker2 # [ 17.840652] systemd[1]: Reached target Network is Online.4099worker1 # [ 17.850718] systemd[1]: Starting Nix worker-protocol over gRPC...4100worker2 # [ 17.847646] systemd[1]: Starting scheduler-token.service...4101client # [ 17.852762] dhcpcd[565]: eth0: leased 10.0.2.15 for 86400 seconds4102client # [ 17.857169] dhcpcd[565]: eth0: adding route to 10.0.2.0/244103client # [ 17.859355] dhcpcd[565]: eth0: adding default route via 10.0.2.24104lb # [ 17.884063] envoy[910]: [2026-09-21 04:23:29.831][910][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4105lb # [ 17.886632] envoy[910]: [2026-09-21 04:23:29.834][910][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4106lb # [ 17.889919] envoy[910]: [2026-09-21 04:23:29.834][910][info][main] [source/server/server.cc:1077] starting main dispatch loop4107worker2 # [ 17.964577] systemd[1]: scheduler-token.service: Deactivated successfully.4108worker2 # [ 17.965835] systemd[1]: Finished scheduler-token.service.4109worker2 # [ 17.967389] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 117ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4110worker2 # [ 17.972433] systemd[1]: Starting Nix worker-protocol over gRPC...4111client # [ 17.985211] systemd[1]: Started DHCP Client.4112client # [ 17.986932] systemd[1]: Reached target Multi-User System.4113client # [ 17.988151] systemd[1]: Startup finished in 854ms (kernel) + 5.015s (initrd) + 12.117s (userspace) = 17.987s.4114worker1 # [ 18.099088] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4115worker1 # [ 18.109296] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4116worker1 # [ 18.116341] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14117worker1 # [ 18.161286] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4118worker1 # [ 18.164891] systemd[1]: Started Nix worker-protocol over gRPC.4119worker2 # [ 18.167407] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4120worker2 # [ 18.177118] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4121worker2 # [ 18.184283] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14122lb # [ 18.233524] envoy[910]: [2026-09-21 04:23:30.181][910][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4123lb # [ 18.238662] envoy[910]: [2026-09-21 04:23:30.181][910][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4124worker2 # [ 18.225769] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4125lb # [ 18.244185] envoy[910]: [2026-09-21 04:23:30.181][910][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4126worker2 # [ 18.228900] systemd[1]: Started Nix worker-protocol over gRPC.4127lb # [ 18.248578] envoy[910]: [2026-09-21 04:23:30.185][910][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4128worker1 # [ 18.266133] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=scheduler_connected addr=lb:500514129worker1 # [ 18.279109] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4130worker2 # [ 18.299528] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4131worker1 # [ 18.480733] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=scheduler_connected addr=lb:500514132worker1 # [ 18.484735] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker2 # [ 18.503743] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=scheduler_connected addr=lb:500514134worker2 # [ 18.507621] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker1 # [ 18.685547] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=scheduler_connected addr=lb:500514136worker1 # [ 18.689654] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker2 # [ 18.708657] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=scheduler_connected addr=lb:500514138worker2 # [ 18.712487] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker1 # [ 18.890805] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=scheduler_connected addr=lb:500514140worker1 # [ 18.895291] nix-grpc-daemon[786]: ts=2026-09-21T04:23:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4141worker2 # [ 18.913957] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=scheduler_connected addr=lb:500514142worker2 # [ 18.917789] nix-grpc-daemon[785]: ts=2026-09-21T04:23:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker1 # [ 19.093220] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514144worker1 # [ 19.097771] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 19.115835] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 19.119663] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker1 # [ 19.300667] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514148worker1 # [ 19.304305] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 19.321739] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 19.324942] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 19.501588] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514152worker1 # [ 19.506417] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker2 # [ 19.522554] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 19.526316] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker1 # [ 19.707352] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514156worker1 # [ 19.711884] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 19.727519] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 19.731510] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 19.912760] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514160worker1 # [ 19.917745] nix-grpc-daemon[786]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker2 # [ 19.932376] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 19.935446] nix-grpc-daemon[785]: ts=2026-09-21T04:23:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker1 # [ 20.118700] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 20.123428] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 20.136798] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 20.140685] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker1 # [ 20.325888] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514168worker1 # [ 20.329348] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker2 # [ 20.343120] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 20.346256] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker1 # [ 20.526986] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514172worker1 # [ 20.531278] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker2 # [ 20.544340] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 20.548150] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker1 # [ 20.732075] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514176worker1 # [ 20.736336] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 20.748595] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514178worker2 # [ 20.750220] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker1 # [ 20.937394] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514180worker1 # [ 20.941964] nix-grpc-daemon[786]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker2 # [ 20.951487] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 20.955239] nix-grpc-daemon[785]: ts=2026-09-21T04:23:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker1 # [ 21.142822] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 21.147330] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker2 # [ 21.156544] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 21.160369] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker1 # [ 21.349741] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 21.353224] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker2 # [ 21.362766] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514190worker2 # [ 21.365894] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker1 # [ 21.550605] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 21.555443] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 21.563722] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 21.567541] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 21.756970] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514196worker1 # [ 21.761436] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker2 # [ 21.768734] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514198worker2 # [ 21.772458] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker1 # [ 21.962546] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 21.966975] nix-grpc-daemon[786]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker2 # [ 21.973607] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 21.977471] nix-grpc-daemon[785]: ts=2026-09-21T04:23:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker1 # [ 22.167826] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 22.171801] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker2 # [ 22.178406] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 22.182240] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 22.374584] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514208worker1 # [ 22.377943] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker2 # [ 22.384498] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 22.387570] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 22.575651] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 22.580322] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 22.585896] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 22.589671] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker1 # [ 22.781112] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 22.785227] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 22.790745] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514218worker2 # [ 22.794542] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker1 # [ 22.986202] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 22.990135] nix-grpc-daemon[786]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 22.995410] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 22.999138] nix-grpc-daemon[785]: ts=2026-09-21T04:23:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 23.190645] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514224worker1 # [ 23.193761] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 23.200240] nix-grpc-daemon[785]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514226worker2 # [ 23.204079] nix-grpc-daemon[785]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker1 # [ 23.395808] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 23.399296] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 23.406698] nix-grpc-daemon[785]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 23.409804] nix-grpc-daemon[785]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker1 # [ 23.600160] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 23.604185] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker2 # [ 23.607819] nix-grpc-daemon[785]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 23.611795] nix-grpc-daemon[785]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker1 # [ 23.804985] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 23.808927] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 23.812901] nix-grpc-daemon[785]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 23.816712] nix-grpc-daemon[785]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker1 # [ 24.009824] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_connected addr=lb:500514240worker1 # [ 24.013799] nix-grpc-daemon[786]: ts=2026-09-21T04:23:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker2 # [ 24.018070] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 24.022730] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker1 # [ 24.214935] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514244worker1 # [ 24.219098] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker2 # [ 24.223698] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514246worker2 # [ 24.227478] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker1 # [ 24.421391] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 24.424524] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker2 # [ 24.428861] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514250worker2 # [ 24.430528] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker1 # [ 24.622166] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514252worker1 # [ 24.626349] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker2 # [ 24.629970] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 24.632070] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker1 # [ 24.827192] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514256worker1 # [ 24.831265] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker2 # [ 24.833227] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514258worker2 # [ 24.836925] nix-grpc-daemon[785]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker1 # [ 25.032353] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_connected addr=lb:500514260worker1 # [ 25.036460] nix-grpc-daemon[786]: ts=2026-09-21T04:23:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker2 # [ 25.038197] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514262worker2 # [ 25.042252] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker1 # [ 25.236715] nix-grpc-daemon[786]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514264worker1 # [ 25.240664] nix-grpc-daemon[786]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker2 # [ 25.243087] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514266worker2 # [ 25.246806] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267lb # [ 25.279872] rustfs-bucket-start[993]: mb s3://farm4268lb # [ 25.285554] systemd[1]: Finished rustfs-bucket.service.4269lb # [ 25.291768] systemd[1]: Starting niks3 server...4270worker1 # [ 25.443316] nix-grpc-daemon[786]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514271worker1 # [ 25.446570] nix-grpc-daemon[786]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker2 # [ 25.449164] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514273worker2 # [ 25.452195] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4274lb # [ 25.512594] postgres[1013]: [1013] ERROR: relation "goose_db_version" does not exist at character 364275lb # [ 25.513814] postgres[1013]: [1013] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4276lb # [ 25.551632] niks3-server[1003]: 2026/09/21 04:23:37 OK 20241026095416_initial_model.sql (20.27ms)4277lb # [ 25.555715] niks3-server[1003]: 2026/09/21 04:23:37 OK 20251210153512_drop_unused_gin_index.sql (2.02ms)4278lb # [ 25.560841] niks3-server[1003]: 2026/09/21 04:23:37 OK 20251218171726_add_pins.sql (3.75ms)4279lb # [ 25.565623] niks3-server[1003]: 2026/09/21 04:23:37 OK 20260628120000_add_object_size_and_stats.sql (4.84ms)4280lb # [ 25.570340] niks3-server[1003]: 2026/09/21 04:23:37 OK 20260905000000_add_claims.sql (3.41ms)4281lb # [ 25.574360] niks3-server[1003]: 2026/09/21 04:23:37 OK 20260920000000_drop_claims.sql (2.87ms)4282lb # [ 25.575771] niks3-server[1003]: 2026/09/21 04:23:37 goose: successfully migrated database to version: 202609200000004283lb # [ 25.581216] niks3-server[1003]: 2026/09/21 04:23:37 OK 1_commit_pending_closure.sql (5.48ms)4284lb # [ 25.584041] niks3-server[1003]: 2026/09/21 04:23:37 OK 2_object_stats_trigger.sql (1.59ms)4285lb # [ 25.585206] niks3-server[1003]: 2026/09/21 04:23:37 goose: up to current file version: 24286lb # [ 25.591941] niks3-server[1003]: 2026/09/21 04:23:37 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4287worker1 # [ 25.643801] nix-grpc-daemon[786]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514288worker1 # [ 25.647863] nix-grpc-daemon[786]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker2 # [ 25.649834] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514290lb # [ 25.672191] niks3-server[1003]: 2026/09/21 04:23:37 INFO Read proxy enabled — serving cache objects from S34291lb # [ 25.673997] niks3-server[1003]: 2026/09/21 04:23:37 INFO Using socket-activated listener address=0.0.0.0:57514292worker2 # [ 25.653691] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293lb # [ 25.678060] niks3-server[1003]: 2026/09/21 04:23:37 INFO systemd watchdog enabled interval=15s4294lb # [ 25.679965] systemd[1]: Started niks3 server.4295lb # [ 25.681701] niks3-server[1003]: 2026/09/21 04:23:37 INFO Starting HTTP server address=0.0.0.0:57514296lb # [ 25.684087] systemd[1]: Reached target Multi-User System.4297lb # [ 25.685501] systemd[1]: Startup finished in 848ms (kernel) + 5.019s (initrd) + 19.816s (userspace) = 25.684s.4298lb # [ 25.691146] niks3-server[1003]: 2026/09/21 04:23:37 INFO lead: acquired remote=192.168.1.4:575184299worker2 # [ 25.678711] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_take_over4300worker1 # [ 25.848929] nix-grpc-daemon[786]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514301worker1 # [ 25.853876] nix-grpc-daemon[786]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4302worker2 # [ 25.855248] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_connected addr=lb:500514303worker2 # [ 25.859199] nix-grpc-daemon[785]: ts=2026-09-21T04:23:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4304worker1 # [ 26.054196] nix-grpc-daemon[786]: ts=2026-09-21T04:23:38Z level=info event=scheduler_connected addr=lb:500514305worker1 # [ 26.058299] nix-grpc-daemon[786]: ts=2026-09-21T04:23:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4306worker2 # [ 26.060161] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=scheduler_connected addr=lb:500514307worker2 # [ 26.063929] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4308worker1 # [ 26.258990] nix-grpc-daemon[786]: ts=2026-09-21T04:23:38Z level=info event=scheduler_connected addr=lb:500514309worker1 # [ 26.262865] nix-grpc-daemon[786]: ts=2026-09-21T04:23:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310worker2 # [ 26.264730] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=scheduler_connected addr=lb:500514311worker2 # [ 26.268393] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4312worker1 # [ 26.464535] nix-grpc-daemon[786]: ts=2026-09-21T04:23:38Z level=info event=scheduler_connected addr=lb:500514313worker2 # [ 26.472675] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=scheduler_connected addr=lb:500514314worker2 # [ 26.476933] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:445664315worker2 # [ 26.482198] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04316worker2 # [ 26.503517] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=oidc_jwks provider=mock keys=14317worker2 # [ 26.507793] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:445664318lb: (finished: waiting for unit niks3.service, in 14.49 seconds)4319lb: waiting for TCP port 5751 on localhost4320worker2 # [ 26.511457] nix-grpc-daemon[785]: ts=2026-09-21T04:23:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04321lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4322lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4323lb: waiting for unit envoy.service4324lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4325lb: waiting for TCP port 50051 on localhost4326lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4327lb # [ 26.684989] envoy[910]: conn peer=[::1]:45864 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24328lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4329worker2 # [ 26.794353] systemd[1]: Starting scheduler-token.service...4330worker2 # [ 26.863564] systemd[1]: scheduler-token.service: Deactivated successfully.4331worker2 # [ 26.865198] systemd[1]: Finished scheduler-token.service.4332worker2 # [ 26.866578] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 70ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4333lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4334lb: must succeed: curl -sf localhost:9901/clusters4335lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4336lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4337subtest: one node schedules and both builders hold a WorkerSession on it4338lb: must succeed: curl -sf localhost:9901/clusters4339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4340worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4341worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4342lb: must succeed: curl -sf localhost:9901/clusters4343lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4344lb: must succeed: curl -sf localhost:9901/clusters4345lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4346lb: must succeed: curl -sf localhost:9901/clusters4347lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4348lb: must succeed: curl -sf localhost:9901/clusters4349lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4350worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4351worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4352worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4353worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4354lb: must succeed: curl -sf localhost:9901/clusters4355lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4356lb: must succeed: curl -sf localhost:9901/clusters4357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4358worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4359worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4360worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4361worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4362worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4363worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4364worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4365worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4366(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.67 seconds)4367subtest: balancer auth: client cert, bearer token, nothing, unknown CN4368worker2 # [ 27.912370] nix-grpc-daemon[785]: ts=2026-09-21T04:23:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:445684369worker2 # [ 27.925273] systemd[1]: Started Nix Daemon.4370worker2 # [ 28.142090] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4371worker2 # [ 28.167677] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 1/5)4372worker2 # [ 28.262263] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 160 ms (attempt 2/5)4373worker2 # [ 28.422787] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 3/5)4374worker2 # [ 28.501590] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 460 ms (attempt 4/5)4375worker2 # [ 28.971839] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4376worker2 # [ 28.975326] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4377worker2 # [ 29.006903] nix-grpc-daemon[785]: ts=2026-09-21T04:23:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:445684378worker2 # [ 29.023161] nix-grpc-daemon[785]: ts=2026-09-21T04:23:41Z level=debug event=path_info_miss path=z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test error="error: path '/nix/store/z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test' is required, but there is no substituter that can build it"4379worker2 # [ 29.034065] nix-grpc-daemon[785]: ts=2026-09-21T04:23:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:445684380worker2 # [ 29.042879] nix-grpc-daemon[785]: ts=2026-09-21T04:23:41Z level=debug event=path_info_miss path=z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test error="error: path '/nix/store/z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test' is required, but there is no substituter that can build it"4381client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4382client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4383client: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4384client: (finished: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4385worker1 # [ 29.453528] nix-grpc-daemon[786]: ts=2026-09-21T04:23:41Z level=info event=oidc_jwks provider=mock keys=14386worker1 # [ 29.457558] nix-grpc-daemon[786]: ts=2026-09-21T04:23:41Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:443124387worker1 # [ 29.471666] systemd[1]: Started Nix Daemon.4388worker1 # [ 29.722642] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4389worker1 # [ 29.753665] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)4390worker1 # [ 29.763298] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 199 ms (attempt 2/5)4391worker1 # [ 29.964907] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 194 ms (attempt 3/5)4392worker1 # [ 30.161879] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 756 ms (attempt 4/5)4393worker1 # [ 30.929247] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4394worker1 # [ 30.932744] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4395worker1 # [ 30.954595] nix-grpc-daemon[786]: ts=2026-09-21T04:23:42Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:443124396worker2 # [ 30.968690] nix-grpc-daemon[785]: ts=2026-09-21T04:23:42Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:445684397worker1 # [ 30.971669] nix-grpc-daemon[786]: ts=2026-09-21T04:23:42Z level=debug event=path_info_miss path=z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test error="error: path '/nix/store/z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test' is required, but there is no substituter that can build it"4398worker2 # [ 30.973932] nix-grpc-daemon[785]: ts=2026-09-21T04:23:42Z level=debug event=path_info_miss path=z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test error="error: path '/nix/store/z2k9xss7v1mgs1j9xkkp4fxc20a53raz-nixos-system-client-test' is required, but there is no substituter that can build it"4399client: must succeed: rm -r /var/lib/nix-grpc-store4400client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4401worker1 # [ 31.259132] nix-grpc-daemon[786]: ts=2026-09-21T04:23:43Z level=info event=denied method=QueryMissing cn=- role=none4402worker2 # [ 31.498555] nix-grpc-daemon[785]: ts=2026-09-21T04:23:43Z level=info event=denied method=QueryMissing cn=stranger role=none4403(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.90 seconds)4404subtest: DAG build is scheduled across workers and lands in the cache4405client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24406worker2 # [ 31.758628] nix-grpc-daemon[785]: ts=2026-09-21T04:23:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:445684407client # don't know how to build these paths:4408client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4409worker1 # [ 31.781634] nix-grpc-daemon[786]: ts=2026-09-21T04:23:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124410worker2 # [ 31.805367] nix-grpc-daemon[785]: ts=2026-09-21T04:23:43Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04411worker2 # [ 31.810280] nix-grpc-daemon[785]: ts=2026-09-21T04:23:43Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14412worker1 # [ 31.864362] nix-grpc-daemon[786]: ts=2026-09-21T04:23:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124413worker1 # [ 31.886702] nix-grpc-daemon[786]: ts=2026-09-21T04:23:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124414client # copying 1 paths to 'lb:50051'...4415worker1 # [ 31.899682] nix-grpc-daemon[786]: ts=2026-09-21T04:23:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443124416client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4417worker1 # [ 31.933528] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4418worker1 # [ 31.960220] nix-grpc-daemon[786]: ts=2026-09-21T04:23:43Z level=info event=niks3_push_started pid=8594419worker1 # [ 32.215806] nix-grpc-daemon[859]: time=2026-09-21T04:23:44.166Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4420worker1 # [ 32.381717] nix-daemon[846]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4421worker1 # [ 32.396724] nix-daemon[846]: reaped child process 874, status = succeeded4422worker1 # [ 32.550435] nix-daemon[846]: accepted connection from pid 876, user nix-grpc-daemon (trusted)4423worker1 # [ 32.564482] nix-daemon[846]: reaped child process 879, status = succeeded4424lb # [ 32.592148] niks3-server[1003]: 2026/09/21 04:23:44 INFO Received uploads request method=POST path=/api/pending_closures4425worker1 # [ 32.606108] nix-grpc-daemon[859]: time=2026-09-21T04:23:44.559Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4426worker1 # [ 32.608354] nix-grpc-daemon[859]: time=2026-09-21T04:23:44.561Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4427lb # [ 32.660831] niks3-server[1003]: 2026/09/21 04:23:44 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4428lb # [ 32.666104] niks3-server[1003]: 2026/09/21 04:23:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4429lb # [ 32.679051] niks3-server[1003]: 2026/09/21 04:23:44 INFO Signed narinfos id=1 count=14430worker1 # [ 32.668542] nix-grpc-daemon[859]: time=2026-09-21T04:23:44.621Z level=INFO msg="Uploading 1 narinfos"4431lb # [ 32.693794] niks3-server[1003]: 2026/09/21 04:23:44 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4432lb # [ 32.696446] niks3-server[1003]: 2026/09/21 04:23:44 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4433lb # [ 32.706693] niks3-server[1003]: 2026/09/21 04:23:44 INFO Completed upload id=14434worker1 # [ 32.696748] nix-grpc-daemon[859]: time=2026-09-21T04:23:44.649Z level=INFO msg="Upload complete. (466ms)"4435worker1 # [ 32.699901] nix-grpc-daemon[786]: ts=2026-09-21T04:23:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=5044436worker1 # [ 32.703157] nix-grpc-daemon[859]: time=2026-09-21T04:23:44.655Z level=WARN msg="Failed to register uploaded object" key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo error="server returned 404: object is not pending upload\n"4437worker1 # [ 32.706384] nix-grpc-daemon[786]: ts=2026-09-21T04:23:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124438worker1 # [ 32.716102] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4439worker1 # [ 32.718448] nix-daemon[846]: reaped child process 857, status = succeeded4440client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4441worker1 # [ 32.725795] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4442client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4443client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4444client # farm-a> LOG-farm-a-t14445worker1 # [ 33.744689] nix-daemon[846]: reaped child process 883, status = succeeded4446worker1 # [ 33.900742] nix-daemon[846]: accepted connection from pid 900, user nix-grpc-daemon (trusted)4447worker1 # [ 33.913585] nix-daemon[846]: reaped child process 903, status = succeeded4448lb # [ 33.939610] niks3-server[1003]: 2026/09/21 04:23:45 INFO Received uploads request method=POST path=/api/pending_closures4449worker1 # [ 33.945601] nix-grpc-daemon[859]: time=2026-09-21T04:23:45.898Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4450worker1 # [ 33.947136] nix-grpc-daemon[859]: time=2026-09-21T04:23:45.900Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4451lb # [ 34.001535] niks3-server[1003]: 2026/09/21 04:23:45 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4452lb # [ 34.008566] niks3-server[1003]: 2026/09/21 04:23:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4453lb # [ 34.014344] niks3-server[1003]: 2026/09/21 04:23:45 INFO Signed narinfos id=2 count=14454worker1 # [ 34.004513] nix-grpc-daemon[859]: time=2026-09-21T04:23:45.956Z level=INFO msg="Uploading 1 narinfos"4455lb # [ 34.017958] niks3-server[1003]: 2026/09/21 04:23:45 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4456lb # [ 34.021755] niks3-server[1003]: 2026/09/21 04:23:45 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4457lb # [ 34.029174] niks3-server[1003]: 2026/09/21 04:23:45 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4458lb # [ 34.035427] niks3-server[1003]: 2026/09/21 04:23:45 INFO Completed upload id=24459worker1 # [ 34.024829] nix-grpc-daemon[859]: time=2026-09-21T04:23:45.978Z level=INFO msg="Upload complete. (284ms)"4460lb # [ 34.037231] niks3-server[1003]: 2026/09/21 04:23:45 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4461worker1 # [ 34.032292] nix-grpc-daemon[786]: ts=2026-09-21T04:23:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14462worker1 # [ 34.044522] nix-daemon[846]: reaped child process 881, status = succeeded4463worker2 # [ 34.043862] nix-grpc-daemon[785]: ts=2026-09-21T04:23:46Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04464worker2 # [ 34.048678] nix-grpc-daemon[785]: ts=2026-09-21T04:23:46Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14465worker1 # [ 34.107499] nix-grpc-daemon[786]: ts=2026-09-21T04:23:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124466worker1 # [ 34.120236] nix-grpc-daemon[786]: ts=2026-09-21T04:23:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124467worker1 # [ 34.133281] nix-grpc-daemon[859]: time=2026-09-21T04:23:46.086Z level=INFO msg="All 1 paths already cached"4468client # copying 1 paths to 'lb:50051'...4469worker1 # [ 34.141455] nix-grpc-daemon[786]: ts=2026-09-21T04:23:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443124470client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4471worker1 # [ 34.160370] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4472worker1 # [ 34.333678] nix-daemon[846]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4473worker1 # [ 34.349182] nix-daemon[846]: reaped child process 911, status = succeeded4474worker1 # [ 34.501164] nix-daemon[846]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4475worker1 # [ 34.514573] nix-daemon[846]: reaped child process 916, status = succeeded4476lb # [ 34.539528] niks3-server[1003]: 2026/09/21 04:23:46 INFO Received uploads request method=POST path=/api/pending_closures4477worker1 # [ 34.538729] nix-grpc-daemon[859]: time=2026-09-21T04:23:46.491Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4478worker1 # [ 34.540147] nix-grpc-daemon[859]: time=2026-09-21T04:23:46.491Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4479lb # [ 34.570082] niks3-server[1003]: 2026/09/21 04:23:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4480lb # [ 34.575778] niks3-server[1003]: 2026/09/21 04:23:46 INFO Signed narinfos id=3 count=14481worker1 # [ 34.568446] nix-grpc-daemon[859]: time=2026-09-21T04:23:46.521Z level=INFO msg="Uploading 1 narinfos"4482lb # [ 34.584158] niks3-server[1003]: 2026/09/21 04:23:46 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4483lb # [ 34.596073] niks3-server[1003]: 2026/09/21 04:23:46 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4484lb # [ 34.602125] niks3-server[1003]: 2026/09/21 04:23:46 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4485worker1 # [ 34.597689] nix-grpc-daemon[859]: time=2026-09-21T04:23:46.551Z level=INFO msg="Upload complete. (421ms)"4486lb # [ 34.608456] niks3-server[1003]: 2026/09/21 04:23:46 INFO Completed upload id=34487worker1 # [ 34.599803] nix-grpc-daemon[786]: ts=2026-09-21T04:23:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=6244488lb # [ 34.612146] niks3-server[1003]: 2026/09/21 04:23:46 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4489worker1 # [ 34.605791] nix-grpc-daemon[786]: ts=2026-09-21T04:23:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124490worker1 # [ 34.614820] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4491worker1 # [ 34.621123] nix-daemon[846]: reaped child process 905, status = succeeded4492worker1 # [ 34.625441] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4493client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4494client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4495client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4496client # farm-b> LOG-farm-b-t14497worker1 # [ 35.654476] nix-daemon[846]: reaped child process 920, status = succeeded4498worker1 # [ 35.809100] nix-daemon[846]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4499worker1 # [ 35.822143] nix-daemon[846]: reaped child process 939, status = succeeded4500lb # [ 35.850424] niks3-server[1003]: 2026/09/21 04:23:47 INFO Received uploads request method=POST path=/api/pending_closures4501worker1 # [ 35.863101] nix-grpc-daemon[859]: time=2026-09-21T04:23:47.816Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4502worker1 # [ 35.865337] nix-grpc-daemon[859]: time=2026-09-21T04:23:47.818Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4503lb # [ 35.894165] niks3-server[1003]: 2026/09/21 04:23:47 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4504lb # [ 35.907863] niks3-server[1003]: 2026/09/21 04:23:47 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4505lb # [ 35.919431] niks3-server[1003]: 2026/09/21 04:23:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4506lb # [ 35.924506] niks3-server[1003]: 2026/09/21 04:23:47 INFO Signed narinfos id=4 count=14507worker1 # [ 35.916594] nix-grpc-daemon[859]: time=2026-09-21T04:23:47.869Z level=INFO msg="Uploading 1 narinfos"4508lb # [ 35.930060] niks3-server[1003]: 2026/09/21 04:23:47 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4509lb # [ 35.939752] niks3-server[1003]: 2026/09/21 04:23:47 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4510lb # [ 35.946055] niks3-server[1003]: 2026/09/21 04:23:47 INFO Completed upload id=44511worker1 # [ 35.935996] nix-grpc-daemon[859]: time=2026-09-21T04:23:47.888Z level=INFO msg="Upload complete. (301ms)"4512lb # [ 35.948343] niks3-server[1003]: 2026/09/21 04:23:47 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4513worker1 # [ 35.938293] nix-grpc-daemon[786]: ts=2026-09-21T04:23:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14514worker2 # [ 35.936078] nix-grpc-daemon[785]: ts=2026-09-21T04:23:47Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04515worker1 # [ 35.946909] nix-daemon[846]: reaped child process 918, status = succeeded4516worker2 # [ 35.938168] nix-grpc-daemon[785]: ts=2026-09-21T04:23:47Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14517worker1 # [ 35.987905] nix-grpc-daemon[786]: ts=2026-09-21T04:23:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124518worker1 # [ 35.996248] nix-grpc-daemon[786]: ts=2026-09-21T04:23:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124519worker1 # [ 36.005769] nix-grpc-daemon[859]: time=2026-09-21T04:23:47.959Z level=INFO msg="All 2 paths already cached"4520client # copying 1 paths to 'lb:50051'...4521worker1 # [ 36.011490] nix-grpc-daemon[786]: ts=2026-09-21T04:23:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443124522client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4523worker1 # [ 36.029734] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4524worker1 # [ 36.188188] nix-daemon[846]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4525worker1 # [ 36.202150] nix-daemon[846]: reaped child process 947, status = succeeded4526worker1 # [ 36.359223] nix-daemon[846]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4527worker1 # [ 36.374821] nix-daemon[846]: reaped child process 952, status = succeeded4528lb # [ 36.399688] niks3-server[1003]: 2026/09/21 04:23:48 INFO Received uploads request method=POST path=/api/pending_closures4529worker1 # [ 36.399406] nix-grpc-daemon[859]: time=2026-09-21T04:23:48.352Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4530worker1 # [ 36.401424] nix-grpc-daemon[859]: time=2026-09-21T04:23:48.354Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4531lb # [ 36.434317] niks3-server[1003]: 2026/09/21 04:23:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4532lb # [ 36.438792] niks3-server[1003]: 2026/09/21 04:23:48 INFO Signed narinfos id=5 count=14533worker1 # [ 36.432882] nix-grpc-daemon[859]: time=2026-09-21T04:23:48.386Z level=INFO msg="Uploading 1 narinfos"4534lb # [ 36.447864] niks3-server[1003]: 2026/09/21 04:23:48 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4535lb # [ 36.454907] niks3-server[1003]: 2026/09/21 04:23:48 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4536lb # [ 36.465991] niks3-server[1003]: 2026/09/21 04:23:48 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4537lb # [ 36.472441] niks3-server[1003]: 2026/09/21 04:23:48 INFO Completed upload id=54538worker1 # [ 36.463939] nix-grpc-daemon[859]: time=2026-09-21T04:23:48.417Z level=INFO msg="Upload complete. (429ms)"4539lb # [ 36.475456] niks3-server[1003]: 2026/09/21 04:23:48 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4540worker1 # [ 36.468133] nix-grpc-daemon[786]: ts=2026-09-21T04:23:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=7524541worker1 # [ 36.472611] nix-grpc-daemon[786]: ts=2026-09-21T04:23:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124542worker1 # [ 36.483141] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4543worker1 # [ 36.487793] nix-daemon[846]: reaped child process 942, status = succeeded4544worker1 # [ 36.492851] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4545client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4546client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4547client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4548client # farm-top> LOG-farm-top-t14549worker1 # [ 37.707890] nix-daemon[846]: reaped child process 956, status = succeeded4550worker1 # [ 37.856308] nix-daemon[846]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4551lb # [ 37.867336] envoy[910]: conn peer=[2001:db8:1::1]:57638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=365 ms=11744552lb # [ 37.870979] envoy[910]: conn peer=[2001:db8:1::1]:57642 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=15654553lb # [ 37.874397] envoy[910]: conn peer=[2001:db8:1::1]:57652 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=104554lb # [ 37.877737] envoy[910]: conn peer=[2001:db8:1::1]:57654 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=94555worker1 # [ 37.870393] nix-daemon[846]: reaped child process 975, status = succeeded4556lb # [ 37.897600] niks3-server[1003]: 2026/09/21 04:23:49 INFO Received uploads request method=POST path=/api/pending_closures4557worker1 # [ 37.895495] nix-grpc-daemon[859]: time=2026-09-21T04:23:49.848Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4558worker1 # [ 37.897056] nix-grpc-daemon[859]: time=2026-09-21T04:23:49.850Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4559lb # [ 37.933184] niks3-server[1003]: 2026/09/21 04:23:49 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4560lb # [ 37.951075] niks3-server[1003]: 2026/09/21 04:23:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4561lb # [ 37.956517] niks3-server[1003]: 2026/09/21 04:23:49 INFO Signed narinfos id=6 count=14562worker1 # [ 37.946097] nix-grpc-daemon[859]: time=2026-09-21T04:23:49.899Z level=INFO msg="Uploading 1 narinfos"4563lb # [ 37.961558] niks3-server[1003]: 2026/09/21 04:23:49 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4564lb # [ 37.970734] niks3-server[1003]: 2026/09/21 04:23:49 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4565lb # [ 37.972443] niks3-server[1003]: 2026/09/21 04:23:49 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4566lb # [ 38.010319] niks3-server[1003]: 2026/09/21 04:23:49 INFO Completed upload id=64567worker1 # [ 37.999621] nix-grpc-daemon[859]: time=2026-09-21T04:23:49.952Z level=INFO msg="Upload complete. (299ms)"4568lb # [ 38.014887] niks3-server[1003]: 2026/09/21 04:23:49 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4569worker1 # [ 38.006708] nix-grpc-daemon[786]: ts=2026-09-21T04:23:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14570worker1 # [ 38.021434] nix-daemon[846]: reaped child process 954, status = succeeded4571client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.53 seconds)4572client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4573client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4574client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4575client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.03 seconds)4576client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4577client # copying 3 paths...4578client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4579client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4580client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4581client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.29 seconds)4582(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.13 seconds)4583subtest: repeat is answered Cached by the scheduler without building4584worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4585worker1 # finding garbage collector roots...4586worker1 # deleting specified paths...4587worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4588worker1 # deleting unused links...4589worker1 # note: hard linking is currently saving 0.0 KiB4590worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4591worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4592worker2 # finding garbage collector roots...4593worker2 # deleting specified paths...4594worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4595worker2 # deleting unused links...4596worker2 # note: hard linking is currently saving 0.0 KiB4597worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.28 seconds)4598client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24599client # don't know how to build these paths:4600client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4601worker2 # [ 39.369927] nix-grpc-daemon[785]: ts=2026-09-21T04:23:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:445684602worker1 # [ 39.382611] nix-grpc-daemon[786]: ts=2026-09-21T04:23:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124603worker1 # [ 39.392625] nix-grpc-daemon[859]: time=2026-09-21T04:23:51.345Z level=INFO msg="All 2 paths already cached"4604worker2 # [ 39.397858] nix-grpc-daemon[785]: ts=2026-09-21T04:23:51Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14605client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.25 seconds)4606client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4607client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4608lb: must succeed: curl -sf localhost:9901/clusters4609lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4610lb: must succeed: curl -sf localhost:9901/clusters4611lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4612worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4613worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4614(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.24 seconds)4615subtest: inputs already in the cache are not sent to the scheduler4616lb: must succeed: curl -sf localhost:9901/clusters4617lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4618lb: must succeed: curl -sf localhost:9901/clusters4619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4620worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4621worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4622client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24623worker2 # [ 40.284291] nix-grpc-daemon[785]: ts=2026-09-21T04:23:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:445684624client # don't know how to build these paths:4625client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4626worker2 # [ 40.292579] nix-grpc-daemon[785]: ts=2026-09-21T04:23:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445684627worker2 # [ 40.320140] nix-grpc-daemon[785]: ts=2026-09-21T04:23:52Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04628worker2 # [ 40.324610] nix-grpc-daemon[785]: ts=2026-09-21T04:23:52Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14629worker1 # [ 40.373630] nix-grpc-daemon[786]: ts=2026-09-21T04:23:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124630worker1 # [ 40.388072] nix-grpc-daemon[786]: ts=2026-09-21T04:23:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124631worker1 # [ 40.401310] nix-grpc-daemon[859]: time=2026-09-21T04:23:52.354Z level=INFO msg="All 2 paths already cached"4632client # copying 1 paths to 'lb:50051'...4633client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4634worker1 # [ 40.409812] nix-grpc-daemon[786]: ts=2026-09-21T04:23:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443124635worker1 # [ 40.421744] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4636worker1 # [ 40.600251] nix-daemon[846]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4637worker1 # [ 40.615691] nix-daemon[846]: reaped child process 996, status = succeeded4638worker1 # [ 40.774212] nix-daemon[846]: accepted connection from pid 998, user nix-grpc-daemon (trusted)4639worker1 # [ 40.787166] nix-daemon[846]: reaped child process 1001, status = succeeded4640lb # [ 40.811829] niks3-server[1003]: 2026/09/21 04:23:52 INFO Received uploads request method=POST path=/api/pending_closures4641worker1 # [ 40.806290] nix-grpc-daemon[859]: time=2026-09-21T04:23:52.759Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4642worker1 # [ 40.808158] nix-grpc-daemon[859]: time=2026-09-21T04:23:52.761Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4643lb # [ 40.827293] niks3-server[1003]: 2026/09/21 04:23:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4644lb # [ 40.829756] niks3-server[1003]: 2026/09/21 04:23:52 INFO Signed narinfos id=7 count=14645worker1 # [ 40.819201] nix-grpc-daemon[859]: time=2026-09-21T04:23:52.772Z level=INFO msg="Uploading 1 narinfos"4646lb # [ 40.833432] niks3-server[1003]: 2026/09/21 04:23:52 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4647lb # [ 40.835937] niks3-server[1003]: 2026/09/21 04:23:52 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4648lb # [ 40.840050] niks3-server[1003]: 2026/09/21 04:23:52 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4649lb # [ 40.842553] niks3-server[1003]: 2026/09/21 04:23:52 INFO Completed upload id=74650worker1 # [ 40.833129] nix-grpc-daemon[859]: time=2026-09-21T04:23:52.786Z level=INFO msg="Upload complete. (399ms)"4651lb # [ 40.844967] niks3-server[1003]: 2026/09/21 04:23:52 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4652worker1 # [ 40.835116] nix-grpc-daemon[786]: ts=2026-09-21T04:23:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=7524653worker1 # [ 40.838732] nix-grpc-daemon[786]: ts=2026-09-21T04:23:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124654worker1 # [ 40.847724] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4655worker1 # [ 40.850853] nix-daemon[846]: reaped child process 991, status = succeeded4656worker1 # [ 40.857304] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4657client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4658client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4659client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4660client # farm-top> LOG-farm-top-t1b4661worker1 # [ 42.274654] nix-daemon[846]: reaped child process 1005, status = succeeded4662worker1 # [ 42.438879] nix-daemon[846]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4663worker1 # [ 42.451074] nix-daemon[846]: reaped child process 1025, status = succeeded4664lb # [ 42.480064] niks3-server[1003]: 2026/09/21 04:23:54 INFO Received uploads request method=POST path=/api/pending_closures4665worker1 # [ 42.476202] nix-grpc-daemon[859]: time=2026-09-21T04:23:54.429Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4666worker1 # [ 42.480884] nix-grpc-daemon[859]: time=2026-09-21T04:23:54.431Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4667lb # [ 42.516171] niks3-server[1003]: 2026/09/21 04:23:54 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4668lb # [ 42.529546] niks3-server[1003]: 2026/09/21 04:23:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4669worker1 # [ 42.525876] nix-grpc-daemon[859]: time=2026-09-21T04:23:54.479Z level=INFO msg="Uploading 1 narinfos"4670lb # [ 42.537321] niks3-server[1003]: 2026/09/21 04:23:54 INFO Signed narinfos id=8 count=14671lb # [ 42.544413] niks3-server[1003]: 2026/09/21 04:23:54 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4672lb # [ 42.548243] niks3-server[1003]: 2026/09/21 04:23:54 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4673lb # [ 42.557333] niks3-server[1003]: 2026/09/21 04:23:54 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4674lb # [ 42.562268] niks3-server[1003]: 2026/09/21 04:23:54 INFO Completed upload id=84675lb # [ 42.565758] niks3-server[1003]: 2026/09/21 04:23:54 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4676worker1 # [ 42.553585] nix-grpc-daemon[859]: time=2026-09-21T04:23:54.506Z level=INFO msg="Upload complete. (287ms)"4677worker1 # [ 42.560426] nix-grpc-daemon[786]: ts=2026-09-21T04:23:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14678worker1 # [ 42.573428] nix-daemon[846]: reaped child process 1003, status = succeeded4679client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.54 seconds)4680client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4681client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.21 seconds)4682lb: must succeed: curl -sf localhost:9901/clusters4683lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4684lb: must succeed: curl -sf localhost:9901/clusters4685lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4686worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4687worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4688(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.08 seconds)4689subtest: two clients wanting the same drv build it once4690client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4691client # [ 43.008987] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4692client # Running as unit: dedup1.service; invocation ID: f75c642a242f43a9b55e386526507aef4693client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.04 seconds)4694client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4695worker1 # [ 43.233612] nix-grpc-daemon[786]: ts=2026-09-21T04:23:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443124696client # [ 43.223931] nix[888]: don't know how to build these paths:4697client # [ 43.224823] nix[888]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4698worker2 # [ 43.235866] nix-grpc-daemon[785]: ts=2026-09-21T04:23:55Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04699worker2 # [ 43.243979] nix-grpc-daemon[785]: ts=2026-09-21T04:23:55Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14700worker2 # [ 43.250444] nix-grpc-daemon[785]: ts=2026-09-21T04:23:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:445684701worker2 # [ 43.262768] nix-grpc-daemon[785]: ts=2026-09-21T04:23:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445684702client # [ 43.268692] nix[888]: copying 1 paths to 'lb:50051'...4703client # [ 43.269456] nix[888]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4704worker2 # [ 43.273647] nix-grpc-daemon[785]: ts=2026-09-21T04:23:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:445684705worker2 # [ 43.284898] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4706worker2 # [ 43.305222] nix-grpc-daemon[785]: ts=2026-09-21T04:23:55Z level=info event=niks3_push_started pid=9204707worker2 # [ 43.547859] nix-grpc-daemon[920]: time=2026-09-21T04:23:55.533Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""4708worker2 # [ 43.703518] nix-daemon[865]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4709worker2 # [ 43.718903] nix-daemon[865]: reaped child process 936, status = succeeded4710worker2 # [ 43.859422] nix-daemon[865]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4711worker2 # [ 43.872406] nix-daemon[865]: reaped child process 941, status = succeeded4712lb # [ 43.906210] niks3-server[1003]: 2026/09/21 04:23:55 INFO Received uploads request method=POST path=/api/pending_closures4713worker2 # [ 43.895886] nix-grpc-daemon[920]: time=2026-09-21T04:23:55.884Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4714worker2 # [ 43.898051] nix-grpc-daemon[920]: time=2026-09-21T04:23:55.886Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4715lb # [ 43.942590] niks3-server[1003]: 2026/09/21 04:23:55 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4716lb # [ 43.954198] niks3-server[1003]: 2026/09/21 04:23:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4717worker2 # [ 43.940663] nix-grpc-daemon[920]: time=2026-09-21T04:23:55.929Z level=INFO msg="Uploading 1 narinfos"4718lb # [ 43.959472] niks3-server[1003]: 2026/09/21 04:23:55 INFO Signed narinfos id=9 count=14719lb # [ 43.970053] niks3-server[1003]: 2026/09/21 04:23:55 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4720lb # [ 43.981191] niks3-server[1003]: 2026/09/21 04:23:55 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4721lb # [ 43.986608] niks3-server[1003]: 2026/09/21 04:23:55 INFO Completed upload id=94722worker2 # [ 43.970082] nix-grpc-daemon[920]: time=2026-09-21T04:23:55.958Z level=INFO msg="Upload complete. (412ms)"4723lb # [ 43.991044] niks3-server[1003]: 2026/09/21 04:23:55 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4724worker2 # [ 43.973604] nix-grpc-daemon[785]: ts=2026-09-21T04:23:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=0 paths=1 nar_bytes_in=4324725worker2 # [ 43.979246] nix-grpc-daemon[785]: ts=2026-09-21T04:23:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:445684726worker2 # [ 43.987996] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4727worker2 # [ 43.990293] nix-daemon[865]: reaped child process 918, status = succeeded4728client # [ 43.990938] nix[888]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4729worker2 # [ 43.995858] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4730client # [ 44.067674] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4731client # Running as unit: dedup2.service; invocation ID: b6c35fe4259249f1902b4426a71fb1234732client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.06 seconds)4733client # [ 44.280521] nix[904]: don't know how to build these paths:4734client # [ 44.281982] nix[904]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4735worker1 # [ 44.292095] nix-grpc-daemon[786]: ts=2026-09-21T04:23:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443124736worker2 # [ 44.294511] nix-grpc-daemon[785]: ts=2026-09-21T04:23:56Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04737worker2 # [ 44.301328] nix-grpc-daemon[785]: ts=2026-09-21T04:23:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:445684738client # [ 44.301835] nix[904]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4739client: must succeed: sleep 24740client: (finished: must succeed: sleep 2, in 2.05 seconds)4741client: waiting for success: ! systemctl is-active dedup1 dedup24742worker2 # [ 47.612809] nix-daemon[865]: reaped child process 945, status = succeeded4743worker2 # [ 47.759678] nix-daemon[865]: accepted connection from pid 977, user nix-grpc-daemon (trusted)4744worker2 # [ 47.771073] nix-daemon[865]: reaped child process 980, status = succeeded4745lb # [ 47.805189] niks3-server[1003]: 2026/09/21 04:23:59 INFO Received uploads request method=POST path=/api/pending_closures4746worker2 # [ 47.795117] nix-grpc-daemon[920]: time=2026-09-21T04:23:59.783Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4747worker2 # [ 47.796613] nix-grpc-daemon[920]: time=2026-09-21T04:23:59.785Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4748lb # [ 47.846303] niks3-server[1003]: 2026/09/21 04:23:59 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4749lb # [ 47.855080] niks3-server[1003]: 2026/09/21 04:23:59 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4750lb # [ 47.865666] niks3-server[1003]: 2026/09/21 04:23:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4751worker2 # [ 47.852652] nix-grpc-daemon[920]: time=2026-09-21T04:23:59.841Z level=INFO msg="Uploading 1 narinfos"4752lb # [ 47.872508] envoy[910]: conn peer=[2001:db8:1::1]:57666 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=4091 ms=62874753lb # [ 47.876226] envoy[910]: conn peer=[2001:db8:1::1]:41888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=503 ms=424754lb # [ 47.878947] envoy[910]: conn peer=[2001:db8:1::1]:41904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1804 ms=22784755lb # [ 47.881817] niks3-server[1003]: 2026/09/21 04:23:59 INFO Signed narinfos id=10 count=14756lb # [ 47.887512] niks3-server[1003]: 2026/09/21 04:23:59 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4757lb # [ 47.894109] niks3-server[1003]: 2026/09/21 04:23:59 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4758lb # [ 47.897136] niks3-server[1003]: 2026/09/21 04:23:59 INFO Completed upload id=104759lb # [ 47.898905] niks3-server[1003]: 2026/09/21 04:23:59 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4760worker2 # [ 47.879681] nix-grpc-daemon[920]: time=2026-09-21T04:23:59.868Z level=INFO msg="Upload complete. (271ms)"4761worker2 # [ 47.890754] nix-grpc-daemon[785]: ts=2026-09-21T04:23:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14762worker2 # [ 47.896812] nix-grpc-daemon[785]: ts=2026-09-21T04:23:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14763worker2 # [ 47.907812] nix-daemon[865]: reaped child process 943, status = succeeded4764client # [ 47.921265] systemd[1]: dedup1.service: Deactivated successfully.4765client # [ 47.923029] systemd[1]: dedup1.service: Consumed 115ms CPU time over 4.914s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.5K outgoing IP traffic.4766client # [ 47.954083] systemd[1]: dedup2.service: Deactivated successfully.4767client # [ 47.956207] systemd[1]: dedup2.service: Consumed 126ms CPU time over 3.887s wall clock time, 35.3M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4768client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4769client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4770client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4771worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4772worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4773worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4774worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4775worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4776worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4777worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4778worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4779(finished: subtest: two clients wanting the same drv build it once, in 5.92 seconds)4780subtest: build hook: nix-daemon with builders = grpc://lb4781client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4782client # [ 49.059741] systemd[1]: Started Nix Daemon.4783client # [ 49.235161] nix-daemon[946]: accepted connection from pid 943, user root (trusted)4784worker1 # [ 49.448954] nix-grpc-daemon[786]: ts=2026-09-21T04:24:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:443124785worker2 # [ 49.446487] nix-grpc-daemon[785]: ts=2026-09-21T04:24:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445684786worker2 # [ 49.460596] nix-grpc-daemon[785]: ts=2026-09-21T04:24:01Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04787worker2 # [ 49.464743] nix-grpc-daemon[785]: ts=2026-09-21T04:24:01Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14788worker1 # [ 49.512732] nix-grpc-daemon[786]: ts=2026-09-21T04:24:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124789worker1 # [ 49.524109] nix-grpc-daemon[786]: ts=2026-09-21T04:24:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124790worker1 # [ 49.534983] nix-grpc-daemon[786]: ts=2026-09-21T04:24:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443124791worker1 # [ 49.545911] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4792worker1 # [ 49.735826] nix-daemon[846]: accepted connection from pid 1050, user nix-grpc-daemon (trusted)4793worker1 # [ 49.750283] nix-daemon[846]: reaped child process 1053, status = succeeded4794worker1 # [ 49.914702] nix-daemon[846]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4795worker1 # [ 49.926446] nix-daemon[846]: reaped child process 1058, status = succeeded4796lb # [ 49.949734] niks3-server[1003]: 2026/09/21 04:24:01 INFO Received uploads request method=POST path=/api/pending_closures4797worker1 # [ 49.944413] nix-grpc-daemon[859]: time=2026-09-21T04:24:01.897Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4798worker1 # [ 49.945898] nix-grpc-daemon[859]: time=2026-09-21T04:24:01.899Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4799lb # [ 49.970195] niks3-server[1003]: 2026/09/21 04:24:01 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4800lb # [ 49.973452] niks3-server[1003]: 2026/09/21 04:24:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4801lb # [ 49.976520] niks3-server[1003]: 2026/09/21 04:24:01 INFO Signed narinfos id=11 count=14802worker1 # [ 49.965916] nix-grpc-daemon[859]: time=2026-09-21T04:24:01.919Z level=INFO msg="Uploading 1 narinfos"4803lb # [ 49.979927] niks3-server[1003]: 2026/09/21 04:24:01 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4804lb # [ 49.986429] niks3-server[1003]: 2026/09/21 04:24:01 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4805lb # [ 49.990441] niks3-server[1003]: 2026/09/21 04:24:01 INFO Completed upload id=114806worker1 # [ 49.979674] nix-grpc-daemon[859]: time=2026-09-21T04:24:01.933Z level=INFO msg="Upload complete. (421ms)"4807worker1 # [ 49.981702] nix-grpc-daemon[786]: ts=2026-09-21T04:24:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=5124808lb # [ 49.993081] niks3-server[1003]: 2026/09/21 04:24:01 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4809worker1 # [ 49.984802] nix-grpc-daemon[786]: ts=2026-09-21T04:24:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124810worker1 # [ 49.992263] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4811worker1 # [ 49.994913] nix-daemon[846]: reaped child process 1048, status = succeeded4812worker1 # [ 50.000065] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4813worker1 # [ 51.214915] nix-daemon[846]: reaped child process 1062, status = succeeded4814worker1 # [ 51.363986] nix-daemon[846]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4815worker1 # [ 51.377413] nix-daemon[846]: reaped child process 1081, status = succeeded4816lb # [ 51.405550] niks3-server[1003]: 2026/09/21 04:24:03 INFO Received uploads request method=POST path=/api/pending_closures4817worker1 # [ 51.403310] nix-grpc-daemon[859]: time=2026-09-21T04:24:03.356Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4818worker1 # [ 51.404794] nix-grpc-daemon[859]: time=2026-09-21T04:24:03.358Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4819lb # [ 51.444820] niks3-server[1003]: 2026/09/21 04:24:03 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4820lb # [ 51.454396] niks3-server[1003]: 2026/09/21 04:24:03 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4821lb # [ 51.462573] niks3-server[1003]: 2026/09/21 04:24:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4822lb # [ 51.468328] niks3-server[1003]: 2026/09/21 04:24:03 INFO Signed narinfos id=12 count=14823worker1 # [ 51.460956] nix-grpc-daemon[859]: time=2026-09-21T04:24:03.414Z level=INFO msg="Uploading 1 narinfos"4824lb # [ 51.474353] niks3-server[1003]: 2026/09/21 04:24:03 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4825lb # [ 51.483665] niks3-server[1003]: 2026/09/21 04:24:03 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4826lb # [ 51.488914] niks3-server[1003]: 2026/09/21 04:24:03 INFO Completed upload id=124827worker1 # [ 51.478437] nix-grpc-daemon[859]: time=2026-09-21T04:24:03.431Z level=INFO msg="Upload complete. (271ms)"4828lb # [ 51.492239] niks3-server[1003]: 2026/09/21 04:24:03 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4829worker1 # [ 51.485341] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14830worker1 # [ 51.491971] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443124831worker1 # [ 51.502157] nix-daemon[846]: reaped child process 1060, status = succeeded4832worker1 # [ 51.505684] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443124833worker1 # [ 51.510126] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_out=1284834worker2 # [ 51.770834] nix-grpc-daemon[785]: ts=2026-09-21T04:24:03Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:445684835worker1 # [ 51.782600] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:443124836worker1 # [ 51.792164] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4837worker1 # [ 51.853857] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124838worker1 # [ 51.863336] nix-grpc-daemon[859]: time=2026-09-21T04:24:03.816Z level=INFO msg="All 1 paths already cached"4839worker2 # [ 51.873287] nix-grpc-daemon[785]: ts=2026-09-21T04:24:03Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04840worker2 # [ 51.877282] nix-grpc-daemon[785]: ts=2026-09-21T04:24:03Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14841worker1 # [ 51.893601] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124842worker1 # [ 51.906107] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124843worker1 # [ 51.917492] nix-grpc-daemon[859]: time=2026-09-21T04:24:03.870Z level=INFO msg="All 1 paths already cached"4844worker1 # [ 51.924492] nix-grpc-daemon[786]: ts=2026-09-21T04:24:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443124845worker1 # [ 51.936380] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4846worker1 # [ 52.111093] nix-daemon[846]: accepted connection from pid 1089, user nix-grpc-daemon (trusted)4847worker1 # [ 52.124723] nix-daemon[846]: reaped child process 1092, status = succeeded4848worker1 # [ 52.277667] nix-daemon[846]: accepted connection from pid 1094, user nix-grpc-daemon (trusted)4849worker1 # [ 52.292230] nix-daemon[846]: reaped child process 1097, status = succeeded4850lb # [ 52.317780] niks3-server[1003]: 2026/09/21 04:24:04 INFO Received uploads request method=POST path=/api/pending_closures4851worker1 # [ 52.311210] nix-grpc-daemon[859]: time=2026-09-21T04:24:04.264Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4852worker1 # [ 52.312673] nix-grpc-daemon[859]: time=2026-09-21T04:24:04.266Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4853lb # [ 52.333769] niks3-server[1003]: 2026/09/21 04:24:04 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4854lb # [ 52.336264] niks3-server[1003]: 2026/09/21 04:24:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4855lb # [ 52.338676] niks3-server[1003]: 2026/09/21 04:24:04 INFO Signed narinfos id=13 count=14856worker1 # [ 52.327700] nix-grpc-daemon[859]: time=2026-09-21T04:24:04.281Z level=INFO msg="Uploading 1 narinfos"4857lb # [ 52.341286] niks3-server[1003]: 2026/09/21 04:24:04 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4858lb # [ 52.345794] niks3-server[1003]: 2026/09/21 04:24:04 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4859lb # [ 52.350293] niks3-server[1003]: 2026/09/21 04:24:04 INFO Completed upload id=134860worker1 # [ 52.339496] nix-grpc-daemon[859]: time=2026-09-21T04:24:04.292Z level=INFO msg="Upload complete. (388ms)"4861lb # [ 52.351501] niks3-server[1003]: 2026/09/21 04:24:04 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4862worker1 # [ 52.341609] nix-grpc-daemon[786]: ts=2026-09-21T04:24:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=6244863worker1 # [ 52.344975] nix-grpc-daemon[786]: ts=2026-09-21T04:24:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124864worker1 # [ 52.351558] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4865worker1 # [ 52.354498] nix-daemon[846]: reaped child process 1087, status = succeeded4866worker1 # [ 52.359391] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4867worker1 # [ 53.374562] nix-daemon[846]: reaped child process 1101, status = succeeded4868worker1 # [ 53.520336] nix-daemon[846]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)4869worker1 # [ 53.534353] nix-daemon[846]: reaped child process 1120, status = succeeded4870lb # [ 53.560212] niks3-server[1003]: 2026/09/21 04:24:05 INFO Received uploads request method=POST path=/api/pending_closures4871worker1 # [ 53.553608] nix-grpc-daemon[859]: time=2026-09-21T04:24:05.506Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4872worker1 # [ 53.555152] nix-grpc-daemon[859]: time=2026-09-21T04:24:05.508Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4873lb # [ 53.577799] niks3-server[1003]: 2026/09/21 04:24:05 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4874lb # [ 53.583366] niks3-server[1003]: 2026/09/21 04:24:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4875lb # [ 53.585358] niks3-server[1003]: 2026/09/21 04:24:05 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4876lb # [ 53.587346] niks3-server[1003]: 2026/09/21 04:24:05 INFO Signed narinfos id=14 count=14877worker1 # [ 53.577888] nix-grpc-daemon[859]: time=2026-09-21T04:24:05.531Z level=INFO msg="Uploading 1 narinfos"4878lb # [ 53.590923] niks3-server[1003]: 2026/09/21 04:24:05 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4879lb # [ 53.594930] niks3-server[1003]: 2026/09/21 04:24:05 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4880lb # [ 53.598289] niks3-server[1003]: 2026/09/21 04:24:05 INFO Completed upload id=144881worker1 # [ 53.587540] nix-grpc-daemon[859]: time=2026-09-21T04:24:05.540Z level=INFO msg="Upload complete. (221ms)"4882lb # [ 53.599488] niks3-server[1003]: 2026/09/21 04:24:05 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4883worker1 # [ 53.591242] nix-grpc-daemon[786]: ts=2026-09-21T04:24:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14884worker2 # [ 53.587182] nix-grpc-daemon[785]: ts=2026-09-21T04:24:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:445684885worker1 # [ 53.597385] nix-daemon[846]: reaped child process 1099, status = succeeded4886worker2 # [ 53.595844] nix-grpc-daemon[785]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4887worker2 # [ 53.603506] nix-grpc-daemon[785]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4888worker1 # [ 53.630499] nix-grpc-daemon[786]: ts=2026-09-21T04:24:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:443124889worker1 # [ 53.632796] nix-grpc-daemon[786]: ts=2026-09-21T04:24:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_out=1764890worker1 # [ 53.682853] nix-grpc-daemon[786]: ts=2026-09-21T04:24:05Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 bytes_in=312 bytes_out=2004891worker1 # [ 53.684933] nix-daemon[846]: reaped child process 1084, status = succeeded4892worker2 # [ 53.881261] nix-grpc-daemon[785]: ts=2026-09-21T04:24:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:445684893worker2 # [ 53.885227] nix-grpc-daemon[785]: ts=2026-09-21T04:24:05Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:445684894worker2 # [ 53.890295] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4895worker2 # [ 53.939891] nix-grpc-daemon[785]: ts=2026-09-21T04:24:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445684896worker2 # [ 53.944588] nix-grpc-daemon[920]: time=2026-09-21T04:24:05.933Z level=INFO msg="All 2 paths already cached"4897worker2 # [ 53.956209] nix-grpc-daemon[785]: ts=2026-09-21T04:24:05Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04898worker2 # [ 53.960853] nix-grpc-daemon[785]: ts=2026-09-21T04:24:05Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14899worker1 # [ 54.013318] nix-grpc-daemon[786]: ts=2026-09-21T04:24:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124900worker1 # [ 54.025537] nix-grpc-daemon[786]: ts=2026-09-21T04:24:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443124901worker1 # [ 54.052975] nix-grpc-daemon[859]: time=2026-09-21T04:24:06.006Z level=INFO msg="All 2 paths already cached"4902worker1 # [ 54.060904] nix-grpc-daemon[786]: ts=2026-09-21T04:24:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443124903worker1 # [ 54.070164] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4904worker1 # [ 54.231145] nix-daemon[846]: accepted connection from pid 1124, user nix-grpc-daemon (trusted)4905worker1 # [ 54.245133] nix-daemon[846]: reaped child process 1127, status = succeeded4906worker1 # [ 54.414667] nix-daemon[846]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)4907worker1 # [ 54.428208] nix-daemon[846]: reaped child process 1132, status = succeeded4908lb # [ 54.455487] niks3-server[1003]: 2026/09/21 04:24:06 INFO Received uploads request method=POST path=/api/pending_closures4909worker1 # [ 54.452183] nix-grpc-daemon[859]: time=2026-09-21T04:24:06.405Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4910worker1 # [ 54.453705] nix-grpc-daemon[859]: time=2026-09-21T04:24:06.407Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4911lb # [ 54.483189] niks3-server[1003]: 2026/09/21 04:24:06 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4912lb # [ 54.491227] niks3-server[1003]: 2026/09/21 04:24:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4913lb # [ 54.495684] niks3-server[1003]: 2026/09/21 04:24:06 INFO Signed narinfos id=15 count=14914worker1 # [ 54.487945] nix-grpc-daemon[859]: time=2026-09-21T04:24:06.441Z level=INFO msg="Uploading 1 narinfos"4915lb # [ 54.508909] niks3-server[1003]: 2026/09/21 04:24:06 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4916lb # [ 54.512663] niks3-server[1003]: 2026/09/21 04:24:06 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4917worker1 # [ 54.507223] nix-grpc-daemon[859]: time=2026-09-21T04:24:06.460Z level=INFO msg="Upload complete. (423ms)"4918lb # [ 54.517935] niks3-server[1003]: 2026/09/21 04:24:06 INFO Completed upload id=154919worker1 # [ 54.509325] nix-grpc-daemon[786]: ts=2026-09-21T04:24:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=0 paths=1 nar_bytes_in=7524920lb # [ 54.522927] niks3-server[1003]: 2026/09/21 04:24:06 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4921worker1 # [ 54.516463] nix-grpc-daemon[786]: ts=2026-09-21T04:24:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443124922worker1 # [ 54.520656] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4923worker1 # [ 54.523164] nix-daemon[846]: reaped child process 1122, status = succeeded4924worker1 # [ 54.528753] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4925worker1 # [ 56.158345] nix-daemon[846]: reaped child process 1136, status = succeeded4926worker1 # [ 56.324504] nix-daemon[846]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)4927worker1 # [ 56.338979] nix-daemon[846]: reaped child process 1155, status = succeeded4928lb # [ 56.367422] niks3-server[1003]: 2026/09/21 04:24:08 INFO Received uploads request method=POST path=/api/pending_closures4929worker1 # [ 56.365274] nix-grpc-daemon[859]: time=2026-09-21T04:24:08.318Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4930worker1 # [ 56.368067] nix-grpc-daemon[859]: time=2026-09-21T04:24:08.320Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4931lb # [ 56.410112] niks3-server[1003]: 2026/09/21 04:24:08 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4932lb # [ 56.421857] niks3-server[1003]: 2026/09/21 04:24:08 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4933lb # [ 56.428858] niks3-server[1003]: 2026/09/21 04:24:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4934lb # [ 56.434056] niks3-server[1003]: 2026/09/21 04:24:08 INFO Signed narinfos id=16 count=14935worker1 # [ 56.426076] nix-grpc-daemon[859]: time=2026-09-21T04:24:08.379Z level=INFO msg="Uploading 1 narinfos"4936lb # [ 56.445156] niks3-server[1003]: 2026/09/21 04:24:08 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4937lb # [ 56.448897] niks3-server[1003]: 2026/09/21 04:24:08 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4938lb # [ 56.451803] niks3-server[1003]: 2026/09/21 04:24:08 INFO Completed upload id=164939worker1 # [ 56.441349] nix-grpc-daemon[859]: time=2026-09-21T04:24:08.394Z level=INFO msg="Upload complete. (301ms)"4940lb # [ 56.454359] niks3-server[1003]: 2026/09/21 04:24:08 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4941worker1 # [ 56.448233] nix-grpc-daemon[786]: ts=2026-09-21T04:24:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14942worker2 # [ 56.443475] nix-grpc-daemon[785]: ts=2026-09-21T04:24:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:445684943worker1 # [ 56.460831] nix-daemon[846]: reaped child process 1134, status = succeeded4944worker2 # [ 56.452597] nix-grpc-daemon[785]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4945worker2 # [ 56.478313] nix-grpc-daemon[785]: ts=2026-09-21T04:24:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:445684946worker2 # [ 56.483965] nix-grpc-daemon[785]: ts=2026-09-21T04:24:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=0 paths=1 nar_bytes_out=2324947worker2 # [ 56.538200] nix-grpc-daemon[785]: ts=2026-09-21T04:24:08Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=2 bytes_in=376 bytes_out=2004948worker2 # [ 56.543555] nix-daemon[865]: reaped child process 998, status = succeeded4949client # [ 56.591065] nix-daemon[946]: reaped child process 948, status = succeeded4950client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.72 seconds)4951client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4952client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4953client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4954client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4955client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4956client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4957(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.81 seconds)4958subtest: an input only one worker has still reaches the builder4959client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4960client # this derivation will be built:4961client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4962client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4963client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.41 seconds)4964client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4965client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.19 seconds)4966worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4967worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4968worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4969worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)4970client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24971client # [ 57.735199] nix-daemon[946]: accepted connection from pid 1033, user root (trusted)4972client # this derivation will be built:4973client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4974lb # [ 57.871549] envoy[910]: conn peer=[2001:db8:1::1]:45868 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2998 tx=1451 ms=46734975lb # [ 57.875373] envoy[910]: conn peer=[2001:db8:1::1]:45872 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=36654976lb # [ 57.878841] envoy[910]: conn peer=[2001:db8:1::1]:45878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3423 tx=1711 ms=21154977lb # [ 57.882310] envoy[910]: conn peer=[2001:db8:1::1]:45894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=564978lb # [ 57.885716] envoy[910]: conn peer=[2001:db8:1::1]:36068 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=564979lb # [ 57.889154] envoy[910]: conn peer=[2001:db8:1::1]:45896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4442 tx=2323 ms=19064980lb # [ 57.893408] envoy[910]: conn peer=[2001:db8:1::1]:36086 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=584981lb # [ 57.896779] envoy[910]: conn peer=[2001:db8:1::1]:36076 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4932 tx=2592 ms=26554982client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...4983worker1 # [ 57.944698] nix-grpc-daemon[786]: ts=2026-09-21T04:24:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:443124984worker1 # [ 57.951926] nix-grpc-daemon[786]: ts=2026-09-21T04:24:09Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:443124985worker1 # [ 57.961070] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4986worker2 # [ 57.971376] nix-grpc-daemon[785]: ts=2026-09-21T04:24:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445684987client # copying 1 paths...4988client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4989worker2 # [ 57.983974] nix-grpc-daemon[785]: ts=2026-09-21T04:24:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:445684990worker2 # [ 57.994733] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4991worker2 # [ 58.181557] nix-daemon[865]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)4992worker2 # [ 58.195206] nix-daemon[865]: reaped child process 1012, status = succeeded4993lb # [ 58.229979] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received uploads request method=POST path=/api/pending_closures4994worker2 # [ 58.215706] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.203Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4995worker2 # [ 58.217169] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.203Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4996lb # [ 58.251707] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4997lb # [ 58.260667] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4998lb # [ 58.265778] niks3-server[1003]: 2026/09/21 04:24:10 INFO Signed narinfos id=17 count=14999worker2 # [ 58.249934] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.238Z level=INFO msg="Uploading 1 narinfos"5000lb # [ 58.271726] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5001lb # [ 58.282273] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5002worker2 # [ 58.268663] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.257Z level=INFO msg="Upload complete. (263ms)"5003lb # [ 58.287853] niks3-server[1003]: 2026/09/21 04:24:10 INFO Completed upload id=175004worker2 # [ 58.270757] nix-grpc-daemon[785]: ts=2026-09-21T04:24:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=0 paths=1 nar_bytes_in=1205005lb # [ 58.291218] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5006worker2 # [ 58.277628] nix-daemon[865]: reaped child process 1007, status = succeeded5007worker2 # [ 58.282976] nix-grpc-daemon[785]: ts=2026-09-21T04:24:10Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05008worker2 # [ 58.284883] nix-grpc-daemon[785]: ts=2026-09-21T04:24:10Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15009worker2 # [ 58.331437] nix-grpc-daemon[785]: ts=2026-09-21T04:24:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:445685010worker2 # [ 58.343931] nix-grpc-daemon[785]: ts=2026-09-21T04:24:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445685011client # copying 2 paths to 'lb:50051'...5012client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5013worker2 # [ 58.360352] nix-grpc-daemon[785]: ts=2026-09-21T04:24:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:445685014client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5015worker2 # [ 58.373159] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5016worker2 # [ 58.546210] nix-daemon[865]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)5017worker2 # [ 58.558480] nix-daemon[865]: reaped child process 1021, status = succeeded5018worker2 # [ 58.717375] nix-daemon[865]: accepted connection from pid 1023, user nix-grpc-daemon (trusted)5019worker2 # [ 58.728935] nix-daemon[865]: reaped child process 1026, status = succeeded5020lb # [ 58.764387] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received uploads request method=POST path=/api/pending_closures5021lb # [ 58.768769] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received uploads request method=POST path=/api/pending_closures5022worker2 # [ 58.753269] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.741Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5023worker2 # [ 58.755457] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.743Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5024worker2 # [ 58.757505] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.743Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5025lb # [ 58.787731] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5026lb # [ 58.790579] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5027lb # [ 58.796413] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5028lb # [ 58.800689] niks3-server[1003]: 2026/09/21 04:24:10 INFO Signed narinfos id=18 count=15029lb # [ 58.801987] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5030lb # [ 58.803952] niks3-server[1003]: 2026/09/21 04:24:10 INFO Signed narinfos id=19 count=25031worker2 # [ 58.786765] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.774Z level=INFO msg="Uploading 3 narinfos"5032lb # [ 58.805750] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5033lb # [ 58.808992] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5034lb # [ 58.822144] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5035lb # [ 58.825409] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5036lb # [ 58.828109] niks3-server[1003]: 2026/09/21 04:24:10 INFO Completed upload id=185037lb # [ 58.830628] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5038lb # [ 58.832149] niks3-server[1003]: 2026/09/21 04:24:10 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5039lb # [ 58.834111] niks3-server[1003]: 2026/09/21 04:24:10 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5040lb # [ 58.836536] niks3-server[1003]: 2026/09/21 04:24:10 INFO Completed upload id=195041worker2 # [ 58.817669] nix-grpc-daemon[920]: time=2026-09-21T04:24:10.806Z level=INFO msg="Upload complete. (430ms)"5042worker2 # [ 58.821068] nix-grpc-daemon[785]: ts=2026-09-21T04:24:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=0 paths=2 nar_bytes_in=9765043worker2 # [ 58.823584] nix-grpc-daemon[785]: ts=2026-09-21T04:24:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:445685044worker2 # [ 58.829261] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5045worker2 # [ 58.832411] nix-daemon[865]: reaped child process 1016, status = succeeded5046worker2 # [ 58.837411] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5047client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5048client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5049client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5050worker2 # [ 59.864639] nix-daemon[865]: reaped child process 1030, status = succeeded5051worker2 # [ 60.031772] nix-daemon[865]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)5052worker2 # [ 60.042899] nix-daemon[865]: reaped child process 1049, status = succeeded5053lb # [ 60.077334] niks3-server[1003]: 2026/09/21 04:24:12 INFO Received uploads request method=POST path=/api/pending_closures5054worker2 # [ 60.062963] nix-grpc-daemon[920]: time=2026-09-21T04:24:12.051Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5055lb # [ 60.093020] niks3-server[1003]: 2026/09/21 04:24:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5056lb # [ 60.094989] niks3-server[1003]: 2026/09/21 04:24:12 INFO Signed narinfos id=20 count=15057worker2 # [ 60.075821] nix-grpc-daemon[920]: time=2026-09-21T04:24:12.064Z level=INFO msg="Uploading 1 narinfos"5058lb # [ 60.097935] niks3-server[1003]: 2026/09/21 04:24:12 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5059lb # [ 60.100732] niks3-server[1003]: 2026/09/21 04:24:12 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5060lb # [ 60.104438] niks3-server[1003]: 2026/09/21 04:24:12 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5061lb # [ 60.111196] niks3-server[1003]: 2026/09/21 04:24:12 INFO Completed upload id=205062worker2 # [ 60.092159] nix-grpc-daemon[920]: time=2026-09-21T04:24:12.080Z level=INFO msg="Upload complete. (247ms)"5063lb # [ 60.112872] niks3-server[1003]: 2026/09/21 04:24:12 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5064worker2 # [ 60.094362] nix-grpc-daemon[785]: ts=2026-09-21T04:24:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15065client # copying 1 paths...5066worker2 # [ 60.101925] nix-grpc-daemon[785]: ts=2026-09-21T04:24:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:445685067worker2 # [ 60.103650] nix-daemon[865]: reaped child process 1028, status = succeeded5068client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5069worker2 # [ 60.120661] nix-grpc-daemon[785]: ts=2026-09-21T04:24:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:445685070worker2 # [ 60.122641] nix-grpc-daemon[785]: ts=2026-09-21T04:24:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=0 paths=1 nar_bytes_out=1205071worker1 # [ 60.186858] nix-grpc-daemon[786]: ts=2026-09-21T04:24:12Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=2 bytes_in=320 bytes_out=2005072worker1 # [ 60.192312] nix-daemon[846]: reaped child process 1163, status = succeeded5073client # [ 60.228391] nix-daemon[946]: reaped child process 1036, status = succeeded5074client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.71 seconds)5075client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25076client # [ 60.421671] nix-daemon[946]: accepted connection from pid 1057, user root (trusted)5077client # this derivation will be built:5078client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5079worker2 # [ 60.619394] nix-grpc-daemon[785]: ts=2026-09-21T04:24:12Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:445685080client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5081worker1 # [ 60.631094] nix-grpc-daemon[786]: ts=2026-09-21T04:24:12Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:443125082worker1 # [ 60.637088] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5083worker2 # [ 60.683895] nix-grpc-daemon[785]: ts=2026-09-21T04:24:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445685084worker2 # [ 60.692157] nix-grpc-daemon[920]: time=2026-09-21T04:24:12.680Z level=INFO msg="All 1 paths already cached"5085worker2 # [ 60.706160] nix-grpc-daemon[785]: ts=2026-09-21T04:24:12Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05086worker1 # [ 60.721524] nix-grpc-daemon[786]: ts=2026-09-21T04:24:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443125087worker2 # [ 60.710865] nix-grpc-daemon[785]: ts=2026-09-21T04:24:12Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15088worker1 # [ 60.730273] nix-grpc-daemon[786]: ts=2026-09-21T04:24:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443125089client # copying 1 paths to 'lb:50051'...5090client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5091worker1 # [ 60.744240] nix-grpc-daemon[786]: ts=2026-09-21T04:24:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443125092worker1 # [ 60.751102] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5093worker1 # [ 60.762621] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)5094worker1 # [ 60.786802] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 2/5)5095worker1 # [ 60.841566] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 220 ms (attempt 3/5)5096worker1 # [ 61.066163] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 255 ms (attempt 4/5)5097worker1 # [ 61.328618] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5098worker1 # [ 61.333247] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5099worker1 # [ 61.338792] nix-grpc-daemon[786]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5100worker1 # [ 61.544573] nix-daemon[846]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5101worker1 # [ 61.560419] nix-daemon[846]: reaped child process 1179, status = succeeded5102worker1 # [ 61.703446] nix-daemon[846]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)5103worker1 # [ 61.718543] nix-daemon[846]: reaped child process 1184, status = succeeded5104lb # [ 61.743805] niks3-server[1003]: 2026/09/21 04:24:13 INFO Received uploads request method=POST path=/api/pending_closures5105worker1 # [ 61.742589] nix-grpc-daemon[859]: time=2026-09-21T04:24:13.695Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5106worker1 # [ 61.744056] nix-grpc-daemon[859]: time=2026-09-21T04:24:13.695Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5107lb # [ 61.773188] niks3-server[1003]: 2026/09/21 04:24:13 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5108lb # [ 61.782508] niks3-server[1003]: 2026/09/21 04:24:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5109lb # [ 61.787705] niks3-server[1003]: 2026/09/21 04:24:13 INFO Signed narinfos id=21 count=15110worker1 # [ 61.779774] nix-grpc-daemon[859]: time=2026-09-21T04:24:13.733Z level=INFO msg="Uploading 1 narinfos"5111lb # [ 61.793784] niks3-server[1003]: 2026/09/21 04:24:13 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5112lb # [ 61.802935] niks3-server[1003]: 2026/09/21 04:24:13 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5113lb # [ 61.807143] niks3-server[1003]: 2026/09/21 04:24:13 INFO Completed upload id=215114worker1 # [ 61.796353] nix-grpc-daemon[859]: time=2026-09-21T04:24:13.749Z level=INFO msg="Upload complete. (434ms)"5115lb # [ 61.809582] niks3-server[1003]: 2026/09/21 04:24:13 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5116worker1 # [ 61.798469] nix-grpc-daemon[786]: ts=2026-09-21T04:24:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 paths=1 nar_bytes_in=5605117worker1 # [ 61.802569] nix-grpc-daemon[786]: ts=2026-09-21T04:24:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443125118worker1 # [ 61.808277] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5119worker1 # [ 61.817297] nix-daemon[846]: reaped child process 1169, status = succeeded5120worker1 # [ 61.820608] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5121client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5122client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5123client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5124worker1 # [ 62.844701] nix-daemon[846]: reaped child process 1188, status = succeeded5125worker1 # [ 62.997098] nix-daemon[846]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5126worker1 # [ 63.010701] nix-daemon[846]: reaped child process 1207, status = succeeded5127lb # [ 63.034579] niks3-server[1003]: 2026/09/21 04:24:14 INFO Received uploads request method=POST path=/api/pending_closures5128worker1 # [ 63.032411] nix-grpc-daemon[859]: time=2026-09-21T04:24:14.985Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5129lb # [ 63.055174] niks3-server[1003]: 2026/09/21 04:24:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5130lb # [ 63.058154] niks3-server[1003]: 2026/09/21 04:24:15 INFO Signed narinfos id=22 count=15131worker1 # [ 63.047443] nix-grpc-daemon[859]: time=2026-09-21T04:24:15.000Z level=INFO msg="Uploading 1 narinfos"5132lb # [ 63.061062] niks3-server[1003]: 2026/09/21 04:24:15 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5133lb # [ 63.064146] niks3-server[1003]: 2026/09/21 04:24:15 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5134lb # [ 63.068056] niks3-server[1003]: 2026/09/21 04:24:15 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5135lb # [ 63.071487] niks3-server[1003]: 2026/09/21 04:24:15 INFO Completed upload id=225136worker1 # [ 63.061754] nix-grpc-daemon[859]: time=2026-09-21T04:24:15.015Z level=INFO msg="Upload complete. (233ms)"5137lb # [ 63.073020] niks3-server[1003]: 2026/09/21 04:24:15 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5138worker1 # [ 63.064451] nix-grpc-daemon[786]: ts=2026-09-21T04:24:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15139client # copying 1 paths...5140worker2 # [ 63.060379] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:445685141worker1 # [ 63.071398] nix-daemon[846]: reaped child process 1186, status = succeeded5142worker2 # [ 63.067546] nix-grpc-daemon[785]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5143client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5144worker2 # [ 63.085076] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:445685145worker2 # [ 63.088628] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=0 paths=1 nar_bytes_out=1205146worker1 # [ 63.145076] nix-grpc-daemon[786]: ts=2026-09-21T04:24:15Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=2 bytes_in=320 bytes_out=2005147worker1 # [ 63.147361] nix-daemon[846]: reaped child process 1166, status = succeeded5148client # [ 63.174620] nix-daemon[946]: reaped child process 1060, status = succeeded5149client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.95 seconds)5150client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25151client # [ 63.354961] nix-daemon[946]: accepted connection from pid 1081, user root (trusted)5152client # this derivation will be built:5153client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5154worker2 # [ 63.548184] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:445685155client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5156worker1 # [ 63.560097] nix-grpc-daemon[786]: ts=2026-09-21T04:24:15Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:443125157worker1 # [ 63.567763] nix-daemon[846]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5158worker2 # [ 63.619569] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445685159worker2 # [ 63.624508] nix-grpc-daemon[920]: time=2026-09-21T04:24:15.612Z level=INFO msg="All 1 paths already cached"5160worker2 # [ 63.635568] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05161worker2 # [ 63.637617] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15162worker2 # [ 63.679229] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:445685163worker2 # [ 63.691393] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:445685164worker2 # [ 63.703646] nix-grpc-daemon[920]: time=2026-09-21T04:24:15.692Z level=INFO msg="All 1 paths already cached"5165client # copying 1 paths to 'lb:50051'...5166client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5167worker2 # [ 63.712359] nix-grpc-daemon[785]: ts=2026-09-21T04:24:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:445685168worker2 # [ 63.723825] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5169worker2 # [ 63.900587] nix-daemon[865]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)5170worker2 # [ 63.913664] nix-daemon[865]: reaped child process 1061, status = succeeded5171worker2 # [ 64.080637] nix-daemon[865]: accepted connection from pid 1063, user nix-grpc-daemon (trusted)5172worker2 # [ 64.092733] nix-daemon[865]: reaped child process 1066, status = succeeded5173lb # [ 64.130258] niks3-server[1003]: 2026/09/21 04:24:16 INFO Received uploads request method=POST path=/api/pending_closures5174worker2 # [ 64.118378] nix-grpc-daemon[920]: time=2026-09-21T04:24:16.106Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5175worker2 # [ 64.119726] nix-grpc-daemon[920]: time=2026-09-21T04:24:16.106Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5176lb # [ 64.153685] niks3-server[1003]: 2026/09/21 04:24:16 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5177lb # [ 64.167293] niks3-server[1003]: 2026/09/21 04:24:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5178lb # [ 64.172957] niks3-server[1003]: 2026/09/21 04:24:16 INFO Signed narinfos id=23 count=15179worker2 # [ 64.157201] nix-grpc-daemon[920]: time=2026-09-21T04:24:16.145Z level=INFO msg="Uploading 1 narinfos"5180lb # [ 64.180974] niks3-server[1003]: 2026/09/21 04:24:16 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5181lb # [ 64.189781] niks3-server[1003]: 2026/09/21 04:24:16 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5182worker2 # [ 64.177200] nix-grpc-daemon[920]: time=2026-09-21T04:24:16.165Z level=INFO msg="Upload complete. (441ms)"5183lb # [ 64.196440] niks3-server[1003]: 2026/09/21 04:24:16 INFO Completed upload id=235184worker2 # [ 64.179754] nix-grpc-daemon[785]: ts=2026-09-21T04:24:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=0 paths=1 nar_bytes_in=5605185lb # [ 64.200080] niks3-server[1003]: 2026/09/21 04:24:16 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5186worker2 # [ 64.186141] nix-grpc-daemon[785]: ts=2026-09-21T04:24:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:445685187worker2 # [ 64.193775] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5188worker2 # [ 64.198294] nix-daemon[865]: reaped child process 1056, status = succeeded5189worker2 # [ 64.203356] nix-daemon[865]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5190client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5191client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5192client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5193worker2 # [ 65.016919] nix-daemon[865]: reaped child process 1070, status = succeeded5194worker2 # [ 65.161830] nix-daemon[865]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)5195worker2 # [ 65.173747] nix-daemon[865]: reaped child process 1089, status = succeeded5196lb # [ 65.208264] niks3-server[1003]: 2026/09/21 04:24:17 INFO Received uploads request method=POST path=/api/pending_closures5197worker2 # [ 65.197819] nix-grpc-daemon[920]: time=2026-09-21T04:24:17.186Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5198lb # [ 65.238281] niks3-server[1003]: 2026/09/21 04:24:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5199lb # [ 65.247446] niks3-server[1003]: 2026/09/21 04:24:17 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5200lb # [ 65.252722] niks3-server[1003]: 2026/09/21 04:24:17 INFO Signed narinfos id=24 count=15201worker2 # [ 65.236930] nix-grpc-daemon[920]: time=2026-09-21T04:24:17.225Z level=INFO msg="Uploading 1 narinfos"5202lb # [ 65.257987] niks3-server[1003]: 2026/09/21 04:24:17 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5203lb # [ 65.269495] niks3-server[1003]: 2026/09/21 04:24:17 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5204lb # [ 65.277435] niks3-server[1003]: 2026/09/21 04:24:17 INFO Completed upload id=245205worker2 # [ 65.258430] nix-grpc-daemon[920]: time=2026-09-21T04:24:17.247Z level=INFO msg="Upload complete. (248ms)"5206lb # [ 65.279845] niks3-server[1003]: 2026/09/21 04:24:17 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5207worker2 # [ 65.262589] nix-grpc-daemon[785]: ts=2026-09-21T04:24:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15208client # copying 1 paths...5209worker2 # [ 65.275394] nix-daemon[865]: reaped child process 1068, status = succeeded5210worker1 # [ 65.281851] nix-grpc-daemon[786]: ts=2026-09-21T04:24:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443125211worker1 # [ 65.295238] nix-grpc-daemon[786]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5212client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5213worker2 # [ 65.322149] nix-grpc-daemon[785]: ts=2026-09-21T04:24:17Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:445685214worker2 # [ 65.326591] nix-grpc-daemon[785]: ts=2026-09-21T04:24:17Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44568 duration_s=0 paths=1 nar_bytes_out=1205215worker1 # [ 65.347430] nix-grpc-daemon[786]: ts=2026-09-21T04:24:17Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:44312 duration_s=1 bytes_in=320 bytes_out=2005216worker1 # [ 65.352897] nix-daemon[846]: reaped child process 1210, status = succeeded5217client # [ 65.385577] nix-daemon[946]: reaped child process 1084, status = succeeded5218client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.21 seconds)5219(finished: subtest: an input only one worker has still reaches the builder, in 8.70 seconds)5220subtest: upload whose reference the worker lacks is completed from the cache5221client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5222client # this derivation will be built:5223client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5224client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5225client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.41 seconds)5226client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5227client # this derivation will be built:5228client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5229client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5230client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.41 seconds)5231client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5232client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.19 seconds)5233worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5234worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.23 seconds)5235worker2 # [ 66.696563] systemd[1]: Stopping Nix worker-protocol over gRPC...5236worker2 # [ 67.250213] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=signal_exit5237worker2 # [ 67.505635] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=scheduler_yield5238worker2 # [ 67.509975] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=scheduler_restarting addr=lb:500515239lb # [ 67.528853] niks3-server[1003]: 2026/09/21 04:24:19 INFO lead: released remote=192.168.1.4:575185240worker2 # [ 67.514111] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:445665241worker1 # [ 67.525566] nix-grpc-daemon[786]: ts=2026-09-21T04:24:19Z level=info event=scheduler_restarting addr=lb:500515242worker2 # [ 67.518868] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=worker_gone addr=192.168.1.4:500515243worker2 # [ 67.521877] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:445665244worker2 # [ 67.525159] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=worker_gone addr=192.168.1.3:500515245worker2 # [ 67.711153] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500515246worker2 # [ 67.715145] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5247worker1 # [ 67.725723] nix-grpc-daemon[786]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500515248worker1 # [ 67.729755] nix-grpc-daemon[786]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5249lb # [ 67.872453] envoy[910]: conn peer=[2001:db8:1::1]:36108 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555250lb # [ 67.875952] envoy[910]: conn peer=[2001:db8:1::1]:36102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4810 tx=2105 ms=22425251lb # [ 67.879511] envoy[910]: conn peer=[2001:db8:1::1]:36550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=505252lb # [ 67.882885] envoy[910]: conn peer=[2001:db8:1::1]:36120 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4241 tx=2290 ms=25205253lb # [ 67.886459] envoy[910]: conn peer=[2001:db8:1::1]:36562 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175254lb # [ 67.889928] envoy[910]: conn peer=[2001:db8:1::1]:36558 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4313 tx=2265 ms=17885255worker2 # [ 67.916205] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500515256worker2 # [ 67.920133] nix-grpc-daemon[785]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5257worker1 # [ 67.930657] nix-grpc-daemon[786]: ts=2026-09-21T04:24:19Z level=info event=scheduler_connected addr=lb:500515258worker1 # [ 67.934768] nix-grpc-daemon[786]: ts=2026-09-21T04:24:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5259worker2 # [ 68.121230] nix-grpc-daemon[785]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500515260worker2 # [ 68.124964] nix-grpc-daemon[785]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5261worker1 # [ 68.135659] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500515262worker1 # [ 68.139687] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5263worker2 # [ 68.327438] nix-grpc-daemon[785]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500515264worker2 # [ 68.330560] nix-grpc-daemon[785]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5265worker1 # [ 68.340373] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500515266worker1 # [ 68.344272] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5267worker2 # [ 68.509709] nix-grpc-daemon[785]: ts=2026-09-21T04:24:20Z level=info event=scheduler_restarting5268worker1 # [ 68.546736] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500515269worker2 # [ 68.540488] nix-grpc-daemon[785]: ts=2026-09-21T04:24:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5270worker1 # [ 68.550164] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5271worker2 # [ 68.550672] nix-daemon[865]: reaped child process 867, status = succeeded5272worker1 # [ 68.747741] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500515273worker1 # [ 68.751818] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5274worker1 # [ 68.952700] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_connected addr=lb:500515275worker1 # [ 68.956708] nix-grpc-daemon[786]: ts=2026-09-21T04:24:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5276worker2 # [ 68.958525] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5277worker2 # [ 68.961192] systemd[1]: Stopped Nix worker-protocol over gRPC.5278worker2 # [ 68.963681] systemd[1]: nix-grpc-daemon.service: Consumed 1.477s CPU time over 50.982s wall clock time, 53M memory peak, 57.2K incoming IP traffic, 160.9K outgoing IP traffic.5279worker2 # [ 68.973822] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5280worker2 # [ 68.978233] systemd[1]: Closed Nix worker-protocol over gRPC.5281worker2 # [ 68.981146] systemd[1]: nix-grpc-daemon.socket: Consumed 337us CPU time over 59.477s wall clock time, 256K memory peak, 72.1K incoming IP traffic, 61.1K outgoing IP traffic.5282lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5283lb: must succeed: curl -sf localhost:9901/clusters5284lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5285lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5286client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25287worker1 # [ 69.157617] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500515288worker1 # [ 69.161839] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5289worker1 # [ 69.304248] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443125290worker1 # [ 69.311665] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:443125291client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5292client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.27 seconds)5293worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5294worker1 # [ 69.362503] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500515295worker1 # [ 69.366444] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5296worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5297worker2 # [ 69.428736] systemd[1]: Listening on Nix worker-protocol over gRPC.5298worker1 # [ 69.501597] systemd[1]: Stopping Nix worker-protocol over gRPC...5299worker1 # [ 69.568584] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500515300worker1 # [ 69.572381] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5301worker1 # [ 69.773338] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500515302worker1 # [ 69.777445] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303worker1 # [ 69.978579] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_connected addr=lb:500515304worker1 # [ 69.982642] nix-grpc-daemon[786]: ts=2026-09-21T04:24:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker1 # [ 70.183744] nix-grpc-daemon[786]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500515306worker1 # [ 70.188774] nix-grpc-daemon[786]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307worker1 # [ 70.192500] nix-grpc-daemon[786]: ts=2026-09-21T04:24:22Z level=info event=signal_exit5308worker2 # [ 70.375963] systemd[1]: Starting scheduler-token.service...5309worker1 # [ 70.388666] nix-grpc-daemon[786]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500515310worker1 # [ 70.392712] nix-grpc-daemon[786]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5311worker2 # [ 70.477146] systemd[1]: scheduler-token.service: Deactivated successfully.5312worker2 # [ 70.478497] systemd[1]: Finished scheduler-token.service.5313worker2 # [ 70.479656] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 101ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5314worker2 # [ 70.485352] systemd[1]: Starting Nix worker-protocol over gRPC...5315worker1 # [ 70.594584] nix-grpc-daemon[786]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500515316worker1 # [ 70.598398] nix-grpc-daemon[786]: ts=2026-09-21T04:24:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker2 # [ 70.651558] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5318worker2 # [ 70.653450] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5319worker2 # [ 70.656935] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15320worker2 # [ 70.664308] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5321worker2 # [ 70.668149] systemd[1]: Started Nix worker-protocol over gRPC.5322worker2 # [ 70.678167] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=scheduler_take_over5323lb # [ 70.696591] niks3-server[1003]: 2026/09/21 04:24:22 INFO lead: acquired remote=192.168.1.4:570765324worker2 # [ 70.697474] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500515325worker2 # [ 70.703825] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=oidc_jwks provider=mock keys=15326worker2 # [ 70.705214] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:590925327worker2 # [ 70.706777] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05328worker1 # [ 70.799403] nix-grpc-daemon[786]: ts=2026-09-21T04:24:22Z level=info event=scheduler_connected addr=lb:500515329worker2 # [ 70.793254] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:590925330worker2 # [ 70.796937] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05331worker1 # [ 71.451949] nix-grpc-daemon[786]: ts=2026-09-21T04:24:23Z level=info event=scheduler_restarting5332worker2 # [ 71.452624] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590925333worker2 # [ 71.456394] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:23Z level=info event=worker_gone addr=192.168.1.3:500515334worker1 # [ 71.477885] nix-grpc-daemon[786]: ts=2026-09-21T04:24:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5335worker1 # [ 71.494939] nix-daemon[846]: reaped child process 849, status = succeeded5336worker1 # [ 72.325850] nix-grpc-daemon[786]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5337worker1 # [ 72.329439] nix-grpc-daemon[786]: E0000 00:00:1789964664.277571 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5338worker1 # [ 72.334239] nix-grpc-daemon[786]: E0000 00:00:1789964664.278345 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5339worker1 # [ 72.915771] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5340worker1 # [ 72.920847] systemd[1]: Stopped Nix worker-protocol over gRPC.5341worker1 # [ 72.924416] systemd[1]: nix-grpc-daemon.service: Consumed 2.648s CPU time over 55.066s wall clock time, 62.9M memory peak, 85K incoming IP traffic, 159.4K outgoing IP traffic.5342worker1 # [ 72.934850] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5343worker1 # [ 72.938550] systemd[1]: Closed Nix worker-protocol over gRPC.5344worker1 # [ 72.941481] systemd[1]: nix-grpc-daemon.socket: Consumed 335us CPU time over 1min 3.409s wall clock time, 256K memory peak, 78.4K incoming IP traffic, 64.4K outgoing IP traffic.5345lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5346lb: must succeed: curl -sf localhost:9901/clusters5347lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5348lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5349client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25350worker2 # [ 73.269747] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:25Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:532765351worker2 # [ 73.281881] nix-daemon[865]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5352worker2 # [ 73.344275] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532765353worker2 # [ 73.354164] nix-daemon[865]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5354worker2 # [ 73.371409] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)5355worker2 # [ 73.426663] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 2/5)5356worker2 # [ 73.484569] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 216 ms (attempt 3/5)5357worker2 # [ 73.706451] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 423 ms (attempt 4/5)5358worker2 # [ 74.135640] nix-grpc-daemon[1113]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5359worker2 # [ 74.139548] nix-grpc-daemon[1113]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5360client # copying 2 paths...5361client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5362worker2 # [ 74.161585] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532765363client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5364worker2 # [ 74.171353] nix-daemon[865]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5365worker2 # [ 74.193525] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:26Z level=info event=niks3_push_started pid=11745366worker2 # [ 74.423048] nix-grpc-daemon[1174]: time=2026-09-21T04:24:26.411Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5367worker2 # [ 74.621901] nix-daemon[865]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)5368worker2 # [ 74.632360] nix-daemon[865]: reaped child process 1190, status = succeeded5369lb # [ 74.673652] niks3-server[1003]: 2026/09/21 04:24:26 INFO Received uploads request method=POST path=/api/pending_closures5370lb # [ 74.681896] niks3-server[1003]: 2026/09/21 04:24:26 INFO Received uploads request method=POST path=/api/pending_closures5371worker2 # [ 74.670038] nix-grpc-daemon[1174]: time=2026-09-21T04:24:26.658Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5372worker2 # [ 74.671628] nix-grpc-daemon[1174]: time=2026-09-21T04:24:26.660Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5373worker2 # [ 74.673584] nix-grpc-daemon[1174]: time=2026-09-21T04:24:26.662Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5374lb # [ 74.733560] niks3-server[1003]: 2026/09/21 04:24:26 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5375lb # [ 74.738417] niks3-server[1003]: 2026/09/21 04:24:26 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5376lb # [ 74.753643] niks3-server[1003]: 2026/09/21 04:24:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5377lb # [ 74.755396] niks3-server[1003]: 2026/09/21 04:24:26 INFO Signed narinfos id=25 count=15378lb # [ 74.758784] niks3-server[1003]: 2026/09/21 04:24:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5379lb # [ 74.761698] niks3-server[1003]: 2026/09/21 04:24:26 INFO Signed narinfos id=26 count=25380worker2 # [ 74.744123] nix-grpc-daemon[1174]: time=2026-09-21T04:24:26.731Z level=INFO msg="Uploading 3 narinfos"5381lb # [ 74.764152] niks3-server[1003]: 2026/09/21 04:24:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5382lb # [ 74.765775] niks3-server[1003]: 2026/09/21 04:24:26 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5383lb # [ 74.782141] niks3-server[1003]: 2026/09/21 04:24:26 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5384lb # [ 74.785802] niks3-server[1003]: 2026/09/21 04:24:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5385lb # [ 74.787867] niks3-server[1003]: 2026/09/21 04:24:26 INFO Completed upload id=255386lb # [ 74.789433] niks3-server[1003]: 2026/09/21 04:24:26 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5387lb # [ 74.791638] niks3-server[1003]: 2026/09/21 04:24:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5388lb # [ 74.794977] niks3-server[1003]: 2026/09/21 04:24:26 INFO Completed upload id=265389worker2 # [ 74.776358] nix-grpc-daemon[1174]: time=2026-09-21T04:24:26.764Z level=INFO msg="Upload complete. (349ms)"5390worker2 # [ 74.783223] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53276 duration_s=0 paths=2 nar_bytes_in=3125391worker2 # [ 74.788940] nix-grpc-daemon[1174]: time=2026-09-21T04:24:26.768Z level=WARN msg="Failed to register uploaded object" key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo error="server returned 404: object is not pending upload\n"5392worker2 # [ 74.806716] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:26Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:53276 duration_s=1 bytes_in=400 bytes_out=2005393worker2 # [ 74.808714] nix-daemon[865]: reaped child process 1163, status = succeeded5394worker2 # [ 74.809673] nix-daemon[865]: reaped child process 1172, status = succeeded5395client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.80 seconds)5396worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5397worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5398worker1 # [ 74.927660] systemd[1]: Listening on Nix worker-protocol over gRPC.5399worker1 # [ 74.935908] systemd[1]: Starting Nix worker-protocol over gRPC...5400worker1 # [ 75.132928] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5401worker1 # [ 75.134856] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5402worker1 # [ 75.138959] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15403worker1 # [ 75.146745] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5404worker1 # [ 75.148450] systemd[1]: Started Nix worker-protocol over gRPC.5405lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5406lb: must succeed: curl -sf localhost:9901/clusters5407worker1 # [ 75.180207] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:27Z level=info event=scheduler_connected addr=lb:500515408worker2 # [ 75.173893] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:590925409worker2 # [ 75.178622] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05410lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5411lb: must succeed: curl -sf localhost:9901/clusters5412lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5413lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.17 seconds)5414(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.92 seconds)5415subtest: scheduler down: the other node takes the lock and keeps it5416lb: must succeed: curl -sf localhost:9901/clusters5417lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5418lb: must succeed: curl -sf localhost:9901/clusters5419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5420lb: must succeed: curl -sf localhost:9901/clusters5421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5422lb: must succeed: curl -sf localhost:9901/clusters5423lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5424worker2 # [ 76.575545] systemd[1]: Stopping Nix worker-protocol over gRPC...5425worker2 # [ 76.670125] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:28Z level=info event=signal_exit5426worker2 # [ 76.880979] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:28Z level=info event=scheduler_yield5427lb # [ 76.903319] niks3-server[1003]: 2026/09/21 04:24:28 INFO lead: released remote=192.168.1.4:570765428worker1 # [ 76.892962] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:28Z level=info event=scheduler_restarting addr=lb:500515429worker2 # [ 76.887750] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590925430worker2 # [ 76.893242] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:28Z level=info event=worker_gone addr=192.168.1.3:500515431worker2 # [ 76.896696] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:28Z level=info event=scheduler_restarting addr=lb:500515432worker2 # [ 76.899674] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590925433worker2 # [ 76.903178] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:28Z level=info event=worker_gone addr=192.168.1.4:500515434worker1 # [ 77.092869] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515435worker1 # [ 77.097106] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5436worker2 # [ 77.093570] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515437worker2 # [ 77.097457] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5438worker1 # [ 77.298077] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515439worker1 # [ 77.302332] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5440worker2 # [ 77.298452] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515441worker2 # [ 77.302267] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442worker1 # [ 77.503233] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515443worker1 # [ 77.507371] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5444worker2 # [ 77.503246] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515445worker2 # [ 77.507065] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5446worker1 # [ 77.708378] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515447worker1 # [ 77.712747] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5448worker2 # [ 77.708921] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515449worker2 # [ 77.712173] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450lb # [ 77.872859] envoy[910]: conn peer=[2001:db8:1::4]:34012 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=37929 tx=1950 ms=502435451lb # [ 77.876815] envoy[910]: conn peer=[2001:db8:1::1]:36568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=225452lb # [ 77.880296] envoy[910]: conn peer=[2001:db8:1::3]:53760 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13273 tx=2727 ms=532155453lb # [ 77.883897] envoy[910]: conn peer=[2001:db8:1::1]:44914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1612 tx=639 ms=15275454worker2 # [ 77.885524] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=scheduler_restarting5455worker2 # [ 77.906970] nix-daemon[865]: reaped child process 1166, status = succeeded5456worker1 # [ 77.916679] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_connected addr=lb:500515457worker1 # [ 77.919932] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5458worker2 # [ 77.920156] nix-grpc-daemon[1113]: ts=2026-09-21T04:24:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5459worker1 # [ 78.117512] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_connected addr=lb:500515460worker1 # [ 78.121610] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5461worker1 # [ 78.322543] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_connected addr=lb:500515462worker1 # [ 78.326631] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5463worker1 # [ 78.527797] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_connected addr=lb:500515464worker1 # [ 78.531863] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5465worker2 # [ 78.693534] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5466worker2 # [ 78.696708] systemd[1]: Stopped Nix worker-protocol over gRPC.5467worker2 # [ 78.701571] systemd[1]: nix-grpc-daemon.service: Consumed 344ms CPU time over 8.207s wall clock time, 33.6M memory peak, 19.5K incoming IP traffic, 26.1K outgoing IP traffic.5468worker2 # [ 78.711964] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5469worker2 # [ 78.715552] systemd[1]: Closed Nix worker-protocol over gRPC.5470worker2 # [ 78.718130] systemd[1]: nix-grpc-daemon.socket: Consumed 279us CPU time over 9.291s wall clock time, 256K memory peak, 21.6K incoming IP traffic, 16.1K outgoing IP traffic.5471worker1 # [ 78.732880] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_connected addr=lb:500515472worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5473worker1 # [ 78.737090] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5474worker1 # [ 78.939246] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_connected addr=lb:500515475worker1 # [ 78.942510] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5476worker1 # [ 79.140158] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_connected addr=lb:500515477worker1 # [ 79.144305] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5478worker1 # [ 79.344985] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_connected addr=lb:500515479worker1 # [ 79.349138] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5480worker1 # [ 79.550196] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_connected addr=lb:500515481worker1 # [ 79.554305] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker1 # [ 79.755479] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_connected addr=lb:500515483worker1 # [ 79.759549] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker1 # [ 79.961876] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_connected addr=lb:500515485worker1 # [ 79.965330] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5486lb # [ 80.169483] niks3-server[1003]: 2026/09/21 04:24:32 INFO lead: acquired remote=192.168.1.3:383265487worker1 # [ 80.159230] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_take_over5488worker1 # [ 80.165697] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_connected addr=lb:500515489worker1 # [ 80.169413] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5490worker1 # [ 80.370691] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_connected addr=lb:500515491worker1 # [ 80.374772] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5492worker1 # [ 80.575759] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_connected addr=lb:500515493worker1 # [ 80.579760] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5494worker1 # [ 80.780643] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_connected addr=lb:500515495worker1 # [ 80.784653] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5496worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 2.23 seconds)5497worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5498worker1 # [ 80.985449] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=scheduler_connected addr=lb:500515499worker1 # [ 80.989459] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:602145500worker1 # [ 80.992347] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05501worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5502client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25503worker1 # [ 81.293084] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:602165504worker1 # [ 81.304357] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5505client # don't know how to build these paths:5506client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5507worker1 # [ 81.319417] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602165508worker1 # [ 81.331863] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)5509worker1 # [ 81.374626] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)5510worker1 # [ 81.389143] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 246 ms (attempt 3/5)5511worker1 # [ 81.641406] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 4/5)5512worker1 # [ 81.692340] nix-grpc-daemon[1235]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5513worker1 # [ 81.695557] nix-grpc-daemon[1235]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5514worker1 # [ 81.727201] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:33Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05515worker1 # [ 81.731653] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:33Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15516worker1 # [ 81.781937] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602165517worker1 # [ 81.795822] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602165518client # copying 1 paths to 'lb:50051'...5519client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5520worker1 # [ 81.809553] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602165521worker1 # [ 81.821192] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5522worker1 # [ 81.846199] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:33Z level=info event=niks3_push_started pid=13155523worker1 # [ 82.103888] nix-grpc-daemon[1315]: time=2026-09-21T04:24:34.056Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5524worker1 # [ 82.324910] nix-daemon[846]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5525worker1 # [ 82.338532] nix-daemon[846]: reaped child process 1331, status = succeeded5526worker1 # [ 82.562256] nix-daemon[846]: accepted connection from pid 1333, user nix-grpc-daemon (trusted)5527worker1 # [ 82.575203] nix-daemon[846]: reaped child process 1336, status = succeeded5528lb # [ 82.607180] niks3-server[1003]: 2026/09/21 04:24:34 INFO Received uploads request method=POST path=/api/pending_closures5529worker1 # [ 82.605345] nix-grpc-daemon[1315]: time=2026-09-21T04:24:34.558Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5530worker1 # [ 82.606921] nix-grpc-daemon[1315]: time=2026-09-21T04:24:34.560Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5531lb # [ 82.646089] niks3-server[1003]: 2026/09/21 04:24:34 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5532lb # [ 82.653990] niks3-server[1003]: 2026/09/21 04:24:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5533worker1 # [ 82.649144] nix-grpc-daemon[1315]: time=2026-09-21T04:24:34.602Z level=INFO msg="Uploading 1 narinfos"5534lb # [ 82.659868] niks3-server[1003]: 2026/09/21 04:24:34 INFO Signed narinfos id=27 count=15535lb # [ 82.674602] niks3-server[1003]: 2026/09/21 04:24:34 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5536lb # [ 82.677566] niks3-server[1003]: 2026/09/21 04:24:34 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5537lb # [ 82.684238] niks3-server[1003]: 2026/09/21 04:24:34 INFO Completed upload id=275538worker1 # [ 82.673556] nix-grpc-daemon[1315]: time=2026-09-21T04:24:34.626Z level=INFO msg="Upload complete. (564ms)"5539lb # [ 82.689683] niks3-server[1003]: 2026/09/21 04:24:34 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5540worker1 # [ 82.677546] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60216 duration_s=0 paths=1 nar_bytes_in=5205541worker1 # [ 82.683751] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602165542worker1 # [ 82.689892] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5543worker1 # [ 82.696925] nix-daemon[846]: reaped child process 1313, status = succeeded5544client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5545worker1 # [ 82.701140] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5546client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5547client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5548client # farm-a> LOG-farm-a-on-standby5549worker1 # [ 83.915616] nix-daemon[846]: reaped child process 1340, status = succeeded5550worker1 # [ 84.101461] nix-daemon[846]: accepted connection from pid 1356, user nix-grpc-daemon (trusted)5551worker1 # [ 84.114985] nix-daemon[846]: reaped child process 1359, status = succeeded5552lb # [ 84.147715] niks3-server[1003]: 2026/09/21 04:24:36 INFO Received uploads request method=POST path=/api/pending_closures5553worker1 # [ 84.145127] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.098Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5554worker1 # [ 84.148062] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.100Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5555lb # [ 84.181888] niks3-server[1003]: 2026/09/21 04:24:36 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5556lb # [ 84.195110] niks3-server[1003]: 2026/09/21 04:24:36 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5557lb # [ 84.206147] niks3-server[1003]: 2026/09/21 04:24:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5558lb # [ 84.210925] niks3-server[1003]: 2026/09/21 04:24:36 INFO Signed narinfos id=28 count=15559worker1 # [ 84.204270] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.156Z level=INFO msg="Uploading 1 narinfos"5560lb # [ 84.215967] niks3-server[1003]: 2026/09/21 04:24:36 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5561lb # [ 84.225648] niks3-server[1003]: 2026/09/21 04:24:36 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5562lb # [ 84.230282] niks3-server[1003]: 2026/09/21 04:24:36 INFO Completed upload id=285563worker1 # [ 84.220883] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.174Z level=INFO msg="Upload complete. (313ms)"5564lb # [ 84.233232] niks3-server[1003]: 2026/09/21 04:24:36 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5565worker1 # [ 84.233225] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60216 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15566worker1 # [ 84.246767] nix-daemon[846]: reaped child process 1338, status = succeeded5567worker1 # [ 84.252052] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:36Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05568worker1 # [ 84.254183] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:36Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15569worker1 # [ 84.298570] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602165570worker1 # [ 84.311220] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602165571worker1 # [ 84.323879] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.277Z level=INFO msg="All 1 paths already cached"5572client # copying 1 paths to 'lb:50051'...5573client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5574worker1 # [ 84.331933] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602165575worker1 # [ 84.340434] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5576worker1 # [ 84.545243] nix-daemon[846]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5577worker1 # [ 84.558639] nix-daemon[846]: reaped child process 1368, status = succeeded5578worker1 # [ 84.769428] nix-daemon[846]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)5579worker1 # [ 84.783210] nix-daemon[846]: reaped child process 1373, status = succeeded5580lb # [ 84.813844] niks3-server[1003]: 2026/09/21 04:24:36 INFO Received uploads request method=POST path=/api/pending_closures5581worker1 # [ 84.810795] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.763Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5582worker1 # [ 84.812204] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.763Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5583lb # [ 84.838060] niks3-server[1003]: 2026/09/21 04:24:36 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5584lb # [ 84.849451] niks3-server[1003]: 2026/09/21 04:24:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5585lb # [ 84.854559] niks3-server[1003]: 2026/09/21 04:24:36 INFO Signed narinfos id=29 count=15586worker1 # [ 84.846674] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.799Z level=INFO msg="Uploading 1 narinfos"5587lb # [ 84.860805] niks3-server[1003]: 2026/09/21 04:24:36 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5588lb # [ 84.871405] niks3-server[1003]: 2026/09/21 04:24:36 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5589worker1 # [ 84.867151] nix-grpc-daemon[1315]: time=2026-09-21T04:24:36.820Z level=INFO msg="Upload complete. (513ms)"5590lb # [ 84.878206] niks3-server[1003]: 2026/09/21 04:24:36 INFO Completed upload id=295591worker1 # [ 84.869216] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60216 duration_s=0 paths=1 nar_bytes_in=6405592lb # [ 84.881866] niks3-server[1003]: 2026/09/21 04:24:36 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5593worker1 # [ 84.875600] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602165594worker1 # [ 84.882125] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5595worker1 # [ 84.884593] nix-daemon[846]: reaped child process 1363, status = succeeded5596worker1 # [ 84.890240] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5597client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5598client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5599client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5600client # farm-b> LOG-farm-b-on-standby5601worker1 # [ 86.106138] nix-daemon[846]: reaped child process 1377, status = succeeded5602worker1 # [ 86.295907] nix-daemon[846]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5603worker1 # [ 86.309093] nix-daemon[846]: reaped child process 1396, status = succeeded5604lb # [ 86.342364] niks3-server[1003]: 2026/09/21 04:24:38 INFO Received uploads request method=POST path=/api/pending_closures5605worker1 # [ 86.340343] nix-grpc-daemon[1315]: time=2026-09-21T04:24:38.293Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5606worker1 # [ 86.341872] nix-grpc-daemon[1315]: time=2026-09-21T04:24:38.295Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5607lb # [ 86.378641] niks3-server[1003]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5608lb # [ 86.388265] niks3-server[1003]: 2026/09/21 04:24:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5609lb # [ 86.393232] niks3-server[1003]: 2026/09/21 04:24:38 INFO Signed narinfos id=30 count=15610worker1 # [ 86.385618] nix-grpc-daemon[1315]: time=2026-09-21T04:24:38.338Z level=INFO msg="Uploading 1 narinfos"5611lb # [ 86.400582] niks3-server[1003]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5612lb # [ 86.406533] niks3-server[1003]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5613lb # [ 86.414962] niks3-server[1003]: 2026/09/21 04:24:38 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5614lb # [ 86.420505] niks3-server[1003]: 2026/09/21 04:24:38 INFO Completed upload id=305615lb # [ 86.424554] niks3-server[1003]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5616worker1 # [ 86.409750] nix-grpc-daemon[1315]: time=2026-09-21T04:24:38.363Z level=INFO msg="Upload complete. (311ms)"5617worker1 # [ 86.416959] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60216 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15618worker1 # [ 86.428369] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:38Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05619worker1 # [ 86.432402] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:38Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15620worker1 # [ 86.440862] nix-daemon[846]: reaped child process 1375, status = succeeded5621worker1 # [ 86.472474] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602165622worker1 # [ 86.484423] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602165623worker1 # [ 86.496953] nix-grpc-daemon[1315]: time=2026-09-21T04:24:38.450Z level=INFO msg="All 2 paths already cached"5624client # copying 1 paths to 'lb:50051'...5625client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5626worker1 # [ 86.505578] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602165627worker1 # [ 86.513961] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5628worker1 # [ 86.729838] nix-daemon[846]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)5629worker1 # [ 86.743366] nix-daemon[846]: reaped child process 1405, status = succeeded5630worker1 # [ 86.962454] nix-daemon[846]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5631worker1 # [ 86.976753] nix-daemon[846]: reaped child process 1410, status = succeeded5632lb # [ 87.009165] niks3-server[1003]: 2026/09/21 04:24:38 INFO Received uploads request method=POST path=/api/pending_closures5633worker1 # [ 87.007802] nix-grpc-daemon[1315]: time=2026-09-21T04:24:38.961Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5634worker1 # [ 87.009743] nix-grpc-daemon[1315]: time=2026-09-21T04:24:38.962Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5635lb # [ 87.037164] niks3-server[1003]: 2026/09/21 04:24:38 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5636lb # [ 87.048515] niks3-server[1003]: 2026/09/21 04:24:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5637lb # [ 87.053777] niks3-server[1003]: 2026/09/21 04:24:39 INFO Signed narinfos id=31 count=15638worker1 # [ 87.045983] nix-grpc-daemon[1315]: time=2026-09-21T04:24:38.999Z level=INFO msg="Uploading 1 narinfos"5639lb # [ 87.060478] niks3-server[1003]: 2026/09/21 04:24:39 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5640lb # [ 87.071303] niks3-server[1003]: 2026/09/21 04:24:39 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5641lb # [ 87.077564] niks3-server[1003]: 2026/09/21 04:24:39 INFO Completed upload id=315642worker1 # [ 87.066832] nix-grpc-daemon[1315]: time=2026-09-21T04:24:39.020Z level=INFO msg="Upload complete. (540ms)"5643lb # [ 87.080488] niks3-server[1003]: 2026/09/21 04:24:39 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5644worker1 # [ 87.070422] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60216 duration_s=0 paths=1 nar_bytes_in=7685645worker1 # [ 87.075938] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602165646worker1 # [ 87.085951] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5647worker1 # [ 87.090981] nix-daemon[846]: reaped child process 1400, status = succeeded5648worker1 # [ 87.096236] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5649client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5650client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5651client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5652lb # [ 87.872550] envoy[910]: conn peer=[2001:db8:1::4]:54866 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4478 tx=349 ms=72175653client # farm-top> LOG-farm-top-on-standby5654worker1 # [ 88.121861] nix-daemon[846]: reaped child process 1414, status = succeeded5655worker1 # [ 88.329656] nix-daemon[846]: accepted connection from pid 1430, user nix-grpc-daemon (trusted)5656worker1 # [ 88.343559] nix-daemon[846]: reaped child process 1433, status = succeeded5657lb # [ 88.377499] niks3-server[1003]: 2026/09/21 04:24:40 INFO Received uploads request method=POST path=/api/pending_closures5658worker1 # [ 88.376515] nix-grpc-daemon[1315]: time=2026-09-21T04:24:40.329Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5659worker1 # [ 88.379736] nix-grpc-daemon[1315]: time=2026-09-21T04:24:40.331Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5660lb # [ 88.409746] niks3-server[1003]: 2026/09/21 04:24:40 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5661lb # [ 88.424613] niks3-server[1003]: 2026/09/21 04:24:40 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5662lb # [ 88.427707] niks3-server[1003]: 2026/09/21 04:24:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5663lb # [ 88.432829] niks3-server[1003]: 2026/09/21 04:24:40 INFO Signed narinfos id=32 count=15664worker1 # [ 88.424760] nix-grpc-daemon[1315]: time=2026-09-21T04:24:40.377Z level=INFO msg="Uploading 1 narinfos"5665lb # [ 88.438931] niks3-server[1003]: 2026/09/21 04:24:40 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5666lb # [ 88.444301] niks3-server[1003]: 2026/09/21 04:24:40 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5667lb # [ 88.447407] niks3-server[1003]: 2026/09/21 04:24:40 INFO Completed upload id=325668lb # [ 88.449601] niks3-server[1003]: 2026/09/21 04:24:40 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5669worker1 # [ 88.438048] nix-grpc-daemon[1315]: time=2026-09-21T04:24:40.391Z level=INFO msg="Upload complete. (332ms)"5670worker1 # [ 88.445306] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60216 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15671worker1 # [ 88.456086] nix-daemon[846]: reaped child process 1412, status = succeeded5672client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.46 seconds)5673client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5674client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5675worker2 # [ 88.776790] systemd[1]: Listening on Nix worker-protocol over gRPC.5676worker2 # [ 88.783907] systemd[1]: Starting scheduler-token.service...5677worker2 # [ 88.861513] systemd[1]: scheduler-token.service: Deactivated successfully.5678worker2 # [ 88.862649] systemd[1]: Finished scheduler-token.service.5679worker2 # [ 88.864209] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5680worker2 # [ 88.869408] systemd[1]: Starting Nix worker-protocol over gRPC...5681worker2 # [ 89.050555] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5682worker2 # [ 89.052355] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5683worker2 # [ 89.055879] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15684worker2 # [ 89.063059] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5685worker2 # [ 89.066174] systemd[1]: Started Nix worker-protocol over gRPC.5686lb: must succeed: curl -sf localhost:9901/clusters5687worker2 # [ 89.094380] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:41Z level=info event=scheduler_connected addr=lb:500515688worker1 # [ 89.112206] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=info event=oidc_jwks provider=mock keys=15689worker1 # [ 89.116364] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:602145690worker1 # [ 89.120273] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05691lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5692worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5693worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5694lb: must succeed: curl -sf localhost:9901/clusters5695lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5696lb: must succeed: curl -sf localhost:9901/clusters5697lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5698worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5699worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5700(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.11 seconds)5701subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5702client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5703client # [ 89.487667] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5704client # Running as unit: rs.service; invocation ID: 49760de0761e42538fcfc571af9b1cc55705client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5706client # [ 89.734028] nix[1188]: don't know how to build these paths:5707client # [ 89.735431] nix[1188]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5708worker1 # [ 89.745399] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:602165709worker1 # [ 89.756536] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05710worker1 # [ 89.761730] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15711worker1 # [ 89.800830] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602165712worker1 # [ 89.813156] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602165713client # [ 89.811372] nix[1188]: copying 1 paths to 'lb:50051'...5714worker1 # [ 89.824462] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602165715client # [ 89.813289] nix[1188]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5716worker1 # [ 89.837612] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5717worker1 # [ 90.065668] nix-daemon[846]: accepted connection from pid 1446, user nix-grpc-daemon (trusted)5718worker1 # [ 90.078831] nix-daemon[846]: reaped child process 1449, status = succeeded5719worker1 # [ 90.291128] nix-daemon[846]: accepted connection from pid 1451, user nix-grpc-daemon (trusted)5720worker1 # [ 90.302721] nix-daemon[846]: reaped child process 1454, status = succeeded5721lb # [ 90.335457] niks3-server[1003]: 2026/09/21 04:24:42 INFO Received uploads request method=POST path=/api/pending_closures5722worker1 # [ 90.332848] nix-grpc-daemon[1315]: time=2026-09-21T04:24:42.285Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5723worker1 # [ 90.334354] nix-grpc-daemon[1315]: time=2026-09-21T04:24:42.287Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5724lb # [ 90.363767] niks3-server[1003]: 2026/09/21 04:24:42 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5725lb # [ 90.375117] niks3-server[1003]: 2026/09/21 04:24:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5726lb # [ 90.379946] niks3-server[1003]: 2026/09/21 04:24:42 INFO Signed narinfos id=33 count=15727worker1 # [ 90.372220] nix-grpc-daemon[1315]: time=2026-09-21T04:24:42.325Z level=INFO msg="Uploading 1 narinfos"5728lb # [ 90.386632] niks3-server[1003]: 2026/09/21 04:24:42 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5729lb # [ 90.397958] niks3-server[1003]: 2026/09/21 04:24:42 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5730worker1 # [ 90.393336] nix-grpc-daemon[1315]: time=2026-09-21T04:24:42.346Z level=INFO msg="Upload complete. (544ms)"5731lb # [ 90.404288] niks3-server[1003]: 2026/09/21 04:24:42 INFO Completed upload id=335732worker1 # [ 90.395424] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60216 duration_s=0 paths=1 nar_bytes_in=4165733lb # [ 90.407932] niks3-server[1003]: 2026/09/21 04:24:42 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5734worker1 # [ 90.401453] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602165735worker1 # [ 90.408412] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5736worker1 # [ 90.409755] nix-daemon[846]: reaped child process 1444, status = succeeded5737worker1 # [ 90.415537] nix-daemon[846]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5738client # [ 90.402513] nix[1188]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5739lb: must succeed: curl -sf localhost:9901/clusters5740lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5741lb: must succeed: curl -sf localhost:9901/clusters5742lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5743lb: must succeed: curl -sf localhost:9901/clusters5744lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5745lb: must succeed: curl -sf localhost:9901/clusters5746lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5747worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5748worker2 # [ 92.540379] systemd-journald[287]: Received client request to rotate journal, rotating.5749worker2 # [ 92.560639] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e5540d06e36e41ceaaee7204253a4a9c.5750worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds)5751worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5752worker1 # [ 92.477908] systemd[1]: Stopping Nix worker-protocol over gRPC...5753worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5754client: must succeed: sleep 5.05755worker1 # [ 93.152840] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:45Z level=info event=draining5756client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5757worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5758worker1 # [ 97.837110] nix-daemon[846]: reaped child process 1458, status = succeeded5759lb # [ 97.873667] envoy[910]: conn peer=[2001:db8:1::1]:44924 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9663 tx=4138 ms=71695760worker1 # [ 98.040606] nix-daemon[846]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5761worker1 # [ 98.054253] nix-daemon[846]: reaped child process 1502, status = succeeded5762lb # [ 98.087789] niks3-server[1003]: 2026/09/21 04:24:50 INFO Received uploads request method=POST path=/api/pending_closures5763worker1 # [ 98.088516] nix-grpc-daemon[1315]: time=2026-09-21T04:24:50.041Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5764worker1 # [ 98.092425] nix-grpc-daemon[1315]: time=2026-09-21T04:24:50.043Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5765lb # [ 98.131292] niks3-server[1003]: 2026/09/21 04:24:50 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5766lb # [ 98.140583] niks3-server[1003]: 2026/09/21 04:24:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5767lb # [ 98.145345] niks3-server[1003]: 2026/09/21 04:24:50 INFO Signed narinfos id=34 count=15768worker1 # [ 98.138285] nix-grpc-daemon[1315]: time=2026-09-21T04:24:50.091Z level=INFO msg="Uploading 1 narinfos"5769lb # [ 98.156784] niks3-server[1003]: 2026/09/21 04:24:50 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5770lb # [ 98.161473] niks3-server[1003]: 2026/09/21 04:24:50 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5771lb # [ 98.169569] niks3-server[1003]: 2026/09/21 04:24:50 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5772lb # [ 98.174520] niks3-server[1003]: 2026/09/21 04:24:50 INFO Completed upload id=345773lb # [ 98.177451] niks3-server[1003]: 2026/09/21 04:24:50 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5774worker1 # [ 98.165361] nix-grpc-daemon[1315]: time=2026-09-21T04:24:50.118Z level=INFO msg="Upload complete. (336ms)"5775worker1 # [ 98.173587] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60216 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15776worker1 # [ 98.187367] nix-daemon[846]: reaped child process 1456, status = succeeded5777client # [ 98.202798] systemd[1]: rs.service: Deactivated successfully.5778client # [ 98.205717] systemd[1]: rs.service: Consumed 124ms CPU time over 8.717s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.5779worker1 # [ 99.156146] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=signal_exit5780worker1 # [ 99.378871] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=scheduler_yield5781lb # [ 99.390708] niks3-server[1003]: 2026/09/21 04:24:51 INFO lead: released remote=192.168.1.3:383265782worker1 # [ 99.384194] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=scheduler_restarting addr=lb:500515783worker2 # [ 99.377623] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:51Z level=info event=scheduler_restarting addr=lb:500515784worker1 # [ 99.389077] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:602145785worker1 # [ 99.392397] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=worker_gone addr=192.168.1.3:500515786worker1 # [ 99.395321] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:602145787worker1 # [ 99.398520] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=worker_gone addr=192.168.1.4:500515788worker1 # [ 99.585403] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=scheduler_connected addr=lb:500515789worker2 # [ 99.578115] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:51Z level=info event=scheduler_connected addr=lb:500515790worker1 # [ 99.591518] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5791worker2 # [ 99.583977] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5792worker1 # [ 99.792787] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=scheduler_connected addr=lb:500515793worker2 # [ 99.785175] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:51Z level=info event=scheduler_connected addr=lb:500515794worker2 # [ 99.789877] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5795worker1 # [ 99.798821] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5796worker2 # [ 99.990905] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:51Z level=info event=scheduler_connected addr=lb:500515797worker1 # [ 100.000082] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=scheduler_connected addr=lb:500515798worker2 # [ 99.994854] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5799worker1 # [ 100.004223] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5800worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.18 seconds)5801client: waiting for success: ! systemctl is-active rs5802client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5803client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5804client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5805client: must fail: journalctl -u rs | grep -q 'reconnecting'5806client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5807lb: must succeed: curl -sf localhost:9901/clusters5808worker2 # [ 100.195203] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:52Z level=info event=scheduler_connected addr=lb:500515809worker1 # [ 100.205465] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:52Z level=info event=scheduler_connected addr=lb:500515810worker2 # [ 100.199189] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5811worker1 # [ 100.209623] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5812lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5813worker1 # [ 100.387458] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:52Z level=info event=scheduler_restarting5814worker2 # [ 100.401485] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:52Z level=info event=scheduler_connected addr=lb:500515815worker2 # [ 100.404826] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5816worker1 # [ 100.423778] nix-daemon[846]: reaped child process 1305, status = succeeded5817worker1 # [ 100.426310] nix-grpc-daemon[1235]: ts=2026-09-21T04:24:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5818worker2 # [ 100.602930] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:52Z level=info event=scheduler_connected addr=lb:500515819worker2 # [ 100.607206] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker2 # [ 100.808329] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:52Z level=info event=scheduler_connected addr=lb:500515821worker2 # [ 100.812524] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822worker1 # [ 100.959098] nix-grpc-daemon[1235]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5823worker1 # [ 100.962741] nix-grpc-daemon[1235]: E0000 00:00:1789964692.912181 1278 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5824worker1 # [ 100.967410] nix-grpc-daemon[1235]: E0000 00:00:1789964692.912362 1278 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5825worker2 # [ 101.013636] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_connected addr=lb:500515826worker2 # [ 101.017528] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5827lb: must succeed: curl -sf localhost:9901/clusters5828worker2 # [ 101.218480] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_connected addr=lb:500515829worker2 # [ 101.222254] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5830worker1 # [ 101.291608] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5831worker1 # [ 101.295851] systemd[1]: Stopped Nix worker-protocol over gRPC.5832worker1 # [ 101.299137] systemd[1]: nix-grpc-daemon.service: Consumed 1.357s CPU time over 26.355s wall clock time, 35.9M memory peak, 48.7K incoming IP traffic, 95.9K outgoing IP traffic.5833lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5834worker1 # [ 101.313667] systemd[1]: Starting Nix worker-protocol over gRPC...5835worker2 # [ 101.424434] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_connected addr=lb:500515836worker2 # [ 101.427537] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5837worker1 # [ 101.535917] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5838worker1 # [ 101.538575] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5839worker1 # [ 101.543161] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15840worker1 # [ 101.550057] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5841worker1 # [ 101.553309] systemd[1]: Started Nix worker-protocol over gRPC.5842worker1 # [ 101.560138] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=scheduler_take_over5843lb # [ 101.570703] niks3-server[1003]: 2026/09/21 04:24:53 INFO lead: acquired remote=192.168.1.3:448125844worker1 # [ 101.579139] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=scheduler_connected addr=lb:500515845worker1 # [ 101.580700] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5846worker2 # [ 101.625803] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_connected addr=lb:500515847worker2 # [ 101.629633] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5848worker1 # [ 101.781880] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=scheduler_connected addr=lb:500515849worker1 # [ 101.786164] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5850worker2 # [ 101.830721] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_connected addr=lb:500515851worker2 # [ 101.834546] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5852worker1 # [ 101.987410] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=scheduler_connected addr=lb:500515853worker1 # [ 101.991627] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5854worker2 # [ 102.035647] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:54Z level=info event=scheduler_connected addr=lb:500515855worker2 # [ 102.039462] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5856worker1 # [ 102.192605] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=info event=scheduler_connected addr=lb:500515857worker1 # [ 102.196750] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5858worker2 # [ 102.240668] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:54Z level=info event=scheduler_connected addr=lb:500515859worker2 # [ 102.244566] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5860lb: must succeed: curl -sf localhost:9901/clusters5861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5862worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5863worker1 # [ 102.397783] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=info event=scheduler_connected addr=lb:500515864worker1 # [ 102.405972] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:356085865worker1 # [ 102.410202] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05866worker2 # [ 102.446447] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:54Z level=info event=scheduler_connected addr=lb:500515867worker1 # [ 102.460269] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=info event=oidc_jwks provider=mock keys=15868worker1 # [ 102.463607] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:356085869worker1 # [ 102.467385] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05870worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)5871(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.04 seconds)5872subtest: scheduler restart mid-build: build finishes, no double build5873client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5874client # [ 102.531748] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5875client # Running as unit: mid.service; invocation ID: 1a43888911dd4eb3b7cbf67f3dc545805876client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5877worker1 # [ 102.798510] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415845878worker1 # [ 102.810776] nix-daemon[846]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)5879client # [ 102.810776] nix[1224]: don't know how to build these paths:5880client # [ 102.812063] nix[1224]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5881worker1 # [ 102.831772] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05882worker1 # [ 102.836501] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15883worker1 # [ 102.881553] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415845884worker1 # [ 102.894704] nix-grpc-daemon[1509]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)5885worker1 # [ 102.928705] nix-grpc-daemon[1509]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 2/5)5886worker1 # [ 103.008643] nix-grpc-daemon[1509]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 3/5)5887worker1 # [ 103.107987] nix-grpc-daemon[1509]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 4/5)5888worker1 # [ 103.131634] nix-grpc-daemon[1509]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5889worker1 # [ 103.134426] nix-grpc-daemon[1509]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5890worker1 # [ 103.154105] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415845891client # [ 103.152165] nix[1224]: copying 1 paths to 'lb:50051'...5892worker1 # [ 103.165642] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415845893client # [ 103.154202] nix[1224]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5894worker1 # [ 103.175772] nix-daemon[846]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)5895worker1 # [ 103.199875] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:55Z level=info event=niks3_push_started pid=15755896worker1 # [ 103.457323] nix-grpc-daemon[1575]: time=2026-09-21T04:24:55.410Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5897worker1 # [ 103.684103] nix-daemon[846]: accepted connection from pid 1588, user nix-grpc-daemon (trusted)5898worker1 # [ 103.695980] nix-daemon[846]: reaped child process 1591, status = succeeded5899worker1 # [ 103.894157] nix-daemon[846]: accepted connection from pid 1597, user nix-grpc-daemon (trusted)5900worker1 # [ 103.906820] nix-daemon[846]: reaped child process 1600, status = succeeded5901lb # [ 103.936565] niks3-server[1003]: 2026/09/21 04:24:55 INFO Received uploads request method=POST path=/api/pending_closures5902worker1 # [ 103.935116] nix-grpc-daemon[1575]: time=2026-09-21T04:24:55.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5903worker1 # [ 103.936726] nix-grpc-daemon[1575]: time=2026-09-21T04:24:55.890Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5904lb # [ 103.969927] niks3-server[1003]: 2026/09/21 04:24:55 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5905lb # [ 103.982690] niks3-server[1003]: 2026/09/21 04:24:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5906worker1 # [ 103.981089] nix-grpc-daemon[1575]: time=2026-09-21T04:24:55.934Z level=INFO msg="Uploading 1 narinfos"5907lb # [ 103.992171] niks3-server[1003]: 2026/09/21 04:24:55 INFO Signed narinfos id=35 count=15908lb # [ 103.998970] niks3-server[1003]: 2026/09/21 04:24:55 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5909lb # [ 104.010295] niks3-server[1003]: 2026/09/21 04:24:55 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5910lb # [ 104.017577] niks3-server[1003]: 2026/09/21 04:24:55 INFO Completed upload id=355911worker1 # [ 104.006388] nix-grpc-daemon[1575]: time=2026-09-21T04:24:55.959Z level=INFO msg="Upload complete. (544ms)"5912worker1 # [ 104.009925] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41584 duration_s=0 paths=1 nar_bytes_in=4245913lb # [ 104.026651] niks3-server[1003]: 2026/09/21 04:24:55 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5914worker1 # [ 104.015401] nix-grpc-daemon[1509]: ts=2026-09-21T04:24:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415845915worker1 # [ 104.024715] nix-daemon[846]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)5916worker1 # [ 104.031128] nix-daemon[846]: reaped child process 1573, status = succeeded5917worker1 # [ 104.036100] nix-daemon[846]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)5918client # [ 104.023760] nix[1224]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5919lb: must succeed: curl -sf localhost:9901/clusters5920lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5921lb: must succeed: curl -sf localhost:9901/clusters5922lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5923worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5924worker1 # [ 106.462475] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1509 (nix-grpc-daemon) on client request.5925worker1 # [ 106.467294] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5926lb # [ 106.481836] niks3-server[1003]: 2026/09/21 04:24:58 INFO lead: released remote=192.168.1.3:448125927worker2 # [ 106.474344] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:58Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5928client # [ 106.471289] nix[1224]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5929worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5930client # [ 106.476122] nix[1224]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5931client: must succeed: sleep 5.05932worker1 # [ 106.497749] nix-daemon[846]: reaped child process 1602, status = succeeded5933worker1 # [ 106.498820] nix-daemon[846]: reaped child process 1565, status = succeeded5934worker1 # [ 106.507209] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5935worker1 # [ 106.508838] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5936worker1 # [ 106.510687] systemd[1]: nix-grpc-daemon.service: Consumed 391ms CPU time over 5.192s wall clock time, 33.3M memory peak, 14.3K incoming IP traffic, 20.6K outgoing IP traffic.5937worker1 # [ 106.515256] nix-daemon[1604]: unexpected Nix daemon error: error: interrupted by the user5938worker1 # [ 106.519988] nix-daemon[846]: reaped child process 1604, status = failed with exit code 15939worker2 # [ 106.674868] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:58Z level=info event=scheduler_connected addr=lb:500515940worker1 # [ 106.686895] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5941worker1 # [ 106.697589] systemd[1]: Starting Nix worker-protocol over gRPC...5942worker1 # [ 106.939260] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5943worker1 # [ 106.941527] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5944worker1 # [ 106.945538] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15945worker1 # [ 106.953058] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5946worker1 # [ 106.957138] systemd[1]: Started Nix worker-protocol over gRPC.5947lb # [ 106.976576] niks3-server[1003]: 2026/09/21 04:24:58 INFO lead: acquired remote=192.168.1.3:401105948worker2 # [ 106.956701] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5949worker1 # [ 106.968989] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=info event=scheduler_take_over5950worker1 # [ 106.984365] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=info event=scheduler_connected addr=lb:500515951worker1 # [ 106.986172] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:415885952worker1 # [ 106.987616] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05953worker1 # [ 106.995026] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05954worker1 # [ 106.997040] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15955worker1 # [ 107.044295] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416025956worker1 # [ 107.048382] nix-daemon[846]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)5957worker1 # [ 107.056162] nix-daemon[846]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)5958worker1 # [ 107.063112] nix-daemon[846]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)5959client # [ 107.050173] nix[1224]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5960worker2 # [ 107.157938] nix-grpc-daemon[1214]: ts=2026-09-21T04:24:59Z level=info event=scheduler_connected addr=lb:500515961worker1 # [ 107.173809] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:59Z level=info event=oidc_jwks provider=mock keys=15962worker1 # [ 107.178312] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:415885963worker1 # [ 107.182386] nix-grpc-daemon[1641]: ts=2026-09-21T04:24:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05964lb # [ 107.874801] envoy[910]: conn peer=[2001:db8:1::1]:56814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1403 ms=84385965lb # [ 107.878575] envoy[910]: conn peer=[2001:db8:1::3]:54170 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=6165 tx=1405 ms=252395966lb # [ 107.882252] envoy[910]: conn peer=[2001:db8:1::3]:49068 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1335 tx=411 ms=49065967client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5968client: waiting for success: ! systemctl is-active mid5969worker1 # [ 111.883121] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:03Z level=info event=niks3_push_started pid=17215970worker1 # [ 111.909635] nix-daemon[846]: reaped child process 1696, status = succeeded5971worker1 # [ 112.138648] nix-grpc-daemon[1721]: time=2026-09-21T04:25:04.091Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5972worker1 # [ 112.352646] nix-daemon[846]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)5973worker1 # [ 112.366429] nix-daemon[846]: reaped child process 1737, status = succeeded5974lb # [ 112.401595] niks3-server[1003]: 2026/09/21 04:25:04 INFO Received uploads request method=POST path=/api/pending_closures5975worker1 # [ 112.398914] nix-grpc-daemon[1721]: time=2026-09-21T04:25:04.352Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5976worker1 # [ 112.400430] nix-grpc-daemon[1721]: time=2026-09-21T04:25:04.353Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5977lb # [ 112.446626] niks3-server[1003]: 2026/09/21 04:25:04 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5978lb # [ 112.457664] niks3-server[1003]: 2026/09/21 04:25:04 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5979lb # [ 112.466400] niks3-server[1003]: 2026/09/21 04:25:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5980lb # [ 112.470736] niks3-server[1003]: 2026/09/21 04:25:04 INFO Signed narinfos id=36 count=15981worker1 # [ 112.460561] nix-grpc-daemon[1721]: time=2026-09-21T04:25:04.413Z level=INFO msg="Uploading 1 narinfos"5982lb # [ 112.475516] niks3-server[1003]: 2026/09/21 04:25:04 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5983lb # [ 112.485034] niks3-server[1003]: 2026/09/21 04:25:04 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5984lb # [ 112.488137] niks3-server[1003]: 2026/09/21 04:25:04 INFO Completed upload id=365985worker1 # [ 112.477602] nix-grpc-daemon[1721]: time=2026-09-21T04:25:04.430Z level=INFO msg="Upload complete. (335ms)"5986worker1 # [ 112.484759] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41602 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15987worker1 # [ 112.489265] nix-grpc-daemon[1721]: time=2026-09-21T04:25:04.437Z level=WARN msg="Failed to register uploaded object" key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo error="server returned 404: object is not pending upload\n"5988worker1 # [ 112.500902] nix-daemon[846]: reaped child process 1694, status = succeeded5989client # [ 112.524860] systemd[1]: mid.service: Deactivated successfully.5990client # [ 112.528375] systemd[1]: mid.service: Consumed 141ms CPU time over 9.996s wall clock time, 41.4M memory peak, 8.4K incoming IP traffic, 10.6K outgoing IP traffic.5991client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5992client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5993client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5994(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.51 seconds)5995subtest: interrupting the client stops the build on the worker5996client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5997client # [ 113.017173] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5998client # Running as unit: intr.service; invocation ID: e11c734c06534b7cb9708e4e9c1a53855999client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6000worker1 # [ 113.278790] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:416026001client # [ 113.272697] nix[1260]: don't know how to build these paths:6002client # [ 113.274221] nix[1260]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6003worker1 # [ 113.296777] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:05Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06004worker1 # [ 113.303242] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:05Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16005worker2 # [ 113.341772] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406006worker2 # [ 113.352583] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6007worker2 # [ 113.369890] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)6008worker2 # [ 113.385703] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 2/5)6009worker2 # [ 113.454585] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 220 ms (attempt 3/5)6010worker2 # [ 113.680776] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 4/5)6011worker2 # [ 113.752112] nix-grpc-daemon[1214]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6012worker2 # [ 113.756077] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6013worker2 # [ 113.774280] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406014client # [ 113.780184] nix[1260]: copying 1 paths to 'lb:50051'...6015worker2 # [ 113.785656] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330406016client # [ 113.784096] nix[1260]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6017worker2 # [ 113.797153] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6018worker2 # [ 113.816966] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:05Z level=info event=niks3_push_started pid=13536019worker2 # [ 114.071094] nix-grpc-daemon[1353]: time=2026-09-21T04:25:06.059Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6020worker2 # [ 114.290197] nix-daemon[865]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)6021worker2 # [ 114.304939] nix-daemon[865]: reaped child process 1372, status = succeeded6022worker2 # [ 114.509125] nix-daemon[865]: accepted connection from pid 1375, user nix-grpc-daemon (trusted)6023worker2 # [ 114.519288] nix-daemon[865]: reaped child process 1378, status = succeeded6024lb # [ 114.561521] niks3-server[1003]: 2026/09/21 04:25:06 INFO Received uploads request method=POST path=/api/pending_closures6025worker2 # [ 114.554173] nix-grpc-daemon[1353]: time=2026-09-21T04:25:06.542Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6026worker2 # [ 114.555672] nix-grpc-daemon[1353]: time=2026-09-21T04:25:06.544Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6027lb # [ 114.604649] niks3-server[1003]: 2026/09/21 04:25:06 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6028lb # [ 114.610366] niks3-server[1003]: 2026/09/21 04:25:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6029lb # [ 114.616470] niks3-server[1003]: 2026/09/21 04:25:06 INFO Signed narinfos id=37 count=16030worker2 # [ 114.602371] nix-grpc-daemon[1353]: time=2026-09-21T04:25:06.590Z level=INFO msg="Uploading 1 narinfos"6031lb # [ 114.624356] niks3-server[1003]: 2026/09/21 04:25:06 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6032lb # [ 114.634550] niks3-server[1003]: 2026/09/21 04:25:06 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6033lb # [ 114.639619] niks3-server[1003]: 2026/09/21 04:25:06 INFO Completed upload id=376034worker2 # [ 114.620162] nix-grpc-daemon[1353]: time=2026-09-21T04:25:06.608Z level=INFO msg="Upload complete. (543ms)"6035lb # [ 114.643069] niks3-server[1003]: 2026/09/21 04:25:06 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6036worker2 # [ 114.625128] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=0 paths=1 nar_bytes_in=4246037worker2 # [ 114.630245] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406038worker2 # [ 114.641234] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6039worker2 # [ 114.642828] nix-daemon[865]: reaped child process 1351, status = succeeded6040worker2 # [ 114.648186] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6041client # [ 114.643887] nix[1260]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6042client: must succeed: systemctl kill -s INT intr6043client # [ 116.658628] systemd[1]: intr.service: Sent signal SIGINT to main process 1260 (nix) on client request.6044worker2 # [ 116.666371] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:08Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6045client # [ 116.665538] nix[1260]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6046client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6047client # [ 116.673372] nix[1260]: error: interrupted by the user6048client # [ 116.710435] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6049client # [ 116.711916] systemd[1]: intr.service: Failed with result 'exit-code'.6050client # [ 116.713805] systemd[1]: intr.service: Consumed 117ms CPU time over 3.696s wall clock time, 41.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.6051worker2 # [ 116.866159] nix-daemon[865]: reaped child process 1380, status = succeeded6052worker2 # [ 116.870262] nix-daemon[1382]: unexpected Nix daemon error: error: interrupted by the user6053worker2 # [ 116.874488] nix-daemon[865]: reaped child process 1382, status = failed with exit code 16054lb # [ 117.875916] envoy[910]: conn peer=[2001:db8:1::1]:48440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1823 ms=97106055lb # [ 117.879816] envoy[910]: conn peer=[2001:db8:1::1]:34590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=852 ms=34206056(finished: subtest: interrupting the client stops the build on the worker, in 4.98 seconds)6057subtest: a deploy mid-build drains: the build finishes, the new generation takes over6058client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6059client # [ 118.022798] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6060client # Running as unit: sw.service; invocation ID: 69abd974d29740058076aa3ba78f93e76061client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6062client # [ 118.290173] nix[1280]: don't know how to build these paths:6063client # [ 118.291585] nix[1280]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6064worker1 # [ 118.302173] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:416026065worker1 # [ 118.316255] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:10Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06066worker1 # [ 118.320130] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:10Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16067worker1 # [ 118.369242] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416026068worker1 # [ 118.382094] nix-grpc-daemon[1641]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)6069worker1 # [ 118.412060] nix-grpc-daemon[1641]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 123 ms (attempt 2/5)6070worker1 # [ 118.540944] nix-grpc-daemon[1641]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 378 ms (attempt 3/5)6071worker1 # [ 118.925147] nix-grpc-daemon[1641]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 230 ms (attempt 4/5)6072worker1 # [ 119.161513] nix-grpc-daemon[1641]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6073worker1 # [ 119.165189] nix-grpc-daemon[1641]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6074worker1 # [ 119.184437] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:416026075client # [ 119.181346] nix[1280]: copying 1 paths to 'lb:50051'...6076worker1 # [ 119.195906] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:416026077client # [ 119.185245] nix[1280]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6078worker1 # [ 119.211387] nix-daemon[846]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)6079worker1 # [ 119.435931] nix-daemon[846]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6080worker1 # [ 119.449267] nix-daemon[846]: reaped child process 1784, status = succeeded6081worker1 # [ 119.670238] nix-daemon[846]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6082worker1 # [ 119.683191] nix-daemon[846]: reaped child process 1789, status = succeeded6083lb # [ 119.717544] niks3-server[1003]: 2026/09/21 04:25:11 INFO Received uploads request method=POST path=/api/pending_closures6084worker1 # [ 119.715267] nix-grpc-daemon[1721]: time=2026-09-21T04:25:11.668Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6085worker1 # [ 119.716764] nix-grpc-daemon[1721]: time=2026-09-21T04:25:11.670Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6086lb # [ 119.750616] niks3-server[1003]: 2026/09/21 04:25:11 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6087lb # [ 119.763910] niks3-server[1003]: 2026/09/21 04:25:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6088worker1 # [ 119.758323] nix-grpc-daemon[1721]: time=2026-09-21T04:25:11.711Z level=INFO msg="Uploading 1 narinfos"6089lb # [ 119.768710] niks3-server[1003]: 2026/09/21 04:25:11 INFO Signed narinfos id=38 count=16090lb # [ 119.775112] niks3-server[1003]: 2026/09/21 04:25:11 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6091lb # [ 119.786288] niks3-server[1003]: 2026/09/21 04:25:11 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6092lb # [ 119.792574] niks3-server[1003]: 2026/09/21 04:25:11 INFO Completed upload id=386093worker1 # [ 119.781810] nix-grpc-daemon[1721]: time=2026-09-21T04:25:11.735Z level=INFO msg="Upload complete. (553ms)"6094lb # [ 119.794665] niks3-server[1003]: 2026/09/21 04:25:11 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6095worker1 # [ 119.786294] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41602 duration_s=0 paths=1 nar_bytes_in=4166096worker1 # [ 119.792291] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:416026097worker1 # [ 119.800937] nix-daemon[846]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)6098worker1 # [ 119.808309] nix-daemon[846]: reaped child process 1777, status = succeeded6099worker1 # [ 119.814543] nix-daemon[846]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)6100client # [ 119.801396] nix[1280]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6101worker1: must succeed: systemctl reload nix-grpc-daemon.service6102worker1 # [ 121.816679] systemd[1]: Reloading Nix worker-protocol over gRPC...6103worker1 # [ 121.866694] systemd[1]: Reloaded Nix worker-protocol over gRPC.6104worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds)6105worker1: must succeed: pgrep -f 'read -t [9]0 x'6106worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)6107client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26108worker1 # [ 121.960806] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:13Z level=info event=draining6109worker2 # [ 122.190827] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330406110client # don't know how to build these paths:6111client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6112worker2 # [ 122.199320] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406113worker1 # [ 122.228768] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:14Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06114worker1 # [ 122.230640] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:14Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16115worker2 # [ 122.264564] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406116worker2 # [ 122.276489] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406117client # copying 1 paths to 'lb:50051'...6118client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6119worker2 # [ 122.289490] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330406120worker2 # [ 122.304942] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6121worker2 # [ 122.532122] nix-daemon[865]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)6122worker2 # [ 122.543663] nix-daemon[865]: reaped child process 1440, status = succeeded6123worker2 # [ 122.761303] nix-daemon[865]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6124worker2 # [ 122.772670] nix-daemon[865]: reaped child process 1445, status = succeeded6125lb # [ 122.815357] niks3-server[1003]: 2026/09/21 04:25:14 INFO Received uploads request method=POST path=/api/pending_closures6126worker2 # [ 122.807572] nix-grpc-daemon[1353]: time=2026-09-21T04:25:14.795Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6127worker2 # [ 122.808962] nix-grpc-daemon[1353]: time=2026-09-21T04:25:14.795Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6128lb # [ 122.844526] niks3-server[1003]: 2026/09/21 04:25:14 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6129lb # [ 122.855922] niks3-server[1003]: 2026/09/21 04:25:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6130worker2 # [ 122.841737] nix-grpc-daemon[1353]: time=2026-09-21T04:25:14.830Z level=INFO msg="Uploading 1 narinfos"6131lb # [ 122.860590] niks3-server[1003]: 2026/09/21 04:25:14 INFO Signed narinfos id=39 count=16132lb # [ 122.868780] niks3-server[1003]: 2026/09/21 04:25:14 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6133lb # [ 122.876989] niks3-server[1003]: 2026/09/21 04:25:14 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6134worker2 # [ 122.867100] nix-grpc-daemon[1353]: time=2026-09-21T04:25:14.855Z level=INFO msg="Upload complete. (550ms)"6135lb # [ 122.886201] niks3-server[1003]: 2026/09/21 04:25:14 INFO Completed upload id=396136worker2 # [ 122.868930] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=0 paths=1 nar_bytes_in=5286137lb # [ 122.889321] niks3-server[1003]: 2026/09/21 04:25:14 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6138worker2 # [ 122.874352] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406139worker2 # [ 122.885276] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6140worker2 # [ 122.887518] nix-daemon[865]: reaped child process 1435, status = succeeded6141worker2 # [ 122.893113] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6142client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6143client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6144client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6145client # farm-a> LOG-farm-a-during-drain6146worker2 # [ 124.507325] nix-daemon[865]: reaped child process 1449, status = succeeded6147worker2 # [ 124.707279] nix-daemon[865]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)6148worker2 # [ 124.718806] nix-daemon[865]: reaped child process 1468, status = succeeded6149lb # [ 124.761410] niks3-server[1003]: 2026/09/21 04:25:16 INFO Received uploads request method=POST path=/api/pending_closures6150worker2 # [ 124.750867] nix-grpc-daemon[1353]: time=2026-09-21T04:25:16.739Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6151worker2 # [ 124.752395] nix-grpc-daemon[1353]: time=2026-09-21T04:25:16.741Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6152lb # [ 124.799160] niks3-server[1003]: 2026/09/21 04:25:16 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6153lb # [ 124.806469] niks3-server[1003]: 2026/09/21 04:25:16 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6154lb # [ 124.819874] niks3-server[1003]: 2026/09/21 04:25:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6155lb # [ 124.825698] niks3-server[1003]: 2026/09/21 04:25:16 INFO Signed narinfos id=40 count=16156worker2 # [ 124.809752] nix-grpc-daemon[1353]: time=2026-09-21T04:25:16.797Z level=INFO msg="Uploading 1 narinfos"6157lb # [ 124.828809] niks3-server[1003]: 2026/09/21 04:25:16 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6158lb # [ 124.839599] niks3-server[1003]: 2026/09/21 04:25:16 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6159lb # [ 124.845833] niks3-server[1003]: 2026/09/21 04:25:16 INFO Completed upload id=406160worker2 # [ 124.827087] nix-grpc-daemon[1353]: time=2026-09-21T04:25:16.815Z level=INFO msg="Upload complete. (326ms)"6161lb # [ 124.848277] niks3-server[1003]: 2026/09/21 04:25:16 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6162worker2 # [ 124.831611] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16163worker1 # [ 124.845658] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:16Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06164worker1 # [ 124.848329] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:16Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16165worker2 # [ 124.845141] nix-daemon[865]: reaped child process 1447, status = succeeded6166worker2 # [ 124.884425] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406167worker2 # [ 124.902591] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406168worker2 # [ 124.915412] nix-grpc-daemon[1353]: time=2026-09-21T04:25:16.904Z level=INFO msg="All 1 paths already cached"6169client # copying 1 paths to 'lb:50051'...6170worker2 # [ 124.923524] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330406171client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6172worker2 # [ 124.943230] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6173worker2 # [ 125.167416] nix-daemon[865]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)6174worker2 # [ 125.179222] nix-daemon[865]: reaped child process 1475, status = succeeded6175worker2 # [ 125.397700] nix-daemon[865]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6176worker2 # [ 125.409118] nix-daemon[865]: reaped child process 1480, status = succeeded6177lb # [ 125.451735] niks3-server[1003]: 2026/09/21 04:25:17 INFO Received uploads request method=POST path=/api/pending_closures6178worker2 # [ 125.440950] nix-grpc-daemon[1353]: time=2026-09-21T04:25:17.429Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6179worker2 # [ 125.442768] nix-grpc-daemon[1353]: time=2026-09-21T04:25:17.431Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6180lb # [ 125.479746] niks3-server[1003]: 2026/09/21 04:25:17 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6181lb # [ 125.490663] niks3-server[1003]: 2026/09/21 04:25:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6182lb # [ 125.494800] niks3-server[1003]: 2026/09/21 04:25:17 INFO Signed narinfos id=41 count=16183worker2 # [ 125.478999] nix-grpc-daemon[1353]: time=2026-09-21T04:25:17.467Z level=INFO msg="Uploading 1 narinfos"6184lb # [ 125.507143] niks3-server[1003]: 2026/09/21 04:25:17 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6185lb # [ 125.513466] niks3-server[1003]: 2026/09/21 04:25:17 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6186lb # [ 125.519737] niks3-server[1003]: 2026/09/21 04:25:17 INFO Completed upload id=416187worker2 # [ 125.500773] nix-grpc-daemon[1353]: time=2026-09-21T04:25:17.489Z level=INFO msg="Upload complete. (545ms)"6188lb # [ 125.523521] niks3-server[1003]: 2026/09/21 04:25:17 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6189worker2 # [ 125.503752] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=0 paths=1 nar_bytes_in=6406190worker2 # [ 125.508915] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406191worker2 # [ 125.520855] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6192worker2 # [ 125.522943] nix-daemon[865]: reaped child process 1470, status = succeeded6193worker2 # [ 125.528203] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6194client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6195client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6196client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6197client # farm-b> LOG-farm-b-during-drain6198worker2 # [ 126.146944] nix-daemon[865]: reaped child process 1484, status = succeeded6199worker2 # [ 126.348167] nix-daemon[865]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6200worker2 # [ 126.359664] nix-daemon[865]: reaped child process 1503, status = succeeded6201lb # [ 126.401626] niks3-server[1003]: 2026/09/21 04:25:18 INFO Received uploads request method=POST path=/api/pending_closures6202worker2 # [ 126.392707] nix-grpc-daemon[1353]: time=2026-09-21T04:25:18.380Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6203worker2 # [ 126.394187] nix-grpc-daemon[1353]: time=2026-09-21T04:25:18.380Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6204lb # [ 126.435762] niks3-server[1003]: 2026/09/21 04:25:18 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6205lb # [ 126.442691] niks3-server[1003]: 2026/09/21 04:25:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6206worker2 # [ 126.428526] nix-grpc-daemon[1353]: time=2026-09-21T04:25:18.417Z level=INFO msg="Uploading 1 narinfos"6207lb # [ 126.447343] niks3-server[1003]: 2026/09/21 04:25:18 INFO Signed narinfos id=42 count=16208lb # [ 126.456609] niks3-server[1003]: 2026/09/21 04:25:18 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6209lb # [ 126.463068] niks3-server[1003]: 2026/09/21 04:25:18 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6210lb # [ 126.466943] niks3-server[1003]: 2026/09/21 04:25:18 INFO Completed upload id=426211worker2 # [ 126.449147] nix-grpc-daemon[1353]: time=2026-09-21T04:25:18.437Z level=INFO msg="Upload complete. (313ms)"6212lb # [ 126.470547] niks3-server[1003]: 2026/09/21 04:25:18 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6213lb # [ 126.473060] niks3-server[1003]: 2026/09/21 04:25:18 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6214worker2 # [ 126.454558] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16215worker1 # [ 126.467748] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:18Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06216worker2 # [ 126.460681] nix-daemon[865]: reaped child process 1482, status = succeeded6217worker1 # [ 126.469768] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:18Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16218worker2 # [ 126.505773] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406219worker2 # [ 126.518306] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406220worker2 # [ 126.530963] nix-grpc-daemon[1353]: time=2026-09-21T04:25:18.519Z level=INFO msg="All 2 paths already cached"6221client # copying 1 paths to 'lb:50051'...6222client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6223worker2 # [ 126.539123] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330406224worker2 # [ 126.547817] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6225worker2 # [ 126.761404] nix-daemon[865]: accepted connection from pid 1507, user nix-grpc-daemon (trusted)6226worker2 # [ 126.773176] nix-daemon[865]: reaped child process 1510, status = succeeded6227worker2 # [ 126.987887] nix-daemon[865]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6228worker2 # [ 126.999916] nix-daemon[865]: reaped child process 1515, status = succeeded6229lb # [ 127.041805] niks3-server[1003]: 2026/09/21 04:25:18 INFO Received uploads request method=POST path=/api/pending_closures6230worker2 # [ 127.030726] nix-grpc-daemon[1353]: time=2026-09-21T04:25:19.019Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6231worker2 # [ 127.032294] nix-grpc-daemon[1353]: time=2026-09-21T04:25:19.020Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6232lb # [ 127.069082] niks3-server[1003]: 2026/09/21 04:25:19 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6233lb # [ 127.081931] niks3-server[1003]: 2026/09/21 04:25:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6234lb # [ 127.088943] niks3-server[1003]: 2026/09/21 04:25:19 INFO Signed narinfos id=43 count=16235worker2 # [ 127.073393] nix-grpc-daemon[1353]: time=2026-09-21T04:25:19.061Z level=INFO msg="Uploading 1 narinfos"6236lb # [ 127.092476] niks3-server[1003]: 2026/09/21 04:25:19 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6237lb # [ 127.105072] niks3-server[1003]: 2026/09/21 04:25:19 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6238lb # [ 127.110469] niks3-server[1003]: 2026/09/21 04:25:19 INFO Completed upload id=436239worker2 # [ 127.091603] nix-grpc-daemon[1353]: time=2026-09-21T04:25:19.080Z level=INFO msg="Upload complete. (531ms)"6240lb # [ 127.115620] niks3-server[1003]: 2026/09/21 04:25:19 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6241worker2 # [ 127.096122] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=0 paths=1 nar_bytes_in=7686242worker2 # [ 127.100788] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406243worker2 # [ 127.108866] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6244worker2 # [ 127.113693] nix-daemon[865]: reaped child process 1505, status = succeeded6245client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6246worker2 # [ 127.119664] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6247client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6248client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6249client # farm-top> LOG-farm-top-during-drain6250worker2 # [ 128.139289] nix-daemon[865]: reaped child process 1519, status = succeeded6251worker2 # [ 128.345401] nix-daemon[865]: accepted connection from pid 1535, user nix-grpc-daemon (trusted)6252worker2 # [ 128.357179] nix-daemon[865]: reaped child process 1538, status = succeeded6253lb # [ 128.400762] niks3-server[1003]: 2026/09/21 04:25:20 INFO Received uploads request method=POST path=/api/pending_closures6254worker2 # [ 128.390928] nix-grpc-daemon[1353]: time=2026-09-21T04:25:20.379Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6255worker2 # [ 128.393143] nix-grpc-daemon[1353]: time=2026-09-21T04:25:20.381Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6256lb # [ 128.431784] niks3-server[1003]: 2026/09/21 04:25:20 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6257lb # [ 128.446207] niks3-server[1003]: 2026/09/21 04:25:20 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6258lb # [ 128.457075] niks3-server[1003]: 2026/09/21 04:25:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6259worker2 # [ 128.442716] nix-grpc-daemon[1353]: time=2026-09-21T04:25:20.431Z level=INFO msg="Uploading 1 narinfos"6260lb # [ 128.461923] niks3-server[1003]: 2026/09/21 04:25:20 INFO Signed narinfos id=44 count=16261lb # [ 128.467625] niks3-server[1003]: 2026/09/21 04:25:20 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6262lb # [ 128.476453] niks3-server[1003]: 2026/09/21 04:25:20 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6263lb # [ 128.482033] niks3-server[1003]: 2026/09/21 04:25:20 INFO Completed upload id=446264lb # [ 128.484199] niks3-server[1003]: 2026/09/21 04:25:20 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6265worker2 # [ 128.464608] nix-grpc-daemon[1353]: time=2026-09-21T04:25:20.453Z level=INFO msg="Upload complete. (339ms)"6266worker2 # [ 128.471158] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16267worker2 # [ 128.483447] nix-daemon[865]: reaped child process 1517, status = succeeded6268client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.60 seconds)6269client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6270client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.26 seconds)6271worker1: must succeed: pkill -f 'read -t [9]0 x'6272worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6273client: waiting for success: ! systemctl is-active sw6274worker1 # [ 129.050441] nix-daemon[846]: reaped child process 1793, status = succeeded6275worker1 # [ 129.264392] nix-daemon[846]: accepted connection from pid 1840, user nix-grpc-daemon (trusted)6276worker1 # [ 129.277315] nix-daemon[846]: reaped child process 1843, status = succeeded6277lb # [ 129.310927] niks3-server[1003]: 2026/09/21 04:25:21 INFO Received uploads request method=POST path=/api/pending_closures6278worker1 # [ 129.310772] nix-grpc-daemon[1721]: time=2026-09-21T04:25:21.263Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6279worker1 # [ 129.312237] nix-grpc-daemon[1721]: time=2026-09-21T04:25:21.263Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6280lb # [ 129.345225] niks3-server[1003]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6281lb # [ 129.357985] niks3-server[1003]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6282lb # [ 129.366638] niks3-server[1003]: 2026/09/21 04:25:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6283worker1 # [ 129.360826] nix-grpc-daemon[1721]: time=2026-09-21T04:25:21.313Z level=INFO msg="Uploading 1 narinfos"6284lb # [ 129.371575] niks3-server[1003]: 2026/09/21 04:25:21 INFO Signed narinfos id=45 count=16285lb # [ 129.377240] niks3-server[1003]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6286lb # [ 129.386154] niks3-server[1003]: 2026/09/21 04:25:21 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6287lb # [ 129.391202] niks3-server[1003]: 2026/09/21 04:25:21 INFO Completed upload id=456288worker1 # [ 129.380487] nix-grpc-daemon[1721]: time=2026-09-21T04:25:21.333Z level=INFO msg="Upload complete. (349ms)"6289lb # [ 129.394143] niks3-server[1003]: 2026/09/21 04:25:21 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6290worker1 # [ 129.388449] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41602 duration_s=9 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16291worker1 # [ 129.401368] nix-daemon[846]: reaped child process 1791, status = succeeded6292client # [ 129.419302] systemd[1]: sw.service: Deactivated successfully.6293client # [ 129.422249] systemd[1]: sw.service: Consumed 128ms CPU time over 11.399s wall clock time, 41.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.6294client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)6295client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6296worker1 # [ 129.963164] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:21Z level=info event=signal_exit6297client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6298worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6299worker1 # [ 130.191424] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_yield6300lb # [ 130.205451] niks3-server[1003]: 2026/09/21 04:25:22 INFO lead: released remote=192.168.1.3:401106301worker2 # [ 130.187542] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_restarting addr=lb:500516302worker1 # [ 130.199477] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:415886303worker1 # [ 130.203989] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=worker_gone addr=192.168.1.4:500516304worker1 # [ 130.207339] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_restarting addr=lb:500516305worker1 # [ 130.210605] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:415886306worker1 # [ 130.213983] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=worker_gone addr=192.168.1.3:500516307worker2 # [ 130.388137] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_connected addr=lb:500516308worker2 # [ 130.392131] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6309worker1 # [ 130.401907] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_connected addr=lb:500516310worker1 # [ 130.405327] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6311worker2 # [ 130.593738] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_connected addr=lb:500516312worker2 # [ 130.597637] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6313worker1 # [ 130.605961] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_connected addr=lb:500516314worker1 # [ 130.610141] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6315worker2 # [ 130.798570] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_connected addr=lb:500516316worker1 # [ 130.810822] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_connected addr=lb:500516317worker2 # [ 130.802584] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6318worker1 # [ 130.814712] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6319worker2 # [ 131.003729] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_connected addr=lb:500516320worker1 # [ 131.015668] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_connected addr=lb:500516321worker2 # [ 131.007613] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6322worker1 # [ 131.019623] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6323worker1 # [ 131.196291] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:23Z level=info event=scheduler_restarting6324worker1 # [ 131.218531] nix-daemon[846]: reaped child process 1692, status = succeeded6325worker2 # [ 131.209630] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:23Z level=info event=scheduler_connected addr=lb:500516326worker2 # [ 131.213428] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6327worker1 # [ 131.234275] nix-grpc-daemon[1641]: ts=2026-09-21T04:25:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6328worker1 # [ 131.344493] nix-grpc-daemon[1641]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6329worker1 # [ 131.348277] nix-grpc-daemon[1641]: E0000 00:00:1789964723.297383 1684 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6330worker1 # [ 131.353476] nix-grpc-daemon[1641]: E0000 00:00:1789964723.297842 1684 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6331worker2 # [ 131.414368] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:23Z level=info event=scheduler_connected addr=lb:500516332worker2 # [ 131.418242] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6333worker2 # [ 131.619362] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:23Z level=info event=scheduler_connected addr=lb:500516334worker2 # [ 131.623557] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6335worker2 # [ 131.824954] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:23Z level=info event=scheduler_connected addr=lb:500516336worker2 # [ 131.829873] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6337worker2 # [ 132.030614] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500516338worker2 # [ 132.034520] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6339worker2 # [ 132.236228] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500516340worker2 # [ 132.240215] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6341worker2 # [ 132.441154] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500516342worker2 # [ 132.445126] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6343worker2 # [ 132.646544] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500516344worker2 # [ 132.650468] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6345worker2 # [ 132.851480] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_connected addr=lb:500516346worker2 # [ 132.855349] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6347worker1 # [ 133.009527] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6348worker1 # [ 133.013871] systemd[1]: nix-grpc-daemon.service: Consumed 716ms CPU time over 26.501s wall clock time, 33.4M memory peak, 25.5K incoming IP traffic, 32.9K outgoing IP traffic.6349worker1 # [ 133.029933] systemd[1]: Starting Nix worker-protocol over gRPC...6350worker2 # [ 133.056524] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516351worker2 # [ 133.060486] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6352worker1 # [ 133.247947] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6353worker1 # [ 133.250473] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6354worker1 # [ 133.254448] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16355worker1 # [ 133.262106] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6356worker1 # [ 133.265581] systemd[1]: Started Nix worker-protocol over gRPC.6357worker2 # [ 133.262161] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516358worker1 # [ 133.274070] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_take_over6359lb # [ 133.283873] niks3-server[1003]: 2026/09/21 04:25:25 INFO lead: acquired remote=192.168.1.3:387166360worker2 # [ 133.266089] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6361worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.30 seconds)6362lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6363lb: must succeed: curl -sf localhost:9901/clusters6364worker1 # [ 133.295246] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516365worker1 # [ 133.296965] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6366lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6367worker2 # [ 133.467449] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516368worker2 # [ 133.471513] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6369worker1 # [ 133.498154] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516370worker1 # [ 133.502457] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6371worker2 # [ 133.672309] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516372worker2 # [ 133.676388] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6373worker1 # [ 133.703398] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516374worker1 # [ 133.707673] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6375worker2 # [ 133.877787] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516376worker2 # [ 133.881552] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6377worker1 # [ 133.908424] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_connected addr=lb:500516378worker1 # [ 133.912567] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6379worker2 # [ 134.082459] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:26Z level=info event=scheduler_connected addr=lb:500516380worker1 # [ 134.102192] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:26Z level=info event=oidc_jwks provider=mock keys=16381worker1 # [ 134.106212] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:26Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:362086382worker1 # [ 134.110123] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06383worker1 # [ 134.116644] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:26Z level=info event=scheduler_connected addr=lb:500516384worker1 # [ 134.120609] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:362086385worker1 # [ 134.124348] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06386lb: must succeed: curl -sf localhost:9901/clusters6387lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6388lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.16 seconds)6389lb: must succeed: curl -sf localhost:9901/clusters6390lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6391worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6392worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6393(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.60 seconds)6394subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6395client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16396worker2 # [ 134.840549] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330406397worker1 # [ 134.861438] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186398worker1 # [ 134.873190] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6399worker1 # [ 134.890853] nix-grpc-daemon[1857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5)6400worker1 # [ 134.919778] nix-grpc-daemon[1857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 2/5)6401worker1 # [ 134.977604] nix-grpc-daemon[1857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 3/5)6402worker1 # [ 135.029289] nix-grpc-daemon[1857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 761 ms (attempt 4/5)6403worker1 # [ 135.796437] nix-grpc-daemon[1857]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6404worker1 # [ 135.799834] nix-grpc-daemon[1857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6405worker1 # [ 135.847275] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:27Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06406worker1 # [ 135.852718] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:27Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16407worker1 # [ 135.857526] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:27Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06408worker1 # [ 135.862084] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:27Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06409worker1 # [ 135.896662] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186410worker1 # [ 135.910257] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186411worker1 # [ 135.921806] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186412worker1 # [ 135.930948] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6413worker1 # [ 135.956146] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:27Z level=info event=niks3_push_started pid=19196414worker1 # [ 136.202836] nix-grpc-daemon[1919]: time=2026-09-21T04:25:28.155Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6415worker1 # [ 136.410594] nix-daemon[846]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6416worker1 # [ 136.424192] nix-daemon[846]: reaped child process 1935, status = succeeded6417worker1 # [ 136.649156] nix-daemon[846]: accepted connection from pid 1937, user nix-grpc-daemon (trusted)6418worker1 # [ 136.662270] nix-daemon[846]: reaped child process 1940, status = succeeded6419lb # [ 136.695824] niks3-server[1003]: 2026/09/21 04:25:28 INFO Received uploads request method=POST path=/api/pending_closures6420worker1 # [ 136.695861] nix-grpc-daemon[1919]: time=2026-09-21T04:25:28.649Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6421worker1 # [ 136.697451] nix-grpc-daemon[1919]: time=2026-09-21T04:25:28.650Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6422lb # [ 136.731349] niks3-server[1003]: 2026/09/21 04:25:28 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6423lb # [ 136.742182] niks3-server[1003]: 2026/09/21 04:25:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6424lb # [ 136.747814] niks3-server[1003]: 2026/09/21 04:25:28 INFO Signed narinfos id=46 count=16425worker1 # [ 136.739393] nix-grpc-daemon[1919]: time=2026-09-21T04:25:28.692Z level=INFO msg="Uploading 1 narinfos"6426lb # [ 136.756354] niks3-server[1003]: 2026/09/21 04:25:28 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6427lb # [ 136.764965] niks3-server[1003]: 2026/09/21 04:25:28 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6428lb # [ 136.768576] niks3-server[1003]: 2026/09/21 04:25:28 INFO Completed upload id=466429worker1 # [ 136.757908] nix-grpc-daemon[1919]: time=2026-09-21T04:25:28.711Z level=INFO msg="Upload complete. (551ms)"6430worker1 # [ 136.761134] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=5046431worker1 # [ 136.767694] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186432lb # [ 136.785856] niks3-server[1003]: 2026/09/21 04:25:28 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6433worker1 # [ 136.777453] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6434worker1 # [ 136.780628] nix-daemon[846]: reaped child process 1917, status = succeeded6435worker1 # [ 136.787092] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6436lb # [ 137.878614] envoy[910]: conn peer=[2001:db8:1::1]:34596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9611 tx=4084 ms=62906437lb # [ 137.882615] envoy[910]: conn peer=[2001:db8:1::1]:34592 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1469 ms=110996438lb # [ 137.886182] envoy[910]: conn peer=[2001:db8:1::3]:51244 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1746 tx=539 ms=242446439worker1 # [ 138.015782] nix-daemon[846]: reaped child process 1944, status = succeeded6440worker1 # [ 138.227827] nix-daemon[846]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)6441worker1 # [ 138.239344] nix-daemon[846]: reaped child process 1963, status = succeeded6442lb # [ 138.274742] niks3-server[1003]: 2026/09/21 04:25:30 INFO Received uploads request method=POST path=/api/pending_closures6443worker1 # [ 138.275369] nix-grpc-daemon[1919]: time=2026-09-21T04:25:30.228Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6444worker1 # [ 138.277131] nix-grpc-daemon[1919]: time=2026-09-21T04:25:30.230Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6445lb # [ 138.319360] niks3-server[1003]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6446lb # [ 138.324146] niks3-server[1003]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6447lb # [ 138.334874] niks3-server[1003]: 2026/09/21 04:25:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6448worker1 # [ 138.328467] nix-grpc-daemon[1919]: time=2026-09-21T04:25:30.281Z level=INFO msg="Uploading 1 narinfos"6449lb # [ 138.339051] niks3-server[1003]: 2026/09/21 04:25:30 INFO Signed narinfos id=47 count=16450lb # [ 138.344555] niks3-server[1003]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6451lb # [ 138.353635] niks3-server[1003]: 2026/09/21 04:25:30 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6452lb # [ 138.356304] niks3-server[1003]: 2026/09/21 04:25:30 INFO Completed upload id=476453worker1 # [ 138.345726] nix-grpc-daemon[1919]: time=2026-09-21T04:25:30.299Z level=INFO msg="Upload complete. (351ms)"6454lb # [ 138.358055] niks3-server[1003]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6455worker1 # [ 138.358756] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16456worker1 # [ 138.364285] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:30Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16457worker1 # [ 138.367716] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:30Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06458worker1 # [ 138.375138] nix-daemon[846]: reaped child process 1942, status = succeeded6459worker1 # [ 138.400995] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186460worker1 # [ 138.413214] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186461worker1 # [ 138.424790] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186462worker1 # [ 138.435134] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6463worker1 # [ 138.654295] nix-daemon[846]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)6464worker1 # [ 138.667529] nix-daemon[846]: reaped child process 1972, status = succeeded6465worker1 # [ 138.887216] nix-daemon[846]: accepted connection from pid 1974, user nix-grpc-daemon (trusted)6466worker1 # [ 138.900477] nix-daemon[846]: reaped child process 1977, status = succeeded6467lb # [ 138.932722] niks3-server[1003]: 2026/09/21 04:25:30 INFO Received uploads request method=POST path=/api/pending_closures6468worker1 # [ 138.930693] nix-grpc-daemon[1919]: time=2026-09-21T04:25:30.883Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6469worker1 # [ 138.932109] nix-grpc-daemon[1919]: time=2026-09-21T04:25:30.883Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6470lb # [ 138.958604] niks3-server[1003]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6471lb # [ 138.971121] niks3-server[1003]: 2026/09/21 04:25:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6472lb # [ 138.975835] niks3-server[1003]: 2026/09/21 04:25:30 INFO Signed narinfos id=48 count=16473worker1 # [ 138.968098] nix-grpc-daemon[1919]: time=2026-09-21T04:25:30.921Z level=INFO msg="Uploading 1 narinfos"6474lb # [ 138.982658] niks3-server[1003]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6475lb # [ 138.994360] niks3-server[1003]: 2026/09/21 04:25:30 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6476lb # [ 139.000619] niks3-server[1003]: 2026/09/21 04:25:30 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6477lb # [ 139.004716] niks3-server[1003]: 2026/09/21 04:25:30 INFO Completed upload id=486478worker1 # [ 138.996246] nix-grpc-daemon[1919]: time=2026-09-21T04:25:30.949Z level=INFO msg="Upload complete. (550ms)"6479worker1 # [ 139.000306] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=5046480worker1 # [ 139.008801] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186481worker1 # [ 139.023114] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6482worker1 # [ 139.025934] nix-daemon[846]: reaped child process 1967, status = succeeded6483worker1 # [ 139.031234] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6484worker1 # [ 140.246064] nix-daemon[846]: reaped child process 1981, status = succeeded6485worker1 # [ 140.446188] nix-daemon[846]: accepted connection from pid 1997, user nix-grpc-daemon (trusted)6486worker1 # [ 140.459414] nix-daemon[846]: reaped child process 2000, status = succeeded6487lb # [ 140.494339] niks3-server[1003]: 2026/09/21 04:25:32 INFO Received uploads request method=POST path=/api/pending_closures6488worker1 # [ 140.491587] nix-grpc-daemon[1919]: time=2026-09-21T04:25:32.444Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6489worker1 # [ 140.493150] nix-grpc-daemon[1919]: time=2026-09-21T04:25:32.446Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6490lb # [ 140.530825] niks3-server[1003]: 2026/09/21 04:25:32 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6491lb # [ 140.538118] niks3-server[1003]: 2026/09/21 04:25:32 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6492lb # [ 140.548763] niks3-server[1003]: 2026/09/21 04:25:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6493lb # [ 140.554714] niks3-server[1003]: 2026/09/21 04:25:32 INFO Signed narinfos id=49 count=16494worker1 # [ 140.546677] nix-grpc-daemon[1919]: time=2026-09-21T04:25:32.499Z level=INFO msg="Uploading 1 narinfos"6495lb # [ 140.560107] niks3-server[1003]: 2026/09/21 04:25:32 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6496lb # [ 140.568589] niks3-server[1003]: 2026/09/21 04:25:32 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6497lb # [ 140.576773] niks3-server[1003]: 2026/09/21 04:25:32 INFO Completed upload id=496498lb # [ 140.578094] niks3-server[1003]: 2026/09/21 04:25:32 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6499worker1 # [ 140.566433] nix-grpc-daemon[1919]: time=2026-09-21T04:25:32.519Z level=INFO msg="Upload complete. (328ms)"6500worker1 # [ 140.578606] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16501worker1 # [ 140.585699] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:32Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16502worker1 # [ 140.590261] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186503worker1 # [ 140.593395] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:32Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06504worker1 # [ 140.603060] nix-daemon[846]: reaped child process 1979, status = succeeded6505worker1 # [ 140.604864] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186506worker1 # [ 140.611953] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186507worker1 # [ 140.617720] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6508worker1 # [ 140.827401] nix-daemon[846]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)6509worker1 # [ 140.841079] nix-daemon[846]: reaped child process 2009, status = succeeded6510worker1 # [ 141.066912] nix-daemon[846]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)6511worker1 # [ 141.080548] nix-daemon[846]: reaped child process 2014, status = succeeded6512lb # [ 141.112662] niks3-server[1003]: 2026/09/21 04:25:33 INFO Received uploads request method=POST path=/api/pending_closures6513worker1 # [ 141.109329] nix-grpc-daemon[1919]: time=2026-09-21T04:25:33.062Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6514worker1 # [ 141.110951] nix-grpc-daemon[1919]: time=2026-09-21T04:25:33.064Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6515lb # [ 141.141172] niks3-server[1003]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6516lb # [ 141.152979] niks3-server[1003]: 2026/09/21 04:25:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6517lb # [ 141.157581] niks3-server[1003]: 2026/09/21 04:25:33 INFO Signed narinfos id=50 count=16518worker1 # [ 141.149583] nix-grpc-daemon[1919]: time=2026-09-21T04:25:33.102Z level=INFO msg="Uploading 1 narinfos"6519lb # [ 141.163928] niks3-server[1003]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6520lb # [ 141.174892] niks3-server[1003]: 2026/09/21 04:25:33 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6521worker1 # [ 141.169683] nix-grpc-daemon[1919]: time=2026-09-21T04:25:33.123Z level=INFO msg="Upload complete. (546ms)"6522lb # [ 141.180471] niks3-server[1003]: 2026/09/21 04:25:33 INFO Completed upload id=506523worker1 # [ 141.172425] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=5046524lb # [ 141.184708] niks3-server[1003]: 2026/09/21 04:25:33 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6525worker1 # [ 141.178228] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186526worker1 # [ 141.187624] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6527worker1 # [ 141.192056] nix-daemon[846]: reaped child process 2004, status = succeeded6528worker1 # [ 141.198069] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6529worker1 # [ 142.626794] nix-daemon[846]: reaped child process 2018, status = succeeded6530worker1 # [ 142.830758] nix-daemon[846]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6531worker1 # [ 142.844087] nix-daemon[846]: reaped child process 2037, status = succeeded6532lb # [ 142.877529] niks3-server[1003]: 2026/09/21 04:25:34 INFO Received uploads request method=POST path=/api/pending_closures6533worker1 # [ 142.873481] nix-grpc-daemon[1919]: time=2026-09-21T04:25:34.826Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6534worker1 # [ 142.877145] nix-grpc-daemon[1919]: time=2026-09-21T04:25:34.828Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6535lb # [ 142.897252] niks3-server[1003]: 2026/09/21 04:25:34 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6536lb # [ 142.901056] niks3-server[1003]: 2026/09/21 04:25:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6537lb # [ 142.902943] niks3-server[1003]: 2026/09/21 04:25:34 INFO Signed narinfos id=51 count=16538worker1 # [ 142.892512] nix-grpc-daemon[1919]: time=2026-09-21T04:25:34.845Z level=INFO msg="Uploading 1 narinfos"6539lb # [ 142.906827] niks3-server[1003]: 2026/09/21 04:25:34 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6540lb # [ 142.911433] niks3-server[1003]: 2026/09/21 04:25:34 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6541lb # [ 142.913703] niks3-server[1003]: 2026/09/21 04:25:34 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6542lb # [ 142.917324] niks3-server[1003]: 2026/09/21 04:25:34 INFO Completed upload id=516543worker1 # [ 142.906646] nix-grpc-daemon[1919]: time=2026-09-21T04:25:34.860Z level=INFO msg="Upload complete. (300ms)"6544lb # [ 142.918969] niks3-server[1003]: 2026/09/21 04:25:34 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6545worker1 # [ 142.909971] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16546worker1 # [ 142.913949] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:34Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16547worker1 # [ 142.916436] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186548worker1 # [ 142.921247] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:34Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06549worker1 # [ 142.926466] nix-daemon[846]: reaped child process 2016, status = succeeded6550worker1 # [ 142.927749] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186551worker1 # [ 142.933723] nix-grpc-daemon[1919]: time=2026-09-21T04:25:34.887Z level=INFO msg="All 1 paths already cached"6552worker1 # [ 142.937451] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186553worker1 # [ 142.941609] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6554worker1 # [ 143.150767] nix-daemon[846]: accepted connection from pid 2043, user nix-grpc-daemon (trusted)6555worker1 # [ 143.164428] nix-daemon[846]: reaped child process 2046, status = succeeded6556worker1 # [ 143.386932] nix-daemon[846]: accepted connection from pid 2048, user nix-grpc-daemon (trusted)6557worker1 # [ 143.400320] nix-daemon[846]: reaped child process 2051, status = succeeded6558lb # [ 143.431945] niks3-server[1003]: 2026/09/21 04:25:35 INFO Received uploads request method=POST path=/api/pending_closures6559worker1 # [ 143.425680] nix-grpc-daemon[1919]: time=2026-09-21T04:25:35.379Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6560worker1 # [ 143.427254] nix-grpc-daemon[1919]: time=2026-09-21T04:25:35.380Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6561lb # [ 143.448482] niks3-server[1003]: 2026/09/21 04:25:35 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6562lb # [ 143.451910] niks3-server[1003]: 2026/09/21 04:25:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6563lb # [ 143.454567] niks3-server[1003]: 2026/09/21 04:25:35 INFO Signed narinfos id=52 count=16564worker1 # [ 143.443850] nix-grpc-daemon[1919]: time=2026-09-21T04:25:35.397Z level=INFO msg="Uploading 1 narinfos"6565lb # [ 143.455818] niks3-server[1003]: 2026/09/21 04:25:35 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6566lb # [ 143.461676] niks3-server[1003]: 2026/09/21 04:25:35 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6567lb # [ 143.465076] niks3-server[1003]: 2026/09/21 04:25:35 INFO Completed upload id=526568worker1 # [ 143.455768] nix-grpc-daemon[1919]: time=2026-09-21T04:25:35.409Z level=INFO msg="Upload complete. (507ms)"6569lb # [ 143.467249] niks3-server[1003]: 2026/09/21 04:25:35 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6570worker1 # [ 143.457763] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=6246571worker1 # [ 143.460809] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186572worker1 # [ 143.469330] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6573worker1 # [ 143.470993] nix-daemon[846]: reaped child process 2041, status = succeeded6574worker1 # [ 143.476405] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6575worker1 # [ 144.490345] nix-daemon[846]: reaped child process 2055, status = succeeded6576worker1 # [ 144.690379] nix-daemon[846]: accepted connection from pid 2071, user nix-grpc-daemon (trusted)6577worker1 # [ 144.703630] nix-daemon[846]: reaped child process 2074, status = succeeded6578lb # [ 144.738224] niks3-server[1003]: 2026/09/21 04:25:36 INFO Received uploads request method=POST path=/api/pending_closures6579worker1 # [ 144.737368] nix-grpc-daemon[1919]: time=2026-09-21T04:25:36.689Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6580worker1 # [ 144.738789] nix-grpc-daemon[1919]: time=2026-09-21T04:25:36.689Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6581lb # [ 144.780469] niks3-server[1003]: 2026/09/21 04:25:36 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6582lb # [ 144.788712] niks3-server[1003]: 2026/09/21 04:25:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6583worker1 # [ 144.787695] nix-grpc-daemon[1919]: time=2026-09-21T04:25:36.740Z level=INFO msg="Uploading 1 narinfos"6584lb # [ 144.798276] niks3-server[1003]: 2026/09/21 04:25:36 INFO Signed narinfos id=53 count=16585lb # [ 144.803122] niks3-server[1003]: 2026/09/21 04:25:36 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6586lb # [ 144.816957] niks3-server[1003]: 2026/09/21 04:25:36 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6587lb # [ 144.820597] niks3-server[1003]: 2026/09/21 04:25:36 INFO Completed upload id=536588worker1 # [ 144.810727] nix-grpc-daemon[1919]: time=2026-09-21T04:25:36.763Z level=INFO msg="Upload complete. (327ms)"6589lb # [ 144.822534] niks3-server[1003]: 2026/09/21 04:25:36 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6590lb # [ 144.829914] niks3-server[1003]: 2026/09/21 04:25:36 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6591worker1 # [ 144.816776] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16592worker1 # [ 144.825696] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:36Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16593worker1 # [ 144.831912] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186594worker1 # [ 144.835073] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:36Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06595worker1 # [ 144.843255] nix-daemon[846]: reaped child process 2053, status = succeeded6596worker1 # [ 144.844832] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186597worker1 # [ 144.851503] nix-grpc-daemon[1919]: time=2026-09-21T04:25:36.804Z level=INFO msg="All 1 paths already cached"6598worker1 # [ 144.855348] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186599worker1 # [ 144.859477] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6600worker1 # [ 145.067913] nix-daemon[846]: accepted connection from pid 2080, user nix-grpc-daemon (trusted)6601worker1 # [ 145.081258] nix-daemon[846]: reaped child process 2083, status = succeeded6602worker1 # [ 145.302377] nix-daemon[846]: accepted connection from pid 2085, user nix-grpc-daemon (trusted)6603worker1 # [ 145.315370] nix-daemon[846]: reaped child process 2088, status = succeeded6604lb # [ 145.349385] niks3-server[1003]: 2026/09/21 04:25:37 INFO Received uploads request method=POST path=/api/pending_closures6605worker1 # [ 145.350565] nix-grpc-daemon[1919]: time=2026-09-21T04:25:37.303Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6606worker1 # [ 145.352153] nix-grpc-daemon[1919]: time=2026-09-21T04:25:37.305Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6607lb # [ 145.384371] niks3-server[1003]: 2026/09/21 04:25:37 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6608lb # [ 145.395081] niks3-server[1003]: 2026/09/21 04:25:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6609lb # [ 145.401194] niks3-server[1003]: 2026/09/21 04:25:37 INFO Signed narinfos id=54 count=16610worker1 # [ 145.393526] nix-grpc-daemon[1919]: time=2026-09-21T04:25:37.346Z level=INFO msg="Uploading 1 narinfos"6611lb # [ 145.404823] niks3-server[1003]: 2026/09/21 04:25:37 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6612lb # [ 145.417471] niks3-server[1003]: 2026/09/21 04:25:37 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6613lb # [ 145.423057] niks3-server[1003]: 2026/09/21 04:25:37 INFO Completed upload id=546614worker1 # [ 145.411860] nix-grpc-daemon[1919]: time=2026-09-21T04:25:37.365Z level=INFO msg="Upload complete. (546ms)"6615lb # [ 145.425822] niks3-server[1003]: 2026/09/21 04:25:37 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6616worker1 # [ 145.415777] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=6246617worker1 # [ 145.420913] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186618worker1 # [ 145.432064] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6619worker1 # [ 145.436370] nix-daemon[846]: reaped child process 2078, status = succeeded6620worker1 # [ 145.441667] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6621worker1 # [ 146.868989] nix-daemon[846]: reaped child process 2092, status = succeeded6622worker1 # [ 147.079073] nix-daemon[846]: accepted connection from pid 2108, user nix-grpc-daemon (trusted)6623worker1 # [ 147.091169] nix-daemon[846]: reaped child process 2111, status = succeeded6624lb # [ 147.124747] niks3-server[1003]: 2026/09/21 04:25:39 INFO Received uploads request method=POST path=/api/pending_closures6625worker1 # [ 147.122571] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.075Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6626worker1 # [ 147.125221] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.077Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6627lb # [ 147.166233] niks3-server[1003]: 2026/09/21 04:25:39 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6628lb # [ 147.171561] niks3-server[1003]: 2026/09/21 04:25:39 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6629lb # [ 147.176875] niks3-server[1003]: 2026/09/21 04:25:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6630worker1 # [ 147.170043] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.123Z level=INFO msg="Uploading 1 narinfos"6631lb # [ 147.180589] niks3-server[1003]: 2026/09/21 04:25:39 INFO Signed narinfos id=55 count=16632lb # [ 147.186142] niks3-server[1003]: 2026/09/21 04:25:39 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6633lb # [ 147.195407] niks3-server[1003]: 2026/09/21 04:25:39 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6634lb # [ 147.200710] niks3-server[1003]: 2026/09/21 04:25:39 INFO Completed upload id=556635worker1 # [ 147.190050] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.143Z level=INFO msg="Upload complete. (340ms)"6636lb # [ 147.201832] niks3-server[1003]: 2026/09/21 04:25:39 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6637worker1 # [ 147.199285] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16638worker1 # [ 147.204952] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:39Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16639worker1 # [ 147.209613] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186640worker1 # [ 147.213193] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:39Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06641worker1 # [ 147.219411] nix-daemon[846]: reaped child process 2090, status = succeeded6642worker1 # [ 147.221265] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186643worker1 # [ 147.230045] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.183Z level=INFO msg="All 1 paths already cached"6644worker1 # [ 147.234892] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186645worker1 # [ 147.239268] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6646worker1 # [ 147.447423] nix-daemon[846]: accepted connection from pid 2117, user nix-grpc-daemon (trusted)6647worker1 # [ 147.460778] nix-daemon[846]: reaped child process 2120, status = succeeded6648worker1 # [ 147.688126] nix-daemon[846]: accepted connection from pid 2122, user nix-grpc-daemon (trusted)6649worker1 # [ 147.700510] nix-daemon[846]: reaped child process 2125, status = succeeded6650lb # [ 147.734217] niks3-server[1003]: 2026/09/21 04:25:39 INFO Received uploads request method=POST path=/api/pending_closures6651worker1 # [ 147.731390] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.684Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6652worker1 # [ 147.733357] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.686Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6653lb # [ 147.760861] niks3-server[1003]: 2026/09/21 04:25:39 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6654lb # [ 147.773248] niks3-server[1003]: 2026/09/21 04:25:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6655worker1 # [ 147.768701] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.721Z level=INFO msg="Uploading 1 narinfos"6656lb # [ 147.779412] niks3-server[1003]: 2026/09/21 04:25:39 INFO Signed narinfos id=56 count=16657lb # [ 147.783566] niks3-server[1003]: 2026/09/21 04:25:39 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6658lb # [ 147.795906] niks3-server[1003]: 2026/09/21 04:25:39 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6659lb # [ 147.801927] niks3-server[1003]: 2026/09/21 04:25:39 INFO Completed upload id=566660worker1 # [ 147.791172] nix-grpc-daemon[1919]: time=2026-09-21T04:25:39.744Z level=INFO msg="Upload complete. (545ms)"6661lb # [ 147.804427] niks3-server[1003]: 2026/09/21 04:25:39 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6662worker1 # [ 147.794533] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=6246663worker1 # [ 147.799944] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186664worker1 # [ 147.810180] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6665worker1 # [ 147.813974] nix-daemon[846]: reaped child process 2115, status = succeeded6666worker1 # [ 147.819541] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6667worker1 # [ 149.035396] nix-daemon[846]: reaped child process 2129, status = succeeded6668worker1 # [ 149.241498] nix-daemon[846]: accepted connection from pid 2145, user nix-grpc-daemon (trusted)6669worker1 # [ 149.254214] nix-daemon[846]: reaped child process 2148, status = succeeded6670lb # [ 149.287510] niks3-server[1003]: 2026/09/21 04:25:41 INFO Received uploads request method=POST path=/api/pending_closures6671worker1 # [ 149.287359] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.240Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6672worker1 # [ 149.289138] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.242Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6673lb # [ 149.329434] niks3-server[1003]: 2026/09/21 04:25:41 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6674lb # [ 149.335166] niks3-server[1003]: 2026/09/21 04:25:41 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6675lb # [ 149.344500] niks3-server[1003]: 2026/09/21 04:25:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6676worker1 # [ 149.338451] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.291Z level=INFO msg="Uploading 1 narinfos"6677lb # [ 149.349367] niks3-server[1003]: 2026/09/21 04:25:41 INFO Signed narinfos id=57 count=16678lb # [ 149.354790] niks3-server[1003]: 2026/09/21 04:25:41 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6679lb # [ 149.364104] niks3-server[1003]: 2026/09/21 04:25:41 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6680lb # [ 149.368994] niks3-server[1003]: 2026/09/21 04:25:41 INFO Completed upload id=576681worker1 # [ 149.358365] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.311Z level=INFO msg="Upload complete. (331ms)"6682lb # [ 149.371989] niks3-server[1003]: 2026/09/21 04:25:41 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6683worker1 # [ 149.363666] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16684worker1 # [ 149.376056] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:41Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16685worker1 # [ 149.379478] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:41Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06686worker1 # [ 149.388032] nix-daemon[846]: reaped child process 2127, status = succeeded6687worker1 # [ 149.415757] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186688worker1 # [ 149.428233] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186689worker1 # [ 149.440819] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.394Z level=INFO msg="All 2 paths already cached"6690worker1 # [ 149.448345] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186691worker1 # [ 149.458351] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6692worker1 # [ 149.684983] nix-daemon[846]: accepted connection from pid 2154, user nix-grpc-daemon (trusted)6693worker1 # [ 149.697842] nix-daemon[846]: reaped child process 2157, status = succeeded6694worker1 # [ 149.924574] nix-daemon[846]: accepted connection from pid 2159, user nix-grpc-daemon (trusted)6695worker1 # [ 149.937196] nix-daemon[846]: reaped child process 2162, status = succeeded6696lb # [ 149.971136] niks3-server[1003]: 2026/09/21 04:25:41 INFO Received uploads request method=POST path=/api/pending_closures6697worker1 # [ 149.967901] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.921Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6698worker1 # [ 149.969471] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.922Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6699lb # [ 149.998286] niks3-server[1003]: 2026/09/21 04:25:41 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6700lb # [ 150.009886] niks3-server[1003]: 2026/09/21 04:25:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6701lb # [ 150.015622] niks3-server[1003]: 2026/09/21 04:25:41 INFO Signed narinfos id=58 count=16702worker1 # [ 150.007713] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.961Z level=INFO msg="Uploading 1 narinfos"6703lb # [ 150.019468] niks3-server[1003]: 2026/09/21 04:25:41 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6704lb # [ 150.032618] niks3-server[1003]: 2026/09/21 04:25:41 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6705lb # [ 150.037846] niks3-server[1003]: 2026/09/21 04:25:41 INFO Completed upload id=586706worker1 # [ 150.029514] nix-grpc-daemon[1919]: time=2026-09-21T04:25:41.982Z level=INFO msg="Upload complete. (560ms)"6707lb # [ 150.041280] niks3-server[1003]: 2026/09/21 04:25:41 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6708worker1 # [ 150.032262] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=7526709worker1 # [ 150.039628] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186710worker1 # [ 150.048421] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6711worker1 # [ 150.053106] nix-daemon[846]: reaped child process 2152, status = succeeded6712worker1 # [ 150.058484] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6713worker1 # [ 151.083984] nix-daemon[846]: reaped child process 2166, status = succeeded6714worker1 # [ 151.291150] nix-daemon[846]: accepted connection from pid 2182, user nix-grpc-daemon (trusted)6715worker1 # [ 151.304339] nix-daemon[846]: reaped child process 2185, status = succeeded6716lb # [ 151.338480] niks3-server[1003]: 2026/09/21 04:25:43 INFO Received uploads request method=POST path=/api/pending_closures6717worker1 # [ 151.336301] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.289Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6718worker1 # [ 151.337807] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.291Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6719lb # [ 151.373165] niks3-server[1003]: 2026/09/21 04:25:43 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6720lb # [ 151.385420] niks3-server[1003]: 2026/09/21 04:25:43 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6721lb # [ 151.394555] niks3-server[1003]: 2026/09/21 04:25:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6722lb # [ 151.399247] niks3-server[1003]: 2026/09/21 04:25:43 INFO Signed narinfos id=59 count=16723worker1 # [ 151.391487] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.344Z level=INFO msg="Uploading 1 narinfos"6724lb # [ 151.412082] niks3-server[1003]: 2026/09/21 04:25:43 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6725lb # [ 151.414873] niks3-server[1003]: 2026/09/21 04:25:43 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6726lb # [ 151.420361] niks3-server[1003]: 2026/09/21 04:25:43 INFO Completed upload id=596727worker1 # [ 151.409685] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.363Z level=INFO msg="Upload complete. (344ms)"6728lb # [ 151.424570] niks3-server[1003]: 2026/09/21 04:25:43 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6729worker1 # [ 151.418442] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16730worker1 # [ 151.425247] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:43Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16731worker1 # [ 151.428744] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186732worker1 # [ 151.439151] nix-daemon[846]: reaped child process 2164, status = succeeded6733worker1 # [ 151.461854] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186734worker1 # [ 151.473987] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.427Z level=INFO msg="All 2 paths already cached"6735worker1 # [ 151.481298] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186736worker1 # [ 151.490263] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6737worker1 # [ 151.692655] nix-daemon[846]: accepted connection from pid 2189, user nix-grpc-daemon (trusted)6738worker1 # [ 151.706569] nix-daemon[846]: reaped child process 2192, status = succeeded6739worker1 # [ 151.924525] nix-daemon[846]: accepted connection from pid 2194, user nix-grpc-daemon (trusted)6740worker1 # [ 151.937665] nix-daemon[846]: reaped child process 2197, status = succeeded6741lb # [ 151.969668] niks3-server[1003]: 2026/09/21 04:25:43 INFO Received uploads request method=POST path=/api/pending_closures6742worker1 # [ 151.966710] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.920Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6743worker1 # [ 151.968612] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.921Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6744lb # [ 151.994935] niks3-server[1003]: 2026/09/21 04:25:43 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6745lb # [ 152.006616] niks3-server[1003]: 2026/09/21 04:25:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6746worker1 # [ 152.001242] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.954Z level=INFO msg="Uploading 1 narinfos"6747lb # [ 152.011753] niks3-server[1003]: 2026/09/21 04:25:43 INFO Signed narinfos id=60 count=16748lb # [ 152.017954] niks3-server[1003]: 2026/09/21 04:25:43 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6749lb # [ 152.028166] niks3-server[1003]: 2026/09/21 04:25:43 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6750worker1 # [ 152.024394] nix-grpc-daemon[1919]: time=2026-09-21T04:25:43.977Z level=INFO msg="Upload complete. (524ms)"6751lb # [ 152.035361] niks3-server[1003]: 2026/09/21 04:25:43 INFO Completed upload id=606752worker1 # [ 152.026497] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=7526753lb # [ 152.038192] niks3-server[1003]: 2026/09/21 04:25:43 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6754worker1 # [ 152.031332] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186755worker1 # [ 152.038444] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6756worker1 # [ 152.041256] nix-daemon[846]: reaped child process 2187, status = succeeded6757worker1 # [ 152.047367] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6758worker1 # [ 153.061546] nix-daemon[846]: reaped child process 2201, status = succeeded6759worker1 # [ 153.259927] nix-daemon[846]: accepted connection from pid 2217, user nix-grpc-daemon (trusted)6760worker1 # [ 153.273586] nix-daemon[846]: reaped child process 2220, status = succeeded6761lb # [ 153.308075] niks3-server[1003]: 2026/09/21 04:25:45 INFO Received uploads request method=POST path=/api/pending_closures6762worker1 # [ 153.308894] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.262Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6763worker1 # [ 153.310470] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.263Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6764lb # [ 153.348656] niks3-server[1003]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6765lb # [ 153.355847] niks3-server[1003]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6766lb # [ 153.366101] niks3-server[1003]: 2026/09/21 04:25:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6767lb # [ 153.372412] niks3-server[1003]: 2026/09/21 04:25:45 INFO Signed narinfos id=61 count=16768lb # [ 153.375820] niks3-server[1003]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6769worker1 # [ 153.365316] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.318Z level=INFO msg="Uploading 1 narinfos"6770lb # [ 153.387719] niks3-server[1003]: 2026/09/21 04:25:45 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6771lb # [ 153.391559] niks3-server[1003]: 2026/09/21 04:25:45 INFO Completed upload id=616772worker1 # [ 153.380963] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.334Z level=INFO msg="Upload complete. (326ms)"6773lb # [ 153.394282] niks3-server[1003]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6774worker1 # [ 153.389750] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16775worker1 # [ 153.396327] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:45Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16776worker1 # [ 153.399934] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186777worker1 # [ 153.409594] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362186778worker1 # [ 153.415548] nix-daemon[846]: reaped child process 2199, status = succeeded6779worker1 # [ 153.420920] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.374Z level=INFO msg="All 2 paths already cached"6780worker1 # [ 153.425367] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362186781worker1 # [ 153.430718] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6782worker1 # [ 153.638346] nix-daemon[846]: accepted connection from pid 2224, user nix-grpc-daemon (trusted)6783worker1 # [ 153.651551] nix-daemon[846]: reaped child process 2227, status = succeeded6784worker1 # [ 153.871521] nix-daemon[846]: accepted connection from pid 2229, user nix-grpc-daemon (trusted)6785worker1 # [ 153.884513] nix-daemon[846]: reaped child process 2232, status = succeeded6786lb # [ 153.917720] niks3-server[1003]: 2026/09/21 04:25:45 INFO Received uploads request method=POST path=/api/pending_closures6787worker1 # [ 153.914655] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.867Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6788worker1 # [ 153.916096] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.867Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6789lb # [ 153.941876] niks3-server[1003]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6790lb # [ 153.953359] niks3-server[1003]: 2026/09/21 04:25:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6791worker1 # [ 153.947501] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.900Z level=INFO msg="Uploading 1 narinfos"6792lb # [ 153.958146] niks3-server[1003]: 2026/09/21 04:25:45 INFO Signed narinfos id=62 count=16793lb # [ 153.965427] niks3-server[1003]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6794lb # [ 153.975550] niks3-server[1003]: 2026/09/21 04:25:45 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6795worker1 # [ 153.969430] nix-grpc-daemon[1919]: time=2026-09-21T04:25:45.922Z level=INFO msg="Upload complete. (533ms)"6796lb # [ 153.980208] niks3-server[1003]: 2026/09/21 04:25:45 INFO Completed upload id=626797worker1 # [ 153.971523] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=0 paths=1 nar_bytes_in=7526798lb # [ 153.983387] niks3-server[1003]: 2026/09/21 04:25:45 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6799worker1 # [ 153.977448] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362186800worker1 # [ 153.982318] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6801worker1 # [ 153.984719] nix-daemon[846]: reaped child process 2222, status = succeeded6802worker1 # [ 153.990427] nix-daemon[846]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6803worker1 # [ 155.605555] nix-daemon[846]: reaped child process 2236, status = succeeded6804worker1 # [ 155.810933] nix-daemon[846]: accepted connection from pid 2252, user nix-grpc-daemon (trusted)6805worker1 # [ 155.824194] nix-daemon[846]: reaped child process 2255, status = succeeded6806lb # [ 155.859180] niks3-server[1003]: 2026/09/21 04:25:47 INFO Received uploads request method=POST path=/api/pending_closures6807worker1 # [ 155.858370] nix-grpc-daemon[1919]: time=2026-09-21T04:25:47.811Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6808worker1 # [ 155.859924] nix-grpc-daemon[1919]: time=2026-09-21T04:25:47.813Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6809lb # [ 155.900907] niks3-server[1003]: 2026/09/21 04:25:47 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6810lb # [ 155.907585] niks3-server[1003]: 2026/09/21 04:25:47 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6811lb # [ 155.917820] niks3-server[1003]: 2026/09/21 04:25:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6812lb # [ 155.921960] niks3-server[1003]: 2026/09/21 04:25:47 INFO Signed narinfos id=63 count=16813worker1 # [ 155.914087] nix-grpc-daemon[1919]: time=2026-09-21T04:25:47.867Z level=INFO msg="Uploading 1 narinfos"6814lb # [ 155.932354] niks3-server[1003]: 2026/09/21 04:25:47 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6815lb # [ 155.938598] niks3-server[1003]: 2026/09/21 04:25:47 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6816lb # [ 155.943713] niks3-server[1003]: 2026/09/21 04:25:47 INFO Completed upload id=636817lb # [ 155.945496] niks3-server[1003]: 2026/09/21 04:25:47 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6818worker1 # [ 155.934452] nix-grpc-daemon[1919]: time=2026-09-21T04:25:47.887Z level=INFO msg="Upload complete. (335ms)"6819worker1 # [ 155.941900] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36218 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16820worker1 # [ 155.955720] nix-daemon[846]: reaped child process 2234, status = succeeded6821client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.42 seconds)6822worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6823worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6824client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'6825client # [ 156.127173] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6826client # Running as unit: gpu.service; invocation ID: 2484d04878b74b3bb6a4f42f720211096827client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.07 seconds)6828lb: must succeed: curl -sf localhost:9901/clusters6829lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6830lb: must succeed: curl -sf localhost:9901/clusters6831lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6832worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6833worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6834client # [ 156.370356] nix[1360]: don't know how to build these paths:6835client # [ 156.371688] nix[1360]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6836worker2 # [ 156.374101] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330406837worker2 # [ 156.379588] nix-grpc-daemon[1214]: ts=2026-09-21T04:25:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406838worker1 # [ 156.406302] nix-grpc-daemon[1857]: ts=2026-09-21T04:25:48Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06839client # [ 156.398215] nix[1360]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6840lb: must succeed: curl -sf localhost:9901/clusters6841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6842lb: must succeed: curl -sf localhost:9901/clusters6843lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6844worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6845worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6846client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6847lb # [ 157.879814] envoy[910]: conn peer=[2001:db8:1::1]:33068 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27128 tx=11461 ms=211066848client # [ 186.399296] nix[1360]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6849client # [ 186.407976] nix[1360]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6850client # [ 186.456354] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6851client # [ 186.459319] systemd[1]: gpu.service: Failed with result 'exit-code'.6852client # [ 186.463556] systemd[1]: gpu.service: Consumed 132ms CPU time over 30.329s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6853client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 28.92 seconds)6854client: must succeed: journalctl -u gpu -o cat6855client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6856lb: must succeed: curl -sf localhost:9901/clusters6857lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6858lb: must succeed: curl -sf localhost:9901/clusters6859lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6860worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6861worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6862(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 52.13 seconds)6863subtest: low disk drains a worker and builds go to the other6864worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6865worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)6866worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6867lb # [ 187.882667] envoy[910]: conn peer=[2001:db8:1::1]:40418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300546868worker1 # [ 188.288732] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:20Z level=info event=unhealthy reason=min_free6869worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6870client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26871worker1 # [ 189.094698] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362186872client # don't know how to build these paths:6873client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6874worker2 # [ 189.098414] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406875worker1 # [ 189.128110] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:21Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06876worker1 # [ 189.132620] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:21Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16877worker2 # [ 189.163835] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406878worker2 # [ 189.176135] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406879client # copying 1 paths to 'lb:50051'...6880client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6881worker2 # [ 189.188378] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330406882worker2 # [ 189.199897] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6883worker2 # [ 189.383915] nix-daemon[865]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6884worker2 # [ 189.395890] nix-daemon[865]: reaped child process 1545, status = succeeded6885worker2 # [ 189.567685] nix-daemon[865]: accepted connection from pid 1547, user nix-grpc-daemon (trusted)6886worker2 # [ 189.580104] nix-daemon[865]: reaped child process 1550, status = succeeded6887lb # [ 189.616570] niks3-server[1003]: 2026/09/21 04:26:21 INFO Received uploads request method=POST path=/api/pending_closures6888worker2 # [ 189.607955] nix-grpc-daemon[1353]: time=2026-09-21T04:26:21.596Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6889worker2 # [ 189.609501] nix-grpc-daemon[1353]: time=2026-09-21T04:26:21.596Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6890lb # [ 189.645303] niks3-server[1003]: 2026/09/21 04:26:21 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6891lb # [ 189.658212] niks3-server[1003]: 2026/09/21 04:26:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6892lb # [ 189.663354] niks3-server[1003]: 2026/09/21 04:26:21 INFO Signed narinfos id=64 count=16893worker2 # [ 189.648290] nix-grpc-daemon[1353]: time=2026-09-21T04:26:21.636Z level=INFO msg="Uploading 1 narinfos"6894lb # [ 189.677185] niks3-server[1003]: 2026/09/21 04:26:21 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6895lb # [ 189.683387] niks3-server[1003]: 2026/09/21 04:26:21 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6896worker2 # [ 189.671623] nix-grpc-daemon[1353]: time=2026-09-21T04:26:21.660Z level=INFO msg="Upload complete. (460ms)"6897lb # [ 189.690584] niks3-server[1003]: 2026/09/21 04:26:21 INFO Completed upload id=646898lb # [ 189.694711] niks3-server[1003]: 2026/09/21 04:26:21 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6899worker2 # [ 189.673972] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=0 paths=1 nar_bytes_in=5126900worker2 # [ 189.681086] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406901worker2 # [ 189.689062] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6902worker2 # [ 189.700362] nix-daemon[865]: reaped child process 1540, status = succeeded6903worker2 # [ 189.703432] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6904client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6905client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6906client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6907client # farm-a> LOG-farm-a-drain6908worker2 # [ 190.727338] nix-daemon[865]: reaped child process 1554, status = succeeded6909worker2 # [ 190.894706] nix-daemon[865]: accepted connection from pid 1570, user nix-grpc-daemon (trusted)6910worker2 # [ 190.906767] nix-daemon[865]: reaped child process 1573, status = succeeded6911lb # [ 190.943343] niks3-server[1003]: 2026/09/21 04:26:22 INFO Received uploads request method=POST path=/api/pending_closures6912worker2 # [ 190.935205] nix-grpc-daemon[1353]: time=2026-09-21T04:26:22.923Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6913worker2 # [ 190.936804] nix-grpc-daemon[1353]: time=2026-09-21T04:26:22.925Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6914lb # [ 190.982176] niks3-server[1003]: 2026/09/21 04:26:22 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6915lb # [ 190.992682] niks3-server[1003]: 2026/09/21 04:26:22 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6916lb # [ 191.002500] niks3-server[1003]: 2026/09/21 04:26:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6917worker2 # [ 190.987716] nix-grpc-daemon[1353]: time=2026-09-21T04:26:22.976Z level=INFO msg="Uploading 1 narinfos"6918lb # [ 191.007103] niks3-server[1003]: 2026/09/21 04:26:22 INFO Signed narinfos id=65 count=16919lb # [ 191.011973] niks3-server[1003]: 2026/09/21 04:26:22 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6920lb # [ 191.017052] niks3-server[1003]: 2026/09/21 04:26:22 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6921lb # [ 191.021055] niks3-server[1003]: 2026/09/21 04:26:22 INFO Completed upload id=656922worker2 # [ 191.002330] nix-grpc-daemon[1353]: time=2026-09-21T04:26:22.990Z level=INFO msg="Upload complete. (291ms)"6923lb # [ 191.022876] niks3-server[1003]: 2026/09/21 04:26:22 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6924worker2 # [ 191.008100] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16925worker1 # [ 191.022858] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:22Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06926worker2 # [ 191.018258] nix-daemon[865]: reaped child process 1552, status = succeeded6927worker1 # [ 191.027666] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:22Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16928worker2 # [ 191.062731] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406929worker2 # [ 191.074063] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406930worker2 # [ 191.086467] nix-grpc-daemon[1353]: time=2026-09-21T04:26:23.075Z level=INFO msg="All 1 paths already cached"6931client # copying 1 paths to 'lb:50051'...6932client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6933worker2 # [ 191.093829] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330406934worker2 # [ 191.102539] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6935worker2 # [ 191.280910] nix-daemon[865]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6936worker2 # [ 191.294151] nix-daemon[865]: reaped child process 1580, status = succeeded6937worker2 # [ 191.460894] nix-daemon[865]: accepted connection from pid 1582, user nix-grpc-daemon (trusted)6938worker2 # [ 191.472656] nix-daemon[865]: reaped child process 1585, status = succeeded6939lb # [ 191.507392] niks3-server[1003]: 2026/09/21 04:26:23 INFO Received uploads request method=POST path=/api/pending_closures6940worker2 # [ 191.496062] nix-grpc-daemon[1353]: time=2026-09-21T04:26:23.484Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6941worker2 # [ 191.497562] nix-grpc-daemon[1353]: time=2026-09-21T04:26:23.486Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6942lb # [ 191.535728] niks3-server[1003]: 2026/09/21 04:26:23 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6943lb # [ 191.546800] niks3-server[1003]: 2026/09/21 04:26:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6944lb # [ 191.551119] niks3-server[1003]: 2026/09/21 04:26:23 INFO Signed narinfos id=66 count=16945worker2 # [ 191.535488] nix-grpc-daemon[1353]: time=2026-09-21T04:26:23.524Z level=INFO msg="Uploading 1 narinfos"6946lb # [ 191.557329] niks3-server[1003]: 2026/09/21 04:26:23 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6947lb # [ 191.568722] niks3-server[1003]: 2026/09/21 04:26:23 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6948worker2 # [ 191.555945] nix-grpc-daemon[1353]: time=2026-09-21T04:26:23.544Z level=INFO msg="Upload complete. (439ms)"6949lb # [ 191.574926] niks3-server[1003]: 2026/09/21 04:26:23 INFO Completed upload id=666950lb # [ 191.577272] niks3-server[1003]: 2026/09/21 04:26:23 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6951worker2 # [ 191.558658] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=0 paths=1 nar_bytes_in=6246952worker2 # [ 191.564639] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406953worker2 # [ 191.574964] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6954worker2 # [ 191.576705] nix-daemon[865]: reaped child process 1575, status = succeeded6955worker2 # [ 191.582181] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6956client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6957client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6958client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6959client # farm-b> LOG-farm-b-drain6960worker2 # [ 192.605563] nix-daemon[865]: reaped child process 1589, status = succeeded6961worker2 # [ 192.768456] nix-daemon[865]: accepted connection from pid 1605, user nix-grpc-daemon (trusted)6962worker2 # [ 192.780384] nix-daemon[865]: reaped child process 1608, status = succeeded6963lb # [ 192.816545] niks3-server[1003]: 2026/09/21 04:26:24 INFO Received uploads request method=POST path=/api/pending_closures6964worker2 # [ 192.805833] nix-grpc-daemon[1353]: time=2026-09-21T04:26:24.794Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6965worker2 # [ 192.807373] nix-grpc-daemon[1353]: time=2026-09-21T04:26:24.796Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6966lb # [ 192.858178] niks3-server[1003]: 2026/09/21 04:26:24 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6967lb # [ 192.865198] niks3-server[1003]: 2026/09/21 04:26:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6968lb # [ 192.870948] niks3-server[1003]: 2026/09/21 04:26:24 INFO Signed narinfos id=67 count=16969worker2 # [ 192.855615] nix-grpc-daemon[1353]: time=2026-09-21T04:26:24.844Z level=INFO msg="Uploading 1 narinfos"6970lb # [ 192.878614] niks3-server[1003]: 2026/09/21 04:26:24 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6971lb # [ 192.891728] niks3-server[1003]: 2026/09/21 04:26:24 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6972lb # [ 192.897495] niks3-server[1003]: 2026/09/21 04:26:24 INFO Completed upload id=676973worker2 # [ 192.880878] nix-grpc-daemon[1353]: time=2026-09-21T04:26:24.869Z level=INFO msg="Upload complete. (292ms)"6974lb # [ 192.901251] niks3-server[1003]: 2026/09/21 04:26:24 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6975lb # [ 192.904645] niks3-server[1003]: 2026/09/21 04:26:24 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6976worker2 # [ 192.887183] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16977worker1 # [ 192.902584] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:24Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06978worker2 # [ 192.900717] nix-daemon[865]: reaped child process 1587, status = succeeded6979worker1 # [ 192.907530] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:24Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16980worker2 # [ 192.937169] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330406981worker2 # [ 192.948458] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330406982worker2 # [ 192.960559] nix-grpc-daemon[1353]: time=2026-09-21T04:26:24.949Z level=INFO msg="All 2 paths already cached"6983client # copying 1 paths to 'lb:50051'...6984client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6985worker2 # [ 192.967537] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330406986worker2 # [ 192.977337] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)6987worker2 # [ 193.156457] nix-daemon[865]: accepted connection from pid 1612, user nix-grpc-daemon (trusted)6988worker2 # [ 193.169286] nix-daemon[865]: reaped child process 1615, status = succeeded6989worker2 # [ 193.336303] nix-daemon[865]: accepted connection from pid 1617, user nix-grpc-daemon (trusted)6990worker2 # [ 193.347672] nix-daemon[865]: reaped child process 1620, status = succeeded6991lb # [ 193.384574] niks3-server[1003]: 2026/09/21 04:26:25 INFO Received uploads request method=POST path=/api/pending_closures6992worker2 # [ 193.373063] nix-grpc-daemon[1353]: time=2026-09-21T04:26:25.361Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6993worker2 # [ 193.374615] nix-grpc-daemon[1353]: time=2026-09-21T04:26:25.363Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6994lb # [ 193.413308] niks3-server[1003]: 2026/09/21 04:26:25 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6995lb # [ 193.423115] niks3-server[1003]: 2026/09/21 04:26:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6996lb # [ 193.428105] niks3-server[1003]: 2026/09/21 04:26:25 INFO Signed narinfos id=68 count=16997worker2 # [ 193.412118] nix-grpc-daemon[1353]: time=2026-09-21T04:26:25.400Z level=INFO msg="Uploading 1 narinfos"6998lb # [ 193.434398] niks3-server[1003]: 2026/09/21 04:26:25 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6999lb # [ 193.447528] niks3-server[1003]: 2026/09/21 04:26:25 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7000lb # [ 193.452515] niks3-server[1003]: 2026/09/21 04:26:25 INFO Completed upload id=687001worker2 # [ 193.433633] nix-grpc-daemon[1353]: time=2026-09-21T04:26:25.422Z level=INFO msg="Upload complete. (442ms)"7002lb # [ 193.454749] niks3-server[1003]: 2026/09/21 04:26:25 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7003worker2 # [ 193.437070] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=0 paths=1 nar_bytes_in=7527004worker2 # [ 193.442240] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330407005worker2 # [ 193.452104] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)7006worker2 # [ 193.456247] nix-daemon[865]: reaped child process 1610, status = succeeded7007client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7008client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7009worker2 # [ 193.461456] nix-daemon[865]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)7010client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7011client # farm-top> LOG-farm-top-drain7012worker2 # [ 194.281206] nix-daemon[865]: reaped child process 1624, status = succeeded7013worker2 # [ 194.439342] nix-daemon[865]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)7014worker2 # [ 194.449980] nix-daemon[865]: reaped child process 1643, status = succeeded7015lb # [ 194.487320] niks3-server[1003]: 2026/09/21 04:26:26 INFO Received uploads request method=POST path=/api/pending_closures7016worker2 # [ 194.477315] nix-grpc-daemon[1353]: time=2026-09-21T04:26:26.465Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7017worker2 # [ 194.479045] nix-grpc-daemon[1353]: time=2026-09-21T04:26:26.467Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7018lb # [ 194.529319] niks3-server[1003]: 2026/09/21 04:26:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7019lb # [ 194.536113] niks3-server[1003]: 2026/09/21 04:26:26 INFO Signed narinfos id=69 count=17020worker2 # [ 194.520032] nix-grpc-daemon[1353]: time=2026-09-21T04:26:26.508Z level=INFO msg="Uploading 1 narinfos"7021lb # [ 194.547375] niks3-server[1003]: 2026/09/21 04:26:26 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7022lb # [ 194.553719] niks3-server[1003]: 2026/09/21 04:26:26 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7023lb # [ 194.558436] niks3-server[1003]: 2026/09/21 04:26:26 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7024lb # [ 194.565986] niks3-server[1003]: 2026/09/21 04:26:26 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7025lb # [ 194.571653] niks3-server[1003]: 2026/09/21 04:26:26 INFO Completed upload id=697026worker2 # [ 194.552856] nix-grpc-daemon[1353]: time=2026-09-21T04:26:26.541Z level=INFO msg="Upload complete. (284ms)"7027lb # [ 194.574481] niks3-server[1003]: 2026/09/21 04:26:26 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7028worker2 # [ 194.559404] nix-grpc-daemon[1214]: ts=2026-09-21T04:26:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33040 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17029worker2 # [ 194.571633] nix-daemon[865]: reaped child process 1622, status = succeeded7030client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.78 seconds)7031client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7032client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.22 seconds)7033worker1: must succeed: rm /nix/.rw-store/fill7034worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.14 seconds)7035worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7036worker1 # [ 195.291800] nix-grpc-daemon[1857]: ts=2026-09-21T04:26:27Z level=info event=healthy reason=min_free7037worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)7038(finished: subtest: low disk drains a worker and builds go to the other, in 9.47 seconds)7039(finished: run the VM test script, in 197.31 seconds)7040test script finished in 197.56s7041cleanup7042kill QemuMachine (pid 49)7043client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7044kill QemuMachine (pid 50)7045lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7046kill QemuMachine (pid 51)7047worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7048kill QemuMachine (pid 52)7049worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7050(finished: cleanup, in 0.92 seconds)7051additionally exposed symbols:7052 client, lb, worker1, worker2,7053 vlan1,7054 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh