vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #491
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client: QEMU running (pid 49)14lb: QEMU running (pid 50)15lb # Disk image does not exist, creating the virtualisation disk image...16client # Formatting '/build/vm-state-client/tmp.W8gIhQ60wj', fmt=raw size=107374182417lb # Formatting '/build/vm-state-lb/tmp.prbQ5G8nCh', fmt=raw size=107374182418client # mke2fs 1.47.4 (6-Mar-2025)19worker2 # Disk image does not exist, creating the virtualisation disk image...20lb # mke2fs 1.47.4 (6-Mar-2025)21worker2: QEMU running (pid 52)22client # Discarding device blocks: 0/262144 done23worker2 # Formatting '/build/vm-state-worker2/tmp.lwfrubCPtC', fmt=raw size=107374182424client # Creating filesystem with 262144 4k blocks and 65536 inodes25worker2 # mke2fs 1.47.4 (6-Mar-2025)26lb # Discarding device blocks: 0/262144 done27client # Filesystem UUID: 32a4384f-20d0-4c8b-81c0-23959cae198228worker2 # Discarding device blocks: 0/262144 done29lb # Creating filesystem with 262144 4k blocks and 65536 inodes30client # Superblock backups stored on blocks:31worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes32lb # Filesystem UUID: 4b5c44a8-1016-48de-acff-1e08138bfec233client # 32768, 98304, 163840, 22937634worker2 # Filesystem UUID: 4c72d6b6-e693-42f5-bd7a-183f76c1c57635lb # Superblock backups stored on blocks:36client # 37worker2 # Superblock backups stored on blocks:38lb # 32768, 98304, 163840, 22937639client # Allocating group tables: 0/8 done40worker2 # 32768, 98304, 163840, 22937641lb # 42client # Writing inode tables: 0/8 done43worker2 # 44lb # Allocating group tables: 0/8 done45client # Creating journal (8192 blocks): done46worker2 # Allocating group tables: 0/8 done47lb # Writing inode tables: 0/8 done48client # Writing superblocks and filesystem accounting information: 0/8 done49worker2 # Writing inode tables: 0/8 done50lb # Creating journal (8192 blocks): done51client # 52worker2 # Creating journal (8192 blocks): done53lb # Writing superblocks and filesystem accounting information: 0/8 done54client # Virtualisation disk image created.55worker2 # Writing superblocks and filesystem accounting information: 0/8 done56lb # 57worker1 # Disk image does not exist, creating the virtualisation disk image...58worker1: QEMU running (pid 51)59worker2 # 60lb # Virtualisation disk image created.61worker2 # Virtualisation disk image created.62worker1 # Formatting '/build/vm-state-worker1/tmp.HpumQfM74j', fmt=raw size=107374182463(finished: start all VMs, in 0.18 seconds)64worker1 # mke2fs 1.47.4 (6-Mar-2025)65client: waiting for unit network-addresses-eth1.service66worker1 # Discarding device blocks: 0/262144 done67client: waiting for the VM to finish booting68worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes69worker1 # Filesystem UUID: 6a7249ba-1da5-4fd6-a67d-b968cb127ad370worker1 # Superblock backups stored on blocks:71worker1 # 32768, 98304, 163840, 22937672worker1 # 73worker1 # Allocating group tables: 0/8 done74worker1 # Writing inode tables: 0/8 done75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83client # 84client # 85worker1 # 86worker1 # 87client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0088worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089lb # 90lb # 91client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...92client # 93client # 94client # 95client # 96lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0097worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...98worker1 # 99worker1 # 100client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00101worker1 # 102worker1 # 103worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00104lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...105lb # 106lb # 107lb # 108lb # 109lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00110client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...111client # 112client # 113worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114worker1 # 115worker1 # 116lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117lb # 118lb # 119worker2 # 120worker2 # 121worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123worker2 # 124worker2 # 125worker2 # 126worker2 # 127worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker2 # 130worker2 # 131client # Booting from ROM...132worker1 # Booting from ROM...133lb # Booting from ROM...134worker2 # Booting from ROM...135client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026137client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test/init regInfo=/nix/store/b8ampkhjhkxcfp3s6f3zcp422b7lvzsc-closure-info/registration console=ttyS0,115200n8 console=tty0138client # [ 0.000000] BIOS-provided physical RAM map:139client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable140client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved141client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mvx6cn1l6frdr55gdj0p01an8nxnxzn-nixos-system-lb-test/init regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration console=ttyS0,115200n8 console=tty0143lb # [ 0.000000] BIOS-provided physical RAM map:144client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable145lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable146client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved147lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved148client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved149lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved150client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved151lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable152client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved153lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved154client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved155worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026156lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved157client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved158lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved159client # [ 0.000000] NX (Execute Disable) protection: active160client # [ 0.000000] APIC: Static calls initialized161lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved162client # [ 0.000000] SMBIOS 2.8 present.163lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved164worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026165client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014166lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved167client # [ 0.000000] DMI: Memory slots populated: 1/1168worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/j7glxpj3rbb8k7aikb154lhfz7amnd7b-nixos-system-worker2-test/init regInfo=/nix/store/6wyi8ra5v5p7s9q73i0ykg14mn0bn7gx-closure-info/registration console=ttyS0,115200n8 console=tty0169client # [ 0.000000] Hypervisor detected: KVM170lb # [ 0.000000] NX (Execute Disable) protection: active171worker2 # [ 0.000000] BIOS-provided physical RAM map:172lb # [ 0.000000] APIC: Static calls initialized173client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000174lb # [ 0.000000] SMBIOS 2.8 present.175worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable176client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00177worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved178client # [ 0.000000] kvm-clock: using sched offset of 464702886 cycles179lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014180worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6iych1bhd6jkvv72xp98ahp6dmm92z7r-nixos-system-worker1-test/init regInfo=/nix/store/sncm61przda863m2mj9d0pcd8iyhxwx4-closure-info/registration console=ttyS0,115200n8 console=tty0181worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved182lb # [ 0.000000] DMI: Memory slots populated: 1/1183worker1 # [ 0.000000] BIOS-provided physical RAM map:184lb # [ 0.000000] Hypervisor detected: KVM185client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns186worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable187lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000188client # [ 0.000006] tsc: Detected 2400.012 MHz processor189worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable190worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved191lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00192client # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000193worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved194worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved195lb # [ 0.000000] kvm-clock: using sched offset of 466856155 cycles196client # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs197worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved198worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved199client # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT200worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable201lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns202worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved203lb # [ 0.000005] tsc: Detected 2400.012 MHz processor204client # [ 0.002745] found SMP MP-table at [mem 0x000f5450-0x000f545f]205worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved206client # [ 0.002760] Using GB pages for direct mapping207lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000208worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved209worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved210client # [ 0.002865] RAMDISK: [mem 0x5e366000-0x5ffcffff]211worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved212lb # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs213client # [ 0.002876] ACPI: Early table checksum verification disabled214worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved215worker2 # [ 0.000000] NX (Execute Disable) protection: active216lb # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT217worker2 # [ 0.000000] APIC: Static calls initialized218client # [ 0.002880] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )219worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved220worker2 # [ 0.000000] SMBIOS 2.8 present.221lb # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]222worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved223client # [ 0.002884] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)224lb # [ 0.002744] Using GB pages for direct mapping225worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014226lb # [ 0.002855] RAMDISK: [mem 0x5e367000-0x5ffcffff]227worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved228client # [ 0.002888] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)229worker2 # [ 0.000000] DMI: Memory slots populated: 1/1230worker1 # [ 0.000000] NX (Execute Disable) protection: active231worker2 # [ 0.000000] Hypervisor detected: KVM232worker1 # [ 0.000000] APIC: Static calls initialized233lb # [ 0.002863] ACPI: Early table checksum verification disabled234client # [ 0.002897] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)235worker1 # [ 0.000000] SMBIOS 2.8 present.236worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000237client # [ 0.002899] ACPI: FACS 0x000000005FFE0000 000040238worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00239lb # [ 0.002865] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )240lb # [ 0.002869] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.000000] kvm-clock: using sched offset of 479895990 cycles242worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014243client # [ 0.002901] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)244worker1 # [ 0.000000] DMI: Memory slots populated: 1/1245lb # [ 0.002872] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)246worker1 # [ 0.000000] Hypervisor detected: KVM247client # [ 0.002902] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)248worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns249worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000250worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor251lb # [ 0.002879] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)252client # [ 0.002904] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)253lb # [ 0.002881] ACPI: FACS 0x000000005FFE0000 000040254worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00255worker2 # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000256worker1 # [ 0.000000] kvm-clock: using sched offset of 476665986 cycles257client # [ 0.002905] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)258lb # [ 0.002882] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)259worker2 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs260client # [ 0.002906] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]261worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns262worker2 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT263lb # [ 0.002884] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)264worker1 # [ 0.000006] tsc: Detected 2400.012 MHz processor265worker2 # [ 0.002740] found SMP MP-table at [mem 0x000f5450-0x000f545f]266client # [ 0.002907] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]267lb # [ 0.002885] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)268worker2 # [ 0.002751] Using GB pages for direct mapping269worker1 # [ 0.000822] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000270client # [ 0.002908] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]271worker2 # [ 0.002900] RAMDISK: [mem 0x5e367000-0x5ffcffff]272lb # [ 0.002887] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)273worker1 # [ 0.000850] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs274client # [ 0.002908] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]275worker2 # [ 0.002908] ACPI: Early table checksum verification disabled276lb # [ 0.002888] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]277worker1 # [ 0.000853] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT278client # [ 0.002909] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]279worker2 # [ 0.002911] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )280worker1 # [ 0.002759] found SMP MP-table at [mem 0x000f5450-0x000f545f]281lb # [ 0.002889] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]282client # [ 0.002909] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]283worker2 # [ 0.002915] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)284worker1 # [ 0.002773] Using GB pages for direct mapping285worker1 # [ 0.002862] RAMDISK: [mem 0x5e367000-0x5ffcffff]286lb # [ 0.002890] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]287client # [ 0.002910] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]288worker2 # [ 0.002919] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)289client # [ 0.003145] No NUMA configuration found290worker1 # [ 0.002871] ACPI: Early table checksum verification disabled291lb # [ 0.002890] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]292client # [ 0.003146] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]293worker1 # [ 0.002875] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )294worker2 # [ 0.002925] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)295lb # [ 0.002891] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]296client # [ 0.003149] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]297worker2 # [ 0.002927] ACPI: FACS 0x000000005FFE0000 000040298worker1 # [ 0.002880] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)299client # [ 0.005555] Zone ranges:300lb # [ 0.002891] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]301worker2 # [ 0.002929] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)302client # [ 0.005556] DMA [mem 0x0000000000001000-0x0000000000ffffff]303lb # [ 0.002892] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]304worker1 # [ 0.002884] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)305lb # [ 0.003120] No NUMA configuration found306client # [ 0.005558] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]307worker2 # [ 0.002930] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)308client # [ 0.005559] Normal empty309worker1 # [ 0.002892] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)310client # [ 0.005560] Device empty311lb # [ 0.003121] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]312worker2 # [ 0.002932] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)313client # [ 0.005560] Movable zone start for each node314worker1 # [ 0.002894] ACPI: FACS 0x000000005FFE0000 000040315lb # [ 0.003123] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]316lb # [ 0.005449] Zone ranges:317client # [ 0.005561] Early memory node ranges318worker2 # [ 0.002934] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)319worker1 # [ 0.002896] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)320client # [ 0.005562] node 0: [mem 0x0000000000001000-0x000000000009efff]321lb # [ 0.005449] DMA [mem 0x0000000000001000-0x0000000000ffffff]322worker2 # [ 0.002935] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]323client # [ 0.005563] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]324worker1 # [ 0.002897] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)325lb # [ 0.005451] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]326lb # [ 0.005452] Normal empty327worker2 # [ 0.002936] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]328lb # [ 0.005453] Device empty329client # [ 0.005564] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]330worker1 # [ 0.002899] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)331lb # [ 0.005453] Movable zone start for each node332client # [ 0.005584] On node 0, zone DMA: 1 pages in unavailable ranges333worker2 # [ 0.002936] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]334lb # [ 0.005454] Early memory node ranges335worker1 # [ 0.002900] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)336client # [ 0.005912] On node 0, zone DMA: 97 pages in unavailable ranges337worker2 # [ 0.002937] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]338lb # [ 0.005454] node 0: [mem 0x0000000000001000-0x000000000009efff]339worker1 # [ 0.002902] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]340client # [ 0.034027] On node 0, zone DMA32: 40 pages in unavailable ranges341lb # [ 0.005455] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]342worker2 # [ 0.002937] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]343client # [ 0.034503] ACPI: PM-Timer IO Port: 0x608344worker1 # [ 0.002903] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]345worker2 # [ 0.002938] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]346lb # [ 0.005456] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]347client # [ 0.034516] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])348worker1 # [ 0.002903] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]349lb # [ 0.005476] On node 0, zone DMA: 1 pages in unavailable ranges350worker2 # [ 0.002938] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]351client # [ 0.034545] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23352worker2 # [ 0.003164] No NUMA configuration found353worker1 # [ 0.002904] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]354lb # [ 0.005748] On node 0, zone DMA: 97 pages in unavailable ranges355client # [ 0.034548] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)356worker2 # [ 0.003165] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]357lb # [ 0.032942] On node 0, zone DMA32: 40 pages in unavailable ranges358worker1 # [ 0.002904] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]359client # [ 0.034550] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)360lb # [ 0.033397] ACPI: PM-Timer IO Port: 0x608361worker2 # [ 0.003168] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]362worker1 # [ 0.002905] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]363worker2 # [ 0.005499] Zone ranges:364client # [ 0.034552] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)365lb # [ 0.033408] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])366worker1 # [ 0.002905] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]367worker2 # [ 0.005500] DMA [mem 0x0000000000001000-0x0000000000ffffff]368client # [ 0.034553] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)369lb # [ 0.033434] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23370worker1 # [ 0.003139] No NUMA configuration found371worker2 # [ 0.005502] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]372client # [ 0.034553] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)373lb # [ 0.033437] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)374worker2 # [ 0.005503] Normal empty375worker1 # [ 0.003140] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]376worker2 # [ 0.005504] Device empty377client # [ 0.034556] ACPI: Using ACPI (MADT) for SMP configuration information378lb # [ 0.033438] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)379worker2 # [ 0.005505] Movable zone start for each node380worker1 # [ 0.003143] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]381client # [ 0.034557] ACPI: HPET id: 0x8086a201 base: 0xfed00000382worker2 # [ 0.005505] Early memory node ranges383worker1 # [ 0.005835] Zone ranges:384lb # [ 0.033439] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)385client # [ 0.034562] TSC deadline timer available386worker1 # [ 0.005836] DMA [mem 0x0000000000001000-0x0000000000ffffff]387worker2 # [ 0.005506] node 0: [mem 0x0000000000001000-0x000000000009efff]388client # [ 0.034566] CPU topo: Max. logical packages: 1389lb # [ 0.033440] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)390worker1 # [ 0.005838] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]391worker2 # [ 0.005507] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]392client # [ 0.034567] CPU topo: Max. logical dies: 1393worker1 # [ 0.005839] Normal empty394lb # [ 0.033441] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)395client # [ 0.034567] CPU topo: Max. dies per package: 1396worker1 # [ 0.005840] Device empty397worker2 # [ 0.005508] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]398lb # [ 0.033444] ACPI: Using ACPI (MADT) for SMP configuration information399client # [ 0.034571] CPU topo: Max. threads per core: 1400worker1 # [ 0.005841] Movable zone start for each node401worker2 # [ 0.005528] On node 0, zone DMA: 1 pages in unavailable ranges402worker1 # [ 0.005842] Early memory node ranges403client # [ 0.034571] CPU topo: Num. cores per package: 1404lb # [ 0.033445] ACPI: HPET id: 0x8086a201 base: 0xfed00000405lb # [ 0.033448] TSC deadline timer available406worker1 # [ 0.005842] node 0: [mem 0x0000000000001000-0x000000000009efff]407worker2 # [ 0.005803] On node 0, zone DMA: 97 pages in unavailable ranges408client # [ 0.034572] CPU topo: Num. threads per package: 1409lb # [ 0.033452] CPU topo: Max. logical packages: 1410worker2 # [ 0.033116] On node 0, zone DMA32: 40 pages in unavailable ranges411client # [ 0.034572] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs412worker1 # [ 0.005843] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]413lb # [ 0.033453] CPU topo: Max. logical dies: 1414worker2 # [ 0.033573] ACPI: PM-Timer IO Port: 0x608415client # [ 0.034595] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()416lb # [ 0.033453] CPU topo: Max. dies per package: 1417worker1 # [ 0.005844] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]418worker2 # [ 0.033585] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])419lb # [ 0.033459] CPU topo: Max. threads per core: 1420worker1 # [ 0.005868] On node 0, zone DMA: 1 pages in unavailable ranges421client # [ 0.034630] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]422worker2 # [ 0.033616] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23423lb # [ 0.033459] CPU topo: Num. cores per package: 1424worker1 # [ 0.006140] On node 0, zone DMA: 97 pages in unavailable ranges425client # [ 0.034632] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]426lb # [ 0.033460] CPU topo: Num. threads per package: 1427worker2 # [ 0.033619] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)428worker1 # [ 0.033614] On node 0, zone DMA32: 40 pages in unavailable ranges429lb # [ 0.033460] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs430client # [ 0.034633] [mem 0x60000000-0xafffffff] available for PCI devices431worker2 # [ 0.033621] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)432worker1 # [ 0.034130] ACPI: PM-Timer IO Port: 0x608433client # [ 0.034635] Booting paravirtualized kernel on KVM434worker1 # [ 0.034145] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])435lb # [ 0.033477] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()436worker2 # [ 0.033622] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)437worker1 # [ 0.034175] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23438worker2 # [ 0.033623] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)439lb # [ 0.033508] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]440client # [ 0.034639] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns441worker1 # [ 0.034178] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)442worker2 # [ 0.033623] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)443lb # [ 0.033509] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]444client # [ 0.039118] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1445worker1 # [ 0.034180] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)446worker2 # [ 0.033626] ACPI: Using ACPI (MADT) for SMP configuration information447lb # [ 0.033511] [mem 0x60000000-0xafffffff] available for PCI devices448client # [ 0.041451] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152449worker2 # [ 0.033627] ACPI: HPET id: 0x8086a201 base: 0xfed00000450lb # [ 0.033512] Booting paravirtualized kernel on KVM451worker1 # [ 0.034181] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)452client # [ 0.041498] kvm-guest: PV spinlocks disabled, single CPU453worker2 # [ 0.033631] TSC deadline timer available454worker1 # [ 0.034182] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)455worker2 # [ 0.033636] CPU topo: Max. logical packages: 1456lb # [ 0.033515] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns457worker2 # [ 0.033636] CPU topo: Max. logical dies: 1458worker1 # [ 0.034183] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)459worker2 # [ 0.033637] CPU topo: Max. dies per package: 1460lb # [ 0.037950] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1461worker1 # [ 0.034185] ACPI: Using ACPI (MADT) for SMP configuration information462worker2 # [ 0.033640] CPU topo: Max. threads per core: 1463lb # [ 0.040416] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152464worker1 # [ 0.034186] ACPI: HPET id: 0x8086a201 base: 0xfed00000465worker2 # [ 0.033641] CPU topo: Num. cores per package: 1466worker1 # [ 0.034190] TSC deadline timer available467lb # [ 0.040460] kvm-guest: PV spinlocks disabled, single CPU468worker2 # [ 0.033641] CPU topo: Num. threads per package: 1469client # [ 0.041499] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test/init regInfo=/nix/store/b8ampkhjhkxcfp3s6f3zcp422b7lvzsc-closure-info/registration console=ttyS0,115200n8 console=tty0470worker1 # [ 0.034194] CPU topo: Max. logical packages: 1471worker2 # [ 0.033641] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs472worker1 # [ 0.034195] CPU topo: Max. logical dies: 1473worker1 # [ 0.034196] CPU topo: Max. dies per package: 1474worker2 # [ 0.033659] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()475client # [ 0.041599] Unknown kernel command line parameters "regInfo=/nix/store/b8ampkhjhkxcfp3s6f3zcp422b7lvzsc-closure-info/registration", will be passed to user space.476worker1 # [ 0.034199] CPU topo: Max. threads per core: 1477client # [ 0.041612] random: crng init done478worker2 # [ 0.033689] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]479worker1 # [ 0.034200] CPU topo: Num. cores per package: 1480worker1 # [ 0.034200] CPU topo: Num. threads per package: 1481client # [ 0.041613] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes482worker2 # [ 0.033691] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]483lb # [ 0.040461] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mvx6cn1l6frdr55gdj0p01an8nxnxzn-nixos-system-lb-test/init regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration console=ttyS0,115200n8 console=tty0484worker1 # [ 0.034200] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs485worker2 # [ 0.033692] [mem 0x60000000-0xafffffff] available for PCI devices486client # [ 0.043705] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)487worker1 # [ 0.034221] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()488worker2 # [ 0.033694] Booting paravirtualized kernel on KVM489client # [ 0.044755] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)490lb # [ 0.040553] Unknown kernel command line parameters "regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration", will be passed to user space.491client # [ 0.044793] Fallback order for Node 0: 0492worker1 # [ 0.034256] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]493lb # [ 0.040565] random: crng init done494worker2 # [ 0.033697] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns495client # [ 0.044797] Built 1 zonelists, mobility grouping on. Total pages: 393078496lb # [ 0.040566] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes497worker1 # [ 0.034257] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]498worker2 # [ 0.038146] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1499client # [ 0.044798] Policy zone: DMA32500worker1 # [ 0.034259] [mem 0x60000000-0xafffffff] available for PCI devices501lb # [ 0.042743] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)502client # [ 0.047250] mem auto-init: stack:all(zero), heap alloc:on, heap free:off503worker2 # [ 0.040358] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152504worker1 # [ 0.034261] Booting paravirtualized kernel on KVM505worker2 # [ 0.040400] kvm-guest: PV spinlocks disabled, single CPU506client # [ 0.050672] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1507lb # [ 0.043831] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)508lb # [ 0.043861] Fallback order for Node 0: 0509client # [ 0.054106] allocated 3145728 bytes of page_ext510worker1 # [ 0.034264] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns511client # [ 0.064211] ftrace: allocating 48731 entries in 192 pages512lb # [ 0.043864] Built 1 zonelists, mobility grouping on. Total pages: 393078513worker1 # [ 0.038802] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1514lb # [ 0.043865] Policy zone: DMA32515client # [ 0.064213] ftrace: allocated 192 pages with 2 groups516client # [ 0.065125] Dynamic Preempt: lazy517worker1 # [ 0.040966] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152518lb # [ 0.046422] mem auto-init: stack:all(zero), heap alloc:on, heap free:off519client # [ 0.065314] rcu: Preemptible hierarchical RCU implementation.520worker1 # [ 0.041008] kvm-guest: PV spinlocks disabled, single CPU521lb # [ 0.049518] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1522client # [ 0.065315] rcu: RCU event tracing is enabled.523worker2 # [ 0.040402] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/j7glxpj3rbb8k7aikb154lhfz7amnd7b-nixos-system-worker2-test/init regInfo=/nix/store/6wyi8ra5v5p7s9q73i0ykg14mn0bn7gx-closure-info/registration console=ttyS0,115200n8 console=tty0524lb # [ 0.053170] allocated 3145728 bytes of page_ext525client # [ 0.065316] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.526lb # [ 0.063004] ftrace: allocating 48731 entries in 192 pages527client # [ 0.065318] Trampoline variant of Tasks RCU enabled.528lb # [ 0.063005] ftrace: allocated 192 pages with 2 groups529worker2 # [ 0.040493] Unknown kernel command line parameters "regInfo=/nix/store/6wyi8ra5v5p7s9q73i0ykg14mn0bn7gx-closure-info/registration", will be passed to user space.530client # [ 0.065318] Rude variant of Tasks RCU enabled.531lb # [ 0.063849] Dynamic Preempt: lazy532worker2 # [ 0.040506] random: crng init done533client # [ 0.065318] Tracing variant of Tasks RCU enabled.534lb # [ 0.064030] rcu: Preemptible hierarchical RCU implementation.535worker2 # [ 0.040507] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes536lb # [ 0.064030] rcu: RCU event tracing is enabled.537client # [ 0.065319] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.538worker1 # [ 0.041010] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6iych1bhd6jkvv72xp98ahp6dmm92z7r-nixos-system-worker1-test/init regInfo=/nix/store/sncm61przda863m2mj9d0pcd8iyhxwx4-closure-info/registration console=ttyS0,115200n8 console=tty0539lb # [ 0.064031] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.540worker2 # [ 0.042013] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)541client # [ 0.065320] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1542lb # [ 0.064033] Trampoline variant of Tasks RCU enabled.543worker2 # [ 0.043094] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)544worker1 # [ 0.041111] Unknown kernel command line parameters "regInfo=/nix/store/sncm61przda863m2mj9d0pcd8iyhxwx4-closure-info/registration", will be passed to user space.545client # [ 0.065337] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.546lb # [ 0.064033] Rude variant of Tasks RCU enabled.547worker1 # [ 0.041124] random: crng init done548worker2 # [ 0.043120] Fallback order for Node 0: 0549lb # [ 0.064034] Tracing variant of Tasks RCU enabled.550client # [ 0.065339] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.551worker2 # [ 0.043123] Built 1 zonelists, mobility grouping on. Total pages: 393078552worker1 # [ 0.041125] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes553worker2 # [ 0.043124] Policy zone: DMA32554lb # [ 0.064034] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.555worker1 # [ 0.043284] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)556client # [ 0.065340] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.557worker2 # [ 0.045406] mem auto-init: stack:all(zero), heap alloc:on, heap free:off558lb # [ 0.064035] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1559client # [ 0.069708] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16560worker1 # [ 0.044359] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)561worker2 # [ 0.048913] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1562lb # [ 0.064049] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.563worker1 # [ 0.044394] Fallback order for Node 0: 0564worker2 # [ 0.052264] allocated 3145728 bytes of page_ext565client # [ 0.069988] rcu: srcu_init: Setting srcu_struct sizes based on contention.566worker1 # [ 0.044397] Built 1 zonelists, mobility grouping on. Total pages: 393078567worker2 # [ 0.061912] ftrace: allocating 48731 entries in 192 pages568lb # [ 0.064051] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.569worker1 # [ 0.044398] Policy zone: DMA32570client # [ 0.069996] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns571worker2 # [ 0.061913] ftrace: allocated 192 pages with 2 groups572worker2 # [ 0.062747] Dynamic Preempt: lazy573worker1 # [ 0.047053] mem auto-init: stack:all(zero), heap alloc:on, heap free:off574lb # [ 0.064052] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.575worker2 # [ 0.062878] rcu: Preemptible hierarchical RCU implementation.576client # [ 0.070105] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)577lb # [ 0.068438] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16578worker1 # [ 0.050496] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1579client # [ 0.073647] Console: colour VGA+ 80x25580worker2 # [ 0.062879] rcu: RCU event tracing is enabled.581worker1 # [ 0.053832] allocated 3145728 bytes of page_ext582lb # [ 0.068705] rcu: srcu_init: Setting srcu_struct sizes based on contention.583client # [ 0.073650] printk: legacy console [tty0] enabled584worker2 # [ 0.062879] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.585worker1 # [ 0.063840] ftrace: allocating 48731 entries in 192 pages586client # [ 0.103019] printk: legacy console [ttyS0] enabled587worker2 # [ 0.062881] Trampoline variant of Tasks RCU enabled.588worker1 # [ 0.063843] ftrace: allocated 192 pages with 2 groups589lb # [ 0.068711] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns590worker1 # [ 0.064758] Dynamic Preempt: lazy591client # [ 0.198894] ACPI: Core revision 20250807592worker2 # [ 0.062881] Rude variant of Tasks RCU enabled.593worker2 # [ 0.062881] Tracing variant of Tasks RCU enabled.594worker1 # [ 0.064934] rcu: Preemptible hierarchical RCU implementation.595lb # [ 0.068814] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)596worker1 # [ 0.064934] rcu: RCU event tracing is enabled.597lb # [ 0.072329] Console: colour VGA+ 80x25598worker2 # [ 0.062882] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.599client # [ 0.199794] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns600lb # [ 0.072331] printk: legacy console [tty0] enabled601worker1 # [ 0.064935] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.602worker2 # [ 0.062883] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1603client # [ 0.201305] APIC: Switch to symmetric I/O mode setup604worker1 # [ 0.064937] Trampoline variant of Tasks RCU enabled.605lb # [ 0.104403] printk: legacy console [ttyS0] enabled606worker1 # [ 0.064937] Rude variant of Tasks RCU enabled.607client # [ 0.202255] x2apic enabled608lb # [ 0.200436] ACPI: Core revision 20250807609worker2 # [ 0.062938] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.610worker1 # [ 0.064938] Tracing variant of Tasks RCU enabled.611client # [ 0.202996] APIC: Switched APIC routing to: physical x2apic612worker2 # [ 0.062940] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.613worker1 # [ 0.064938] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.614lb # [ 0.201287] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns615worker2 # [ 0.062941] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.616worker1 # [ 0.064939] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1617lb # [ 0.202792] APIC: Switch to symmetric I/O mode setup618worker2 # [ 0.067333] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16619client # [ 0.204780] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1620lb # [ 0.203732] x2apic enabled621worker1 # [ 0.064961] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.622worker2 # [ 0.067612] rcu: srcu_init: Setting srcu_struct sizes based on contention.623lb # [ 0.204467] APIC: Switched APIC routing to: physical x2apic624worker1 # [ 0.064962] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.625client # [ 0.205736] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns626worker2 # [ 0.067618] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns627worker1 # [ 0.064963] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.628client # [ 0.207345] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)629worker2 # [ 0.067725] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)630worker1 # [ 0.069501] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16631lb # [ 0.206251] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1632worker2 # [ 0.071233] Console: colour VGA+ 80x25633worker1 # [ 0.069795] rcu: srcu_init: Setting srcu_struct sizes based on contention.634worker2 # [ 0.071235] printk: legacy console [tty0] enabled635client # [ 0.208431] x86/cpu: User Mode Instruction Prevention (UMIP) activated636worker2 # [ 0.100593] printk: legacy console [ttyS0] enabled637lb # [ 0.207221] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns638worker1 # [ 0.069802] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns639client # [ 0.209467] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127640worker2 # [ 0.195523] ACPI: Core revision 20250807641client # [ 0.210343] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0642worker1 # [ 0.069907] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)643lb # [ 0.208882] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)644worker1 # [ 0.073821] Console: colour VGA+ 80x25645worker2 # [ 0.196409] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns646worker1 # [ 0.073824] printk: legacy console [tty0] enabled647lb # [ 0.209967] x86/cpu: User Mode Instruction Prevention (UMIP) activated648worker1 # [ 0.105461] printk: legacy console [ttyS0] enabled649client # [ 0.211346] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto650worker2 # [ 0.197960] APIC: Switch to symmetric I/O mode setup651worker1 # [ 0.200737] ACPI: Core revision 20250807652worker2 # [ 0.198893] x2apic enabled653lb # [ 0.211074] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127654client # [ 0.212348] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl655lb # [ 0.211881] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0656worker2 # [ 0.199629] APIC: Switched APIC routing to: physical x2apic657client # [ 0.213342] Transient Scheduler Attacks: Vulnerable: No microcode658worker1 # [ 0.201660] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns659client # [ 0.214342] Spectre V2 : Mitigation: Enhanced / Automatic IBRS660worker1 # [ 0.203200] APIC: Switch to symmetric I/O mode setup661lb # [ 0.212883] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto662worker2 # [ 0.201419] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1663worker1 # [ 0.204130] x2apic enabled664client # [ 0.215342] Speculative Return Stack Overflow: Mitigation: Safe RET665lb # [ 0.214880] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl666worker1 # [ 0.204891] APIC: Switched APIC routing to: physical x2apic667worker2 # [ 0.202373] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns668lb # [ 0.215880] Transient Scheduler Attacks: Vulnerable: No microcode669worker2 # [ 0.204082] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)670lb # [ 0.217639] Spectre V2 : Mitigation: Enhanced / Automatic IBRS671worker1 # [ 0.206672] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1672lb # [ 0.218566] Speculative Return Stack Overflow: Mitigation: Safe RET673worker2 # [ 0.205166] x86/cpu: User Mode Instruction Prevention (UMIP) activated674client # [ 0.216342] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization675worker1 # [ 0.207613] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns676client # [ 0.218350] Spectre V2 : Enabling IBPB for BPF677worker2 # [ 0.206219] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127678worker1 # [ 0.209197] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)679worker2 # [ 0.207080] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0680client # [ 0.219059] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier681client # [ 0.220343] active return thunk: srso_alias_return_thunk682worker1 # [ 0.210280] x86/cpu: User Mode Instruction Prevention (UMIP) activated683worker2 # [ 0.208083] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto684lb # [ 0.218880] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization685worker1 # [ 0.211337] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127686client # [ 0.221363] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'687lb # [ 0.220887] Spectre V2 : Enabling IBPB for BPF688worker2 # [ 0.209080] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl689worker1 # [ 0.212194] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0690client # [ 0.222342] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'691worker2 # [ 0.210080] Transient Scheduler Attacks: Vulnerable: No microcode692lb # [ 0.221881] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier693client # [ 0.223342] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'694worker1 # [ 0.213198] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto695lb # [ 0.222880] active return thunk: srso_alias_return_thunk696worker2 # [ 0.211079] Spectre V2 : Mitigation: Enhanced / Automatic IBRS697client # [ 0.224342] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'698worker2 # [ 0.212080] Speculative Return Stack Overflow: Mitigation: Safe RET699lb # [ 0.223901] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'700worker1 # [ 0.214194] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl701client # [ 0.225342] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'702lb # [ 0.224880] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'703worker1 # [ 0.215194] Transient Scheduler Attacks: Vulnerable: No microcode704client # [ 0.226342] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'705worker1 # [ 0.216193] Spectre V2 : Mitigation: Enhanced / Automatic IBRS706lb # [ 0.226630] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'707client # [ 0.227342] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'708worker1 # [ 0.217194] Speculative Return Stack Overflow: Mitigation: Safe RET709lb # [ 0.227651] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'710client # [ 0.228342] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256711worker2 # [ 0.213079] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization712lb # [ 0.228626] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'713client # [ 0.229342] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64714worker2 # [ 0.215087] Spectre V2 : Enabling IBPB for BPF715lb # [ 0.229610] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'716client # [ 0.230342] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512717worker2 # [ 0.215797] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier718client # [ 0.231342] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024719lb # [ 0.230674] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'720worker2 # [ 0.217080] active return thunk: srso_alias_return_thunk721worker1 # [ 0.218193] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization722client # [ 0.232342] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8723lb # [ 0.231880] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256724worker1 # [ 0.220201] Spectre V2 : Enabling IBPB for BPF725worker2 # [ 0.218100] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'726lb # [ 0.232880] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64727client # [ 0.233342] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.728worker2 # [ 0.219079] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'729worker1 # [ 0.220912] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier730lb # [ 0.233799] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512731worker2 # [ 0.220079] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'732worker1 # [ 0.222194] active return thunk: srso_alias_return_thunk733lb # [ 0.234583] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024734worker2 # [ 0.221079] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'735worker1 # [ 0.223214] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'736lb # [ 0.235619] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8737worker2 # [ 0.222079] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'738worker1 # [ 0.224194] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'739lb # [ 0.236607] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.740worker2 # [ 0.223079] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'741worker1 # [ 0.225193] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'742worker2 # [ 0.224079] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'743worker1 # [ 0.226193] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'744worker2 # [ 0.225080] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256745worker1 # [ 0.227193] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'746worker2 # [ 0.226079] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64747worker1 # [ 0.228194] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'748worker2 # [ 0.227079] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512749worker1 # [ 0.229193] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'750worker2 # [ 0.228079] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024751worker1 # [ 0.230194] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256752worker2 # [ 0.229079] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8753worker1 # [ 0.231193] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64754worker2 # [ 0.230079] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.755worker1 # [ 0.232193] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756worker1 # [ 0.233193] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757worker1 # [ 0.234193] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker1 # [ 0.235193] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759client # [ 0.268042] Freeing SMP alternatives memory: 44K760client # [ 0.268345] pid_max: default: 32768 minimum: 301761client # [ 0.269433] LSM: initializing lsm=capability,landlock,yama,bpf,ima762client # [ 0.270447] landlock: Up and running.763client # [ 0.271343] Yama: becoming mindful.764lb # [ 0.271322] Freeing SMP alternatives memory: 44K765client # [ 0.272143] LSM support for eBPF active766lb # [ 0.271883] pid_max: default: 32768 minimum: 301767client # [ 0.272922] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)768lb # [ 0.272967] LSM: initializing lsm=capability,landlock,yama,bpf,ima769lb # [ 0.273980] landlock: Up and running.770client # [ 0.273382] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)771lb # [ 0.274660] Yama: becoming mindful.772lb # [ 0.275528] LSM support for eBPF active773client # [ 0.276764] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)774lb # [ 0.276007] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)775worker2 # [ 0.264019] Freeing SMP alternatives memory: 44K776lb # [ 0.276922] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)777client # [ 0.277954] Performance Events: Fam17h+ core perfctr, AMD PMU driver.778worker2 # [ 0.264647] pid_max: default: 32768 minimum: 301779client # [ 0.278351] ... version: 2780worker2 # [ 0.265173] LSM: initializing lsm=capability,landlock,yama,bpf,ima781client # [ 0.279121] ... bit width: 48782worker2 # [ 0.266172] landlock: Up and running.783lb # [ 0.280042] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)784client # [ 0.279344] ... generic counters: 6785worker2 # [ 0.267080] Yama: becoming mindful.786client # [ 0.280016] ... generic bitmap: 000000000000003f787worker2 # [ 0.268120] LSM support for eBPF active788lb # [ 0.281456] Performance Events: Fam17h+ core perfctr, AMD PMU driver.789client # [ 0.280344] ... fixed-purpose counters: 0790worker1 # [ 0.270555] Freeing SMP alternatives memory: 44K791lb # [ 0.281888] ... version: 2792client # [ 0.281053] ... fixed-purpose bitmap: 0000000000000000793worker2 # [ 0.268865] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)794worker1 # [ 0.271198] pid_max: default: 32768 minimum: 301795lb # [ 0.282563] ... bit width: 48796client # [ 0.281344] ... value mask: 0000ffffffffffff797lb # [ 0.282882] ... generic counters: 6798worker1 # [ 0.272005] LSM: initializing lsm=capability,landlock,yama,bpf,ima799worker2 # [ 0.269968] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)800client # [ 0.282206] ... max period: 00007fffffffffff801worker1 # [ 0.273004] landlock: Up and running.802lb # [ 0.283581] ... generic bitmap: 000000000000003f803client # [ 0.283030] ... global_ctrl mask: 000000000000003f804worker1 # [ 0.273702] Yama: becoming mindful.805lb # [ 0.283882] ... fixed-purpose counters: 0806client # [ 0.283458] signal: max sigframe size: 3376807worker1 # [ 0.274418] LSM support for eBPF active808lb # [ 0.284588] ... fixed-purpose bitmap: 0000000000000000809worker2 # [ 0.272312] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)810client # [ 0.284219] rcu: Hierarchical SRCU implementation.811lb # [ 0.284882] ... value mask: 0000ffffffffffff812worker1 # [ 0.275325] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)813client # [ 0.284913] rcu: Max phase no-delay instances is 400.814worker2 # [ 0.273655] Performance Events: Fam17h+ core perfctr, AMD PMU driver.815lb # [ 0.285762] ... max period: 00007fffffffffff816worker1 # [ 0.276241] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)817worker2 # [ 0.274088] ... version: 2818lb # [ 0.286526] ... global_ctrl mask: 000000000000003f819worker2 # [ 0.274769] ... bit width: 48820lb # [ 0.286978] signal: max sigframe size: 3376821worker2 # [ 0.275082] ... generic counters: 6822lb # [ 0.287788] rcu: Hierarchical SRCU implementation.823worker2 # [ 0.275777] ... generic bitmap: 000000000000003f824worker1 # [ 0.278924] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)825lb # [ 0.288465] rcu: Max phase no-delay instances is 400.826worker2 # [ 0.276082] ... fixed-purpose counters: 0827worker2 # [ 0.276766] ... fixed-purpose bitmap: 0000000000000000828worker1 # [ 0.279794] Performance Events: Fam17h+ core perfctr, AMD PMU driver.829client # [ 0.289904] smp: Bringing up secondary CPUs ...830worker2 # [ 0.277082] ... value mask: 0000ffffffffffff831worker1 # [ 0.280199] ... version: 2832client # [ 0.290359] smp: Brought up 1 node, 1 CPU833worker1 # [ 0.280878] ... bit width: 48834worker2 # [ 0.277935] ... max period: 00007fffffffffff835worker1 # [ 0.281196] ... generic counters: 6836client # [ 0.291020] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)837worker2 # [ 0.278740] ... global_ctrl mask: 000000000000003f838worker1 # [ 0.281877] ... generic bitmap: 000000000000003f839worker2 # [ 0.279191] signal: max sigframe size: 3376840worker1 # [ 0.282208] ... fixed-purpose counters: 0841lb # [ 0.293369] smp: Bringing up secondary CPUs ...842worker2 # [ 0.279959] rcu: Hierarchical SRCU implementation.843client # [ 0.291536] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109352K reserved, 0K cma-reserved)844worker1 # [ 0.282880] ... fixed-purpose bitmap: 0000000000000000845lb # [ 0.293897] smp: Brought up 1 node, 1 CPU846worker2 # [ 0.280637] rcu: Max phase no-delay instances is 400.847client # [ 0.292591] devtmpfs: initialized848worker1 # [ 0.283196] ... value mask: 0000ffffffffffff849lb # [ 0.294524] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)850client # [ 0.293377] x86/mm: Memory block size: 128MB851worker1 # [ 0.284046] ... max period: 00007fffffffffff852worker1 # [ 0.284836] ... global_ctrl mask: 000000000000003f853worker1 # [ 0.285312] signal: max sigframe size: 3376854lb # [ 0.295009] Memory: 1454580K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)855client # [ 0.295153] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)856worker1 # [ 0.286078] rcu: Hierarchical SRCU implementation.857lb # [ 0.296101] devtmpfs: initialized858worker1 # [ 0.286764] rcu: Max phase no-delay instances is 400.859lb # [ 0.296976] x86/mm: Memory block size: 128MB860client # [ 0.296282] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).861worker2 # [ 0.285939] smp: Bringing up secondary CPUs ...862client # [ 0.297419] pinctrl core: initialized pinctrl subsystem863worker2 # [ 0.286639] smp: Brought up 1 node, 1 CPU864client # [ 0.298516] PM: RTC time: 10:55:11, date: 2026-09-22865lb # [ 0.298692] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)866worker2 # [ 0.287083] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)867lb # [ 0.299831] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).868worker2 # [ 0.288275] Memory: 1454580K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)869lb # [ 0.300966] pinctrl core: initialized pinctrl subsystem870worker1 # [ 0.291791] smp: Bringing up secondary CPUs ...871worker2 # [ 0.289323] devtmpfs: initialized872worker1 # [ 0.292210] smp: Brought up 1 node, 1 CPU873lb # [ 0.302220] PM: RTC time: 10:55:11, date: 2026-09-22874client # [ 0.302499] NET: Registered PF_NETLINK/PF_ROUTE protocol family875worker2 # [ 0.290107] x86/mm: Memory block size: 128MB876worker1 # [ 0.292867] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)877client # [ 0.303862] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations878worker2 # [ 0.291895] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)879worker1 # [ 0.293451] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)880client # [ 0.304369] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations881lb # [ 0.305620] NET: Registered PF_NETLINK/PF_ROUTE protocol family882worker1 # [ 0.294468] devtmpfs: initialized883worker2 # [ 0.292957] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).884client # [ 0.305626] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations885worker1 # [ 0.295271] x86/mm: Memory block size: 128MB886worker2 # [ 0.294169] pinctrl core: initialized pinctrl subsystem887lb # [ 0.306404] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations888client # [ 0.306355] audit: initializing netlink subsys (disabled)889worker2 # [ 0.295295] PM: RTC time: 10:55:11, date: 2026-09-22890lb # [ 0.306905] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations891client # [ 0.307535] thermal_sys: Registered thermal governor 'fair_share'892worker1 # [ 0.297036] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)893client # [ 0.307537] thermal_sys: Registered thermal governor 'bang_bang'894lb # [ 0.308491] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations895worker1 # [ 0.298064] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).896client # [ 0.308346] thermal_sys: Registered thermal governor 'step_wise'897lb # [ 0.308893] audit: initializing netlink subsys (disabled)898worker1 # [ 0.299273] pinctrl core: initialized pinctrl subsystem899client # [ 0.309304] thermal_sys: Registered thermal governor 'user_space'900worker2 # [ 0.298923] NET: Registered PF_NETLINK/PF_ROUTE protocol family901worker1 # [ 0.300301] PM: RTC time: 10:55:11, date: 2026-09-22902lb # [ 0.309985] thermal_sys: Registered thermal governor 'fair_share'903client # [ 0.310081] audit: type=2000 audit(1790074512.054:1): state=initialized audit_enabled=0 res=1904worker2 # [ 0.300303] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations905lb # [ 0.309987] thermal_sys: Registered thermal governor 'bang_bang'906client # [ 0.311347] thermal_sys: Registered thermal governor 'power_allocator'907lb # [ 0.310883] thermal_sys: Registered thermal governor 'step_wise'908worker2 # [ 0.301104] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations909client # [ 0.311361] cpuidle: using governor menu910lb # [ 0.311801] thermal_sys: Registered thermal governor 'user_space'911worker1 # [ 0.303983] NET: Registered PF_NETLINK/PF_ROUTE protocol family912worker2 # [ 0.302382] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations913client # [ 0.313618] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5914lb # [ 0.312613] audit: type=2000 audit(1790074512.055:1): state=initialized audit_enabled=0 res=1915worker2 # [ 0.303091] audit: initializing netlink subsys (disabled)916lb # [ 0.313885] thermal_sys: Registered thermal governor 'power_allocator'917worker1 # [ 0.305415] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations918client # [ 0.314597] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]919worker2 # [ 0.304196] thermal_sys: Registered thermal governor 'fair_share'920lb # [ 0.313898] cpuidle: using governor menu921worker1 # [ 0.306220] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations922client # [ 0.315348] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry923worker2 # [ 0.304198] thermal_sys: Registered thermal governor 'bang_bang'924lb # [ 0.316089] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5925client # [ 0.316356] PCI: Using configuration type 1 for base access926worker2 # [ 0.305083] thermal_sys: Registered thermal governor 'step_wise'927worker1 # [ 0.307487] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations928worker2 # [ 0.305983] thermal_sys: Registered thermal governor 'user_space'929worker1 # [ 0.308208] audit: initializing netlink subsys (disabled)930lb # [ 0.317146] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]931client # [ 0.317436] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.932worker2 # [ 0.306768] audit: type=2000 audit(1790074512.066:1): state=initialized audit_enabled=0 res=1933lb # [ 0.317885] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry934worker1 # [ 0.309293] thermal_sys: Registered thermal governor 'fair_share'935worker2 # [ 0.308085] thermal_sys: Registered thermal governor 'power_allocator'936lb # [ 0.318893] PCI: Using configuration type 1 for base access937worker1 # [ 0.309296] thermal_sys: Registered thermal governor 'bang_bang'938worker2 # [ 0.308100] cpuidle: using governor menu939worker1 # [ 0.310197] thermal_sys: Registered thermal governor 'step_wise'940lb # [ 0.320025] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.941worker1 # [ 0.311138] thermal_sys: Registered thermal governor 'user_space'942worker2 # [ 0.310294] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5943worker1 # [ 0.311895] audit: type=2000 audit(1790074512.066:1): state=initialized audit_enabled=0 res=1944client # [ 0.322479] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages945worker2 # [ 0.311333] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]946worker1 # [ 0.313198] thermal_sys: Registered thermal governor 'power_allocator'947client # [ 0.323346] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page948worker2 # [ 0.312085] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry949worker1 # [ 0.313211] cpuidle: using governor menu950worker2 # [ 0.313095] PCI: Using configuration type 1 for base access951worker1 # [ 0.315217] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5952lb # [ 0.325153] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages953worker2 # [ 0.314148] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.954worker1 # [ 0.316465] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]955lb # [ 0.325884] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page956worker1 # [ 0.317199] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry957client # [ 0.328346] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages958worker1 # [ 0.318209] PCI: Using configuration type 1 for base access959client # [ 0.329345] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page960worker1 # [ 0.319272] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.961worker2 # [ 0.319367] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages962lb # [ 0.330884] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages963worker2 # [ 0.320084] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page964lb # [ 0.331883] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page965worker2 # [ 0.325083] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages966worker1 # [ 0.326429] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages967worker2 # [ 0.326082] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page968worker1 # [ 0.327197] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page969client # [ 0.339740] ACPI: Added _OSI(Module Device)970client # [ 0.340346] ACPI: Added _OSI(Processor Device)971client # [ 0.341116] ACPI: Added _OSI(Processor Aggregator Device)972lb # [ 0.342265] ACPI: Added _OSI(Module Device)973worker1 # [ 0.328191] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages974lb # [ 0.342884] ACPI: Added _OSI(Processor Device)975worker1 # [ 0.333197] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page976lb # [ 0.343677] ACPI: Added _OSI(Processor Aggregator Device)977worker2 # [ 0.336478] ACPI: Added _OSI(Module Device)978worker2 # [ 0.337084] ACPI: Added _OSI(Processor Device)979worker2 # [ 0.337787] ACPI: Added _OSI(Processor Aggregator Device)980client # [ 0.349651] ACPI: 1 ACPI AML tables successfully acquired and loaded981worker1 # [ 0.341836] ACPI: Added _OSI(Module Device)982worker1 # [ 0.342198] ACPI: Added _OSI(Processor Device)983lb # [ 0.353122] ACPI: 1 ACPI AML tables successfully acquired and loaded984worker1 # [ 0.342900] ACPI: Added _OSI(Processor Aggregator Device)985client # [ 0.353752] ACPI: Interpreter enabled986client # [ 0.354361] ACPI: PM: (supports S0 S3 S4 S5)987client # [ 0.355169] ACPI: Using IOAPIC for interrupt routing988lb # [ 0.356859] ACPI: Interpreter enabled989lb # [ 0.357388] ACPI: PM: (supports S0 S3 S4 S5)990lb # [ 0.357883] ACPI: Using IOAPIC for interrupt routing991worker2 # [ 0.346652] ACPI: 1 ACPI AML tables successfully acquired and loaded992client # [ 0.358417] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug993lb # [ 0.358739] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug994client # [ 0.359345] PCI: Using E820 reservations for host bridge windows995worker1 # [ 0.349760] ACPI: 1 ACPI AML tables successfully acquired and loaded996worker2 # [ 0.350487] ACPI: Interpreter enabled997client # [ 0.362483] ACPI: Enabled 2 GPEs in block 00 to 3F998worker2 # [ 0.351098] ACPI: PM: (supports S0 S3 S4 S5)999worker2 # [ 0.351783] ACPI: Using IOAPIC for interrupt routing1000worker1 # [ 0.353170] ACPI: Interpreter enabled1001lb # [ 0.363883] PCI: Using E820 reservations for host bridge windows1002worker1 # [ 0.353639] ACPI: PM: (supports S0 S3 S4 S5)1003worker2 # [ 0.352143] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1004lb # [ 0.364999] ACPI: Enabled 2 GPEs in block 00 to 3F1005worker1 # [ 0.354197] ACPI: Using IOAPIC for interrupt routing1006worker2 # [ 0.353082] PCI: Using E820 reservations for host bridge windows1007worker1 # [ 0.355066] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1008worker2 # [ 0.356145] ACPI: Enabled 2 GPEs in block 00 to 3F1009client # [ 0.369725] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1010client # [ 0.370351] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1011worker1 # [ 0.360197] PCI: Using E820 reservations for host bridge windows1012worker1 # [ 0.361244] ACPI: Enabled 2 GPEs in block 00 to 3F1013client # [ 0.371424] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1014lb # [ 0.372656] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1015client # [ 0.372470] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1016lb # [ 0.373672] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1017client # [ 0.373705] PCI host bridge to bus 0000:001018lb # [ 0.374975] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1019client # [ 0.374349] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1020lb # [ 0.376009] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1021client # [ 0.375345] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1022worker2 # [ 0.364803] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1023lb # [ 0.377240] PCI host bridge to bus 0000:001024client # [ 0.376345] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1025worker2 # [ 0.365088] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1026lb # [ 0.377887] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1027client # [ 0.377345] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1028lb # [ 0.378883] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1029worker2 # [ 0.366642] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1030client # [ 0.378344] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1031worker1 # [ 0.369836] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1032lb # [ 0.379882] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1033worker2 # [ 0.367203] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1034client # [ 0.379350] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1035lb # [ 0.380882] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1036worker2 # [ 0.368452] PCI host bridge to bus 0000:001037client # [ 0.380345] pci_bus 0000:00: root bus resource [bus 00-ff]1038worker1 # [ 0.370789] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1039worker2 # [ 0.369087] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1040lb # [ 0.381882] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1041client # [ 0.381293] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1042worker1 # [ 0.371275] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1043worker2 # [ 0.370082] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1044lb # [ 0.382883] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1045worker1 # [ 0.372318] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1046lb # [ 0.383883] pci_bus 0000:00: root bus resource [bus 00-ff]1047worker2 # [ 0.371082] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1048client # [ 0.382794] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1049worker1 # [ 0.373572] PCI host bridge to bus 0000:001050worker2 # [ 0.372083] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1051lb # [ 0.384799] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1052worker1 # [ 0.374200] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1053worker2 # [ 0.373082] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1054worker1 # [ 0.375196] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1055lb # [ 0.386297] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1056worker2 # [ 0.374082] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1057worker1 # [ 0.376196] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1058worker2 # [ 0.375082] pci_bus 0000:00: root bus resource [bus 00-ff]1059worker1 # [ 0.377197] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1060worker2 # [ 0.376006] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1061worker1 # [ 0.378196] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1062worker1 # [ 0.379196] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1063worker2 # [ 0.377523] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1064worker1 # [ 0.380197] pci_bus 0000:00: root bus resource [bus 00-ff]1065client # [ 0.385410] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1066worker1 # [ 0.381109] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1067client # [ 0.386357] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1068client # [ 0.387331] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1069worker1 # [ 0.382600] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1070lb # [ 0.389063] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1071client # [ 0.388426] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1072lb # [ 0.389896] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1073lb # [ 0.390869] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1074client # [ 0.390069] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1075lb # [ 0.392204] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1076worker2 # [ 0.380166] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1077worker2 # [ 0.381095] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1078client # [ 0.393373] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1079worker2 # [ 0.382044] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1080lb # [ 0.393599] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1081client # [ 0.394190] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1082worker2 # [ 0.383035] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1083worker1 # [ 0.385335] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1084client # [ 0.395171] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1085worker1 # [ 0.386208] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1086lb # [ 0.395914] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1087client # [ 0.396144] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1088worker1 # [ 0.387122] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1089lb # [ 0.396786] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1090worker2 # [ 0.384802] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1091lb # [ 0.397607] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1092client # [ 0.397767] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1093worker1 # [ 0.388076] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1094lb # [ 0.397888] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1095worker2 # [ 0.388116] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1096client # [ 0.399372] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1097worker1 # [ 0.389955] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1098worker2 # [ 0.388964] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1099lb # [ 0.399766] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1100client # [ 0.400219] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1101worker2 # [ 0.389817] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1102client # [ 0.401073] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1103worker2 # [ 0.390088] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1104worker1 # [ 0.392224] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1105lb # [ 0.401924] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1106worker1 # [ 0.393043] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1107client # [ 0.401884] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1108lb # [ 0.402818] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1109worker1 # [ 0.393913] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1110worker2 # [ 0.391807] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1111lb # [ 0.403636] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1112worker1 # [ 0.394202] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1113lb # [ 0.404439] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1114client # [ 0.404458] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1115worker2 # [ 0.395112] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1116worker1 # [ 0.395895] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1117client # [ 0.405351] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1118worker2 # [ 0.395930] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1119client # [ 0.406291] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1120worker2 # [ 0.396787] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1121worker1 # [ 0.398224] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1122worker1 # [ 0.399066] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1123lb # [ 0.407473] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1124client # [ 0.407738] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125worker2 # [ 0.397654] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1126lb # [ 0.407889] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1127worker1 # [ 0.399909] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1128client # [ 0.409373] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1129lb # [ 0.408905] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1130worker1 # [ 0.400820] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1131client # [ 0.410240] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1132lb # [ 0.410467] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1133client # [ 0.411116] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1134worker2 # [ 0.400370] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1135worker2 # [ 0.401706] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1136lb # [ 0.411914] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1137client # [ 0.412877] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1138worker1 # [ 0.403225] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1139worker2 # [ 0.402104] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1140lb # [ 0.412761] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1141worker1 # [ 0.404203] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1142lb # [ 0.413598] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1143client # [ 0.414352] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1144worker1 # [ 0.405144] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1145worker2 # [ 0.403649] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1146client # [ 0.415194] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1147client # [ 0.416118] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1148lb # [ 0.414416] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1149worker1 # [ 0.406544] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150worker2 # [ 0.405830] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1151client # [ 0.417784] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1152worker2 # [ 0.406089] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1153lb # [ 0.416890] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1154worker1 # [ 0.408225] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1155worker2 # [ 0.407038] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1156lb # [ 0.417810] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1157worker1 # [ 0.409134] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158lb # [ 0.418629] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1159worker1 # [ 0.409926] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1160client # [ 0.420372] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1161worker2 # [ 0.408477] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1162lb # [ 0.419484] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1163client # [ 0.421353] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1164worker1 # [ 0.410747] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1165client # [ 0.422320] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1166worker2 # [ 0.411879] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1167worker1 # [ 0.412231] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1168lb # [ 0.421844] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1169worker2 # [ 0.412695] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1170client # [ 0.423797] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1171worker1 # [ 0.413075] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1172worker2 # [ 0.413104] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1173lb # [ 0.422532] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1174worker1 # [ 0.413970] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1175client # [ 0.425383] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1176lb # [ 0.423659] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1177client # [ 0.426237] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1178worker2 # [ 0.414667] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1179worker1 # [ 0.415605] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1180client # [ 0.427060] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1181lb # [ 0.425336] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1182client # [ 0.427350] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1183lb # [ 0.426920] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1184worker1 # [ 0.418189] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1185worker2 # [ 0.417115] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1186lb # [ 0.427805] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1187client # [ 0.428933] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1188worker2 # [ 0.418089] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1189worker1 # [ 0.419028] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1190lb # [ 0.428630] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1191worker2 # [ 0.419023] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1192worker1 # [ 0.420218] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1193client # [ 0.431191] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1194lb # [ 0.428889] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1195client # [ 0.432068] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1196worker2 # [ 0.420477] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1197worker1 # [ 0.421798] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1198lb # [ 0.430398] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1199client # [ 0.432907] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1200worker1 # [ 0.423204] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1201worker2 # [ 0.422832] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1202lb # [ 0.431654] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1203worker1 # [ 0.424049] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1204worker2 # [ 0.423089] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1205client # [ 0.434353] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1206lb # [ 0.431905] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1207worker1 # [ 0.424915] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1208client # [ 0.435186] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1209worker2 # [ 0.424025] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1210worker1 # [ 0.425202] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1211worker2 # [ 0.424905] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1212lb # [ 0.433447] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1213client # [ 0.436078] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1214worker1 # [ 0.426679] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1215lb # [ 0.434890] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1216client # [ 0.436940] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1217worker2 # [ 0.426978] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1218lb # [ 0.435709] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1219worker1 # [ 0.428072] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1220client # [ 0.438373] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1221lb # [ 0.436626] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1222worker1 # [ 0.428930] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1223worker2 # [ 0.429095] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1224client # [ 0.439213] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1225worker2 # [ 0.430034] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1226worker1 # [ 0.429748] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1227client # [ 0.440128] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1228lb # [ 0.438691] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1229worker2 # [ 0.431460] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1230client # [ 0.441782] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1231worker1 # [ 0.431205] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1232lb # [ 0.440917] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1233worker1 # [ 0.432036] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1234lb # [ 0.441794] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1235client # [ 0.442920] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1236worker2 # [ 0.433121] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1237worker1 # [ 0.432923] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1238lb # [ 0.442628] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1239worker2 # [ 0.433968] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1240client # [ 0.443542] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1241lb # [ 0.443434] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1242worker2 # [ 0.435104] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1243worker1 # [ 0.433765] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1244client # [ 0.445908] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1245lb # [ 0.444407] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1246worker2 # [ 0.436639] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1247worker1 # [ 0.436224] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1248worker1 # [ 0.437067] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1249lb # [ 0.445079] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1250worker1 # [ 0.437923] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1251worker2 # [ 0.438089] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1252lb # [ 0.446441] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1253worker2 # [ 0.438922] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1254worker1 # [ 0.438760] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1255worker2 # [ 0.439810] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1256worker1 # [ 0.439797] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1257worker2 # [ 0.440670] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1258worker1 # [ 0.440397] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1259worker2 # [ 0.442553] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1260worker1 # [ 0.442245] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1261worker2 # [ 0.443282] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1262client # [ 0.446547] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1263worker2 # [ 0.444772] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1264client # [ 0.448893] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1265client # [ 0.449616] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1266client # [ 0.450879] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1267lb # [ 0.447082] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1268lb # [ 0.449451] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1269client # [ 0.451862] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1270lb # [ 0.450115] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1271client # [ 0.452678] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1272lb # [ 0.451389] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1273client # [ 0.453609] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1274worker1 # [ 0.443305] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1275client # [ 0.455165] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1276lb # [ 0.452488] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1277worker1 # [ 0.445745] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1278client # [ 0.455998] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1279lb # [ 0.453168] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1280worker2 # [ 0.445280] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1281worker1 # [ 0.446434] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1282client # [ 0.456663] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1283lb # [ 0.454128] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1284worker2 # [ 0.447842] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1285worker1 # [ 0.447705] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1286client # [ 0.457902] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1287lb # [ 0.455690] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1288worker2 # [ 0.448331] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1289worker1 # [ 0.448466] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1290lb # [ 0.456521] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1291worker2 # [ 0.449749] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1292worker1 # [ 0.449483] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1293lb # [ 0.457198] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1294client # [ 0.461324] ACPI: PCI: Interrupt link LNKA configured for IRQ 101295worker1 # [ 0.450448] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1296worker2 # [ 0.450355] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1297lb # [ 0.458428] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1298client # [ 0.462178] ACPI: PCI: Interrupt link LNKB configured for IRQ 101299worker1 # [ 0.452021] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1300worker2 # [ 0.451397] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1301client # [ 0.463201] ACPI: PCI: Interrupt link LNKC configured for IRQ 111302worker1 # [ 0.452819] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1303client # [ 0.464172] ACPI: PCI: Interrupt link LNKD configured for IRQ 111304worker2 # [ 0.452340] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1305client # [ 0.465191] ACPI: PCI: Interrupt link LNKE configured for IRQ 101306worker1 # [ 0.453520] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1307lb # [ 0.462098] ACPI: PCI: Interrupt link LNKA configured for IRQ 101308client # [ 0.466200] ACPI: PCI: Interrupt link LNKF configured for IRQ 101309worker2 # [ 0.454133] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1310worker1 # [ 0.454786] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1311lb # [ 0.463004] ACPI: PCI: Interrupt link LNKB configured for IRQ 101312client # [ 0.467182] ACPI: PCI: Interrupt link LNKG configured for IRQ 111313worker2 # [ 0.454992] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1314lb # [ 0.463999] ACPI: PCI: Interrupt link LNKC configured for IRQ 111315client # [ 0.468328] ACPI: PCI: Interrupt link LNKH configured for IRQ 111316lb # [ 0.464979] ACPI: PCI: Interrupt link LNKD configured for IRQ 111317client # [ 0.469227] ACPI: PCI: Interrupt link GSIA configured for IRQ 161318worker2 # [ 0.456143] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1319lb # [ 0.465979] ACPI: PCI: Interrupt link LNKE configured for IRQ 101320client # [ 0.470094] ACPI: PCI: Interrupt link GSIB configured for IRQ 171321worker1 # [ 0.458495] ACPI: PCI: Interrupt link LNKA configured for IRQ 101322lb # [ 0.466999] ACPI: PCI: Interrupt link LNKF configured for IRQ 101323worker2 # [ 0.458884] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1324client # [ 0.470359] ACPI: PCI: Interrupt link GSIC configured for IRQ 181325worker1 # [ 0.459303] ACPI: PCI: Interrupt link LNKB configured for IRQ 101326client # [ 0.471297] ACPI: PCI: Interrupt link GSID configured for IRQ 191327lb # [ 0.467983] ACPI: PCI: Interrupt link LNKG configured for IRQ 111328worker1 # [ 0.460298] ACPI: PCI: Interrupt link LNKC configured for IRQ 111329client # [ 0.472108] ACPI: PCI: Interrupt link GSIE configured for IRQ 201330lb # [ 0.468999] ACPI: PCI: Interrupt link LNKH configured for IRQ 111331worker1 # [ 0.461292] ACPI: PCI: Interrupt link LNKD configured for IRQ 111332client # [ 0.472357] ACPI: PCI: Interrupt link GSIF configured for IRQ 211333lb # [ 0.469921] ACPI: PCI: Interrupt link GSIA configured for IRQ 161334worker1 # [ 0.462297] ACPI: PCI: Interrupt link LNKE configured for IRQ 101335client # [ 0.473312] ACPI: PCI: Interrupt link GSIG configured for IRQ 221336worker2 # [ 0.462735] ACPI: PCI: Interrupt link LNKA configured for IRQ 101337lb # [ 0.470898] ACPI: PCI: Interrupt link GSIB configured for IRQ 171338client # [ 0.474056] ACPI: PCI: Interrupt link GSIH configured for IRQ 231339worker1 # [ 0.463295] ACPI: PCI: Interrupt link LNKF configured for IRQ 101340worker2 # [ 0.463187] ACPI: PCI: Interrupt link LNKB configured for IRQ 101341lb # [ 0.471846] ACPI: PCI: Interrupt link GSIC configured for IRQ 181342worker1 # [ 0.464295] ACPI: PCI: Interrupt link LNKG configured for IRQ 111343worker2 # [ 0.464185] ACPI: PCI: Interrupt link LNKC configured for IRQ 111344lb # [ 0.472623] ACPI: PCI: Interrupt link GSID configured for IRQ 191345client # [ 0.475479] iommu: Default domain type: Translated1346worker1 # [ 0.465293] ACPI: PCI: Interrupt link LNKH configured for IRQ 111347worker2 # [ 0.465195] ACPI: PCI: Interrupt link LNKD configured for IRQ 111348lb # [ 0.472898] ACPI: PCI: Interrupt link GSIE configured for IRQ 201349client # [ 0.476270] iommu: DMA domain TLB invalidation policy: lazy mode1350worker1 # [ 0.466229] ACPI: PCI: Interrupt link GSIA configured for IRQ 161351lb # [ 0.473842] ACPI: PCI: Interrupt link GSIF configured for IRQ 211352worker2 # [ 0.466182] ACPI: PCI: Interrupt link LNKE configured for IRQ 101353client # [ 0.477395] ACPI: bus type USB registered1354worker1 # [ 0.467215] ACPI: PCI: Interrupt link GSIB configured for IRQ 171355lb # [ 0.474637] ACPI: PCI: Interrupt link GSIG configured for IRQ 221356worker2 # [ 0.467184] ACPI: PCI: Interrupt link LNKF configured for IRQ 101357client # [ 0.478156] usbcore: registered new interface driver usbfs1358worker1 # [ 0.468202] ACPI: PCI: Interrupt link GSIC configured for IRQ 181359lb # [ 0.474901] ACPI: PCI: Interrupt link GSIH configured for IRQ 231360worker2 # [ 0.468192] ACPI: PCI: Interrupt link LNKG configured for IRQ 111361client # [ 0.479042] usbcore: registered new interface driver hub1362worker1 # [ 0.469163] ACPI: PCI: Interrupt link GSID configured for IRQ 191363client # [ 0.479364] usbcore: registered new device driver usb1364worker2 # [ 0.469188] ACPI: PCI: Interrupt link LNKH configured for IRQ 111365worker1 # [ 0.469934] ACPI: PCI: Interrupt link GSIE configured for IRQ 201366lb # [ 0.476868] iommu: Default domain type: Translated1367worker1 # [ 0.470212] ACPI: PCI: Interrupt link GSIF configured for IRQ 211368worker2 # [ 0.470118] ACPI: PCI: Interrupt link GSIA configured for IRQ 161369lb # [ 0.477479] iommu: DMA domain TLB invalidation policy: lazy mode1370client # [ 0.481032] NetLabel: Initializing1371client # [ 0.481346] NetLabel: domain hash size = 1281372worker1 # [ 0.471145] ACPI: PCI: Interrupt link GSIG configured for IRQ 221373worker2 # [ 0.471094] ACPI: PCI: Interrupt link GSIB configured for IRQ 171374lb # [ 0.478135] ACPI: bus type USB registered1375client # [ 0.482046] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1376worker2 # [ 0.472045] ACPI: PCI: Interrupt link GSIC configured for IRQ 181377worker1 # [ 0.471913] ACPI: PCI: Interrupt link GSIH configured for IRQ 231378lb # [ 0.478868] usbcore: registered new interface driver usbfs1379client # [ 0.482395] NetLabel: unlabeled traffic allowed by default1380worker2 # [ 0.472806] ACPI: PCI: Interrupt link GSID configured for IRQ 191381lb # [ 0.479559] usbcore: registered new interface driver hub1382client # [ 0.483253] PCI: Using ACPI for IRQ routing1383lb # [ 0.479899] usbcore: registered new device driver usb1384worker2 # [ 0.473097] ACPI: PCI: Interrupt link GSIE configured for IRQ 201385worker1 # [ 0.473305] iommu: Default domain type: Translated1386worker2 # [ 0.474036] ACPI: PCI: Interrupt link GSIF configured for IRQ 211387worker1 # [ 0.474076] iommu: DMA domain TLB invalidation policy: lazy mode1388lb # [ 0.481562] NetLabel: Initializing1389worker1 # [ 0.475231] ACPI: bus type USB registered1390lb # [ 0.481884] NetLabel: domain hash size = 1281391worker2 # [ 0.474854] ACPI: PCI: Interrupt link GSIG configured for IRQ 221392worker1 # [ 0.475956] usbcore: registered new interface driver usbfs1393lb # [ 0.482584] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1394worker2 # [ 0.476020] ACPI: PCI: Interrupt link GSIH configured for IRQ 231395worker1 # [ 0.476214] usbcore: registered new interface driver hub1396lb # [ 0.482965] NetLabel: unlabeled traffic allowed by default1397worker1 # [ 0.477068] usbcore: registered new device driver usb1398lb # [ 0.483870] PCI: Using ACPI for IRQ routing1399worker2 # [ 0.477947] iommu: Default domain type: Translated1400worker1 # [ 0.478677] NetLabel: Initializing1401worker2 # [ 0.478699] iommu: DMA domain TLB invalidation policy: lazy mode1402worker1 # [ 0.479197] NetLabel: domain hash size = 1281403worker2 # [ 0.479362] ACPI: bus type USB registered1404worker1 # [ 0.479922] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1405worker2 # [ 0.480130] usbcore: registered new interface driver usbfs1406worker1 # [ 0.480260] NetLabel: unlabeled traffic allowed by default1407worker2 # [ 0.481238] usbcore: registered new interface driver hub1408worker1 # [ 0.481130] PCI: Using ACPI for IRQ routing1409worker2 # [ 0.482093] usbcore: registered new device driver usb1410worker2 # [ 0.483893] NetLabel: Initializing1411worker2 # [ 0.484652] NetLabel: domain hash size = 1281412worker2 # [ 0.485082] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413worker2 # [ 0.486115] NetLabel: unlabeled traffic allowed by default1414worker2 # [ 0.487094] PCI: Using ACPI for IRQ routing1415client # [ 0.527389] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416client # [ 0.528341] pci 0000:00:01.0: vgaarb: bridge control possible1417client # [ 0.528341] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418client # [ 0.528351] vgaarb: loaded1419client # [ 0.529104] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420client # [ 0.529346] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421lb # [ 0.528263] pci 0000:00:01.0: vgaarb: setting as boot VGA device1422lb # [ 0.528879] pci 0000:00:01.0: vgaarb: bridge control possible1423lb # [ 0.528879] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1424lb # [ 0.528888] vgaarb: loaded1425worker1 # [ 0.525338] pci 0000:00:01.0: vgaarb: setting as boot VGA device1426lb # [ 0.529631] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01427worker1 # [ 0.526192] pci 0000:00:01.0: vgaarb: bridge control possible1428client # [ 0.534434] clocksource: Switched to clocksource kvm-clock1429lb # [ 0.529884] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1430worker1 # [ 0.526192] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1431worker1 # [ 0.526200] vgaarb: loaded1432worker1 # [ 0.526940] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01433client # [ 0.537992] VFS: Disk quotas dquot_6.6.01434worker1 # [ 0.527198] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1435lb # [ 0.532971] clocksource: Switched to clocksource kvm-clock1436client # [ 0.538763] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1437client # [ 0.540164] pnp: PnP ACPI init1438worker2 # [ 0.532054] pci 0000:00:01.0: vgaarb: setting as boot VGA device1439client # [ 0.540996] ACPI: IRQ 4 override to edge(!), high(!)1440worker2 # [ 0.532078] pci 0000:00:01.0: vgaarb: bridge control possible1441worker1 # [ 0.531284] clocksource: Switched to clocksource kvm-clock1442lb # [ 0.536546] VFS: Disk quotas dquot_6.6.01443client # [ 0.542004] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1444worker2 # [ 0.532078] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1445worker2 # [ 0.532086] vgaarb: loaded1446lb # [ 0.537279] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1447client # [ 0.543540] pnp: PnP ACPI: found 6 devices1448worker1 # [ 0.533127] VFS: Disk quotas dquot_6.6.01449lb # [ 0.538598] pnp: PnP ACPI init1450worker2 # [ 0.532791] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01451lb # [ 0.539420] ACPI: IRQ 4 override to edge(!), high(!)1452worker2 # [ 0.533082] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1453worker1 # [ 0.535648] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1454lb # [ 0.540434] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1455worker1 # [ 0.537189] pnp: PnP ACPI init1456lb # [ 0.541989] pnp: PnP ACPI: found 6 devices1457worker1 # [ 0.538046] ACPI: IRQ 4 override to edge(!), high(!)1458worker1 # [ 0.539144] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1459worker2 # [ 0.537172] clocksource: Switched to clocksource kvm-clock1460worker1 # [ 0.540770] pnp: PnP ACPI: found 6 devices1461client # [ 0.551125] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1462client # [ 0.552678] clocksource: Switched to clocksource acpi_pm1463client # [ 0.553748] NET: Registered PF_INET protocol family1464worker2 # [ 0.541017] VFS: Disk quotas dquot_6.6.01465client # [ 0.554940] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1466worker2 # [ 0.541812] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1467lb # [ 0.549615] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1468worker2 # [ 0.543148] pnp: PnP ACPI init1469lb # [ 0.551099] clocksource: Switched to clocksource acpi_pm1470worker2 # [ 0.543984] ACPI: IRQ 4 override to edge(!), high(!)1471lb # [ 0.552121] NET: Registered PF_INET protocol family1472worker2 # [ 0.544993] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1473lb # [ 0.553361] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1474worker1 # [ 0.548427] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1475worker2 # [ 0.546615] pnp: PnP ACPI: found 6 devices1476worker1 # [ 0.549936] clocksource: Switched to clocksource acpi_pm1477worker1 # [ 0.550974] NET: Registered PF_INET protocol family1478worker1 # [ 0.552190] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1479worker2 # [ 0.554210] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1480worker2 # [ 0.555734] clocksource: Switched to clocksource acpi_pm1481worker2 # [ 0.556736] NET: Registered PF_INET protocol family1482worker2 # [ 0.557963] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1483client # [ 0.571575] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1484client # [ 0.573086] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1485client # [ 0.574514] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1486client # [ 0.575908] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1487lb # [ 0.570310] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1488client # [ 0.577179] TCP: Hash tables configured (established 16384 bind 16384)1489lb # [ 0.571762] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1490client # [ 0.578284] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1491lb # [ 0.573077] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1492client # [ 0.579575] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1493lb # [ 0.574424] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1494client # [ 0.580663] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1495worker1 # [ 0.569668] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1496lb # [ 0.575631] TCP: Hash tables configured (established 16384 bind 16384)1497client # [ 0.581834] NET: Registered PF_UNIX/PF_LOCAL protocol family1498worker1 # [ 0.571267] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1499client # [ 0.582783] NET: Registered PF_XDP protocol family1500lb # [ 0.576710] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1501worker1 # [ 0.572564] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1502client # [ 0.583614] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1503lb # [ 0.577931] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1504client # [ 0.584621] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1505worker1 # [ 0.573889] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1506lb # [ 0.579000] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1507client # [ 0.585671] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1508worker1 # [ 0.575100] TCP: Hash tables configured (established 16384 bind 16384)1509lb # [ 0.580206] NET: Registered PF_UNIX/PF_LOCAL protocol family1510client # [ 0.586758] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1511lb # [ 0.581137] NET: Registered PF_XDP protocol family1512worker1 # [ 0.576281] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1513client # [ 0.587835] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1514lb # [ 0.582012] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1515worker2 # [ 0.574443] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1516worker1 # [ 0.577614] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1517lb # [ 0.583028] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1518client # [ 0.588891] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1519worker1 # [ 0.578727] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1520worker2 # [ 0.575951] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1521lb # [ 0.584047] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1522client # [ 0.590613] ACPI: \_SB_.GSIA: Enabled at IRQ 161523worker1 # [ 0.579919] NET: Registered PF_UNIX/PF_LOCAL protocol family1524worker2 # [ 0.577222] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1525lb # [ 0.585136] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1526worker1 # [ 0.580847] NET: Registered PF_XDP protocol family1527worker2 # [ 0.578526] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1528lb # [ 0.586234] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1529worker1 # [ 0.581686] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1530client # [ 0.592779] ACPI: \_SB_.GSIB: Enabled at IRQ 171531worker2 # [ 0.579747] TCP: Hash tables configured (established 16384 bind 16384)1532worker1 # [ 0.582661] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1533lb # [ 0.587385] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1534worker2 # [ 0.580823] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1535worker1 # [ 0.583639] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1536lb # [ 0.589044] ACPI: \_SB_.GSIA: Enabled at IRQ 161537client # [ 0.595062] ACPI: \_SB_.GSIC: Enabled at IRQ 181538worker1 # [ 0.584708] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1539worker2 # [ 0.582066] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1540worker1 # [ 0.585770] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1541worker2 # [ 0.583158] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1542client # [ 0.597031] ACPI: \_SB_.GSID: Enabled at IRQ 191543lb # [ 0.591659] ACPI: \_SB_.GSIB: Enabled at IRQ 171544worker1 # [ 0.586846] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1545worker2 # [ 0.584381] NET: Registered PF_UNIX/PF_LOCAL protocol family1546worker2 # [ 0.585314] NET: Registered PF_XDP protocol family1547client # [ 0.598879] PCI: CLS 0 bytes, default 641548worker1 # [ 0.588554] ACPI: \_SB_.GSIA: Enabled at IRQ 161549lb # [ 0.593617] ACPI: \_SB_.GSIC: Enabled at IRQ 181550worker2 # [ 0.586193] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1551client # [ 0.599802] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1552worker2 # [ 0.587203] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1553lb # [ 0.595506] ACPI: \_SB_.GSID: Enabled at IRQ 191554worker1 # [ 0.590790] ACPI: \_SB_.GSIB: Enabled at IRQ 171555client # [ 0.601553] Trying to unpack rootfs image as initramfs...1556worker2 # [ 0.588216] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1557lb # [ 0.597144] PCI: CLS 0 bytes, default 641558worker2 # [ 0.589302] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1559worker1 # [ 0.593016] ACPI: \_SB_.GSIC: Enabled at IRQ 181560worker2 # [ 0.590481] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1561lb # [ 0.598080] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1562worker2 # [ 0.591606] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1563worker1 # [ 0.595049] ACPI: \_SB_.GSID: Enabled at IRQ 191564lb # [ 0.599898] Trying to unpack rootfs image as initramfs...1565worker2 # [ 0.593298] ACPI: \_SB_.GSIA: Enabled at IRQ 161566worker1 # [ 0.596841] PCI: CLS 0 bytes, default 641567worker1 # [ 0.597846] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1568worker2 # [ 0.596019] ACPI: \_SB_.GSIB: Enabled at IRQ 171569worker1 # [ 0.599610] Trying to unpack rootfs image as initramfs...1570worker2 # [ 0.597977] ACPI: \_SB_.GSIC: Enabled at IRQ 181571worker2 # [ 0.599972] ACPI: \_SB_.GSID: Enabled at IRQ 191572worker2 # [ 0.601737] PCI: CLS 0 bytes, default 641573worker2 # [ 0.602634] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574worker2 # [ 0.604412] Trying to unpack rootfs image as initramfs...1575client # [ 0.648790] Initialise system trusted keyrings1576client # [ 0.649760] workingset: timestamp_bits=40 max_order=19 bucket_order=01577lb # [ 0.644600] Initialise system trusted keyrings1578lb # [ 0.645533] workingset: timestamp_bits=40 max_order=19 bucket_order=01579worker1 # [ 0.644864] Initialise system trusted keyrings1580worker1 # [ 0.645855] workingset: timestamp_bits=40 max_order=19 bucket_order=01581worker2 # [ 0.650613] Initialise system trusted keyrings1582worker2 # [ 0.653528] workingset: timestamp_bits=40 max_order=19 bucket_order=01583client # [ 0.673403] Key type asymmetric registered1584client # [ 0.674090] Asymmetric key parser 'x509' registered1585lb # [ 0.669260] Key type asymmetric registered1586client # [ 0.674933] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1587lb # [ 0.669970] Asymmetric key parser 'x509' registered1588lb # [ 0.670858] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1589client # [ 0.677544] io scheduler mq-deadline registered1590client # [ 0.678264] io scheduler kyber registered1591worker1 # [ 0.667784] Key type asymmetric registered1592worker1 # [ 0.670433] Asymmetric key parser 'x509' registered1593client # [ 0.680959] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1594lb # [ 0.675319] io scheduler mq-deadline registered1595lb # [ 0.676073] io scheduler kyber registered1596worker1 # [ 0.671246] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1597client # [ 0.682183] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1598worker1 # [ 0.674548] io scheduler mq-deadline registered1599worker1 # [ 0.675329] io scheduler kyber registered1600lb # [ 0.680347] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1601client # [ 0.687523] Linux agpgart interface v0.1031602lb # [ 0.681645] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1603client # [ 0.688236] ACPI: bus type drm_connector registered1604worker2 # [ 0.675822] Key type asymmetric registered1605worker2 # [ 0.676562] Asymmetric key parser 'x509' registered1606worker1 # [ 0.679593] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1607worker2 # [ 0.677365] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1608client # [ 0.691557] usbcore: registered new interface driver usbserial_generic1609worker1 # [ 0.680853] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1610client # [ 0.692611] usbserial: USB Serial support registered for generic1611lb # [ 0.686989] Linux agpgart interface v0.1031612lb # [ 0.687766] ACPI: bus type drm_connector registered1613worker2 # [ 0.680619] io scheduler mq-deadline registered1614worker2 # [ 0.681351] io scheduler kyber registered1615client # [ 0.695391] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1616worker1 # [ 0.686181] Linux agpgart interface v0.1031617client # [ 0.695391] Please enable it if your BIOS has the CPPC option.1618lb # [ 0.690663] usbcore: registered new interface driver usbserial_generic1619worker1 # [ 0.686923] ACPI: bus type drm_connector registered1620lb # [ 0.691706] usbserial: USB Serial support registered for generic1621worker2 # [ 0.684231] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1622client # [ 0.697579] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1623client # [ 0.700535] drop_monitor: Initializing network drop monitor service1624worker1 # [ 0.689915] usbcore: registered new interface driver usbserial_generic1625worker2 # [ 0.687490] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626client # [ 0.701713] NET: Registered PF_INET6 protocol family1627worker1 # [ 0.690967] usbserial: USB Serial support registered for generic1628lb # [ 0.695198] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1629lb # [ 0.695198] Please enable it if your BIOS has the CPPC option.1630lb # [ 0.697490] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1631worker2 # [ 0.691147] Linux agpgart interface v0.1031632worker2 # [ 0.691906] ACPI: bus type drm_connector registered1633worker1 # [ 0.694428] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1634client # [ 0.706523] Segment Routing with IPv61635worker1 # [ 0.694428] Please enable it if your BIOS has the CPPC option.1636client # [ 0.707199] In-situ OAM (IOAM) with IPv61637lb # [ 0.701353] drop_monitor: Initializing network drop monitor service1638worker1 # [ 0.696691] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1639lb # [ 0.702536] NET: Registered PF_INET6 protocol family1640worker2 # [ 0.694909] usbcore: registered new interface driver usbserial_generic1641client # [ 0.709536] IPI shorthand broadcast: enabled1642worker2 # [ 0.695948] usbserial: USB Serial support registered for generic1643lb # [ 0.705632] Segment Routing with IPv61644worker1 # [ 0.700542] drop_monitor: Initializing network drop monitor service1645worker1 # [ 0.701769] NET: Registered PF_INET6 protocol family1646worker2 # [ 0.699492] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1647lb # [ 0.708210] In-situ OAM (IOAM) with IPv61648worker2 # [ 0.699492] Please enable it if your BIOS has the CPPC option.1649lb # [ 0.709262] IPI shorthand broadcast: enabled1650worker2 # [ 0.701622] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1651worker1 # [ 0.704902] Segment Routing with IPv61652worker1 # [ 0.707444] In-situ OAM (IOAM) with IPv61653worker2 # [ 0.704625] drop_monitor: Initializing network drop monitor service1654client # [ 0.717805] sched_clock: Marking stable (584016184, 133345863)->(789328523, -71966476)1655worker1 # [ 0.708465] IPI shorthand broadcast: enabled1656worker2 # [ 0.705760] NET: Registered PF_INET6 protocol family1657worker2 # [ 0.708925] Segment Routing with IPv61658client # [ 0.722461] registered taskstats version 11659client # [ 0.723403] Loading compiled-in X.509 certificates1660lb # [ 0.717470] sched_clock: Marking stable (581013862, 136167899)->(793261873, -76080112)1661worker2 # [ 0.711506] In-situ OAM (IOAM) with IPv61662worker2 # [ 0.712516] IPI shorthand broadcast: enabled1663worker1 # [ 0.716730] sched_clock: Marking stable (581015811, 135391303)->(786242759, -69835645)1664lb # [ 0.722436] registered taskstats version 11665lb # [ 0.723414] Loading compiled-in X.509 certificates1666worker1 # [ 0.720637] registered taskstats version 11667worker1 # [ 0.721592] Loading compiled-in X.509 certificates1668worker2 # [ 0.720620] sched_clock: Marking stable (588014037, 132460758)->(790977715, -70502920)1669worker2 # [ 0.725711] registered taskstats version 11670client # [ 0.739882] Demotion targets for Node 0: null1671worker2 # [ 0.726632] Loading compiled-in X.509 certificates1672client # [ 0.743412] Key type .fscrypt registered1673client # [ 0.744117] Key type fscrypt-provisioning registered1674client # [ 0.745093] ima: No TPM chip found, activating TPM-bypass!1675client # [ 0.747381] ima: Allocated hash algorithm: sha11676lb # [ 0.742205] Demotion targets for Node 0: null1677client # [ 0.748189] ima: No architecture policies found1678lb # [ 0.743082] Key type .fscrypt registered1679lb # [ 0.743806] Key type fscrypt-provisioning registered1680worker1 # [ 0.739421] Demotion targets for Node 0: null1681client # [ 0.750373] PM: Magic number: 10:622:9321682client # [ 0.752058] RAS: Correctable Errors collector initialized.1683lb # [ 0.746306] ima: No TPM chip found, activating TPM-bypass!1684worker1 # [ 0.742462] Key type .fscrypt registered1685lb # [ 0.747222] ima: Allocated hash algorithm: sha11686worker1 # [ 0.743145] Key type fscrypt-provisioning registered1687lb # [ 0.748034] ima: No architecture policies found1688worker1 # [ 0.744104] ima: No TPM chip found, activating TPM-bypass!1689worker2 # [ 0.743031] Demotion targets for Node 0: null1690worker1 # [ 0.746426] ima: Allocated hash algorithm: sha11691worker1 # [ 0.747238] ima: No architecture policies found1692lb # [ 0.752447] PM: Magic number: 10:622:9321693worker2 # [ 0.745530] Key type .fscrypt registered1694worker2 # [ 0.746177] Key type fscrypt-provisioning registered1695lb # [ 0.754080] RAS: Correctable Errors collector initialized.1696worker2 # [ 0.747083] ima: No TPM chip found, activating TPM-bypass!1697client # [ 0.762087] clk: Disabling unused clocks1698worker1 # [ 0.751422] PM: Magic number: 10:622:9321699worker2 # [ 0.749499] ima: Allocated hash algorithm: sha11700worker2 # [ 0.750263] ima: No architecture policies found1701worker1 # [ 0.753098] RAS: Correctable Errors collector initialized.1702client # [ 0.766388] PM: genpd: Disabling unused power domains1703lb # [ 0.761881] clk: Disabling unused clocks1704worker2 # [ 0.754489] PM: Magic number: 10:622:9321705worker2 # [ 0.756033] RAS: Correctable Errors collector initialized.1706lb # [ 0.764198] PM: genpd: Disabling unused power domains1707worker1 # [ 0.765097] clk: Disabling unused clocks1708worker1 # [ 0.765908] PM: genpd: Disabling unused power domains1709worker2 # [ 0.764129] clk: Disabling unused clocks1710worker2 # [ 0.766493] PM: genpd: Disabling unused power domains1711client # [ 0.934933] Freeing initrd memory: 29096K1712lb # [ 0.929503] Freeing initrd memory: 29092K1713client # [ 0.938219] Freeing unused decrypted memory: 2028K1714lb # [ 0.932657] Freeing unused decrypted memory: 2028K1715client # [ 0.940930] Freeing unused kernel image (initmem) memory: 3644K1716lb # [ 0.935291] Freeing unused kernel image (initmem) memory: 3644K1717client # [ 0.942027] Write protecting the kernel read-only data: 32768k1718lb # [ 0.936373] Write protecting the kernel read-only data: 32768k1719client # [ 0.943925] Freeing unused kernel image (text/rodata gap) memory: 1208K1720lb # [ 0.938257] Freeing unused kernel image (text/rodata gap) memory: 1208K1721worker2 # [ 0.931211] Freeing initrd memory: 29092K1722client # [ 0.945501] Freeing unused kernel image (rodata/data gap) memory: 752K1723lb # [ 0.939748] Freeing unused kernel image (rodata/data gap) memory: 752K1724worker2 # [ 0.934307] Freeing unused decrypted memory: 2028K1725worker2 # [ 0.936814] Freeing unused kernel image (initmem) memory: 3644K1726worker2 # [ 0.937857] Write protecting the kernel read-only data: 32768k1727worker2 # [ 0.939697] Freeing unused kernel image (text/rodata gap) memory: 1208K1728worker2 # [ 0.941134] Freeing unused kernel image (rodata/data gap) memory: 752K1729worker1 # [ 0.975078] Freeing initrd memory: 29092K1730worker1 # [ 0.978338] Freeing unused decrypted memory: 2028K1731worker1 # [ 0.981119] Freeing unused kernel image (initmem) memory: 3644K1732worker1 # [ 0.982274] Write protecting the kernel read-only data: 32768k1733worker1 # [ 0.984213] Freeing unused kernel image (text/rodata gap) memory: 1208K1734worker1 # [ 0.985774] Freeing unused kernel image (rodata/data gap) memory: 752K1735client # [ 0.996667] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736lb # [ 0.990947] x86/mm: Checked W+X mappings: passed, no W+X pages found.1737client # [ 0.997704] Run /init as init process1738lb # [ 0.991987] Run /init as init process1739worker2 # [ 0.992366] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740worker2 # [ 0.993413] Run /init as init process1741client # [ 1.007962] systemd[1]: Inserted module 'autofs4'1742lb # [ 1.002133] systemd[1]: Inserted module 'autofs4'1743worker2 # [ 1.004107] systemd[1]: Inserted module 'autofs4'1744client # [ 1.024248] fuse: init (API version 7.45)1745lb # [ 1.018489] fuse: init (API version 7.45)1746client # [ 1.030006] ACPI: \_SB_.GSIG: Enabled at IRQ 221747lb # [ 1.024318] ACPI: \_SB_.GSIG: Enabled at IRQ 221748client # [ 1.032456] ACPI: \_SB_.GSIH: Enabled at IRQ 231749lb # [ 1.026865] ACPI: \_SB_.GSIH: Enabled at IRQ 231750worker2 # [ 1.020352] fuse: init (API version 7.45)1751client # [ 1.035220] ACPI: \_SB_.GSIE: Enabled at IRQ 201752lb # [ 1.030704] ACPI: \_SB_.GSIE: Enabled at IRQ 201753client # [ 1.037482] ACPI: \_SB_.GSIF: Enabled at IRQ 211754lb # [ 1.033233] ACPI: \_SB_.GSIF: Enabled at IRQ 211755worker2 # [ 1.026130] ACPI: \_SB_.GSIG: Enabled at IRQ 221756worker2 # [ 1.028842] ACPI: \_SB_.GSIH: Enabled at IRQ 231757worker2 # [ 1.031957] ACPI: \_SB_.GSIE: Enabled at IRQ 201758worker2 # [ 1.034401] ACPI: \_SB_.GSIF: Enabled at IRQ 211759worker1 # [ 1.036925] x86/mm: Checked W+X mappings: passed, no W+X pages found.1760worker1 # [ 1.038053] Run /init as init process1761worker1 # [ 1.048685] systemd[1]: Inserted module 'autofs4'1762client # [ 1.064159] systemd[1]: Successfully made /usr/ read-only.1763lb # [ 1.061001] systemd[1]: Successfully made /usr/ read-only.1764worker1 # [ 1.065114] fuse: init (API version 7.45)1765worker2 # [ 1.062352] systemd[1]: Successfully made /usr/ read-only.1766worker1 # [ 1.070891] ACPI: \_SB_.GSIG: Enabled at IRQ 221767worker1 # [ 1.073369] ACPI: \_SB_.GSIH: Enabled at IRQ 231768worker1 # [ 1.076341] ACPI: \_SB_.GSIE: Enabled at IRQ 201769worker1 # [ 1.078649] ACPI: \_SB_.GSIF: Enabled at IRQ 211770worker1 # [ 1.105224] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.399718] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.404457] systemd[1]: Detected virtualization kvm.1773client # [ 1.405257] systemd[1]: Detected architecture x86-64.1774client # [ 1.406092] systemd[1]: Running in initrd.1775client # [ 1.407162] systemd[1]: Initializing machine ID from random generator.1776lb # [ 1.396753] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1777client # [ 1.408283] systemd[1]: Hostname set to <client>.1778lb # [ 1.402674] systemd[1]: Detected virtualization kvm.1779lb # [ 1.403750] systemd[1]: Detected architecture x86-64.1780lb # [ 1.404859] systemd[1]: Running in initrd.1781lb # [ 1.406292] systemd[1]: Initializing machine ID from random generator.1782lb # [ 1.407741] systemd[1]: Hostname set to <lb>.1783worker2 # [ 1.397841] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784worker2 # [ 1.402758] systemd[1]: Detected virtualization kvm.1785worker2 # [ 1.403624] systemd[1]: Detected architecture x86-64.1786worker2 # [ 1.404430] systemd[1]: Running in initrd.1787worker2 # [ 1.405546] systemd[1]: Initializing machine ID from random generator.1788worker2 # [ 1.406738] systemd[1]: Hostname set to <worker2>.1789worker1 # [ 1.441718] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker1 # [ 1.453838] systemd[1]: Detected virtualization kvm.1791worker1 # [ 1.455920] systemd[1]: Detected architecture x86-64.1792worker1 # [ 1.458041] systemd[1]: Running in initrd.1793worker1 # [ 1.460651] systemd[1]: Initializing machine ID from random generator.1794worker1 # [ 1.463584] systemd[1]: Hostname set to <worker1>.1795worker2 # [ 1.594628] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796client # [ 1.611089] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797lb # [ 1.624514] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798worker2 # [ 1.632593] systemd[1]: Queued start job for default target Initrd Default Target.1799worker2 # [ 1.635770] systemd[1]: Created slice Slice /system/modprobe.1800worker2 # [ 1.636911] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1801worker2 # [ 1.638260] systemd[1]: Expecting device /dev/disk/by-label/nixos...1802worker2 # [ 1.639296] systemd[1]: Reached target Path Units.1803worker2 # [ 1.640132] systemd[1]: Reached target Slice Units.1804worker2 # [ 1.640973] systemd[1]: Reached target Swaps.1805worker2 # [ 1.641748] systemd[1]: Reached target Timer Units.1806worker2 # [ 1.642683] systemd[1]: Listening on D-Bus System Message Bus Socket.1807worker2 # [ 1.643836] systemd[1]: Listening on Journal Socket (/dev/log).1808worker2 # [ 1.644993] systemd[1]: Listening on Journal Sockets.1809worker2 # [ 1.646057] systemd[1]: Listening on udev Control Socket.1810worker2 # [ 1.647048] systemd[1]: Listening on udev Kernel Socket.1811worker2 # [ 1.647985] systemd[1]: Reached target Socket Units.1812worker2 # [ 1.649786] systemd[1]: Starting Create List of Static Device Nodes...1813client # [ 1.664144] systemd[1]: Queued start job for default target Initrd Default Target.1814worker2 # [ 1.653301] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1815client # [ 1.668682] systemd[1]: Created slice Slice /system/modprobe.1816client # [ 1.669879] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1817client # [ 1.671312] systemd[1]: Expecting device /dev/disk/by-label/nixos...1818client # [ 1.672479] systemd[1]: Reached target Path Units.1819client # [ 1.673328] systemd[1]: Reached target Slice Units.1820worker2 # [ 1.660154] systemd[1]: Starting Load Kernel Module configfs...1821client # [ 1.674239] systemd[1]: Reached target Swaps.1822client # [ 1.675033] systemd[1]: Reached target Timer Units.1823client # [ 1.676024] systemd[1]: Listening on D-Bus System Message Bus Socket.1824client # [ 1.677262] systemd[1]: Listening on Journal Socket (/dev/log).1825client # [ 1.678454] systemd[1]: Listening on Journal Sockets.1826lb # [ 1.672289] systemd[1]: Queued start job for default target Initrd Default Target.1827client # [ 1.679420] systemd[1]: Listening on udev Control Socket.1828client # [ 1.680454] systemd[1]: Listening on udev Kernel Socket.1829client # [ 1.681439] systemd[1]: Reached target Socket Units.1830lb # [ 1.675513] systemd[1]: Created slice Slice /system/modprobe.1831lb # [ 1.676734] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1832client # [ 1.683178] systemd[1]: Starting Create List of Static Device Nodes...1833lb # [ 1.678133] systemd[1]: Expecting device /dev/disk/by-label/nixos...1834lb # [ 1.679281] systemd[1]: Reached target Path Units.1835lb # [ 1.680144] systemd[1]: Reached target Slice Units.1836lb # [ 1.681043] systemd[1]: Reached target Swaps.1837client # [ 1.687432] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1838lb # [ 1.681854] systemd[1]: Reached target Timer Units.1839lb # [ 1.682799] systemd[1]: Listening on D-Bus System Message Bus Socket.1840worker2 # [ 1.675693] systemd[1]: Starting Journal Service...1841lb # [ 1.684079] systemd[1]: Listening on Journal Socket (/dev/log).1842lb # [ 1.685262] systemd[1]: Listening on Journal Sockets.1843lb # [ 1.686305] systemd[1]: Listening on udev Control Socket.1844lb # [ 1.687395] systemd[1]: Listening on udev Kernel Socket.1845lb # [ 1.688386] systemd[1]: Reached target Socket Units.1846client # [ 1.695465] systemd[1]: Starting Load Kernel Module configfs...1847lb # [ 1.690228] systemd[1]: Starting Create List of Static Device Nodes...1848worker1 # [ 1.685253] systemd[1]: bpf-restrict-fs: LSM BPF program attached1849lb # [ 1.694005] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1850worker2 # [ 1.689580] systemd[1]: Starting Load Kernel Modules...1851lb # [ 1.700748] systemd[1]: Starting Load Kernel Module configfs...1852client # [ 1.709684] systemd[1]: Starting Journal Service...1853client # [ 1.718627] systemd[1]: Starting Load Kernel Modules...1854worker2 # [ 1.694545] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1855lb # [ 1.714367] systemd[1]: Starting Journal Service...1856worker2 # [ 1.710390] netfs: FS-Cache loaded1857worker2 # [ 1.715081] 9pnet: Installing 9P2000 support1858lb # [ 1.728268] systemd[1]: Starting Load Kernel Modules...1859client # [ 1.726441] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1860worker2 # [ 1.727576] systemd[1]: Starting Coldplug All udev Devices...1861client # [ 1.743810] netfs: FS-Cache loaded1862client # [ 1.748464] 9pnet: Installing 9P2000 support1863worker1 # [ 1.739628] systemd[1]: Queued start job for default target Initrd Default Target.1864lb # [ 1.735260] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1865worker1 # [ 1.743738] systemd[1]: Created slice Slice /system/modprobe.1866worker2 # [ 1.742624] systemd[1]: Finished Create List of Static Device Nodes.1867worker1 # [ 1.744959] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1868lb # [ 1.751497] netfs: FS-Cache loaded1869worker1 # [ 1.746445] systemd[1]: Expecting device /dev/disk/by-label/nixos...1870worker1 # [ 1.747582] systemd[1]: Reached target Path Units.1871worker1 # [ 1.748527] systemd[1]: Reached target Slice Units.1872worker1 # [ 1.749490] systemd[1]: Reached target Swaps.1873client # [ 1.760563] systemd[1]: Starting Coldplug All udev Devices...1874worker1 # [ 1.750324] systemd[1]: Reached target Timer Units.1875lb # [ 1.756247] 9pnet: Installing 9P2000 support1876worker1 # [ 1.751308] systemd[1]: Listening on D-Bus System Message Bus Socket.1877worker1 # [ 1.752574] systemd[1]: Listening on Journal Socket (/dev/log).1878worker2 # [ 1.750176] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1879worker1 # [ 1.753765] systemd[1]: Listening on Journal Sockets.1880worker1 # [ 1.754807] systemd[1]: Listening on udev Control Socket.1881worker1 # [ 1.755896] systemd[1]: Listening on udev Kernel Socket.1882worker1 # [ 1.756875] systemd[1]: Reached target Socket Units.1883worker1 # [ 1.758687] systemd[1]: Starting Create List of Static Device Nodes...1884lb # [ 1.764967] systemd[1]: Starting Coldplug All udev Devices...1885worker2 # [ 1.757925] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1886worker1 # [ 1.762480] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1887client # [ 1.774033] systemd[1]: Finished Create List of Static Device Nodes.1888worker2 # [ 1.762926] systemd[1]: modprobe@configfs.service: Deactivated successfully.1889client # [ 1.778964] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1890worker2 # [ 1.765803] systemd-journald[67]: Collecting audit messages is disabled.1891worker1 # [ 1.771608] systemd[1]: Starting Load Kernel Module configfs...1892worker2 # [ 1.770895] systemd[1]: Finished Load Kernel Module configfs.1893client # [ 1.787424] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1894lb # [ 1.782268] systemd[1]: Finished Create List of Static Device Nodes.1895worker2 # [ 1.775992] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1896client # [ 1.792325] systemd-journald[67]: Collecting audit messages is disabled.1897worker2 # [ 1.779398] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1898client # [ 1.793923] systemd[1]: modprobe@configfs.service: Deactivated successfully.1899lb # [ 1.788012] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1900worker1 # [ 1.788489] systemd[1]: Starting Journal Service...1901worker2 # [ 1.787705] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1902client # [ 1.801755] systemd[1]: Finished Load Kernel Module configfs.1903lb # [ 1.797127] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1904worker2 # [ 1.790678] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1905client # [ 1.805854] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1906client # [ 1.808286] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1907lb # [ 1.805728] systemd[1]: modprobe@configfs.service: Deactivated successfully.1908worker1 # [ 1.802488] systemd[1]: Starting Load Kernel Modules...1909lb # [ 1.809518] systemd-journald[67]: Collecting audit messages is disabled.1910client # [ 1.815234] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1911lb # [ 1.812675] systemd[1]: Finished Load Kernel Module configfs.1912client # [ 1.819450] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1913lb # [ 1.818729] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1914worker1 # [ 1.808468] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1915lb # [ 1.824793] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1916worker1 # [ 1.823365] netfs: FS-Cache loaded1917worker2 # [ 1.820739] systemd[1]: Finished Load Kernel Modules.1918worker1 # [ 1.828160] 9pnet: Installing 9P2000 support1919lb # [ 1.833377] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1920lb # [ 1.836279] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1921worker2 # [ 1.829702] systemd[1]: Starting Apply Kernel Variables...1922client # [ 1.845044] systemd[1]: Finished Load Kernel Modules.1923worker1 # [ 1.837217] systemd[1]: Starting Coldplug All udev Devices...1924worker2 # [ 1.839643] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1925client # [ 1.853598] systemd[1]: Starting Apply Kernel Variables...1926worker1 # [ 1.852546] systemd[1]: Finished Create List of Static Device Nodes.1927worker2 # [ 1.850911] systemd[1]: Starting Create Static Device Nodes in /dev...1928worker1 # [ 1.857180] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1929lb # [ 1.864695] systemd[1]: Finished Load Kernel Modules.1930client # [ 1.871712] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1931worker1 # [ 1.863852] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1932lb # [ 1.872383] systemd[1]: Starting Apply Kernel Variables...1933worker1 # [ 1.870072] systemd[1]: modprobe@configfs.service: Deactivated successfully.1934worker1 # [ 1.873913] systemd-journald[67]: Collecting audit messages is disabled.1935client # [ 1.885463] systemd[1]: Starting Create Static Device Nodes in /dev...1936worker2 # [ 1.873105] systemd[1]: Finished Apply Kernel Variables.1937worker1 # [ 1.876774] systemd[1]: Finished Load Kernel Module configfs.1938lb # [ 1.882327] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1939client # [ 1.894457] systemd[1]: Finished Apply Kernel Variables.1940worker1 # [ 1.882904] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1941worker2 # [ 1.883494] systemd[1]: Started Journal Service.1942lb # [ 1.891641] systemd[1]: Starting Create Static Device Nodes in /dev...1943worker1 # [ 1.887998] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1944worker2 # [ 1.754210] systemd-modules-load[68]: Inserted module 'dm_mod'1945worker2 # [ 1.756460] systemd-modules-load[68]: Inserted module 'virtio_balloon'1946worker2 # [ 1.758271] systemd-modules-load[68]: Inserted module 'virtio_gpu'1947worker1 # [ 1.894471] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1948worker2 # [ 1.760325] systemd[1]: Finished Create Static Device Nodes in /dev.1949client # [ 1.907480] systemd[1]: Started Journal Service.1950worker2 # [ 1.762359] systemd[1]: Reached target Preparation for Local File Systems.1951worker1 # [ 1.897576] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1952client # [ 1.772732] systemd-modules-load[68]: Inserted module 'dm_mod'1953client # [ 1.776949] systemd-modules-load[68]: Inserted module 'virtio_balloon'1954worker2 # [ 1.765215] systemd[1]: Reached target Local File Systems.1955client # [ 1.778302] systemd-modules-load[68]: Inserted module 'virtio_gpu'1956worker2 # [ 1.768067] systemd[1]: Starting Create System Files and Directories...1957lb # [ 1.911253] systemd[1]: Finished Apply Kernel Variables.1958worker2 # [ 1.773066] systemd[1]: Starting Rule-based Manager for Device Events and Files...1959client # [ 1.792126] systemd[1]: Finished Create Static Device Nodes in /dev.1960client # [ 1.793079] systemd[1]: Reached target Preparation for Local File Systems.1961client # [ 1.795069] systemd[1]: Reached target Local File Systems.1962client # [ 1.796659] systemd[1]: Starting Create System Files and Directories...1963lb # [ 1.925730] systemd[1]: Finished Create Static Device Nodes in /dev.1964lb # [ 1.792682] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.931363] systemd[1]: Started Journal Service.1965lb # 1966lb # [ 1.796281] systemd-modules-load[68]: Inserted module 'virtio_balloon'1967client # [ 1.805070] systemd[1]: Starting Rule-based Manager for Device Events and Files...1968lb # [ 1.797370] systemd-modules-load[68]: Inserted module 'virtio_gpu'1969worker1 # [ 1.929934] systemd[1]: Finished Load Kernel Modules.1970lb # [ 1.799255] systemd[1]: Reached target Preparation for Local File Systems.1971lb # [ 1.801347] systemd[1]: Reached target Local File Systems.1972worker1 # [ 1.935598] systemd[1]: Starting Apply Kernel Variables...1973lb # [ 1.806064] systemd[1]: Starting Create System Files and Directories...1974worker2 # [ 1.805242] systemd[1]: Finished Create System Files and Directories.1975lb # [ 1.811136] systemd[1]: Starting Rule-based Manager for Device Events and Files...1976worker1 # [ 1.946004] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1977client # [ 1.828546] systemd[1]: Finished Create System Files and Directories.1978worker1 # [ 1.957024] systemd[1]: Starting Create Static Device Nodes in /dev...1979worker2 # [ 1.830428] systemd-udevd[78]: Using default interface naming scheme 'v261'.1980lb # [ 1.840527] systemd[1]: Finished Create System Files and Directories.1981client # [ 1.854464] systemd-udevd[83]: Using default interface naming scheme 'v261'.1982worker1 # [ 1.980089] systemd[1]: Finished Apply Kernel Variables.1983worker2 # [ 1.855122] systemd[1]: Started Rule-based Manager for Device Events and Files.1984worker1 # [ 1.857301] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.993879] systemd[1]: Started Journal Service.1985worker1 # 1986lb # [ 1.864177] systemd-udevd[77]: Using default interface naming scheme 'v261'.1987worker1 # [ 1.860383] systemd-modules-load[68]: Inserted module 'virtio_balloon'1988worker1 # [ 1.862496] systemd-modules-load[68]: Inserted module 'virtio_gpu'1989client # [ 1.881371] systemd[1]: Started Rule-based Manager for Device Events and Files.1990worker1 # [ 1.870077] systemd[1]: Finished Create Static Device Nodes in /dev.1991worker1 # [ 1.875067] systemd[1]: Reached target Preparation for Local File Systems.1992worker1 # [ 1.877086] systemd[1]: Reached target Local File Systems.1993worker1 # [ 1.880072] systemd[1]: Starting Create System Files and Directories...1994worker1 # [ 1.885075] systemd[1]: Starting Rule-based Manager for Device Events and Files...1995lb # [ 1.891323] systemd[1]: Started Rule-based Manager for Device Events and Files.1996worker2 # [ 1.903068] systemd[1]: Finished Coldplug All udev Devices.1997worker2 # [ 1.905424] systemd[1]: Reached target System Initialization.1998worker2 # [ 1.906265] systemd[1]: Reached target Basic System.1999client # [ 1.924723] systemd[1]: Finished Coldplug All udev Devices.2000worker1 # [ 1.914462] systemd[1]: Finished Create System Files and Directories.
loading 2,672 hidden lines…
4673worker2 # [ 41.677608] nix-grpc-daemon[785]: ts=2026-09-22T10:55:52Z level=info event=scheduler_connected addr=lb:500514674worker2 # [ 41.689913] nix-grpc-daemon[785]: ts=2026-09-22T10:55:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:455124675worker2 # [ 41.695415] nix-grpc-daemon[785]: ts=2026-09-22T10:55:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04676worker2 # [ 41.716827] nix-grpc-daemon[785]: ts=2026-09-22T10:55:53Z level=info event=oidc_jwks provider=mock keys=14677worker2 # [ 41.721509] nix-grpc-daemon[785]: ts=2026-09-22T10:55:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:455124678worker2 # [ 41.725408] nix-grpc-daemon[785]: ts=2026-09-22T10:55:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04679lb: must succeed: curl -sf localhost:9901/clusters4680lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4681worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4682worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4683lb: must succeed: curl -sf localhost:9901/clusters4684lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4685lb: must succeed: curl -sf localhost:9901/clusters4686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4687lb: must succeed: curl -sf localhost:9901/clusters4688lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4689lb: must succeed: curl -sf localhost:9901/clusters4690lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4691worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4692worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)4693worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4694worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4695lb: must succeed: curl -sf localhost:9901/clusters4696lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4697lb: must succeed: curl -sf localhost:9901/clusters4698lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4699worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4700worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4701worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4702worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4703worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4704worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4705worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4706worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4707(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 11.25 seconds)4708subtest: balancer auth: client cert, bearer token, nothing, unknown CN4709worker1 # [ 43.101909] nix-grpc-daemon[784]: ts=2026-09-22T10:55:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478324710worker1 # [ 43.112521] systemd[1]: Started Nix Daemon.4711worker1 # [ 43.321608] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4712worker1 # [ 43.350119] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5)4713worker1 # [ 43.415597] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 137 ms (attempt 2/5)4714worker1 # [ 43.552881] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 3/5)4715worker1 # [ 43.586304] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 4/5)4716worker1 # [ 43.763310] nix-grpc-daemon[784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4717worker1 # [ 43.767272] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4718worker2 # [ 43.809493] nix-grpc-daemon[785]: ts=2026-09-22T10:55:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550404719worker2 # [ 43.821990] systemd[1]: Started Nix Daemon.4720worker2 # [ 44.036698] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4721worker2 # [ 44.061872] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)4722worker2 # [ 44.124470] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5)4723worker2 # [ 44.318106] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 3/5)4724worker2 # [ 44.401996] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 737 ms (attempt 4/5)4725worker2 # [ 45.151682] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4726worker2 # [ 45.155852] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4727worker2 # [ 45.176160] nix-grpc-daemon[785]: ts=2026-09-22T10:55:56Z level=debug event=path_info_miss path=32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test error="error: path '/nix/store/32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test' is required, but there is no substituter that can build it"4728worker2 # [ 45.197253] nix-grpc-daemon[785]: ts=2026-09-22T10:55:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550404729worker2 # [ 45.205984] nix-grpc-daemon[785]: ts=2026-09-22T10:55:56Z level=debug event=path_info_miss path=32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test error="error: path '/nix/store/32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test' is required, but there is no substituter that can build it"4730client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4731client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4732client: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4733client: (finished: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4734worker2 # [ 45.541922] nix-grpc-daemon[785]: ts=2026-09-22T10:55:56Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:550404735worker1 # [ 45.573105] nix-grpc-daemon[784]: ts=2026-09-22T10:55:57Z level=info event=oidc_jwks provider=mock keys=14736worker1 # [ 45.576265] nix-grpc-daemon[784]: ts=2026-09-22T10:55:57Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:478324737worker2 # [ 45.593373] nix-grpc-daemon[785]: ts=2026-09-22T10:55:56Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:550404738worker1 # [ 45.590069] nix-grpc-daemon[784]: ts=2026-09-22T10:55:57Z level=debug event=path_info_miss path=32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test error="error: path '/nix/store/32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test' is required, but there is no substituter that can build it"4739worker2 # [ 45.601409] nix-grpc-daemon[785]: ts=2026-09-22T10:55:56Z level=debug event=path_info_miss path=32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test error="error: path '/nix/store/32wy8ar5zjxjxi6xl8k9avi2i4758n86-nixos-system-client-test' is required, but there is no substituter that can build it"4740client: must succeed: rm -r /var/lib/nix-grpc-store4741client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.02 seconds)4742worker2 # [ 45.842547] nix-grpc-daemon[785]: ts=2026-09-22T10:55:57Z level=info event=denied method=QueryMissing cn=- role=none4743lb # [ 46.046656] envoy[910]: conn peer=[2001:db8:1::1]:35196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=21524744lb # [ 46.050795] envoy[910]: conn peer=[2001:db8:1::1]:35206 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=704745lb # [ 46.054256] envoy[910]: conn peer=[2001:db8:1::1]:35212 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=144746worker2 # [ 46.067168] nix-grpc-daemon[785]: ts=2026-09-22T10:55:57Z level=info event=denied method=QueryMissing cn=stranger role=none4747(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.31 seconds)4748subtest: DAG build is scheduled across workers and lands in the cache4749client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24750worker2 # [ 46.312889] nix-grpc-daemon[785]: ts=2026-09-22T10:55:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550404751client # don't know how to build these paths:4752client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4753worker1 # [ 46.325711] nix-grpc-daemon[784]: ts=2026-09-22T10:55:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478324754worker1 # [ 46.337146] nix-grpc-daemon[784]: E0000 00:00:1790074558.454409 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4755worker1 # [ 46.369049] nix-grpc-daemon[784]: E0000 00:00:1790074558.486434 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4756worker2 # [ 46.370244] nix-grpc-daemon[785]: ts=2026-09-22T10:55:57Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04757worker2 # [ 46.375221] nix-grpc-daemon[785]: ts=2026-09-22T10:55:57Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14758worker1 # [ 46.375296] nix-grpc-daemon[784]: E0000 00:00:1790074558.490932 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4759worker1 # [ 46.395931] nix-grpc-daemon[784]: E0000 00:00:1790074558.513036 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4760worker1 # [ 46.434372] nix-grpc-daemon[784]: ts=2026-09-22T10:55:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478324761worker1 # [ 46.457319] nix-grpc-daemon[784]: ts=2026-09-22T10:55:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478324762client # copying 1 paths to 'lb:50051'...4763worker1 # [ 46.471069] nix-grpc-daemon[784]: ts=2026-09-22T10:55:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478324764client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4765worker1 # [ 46.498698] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4766worker1 # [ 46.524067] nix-grpc-daemon[784]: ts=2026-09-22T10:55:58Z level=info event=niks3_push_started pid=8654767worker1 # [ 46.775892] nix-grpc-daemon[865]: time=2026-09-22T10:55:58.890Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4768worker1 # [ 46.960484] nix-daemon[853]: accepted connection from pid 877, user nix-grpc-daemon (trusted)4769worker1 # [ 46.976061] nix-daemon[853]: reaped child process 880, status = succeeded4770worker1 # [ 47.122710] nix-daemon[853]: accepted connection from pid 882, user nix-grpc-daemon (trusted)4771worker1 # [ 47.137302] nix-daemon[853]: reaped child process 885, status = succeeded4772lb # [ 47.152558] niks3-server[1067]: 2026/09/22 10:55:59 INFO Received uploads request method=POST path=/api/pending_closures4773worker1 # [ 47.171728] nix-grpc-daemon[865]: time=2026-09-22T10:55:59.289Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4774worker1 # [ 47.173795] nix-grpc-daemon[865]: time=2026-09-22T10:55:59.291Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4775lb # [ 47.213151] niks3-server[1067]: 2026/09/22 10:55:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4776lb # [ 47.235072] niks3-server[1067]: 2026/09/22 10:55:59 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4777lb # [ 47.244362] niks3-server[1067]: 2026/09/22 10:55:59 INFO Signed narinfos id=1 count=14778worker1 # [ 47.242381] nix-grpc-daemon[865]: time=2026-09-22T10:55:59.359Z level=INFO msg="Uploading 1 narinfos"4779lb # [ 47.255378] niks3-server[1067]: 2026/09/22 10:55:59 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4780lb # [ 47.261097] niks3-server[1067]: 2026/09/22 10:55:59 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4781lb # [ 47.271993] niks3-server[1067]: 2026/09/22 10:55:59 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4782lb # [ 47.277293] niks3-server[1067]: 2026/09/22 10:55:59 INFO Completed upload id=14783worker1 # [ 47.273733] nix-grpc-daemon[865]: time=2026-09-22T10:55:59.391Z level=INFO msg="Upload complete. (484ms)"4784worker1 # [ 47.280367] nix-grpc-daemon[784]: ts=2026-09-22T10:55:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_in=5044785worker1 # [ 47.286258] nix-grpc-daemon[784]: ts=2026-09-22T10:55:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478324786worker1 # [ 47.299159] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4787worker1 # [ 47.305838] nix-daemon[853]: reaped child process 863, status = succeeded4788client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4789worker1 # [ 47.315841] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4790client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4791client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4792client # farm-a> LOG-farm-a-t14793worker1 # [ 48.732916] nix-daemon[853]: reaped child process 889, status = succeeded4794worker1 # [ 48.875839] nix-daemon[853]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4795worker1 # [ 48.887336] nix-daemon[853]: reaped child process 909, status = succeeded4796lb # [ 48.907768] niks3-server[1067]: 2026/09/22 10:56:00 INFO Received uploads request method=POST path=/api/pending_closures4797worker1 # [ 48.915388] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.032Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4798worker1 # [ 48.917064] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.034Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4799lb # [ 48.959735] niks3-server[1067]: 2026/09/22 10:56:00 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4800lb # [ 48.966536] niks3-server[1067]: 2026/09/22 10:56:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4801lb # [ 48.972051] niks3-server[1067]: 2026/09/22 10:56:00 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4802worker1 # [ 48.974581] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.091Z level=INFO msg="Uploading 1 narinfos"4803lb # [ 48.977978] niks3-server[1067]: 2026/09/22 10:56:00 INFO Signed narinfos id=2 count=14804lb # [ 48.980712] niks3-server[1067]: 2026/09/22 10:56:00 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4805lb # [ 48.991210] niks3-server[1067]: 2026/09/22 10:56:00 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4806lb # [ 48.998104] niks3-server[1067]: 2026/09/22 10:56:00 INFO Completed upload id=24807lb # [ 49.000290] niks3-server[1067]: 2026/09/22 10:56:00 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4808worker1 # [ 48.994537] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.111Z level=INFO msg="Upload complete. (267ms)"4809worker1 # [ 49.002253] nix-grpc-daemon[784]: ts=2026-09-22T10:56:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14810worker1 # [ 49.014370] nix-daemon[853]: reaped child process 887, status = succeeded4811worker2 # [ 49.019378] nix-grpc-daemon[785]: ts=2026-09-22T10:56:00Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04812worker2 # [ 49.023425] nix-grpc-daemon[785]: ts=2026-09-22T10:56:00Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14813worker1 # [ 49.064100] nix-grpc-daemon[784]: ts=2026-09-22T10:56:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478324814worker1 # [ 49.076727] nix-grpc-daemon[784]: ts=2026-09-22T10:56:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478324815worker1 # [ 49.091072] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.208Z level=INFO msg="All 1 paths already cached"4816client # copying 1 paths to 'lb:50051'...4817worker1 # [ 49.098801] nix-grpc-daemon[784]: ts=2026-09-22T10:56:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478324818client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4819worker1 # [ 49.117369] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4820worker1 # [ 49.294590] nix-daemon[853]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4821worker1 # [ 49.307755] nix-daemon[853]: reaped child process 916, status = succeeded4822worker1 # [ 49.462722] nix-daemon[853]: accepted connection from pid 918, user nix-grpc-daemon (trusted)4823worker1 # [ 49.475232] nix-daemon[853]: reaped child process 921, status = succeeded4824lb # [ 49.492950] niks3-server[1067]: 2026/09/22 10:56:01 INFO Received uploads request method=POST path=/api/pending_closures4825worker1 # [ 49.501873] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.619Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4826worker1 # [ 49.503399] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.620Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4827lb # [ 49.528344] niks3-server[1067]: 2026/09/22 10:56:01 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4828lb # [ 49.537514] niks3-server[1067]: 2026/09/22 10:56:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4829lb # [ 49.544494] niks3-server[1067]: 2026/09/22 10:56:01 INFO Signed narinfos id=3 count=14830worker1 # [ 49.543504] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.661Z level=INFO msg="Uploading 1 narinfos"4831lb # [ 49.551055] niks3-server[1067]: 2026/09/22 10:56:01 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4832lb # [ 49.560296] niks3-server[1067]: 2026/09/22 10:56:01 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4833lb # [ 49.570357] niks3-server[1067]: 2026/09/22 10:56:01 INFO Completed upload id=34834lb # [ 49.572110] niks3-server[1067]: 2026/09/22 10:56:01 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4835worker1 # [ 49.566800] nix-grpc-daemon[865]: time=2026-09-22T10:56:01.684Z level=INFO msg="Upload complete. (434ms)"4836worker1 # [ 49.572179] nix-grpc-daemon[784]: ts=2026-09-22T10:56:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_in=6244837worker1 # [ 49.578324] nix-grpc-daemon[784]: ts=2026-09-22T10:56:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478324838worker1 # [ 49.589615] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4839worker1 # [ 49.601429] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4840client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4841worker1 # [ 49.603554] nix-daemon[853]: reaped child process 911, status = succeeded4842client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4843client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4844client # farm-b> LOG-farm-b-t14845worker1 # [ 50.829755] nix-daemon[853]: reaped child process 925, status = succeeded4846worker1 # [ 50.985635] nix-daemon[853]: accepted connection from pid 941, user nix-grpc-daemon (trusted)4847worker1 # [ 50.997996] nix-daemon[853]: reaped child process 944, status = succeeded4848lb # [ 51.019099] niks3-server[1067]: 2026/09/22 10:56:03 INFO Received uploads request method=POST path=/api/pending_closures4849worker1 # [ 51.022032] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.139Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4850worker1 # [ 51.023540] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.141Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4851lb # [ 51.055888] niks3-server[1067]: 2026/09/22 10:56:03 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4852lb # [ 51.062641] niks3-server[1067]: 2026/09/22 10:56:03 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4853lb # [ 51.076054] niks3-server[1067]: 2026/09/22 10:56:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4854worker1 # [ 51.078643] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.195Z level=INFO msg="Uploading 1 narinfos"4855lb # [ 51.081991] niks3-server[1067]: 2026/09/22 10:56:03 INFO Signed narinfos id=4 count=14856lb # [ 51.085713] niks3-server[1067]: 2026/09/22 10:56:03 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4857lb # [ 51.096234] niks3-server[1067]: 2026/09/22 10:56:03 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4858lb # [ 51.099042] niks3-server[1067]: 2026/09/22 10:56:03 INFO Completed upload id=44859lb # [ 51.100591] niks3-server[1067]: 2026/09/22 10:56:03 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4860worker1 # [ 51.095394] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.212Z level=INFO msg="Upload complete. (284ms)"4861worker2 # [ 51.105099] nix-grpc-daemon[785]: ts=2026-09-22T10:56:02Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04862worker1 # [ 51.102141] nix-grpc-daemon[784]: ts=2026-09-22T10:56:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14863worker2 # [ 51.106969] nix-grpc-daemon[785]: ts=2026-09-22T10:56:02Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14864worker1 # [ 51.115549] nix-daemon[853]: reaped child process 923, status = succeeded4865worker1 # [ 51.149435] nix-grpc-daemon[784]: ts=2026-09-22T10:56:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478324866worker1 # [ 51.161147] nix-grpc-daemon[784]: ts=2026-09-22T10:56:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478324867worker1 # [ 51.174311] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.291Z level=INFO msg="All 2 paths already cached"4868client # copying 1 paths to 'lb:50051'...4869worker1 # [ 51.181314] nix-grpc-daemon[784]: ts=2026-09-22T10:56:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478324870client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4871worker1 # [ 51.192168] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4872worker1 # [ 51.352114] nix-daemon[853]: accepted connection from pid 948, user nix-grpc-daemon (trusted)4873worker1 # [ 51.364350] nix-daemon[853]: reaped child process 951, status = succeeded4874worker1 # [ 51.519633] nix-daemon[853]: accepted connection from pid 953, user nix-grpc-daemon (trusted)4875worker1 # [ 51.531595] nix-daemon[853]: reaped child process 956, status = succeeded4876lb # [ 51.551182] niks3-server[1067]: 2026/09/22 10:56:03 INFO Received uploads request method=POST path=/api/pending_closures4877worker1 # [ 51.557853] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.675Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4878worker1 # [ 51.559248] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.675Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4879lb # [ 51.585458] niks3-server[1067]: 2026/09/22 10:56:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4880worker1 # [ 51.588928] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.706Z level=INFO msg="Uploading 1 narinfos"4881lb # [ 51.592864] niks3-server[1067]: 2026/09/22 10:56:03 INFO Signed narinfos id=5 count=14882lb # [ 51.597766] niks3-server[1067]: 2026/09/22 10:56:03 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4883lb # [ 51.602969] niks3-server[1067]: 2026/09/22 10:56:03 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4884lb # [ 51.615829] niks3-server[1067]: 2026/09/22 10:56:03 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4885lb # [ 51.622621] niks3-server[1067]: 2026/09/22 10:56:03 INFO Completed upload id=54886worker1 # [ 51.621198] nix-grpc-daemon[865]: time=2026-09-22T10:56:03.738Z level=INFO msg="Upload complete. (417ms)"4887lb # [ 51.625211] niks3-server[1067]: 2026/09/22 10:56:03 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4888worker1 # [ 51.625116] nix-grpc-daemon[784]: ts=2026-09-22T10:56:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_in=7524889worker1 # [ 51.630365] nix-grpc-daemon[784]: ts=2026-09-22T10:56:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478324890worker1 # [ 51.639540] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4891worker1 # [ 51.647932] nix-daemon[853]: reaped child process 946, status = succeeded4892worker1 # [ 51.653273] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4893client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4894client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4895client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4896client # farm-top> LOG-farm-top-t14897worker1 # [ 52.676834] nix-daemon[853]: reaped child process 960, status = succeeded4898worker1 # [ 52.814858] nix-daemon[853]: accepted connection from pid 976, user nix-grpc-daemon (trusted)4899worker1 # [ 52.826721] nix-daemon[853]: reaped child process 979, status = succeeded4900lb # [ 52.847396] niks3-server[1067]: 2026/09/22 10:56:04 INFO Received uploads request method=POST path=/api/pending_closures4901worker1 # [ 52.852985] nix-grpc-daemon[865]: time=2026-09-22T10:56:04.970Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4902worker1 # [ 52.854648] nix-grpc-daemon[865]: time=2026-09-22T10:56:04.971Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4903lb # [ 52.891563] niks3-server[1067]: 2026/09/22 10:56:04 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4904lb # [ 52.897117] niks3-server[1067]: 2026/09/22 10:56:04 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4905lb # [ 52.909491] niks3-server[1067]: 2026/09/22 10:56:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4906lb # [ 52.915568] niks3-server[1067]: 2026/09/22 10:56:04 INFO Signed narinfos id=6 count=14907worker1 # [ 52.914272] nix-grpc-daemon[865]: time=2026-09-22T10:56:05.031Z level=INFO msg="Uploading 1 narinfos"4908lb # [ 52.918203] niks3-server[1067]: 2026/09/22 10:56:04 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4909lb # [ 52.927683] niks3-server[1067]: 2026/09/22 10:56:04 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4910lb # [ 52.931711] niks3-server[1067]: 2026/09/22 10:56:04 INFO Completed upload id=64911worker1 # [ 52.927868] nix-grpc-daemon[865]: time=2026-09-22T10:56:05.045Z level=INFO msg="Upload complete. (265ms)"4912lb # [ 52.932973] niks3-server[1067]: 2026/09/22 10:56:04 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4913worker1 # [ 52.930662] nix-grpc-daemon[784]: ts=2026-09-22T10:56:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14914worker1 # [ 52.937275] nix-daemon[853]: reaped child process 958, status = succeeded4915client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.88 seconds)4916client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4917client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4918client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4919client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4920client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4921client # copying 3 paths...4922client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4923client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4924client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4925client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4926(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.42 seconds)4927subtest: repeat is answered Cached by the scheduler without building4928worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4929worker1 # finding garbage collector roots...4930worker1 # deleting specified paths...4931worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4932worker1 # deleting unused links...4933worker1 # note: hard linking is currently saving 0.0 KiB4934worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.21 seconds)4935worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4936worker2 # finding garbage collector roots...4937worker2 # deleting specified paths...4938worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4939worker2 # deleting unused links...4940worker2 # note: hard linking is currently saving 0.0 KiB4941worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4942client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24943worker1 # [ 54.207993] nix-grpc-daemon[784]: ts=2026-09-22T10:56:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478324944client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4945client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4946worker1 # [ 54.221679] nix-grpc-daemon[784]: ts=2026-09-22T10:56:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478324947worker1 # [ 54.227437] nix-grpc-daemon[865]: time=2026-09-22T10:56:06.344Z level=INFO msg="All 2 paths already cached"4948worker2 # [ 54.239539] nix-grpc-daemon[785]: ts=2026-09-22T10:56:05Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14949client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.27 seconds)4950client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4951client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4952lb: must succeed: curl -sf localhost:9901/clusters4953lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4954lb: must succeed: curl -sf localhost:9901/clusters4955lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4956worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4957worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4958(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.16 seconds)4959subtest: inputs already in the cache are not sent to the scheduler4960lb: must succeed: curl -sf localhost:9901/clusters4961lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4962lb: must succeed: curl -sf localhost:9901/clusters4963lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4964worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4965worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4966client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24967client # don't know how to build these paths:4968worker1 # [ 55.020738] nix-grpc-daemon[784]: ts=2026-09-22T10:56:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478324969client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4970worker2 # [ 55.025324] nix-grpc-daemon[785]: ts=2026-09-22T10:56:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550404971worker2 # [ 55.046243] nix-grpc-daemon[785]: ts=2026-09-22T10:56:06Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04972worker2 # [ 55.048129] nix-grpc-daemon[785]: ts=2026-09-22T10:56:06Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14973worker1 # [ 55.095921] nix-grpc-daemon[784]: ts=2026-09-22T10:56:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478324974worker1 # [ 55.108539] nix-grpc-daemon[784]: ts=2026-09-22T10:56:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478324975worker1 # [ 55.121318] nix-grpc-daemon[865]: time=2026-09-22T10:56:07.238Z level=INFO msg="All 2 paths already cached"4976client # copying 1 paths to 'lb:50051'...4977client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4978worker1 # [ 55.129068] nix-grpc-daemon[784]: ts=2026-09-22T10:56:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478324979worker1 # [ 55.137891] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4980worker1 # [ 55.309362] nix-daemon[853]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4981worker1 # [ 55.322732] nix-daemon[853]: reaped child process 999, status = succeeded4982worker1 # [ 55.477238] nix-daemon[853]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4983worker1 # [ 55.489570] nix-daemon[853]: reaped child process 1004, status = succeeded4984lb # [ 55.507664] niks3-server[1067]: 2026/09/22 10:56:07 INFO Received uploads request method=POST path=/api/pending_closures4985worker1 # [ 55.508680] nix-grpc-daemon[865]: time=2026-09-22T10:56:07.625Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4986worker1 # [ 55.510213] nix-grpc-daemon[865]: time=2026-09-22T10:56:07.627Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4987lb # [ 55.525644] niks3-server[1067]: 2026/09/22 10:56:07 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4988lb # [ 55.528471] niks3-server[1067]: 2026/09/22 10:56:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4989lb # [ 55.530738] niks3-server[1067]: 2026/09/22 10:56:07 INFO Signed narinfos id=7 count=14990worker1 # [ 55.526830] nix-grpc-daemon[865]: time=2026-09-22T10:56:07.644Z level=INFO msg="Uploading 1 narinfos"4991lb # [ 55.533495] niks3-server[1067]: 2026/09/22 10:56:07 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4992lb # [ 55.537477] niks3-server[1067]: 2026/09/22 10:56:07 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4993lb # [ 55.541028] niks3-server[1067]: 2026/09/22 10:56:07 INFO Completed upload id=74994worker1 # [ 55.537145] nix-grpc-daemon[865]: time=2026-09-22T10:56:07.654Z level=INFO msg="Upload complete. (385ms)"4995lb # [ 55.543674] niks3-server[1067]: 2026/09/22 10:56:07 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4996worker1 # [ 55.539822] nix-grpc-daemon[784]: ts=2026-09-22T10:56:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_in=7524997worker1 # [ 55.542299] nix-grpc-daemon[784]: ts=2026-09-22T10:56:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478324998worker1 # [ 55.549372] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4999worker1 # [ 55.551150] nix-daemon[853]: reaped child process 994, status = succeeded5000worker1 # [ 55.556178] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5001client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...5002client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv5003lb # [ 56.046911] envoy[910]: conn peer=[2001:db8:1::1]:35218 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=135004lb # [ 56.050723] envoy[910]: conn peer=[2001:db8:1::1]:35228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9554 tx=4068 ms=66635005lb # [ 56.054267] envoy[910]: conn peer=[2001:db8:1::1]:41910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=479 ms=445006client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5007client # farm-top> LOG-farm-top-t1b5008worker1 # [ 56.972627] nix-daemon[853]: reaped child process 1008, status = succeeded5009worker1 # [ 57.104815] nix-daemon[853]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)5010worker1 # [ 57.116860] nix-daemon[853]: reaped child process 1027, status = succeeded5011lb # [ 57.136615] niks3-server[1067]: 2026/09/22 10:56:09 INFO Received uploads request method=POST path=/api/pending_closures5012worker1 # [ 57.141700] nix-grpc-daemon[865]: time=2026-09-22T10:56:09.258Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5013worker1 # [ 57.143752] nix-grpc-daemon[865]: time=2026-09-22T10:56:09.260Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"5014lb # [ 57.179165] niks3-server[1067]: 2026/09/22 10:56:09 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv5015lb # [ 57.188279] niks3-server[1067]: 2026/09/22 10:56:09 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst5016lb # [ 57.192453] niks3-server[1067]: 2026/09/22 10:56:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign5017lb # [ 57.197373] niks3-server[1067]: 2026/09/22 10:56:09 INFO Signed narinfos id=8 count=15018worker1 # [ 57.196467] nix-grpc-daemon[865]: time=2026-09-22T10:56:09.313Z level=INFO msg="Uploading 1 narinfos"5019lb # [ 57.204581] niks3-server[1067]: 2026/09/22 10:56:09 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls5020lb # [ 57.214327] niks3-server[1067]: 2026/09/22 10:56:09 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete5021lb # [ 57.219093] niks3-server[1067]: 2026/09/22 10:56:09 INFO Completed upload id=85022worker1 # [ 57.217465] nix-grpc-daemon[865]: time=2026-09-22T10:56:09.334Z level=INFO msg="Upload complete. (253ms)"5023lb # [ 57.222840] niks3-server[1067]: 2026/09/22 10:56:09 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo5024worker1 # [ 57.219850] nix-grpc-daemon[784]: ts=2026-09-22T10:56:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=15025worker1 # [ 57.226860] nix-daemon[853]: reaped child process 1006, status = succeeded5026client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.45 seconds)5027client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath5028client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)5029lb: must succeed: curl -sf localhost:9901/clusters5030lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5031lb: must succeed: curl -sf localhost:9901/clusters5032lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5033worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true5034worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)5035(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.95 seconds)5036subtest: two clients wanting the same drv build it once5037client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup5038client # [ 57.684303] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.5039client # Running as unit: dedup1.service; invocation ID: 2f14bb8dca5548e782e90fb410bbd44e5040client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)5041client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup5042client # [ 57.898802] nix[889]: don't know how to build these paths:5043client # [ 57.899761] nix[889]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5044worker2 # [ 57.885439] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550405045worker2 # [ 57.896538] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05046worker2 # [ 57.902889] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=15047worker2 # [ 57.909148] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405048worker2 # [ 57.925120] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550405049client # [ 57.948149] nix[889]: copying 1 paths to 'lb:50051'...5050worker2 # [ 57.936135] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550405051client # [ 57.949088] nix[889]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5052worker2 # [ 57.947632] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5053worker2 # [ 57.965686] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=info event=niks3_push_started pid=9235054worker2 # [ 58.204853] nix-grpc-daemon[923]: time=2026-09-22T10:56:09.501Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5055worker2 # [ 58.395087] nix-daemon[873]: accepted connection from pid 935, user nix-grpc-daemon (trusted)5056worker2 # [ 58.410732] nix-daemon[873]: reaped child process 938, status = succeeded5057worker2 # [ 58.576161] nix-daemon[873]: accepted connection from pid 941, user nix-grpc-daemon (trusted)5058worker2 # [ 58.588423] nix-daemon[873]: reaped child process 944, status = succeeded5059lb # [ 58.610275] niks3-server[1067]: 2026/09/22 10:56:10 INFO Received uploads request method=POST path=/api/pending_closures5060worker2 # [ 58.612404] nix-grpc-daemon[923]: time=2026-09-22T10:56:09.911Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5061worker2 # [ 58.614477] nix-grpc-daemon[923]: time=2026-09-22T10:56:09.913Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5062lb # [ 58.636718] niks3-server[1067]: 2026/09/22 10:56:10 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5063lb # [ 58.639975] niks3-server[1067]: 2026/09/22 10:56:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign5064lb # [ 58.641970] niks3-server[1067]: 2026/09/22 10:56:10 INFO Signed narinfos id=9 count=15065worker2 # [ 58.638443] nix-grpc-daemon[923]: time=2026-09-22T10:56:09.937Z level=INFO msg="Uploading 1 narinfos"5066lb # [ 58.650117] niks3-server[1067]: 2026/09/22 10:56:10 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete5067lb # [ 58.653062] niks3-server[1067]: 2026/09/22 10:56:10 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5068lb # [ 58.656221] niks3-server[1067]: 2026/09/22 10:56:10 INFO Completed upload id=95069worker2 # [ 58.652584] nix-grpc-daemon[923]: time=2026-09-22T10:56:09.951Z level=INFO msg="Upload complete. (434ms)"5070worker2 # [ 58.654579] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_in=4325071lb # [ 58.659027] niks3-server[1067]: 2026/09/22 10:56:10 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5072worker2 # [ 58.658046] nix-grpc-daemon[785]: ts=2026-09-22T10:56:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405073worker2 # [ 58.665109] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5074worker2 # [ 58.666362] nix-daemon[873]: reaped child process 921, status = succeeded5075worker2 # [ 58.671614] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5076client # [ 58.683319] nix[889]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5077client # [ 58.762361] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.5078client # Running as unit: dedup2.service; invocation ID: e7cd6ba1bece45289a37147c448e42555079client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)5080worker2 # [ 59.010354] nix-grpc-daemon[785]: ts=2026-09-22T10:56:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550405081client # [ 59.029373] nix[905]: this derivation will be built:5082worker2 # [ 59.017282] nix-grpc-daemon[785]: ts=2026-09-22T10:56:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550405083client # [ 59.034202] nix[905]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5084worker2 # [ 59.029589] nix-grpc-daemon[785]: ts=2026-09-22T10:56:10Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05085client # [ 59.047565] nix[905]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5086worker2 # [ 59.034744] nix-grpc-daemon[785]: ts=2026-09-22T10:56:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405087client: must succeed: sleep 25088client: (finished: must succeed: sleep 2, in 2.05 seconds)5089client: waiting for success: ! systemctl is-active dedup1 dedup25090worker2 # [ 62.286870] nix-daemon[873]: reaped child process 948, status = succeeded5091worker2 # [ 62.441291] nix-daemon[873]: accepted connection from pid 980, user nix-grpc-daemon (trusted)5092worker2 # [ 62.452125] nix-daemon[873]: reaped child process 983, status = succeeded5093lb # [ 62.473975] niks3-server[1067]: 2026/09/22 10:56:14 INFO Received uploads request method=POST path=/api/pending_closures5094worker2 # [ 62.478133] nix-grpc-daemon[923]: time=2026-09-22T10:56:13.777Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5095worker2 # [ 62.479848] nix-grpc-daemon[923]: time=2026-09-22T10:56:13.779Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5096lb # [ 62.511172] niks3-server[1067]: 2026/09/22 10:56:14 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5097lb # [ 62.526278] niks3-server[1067]: 2026/09/22 10:56:14 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5098lb # [ 62.535898] niks3-server[1067]: 2026/09/22 10:56:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign5099lb # [ 62.542426] niks3-server[1067]: 2026/09/22 10:56:14 INFO Signed narinfos id=10 count=15100worker2 # [ 62.539427] nix-grpc-daemon[923]: time=2026-09-22T10:56:13.838Z level=INFO msg="Uploading 1 narinfos"5101lb # [ 62.546095] niks3-server[1067]: 2026/09/22 10:56:14 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5102lb # [ 62.556320] niks3-server[1067]: 2026/09/22 10:56:14 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete5103lb # [ 62.560048] niks3-server[1067]: 2026/09/22 10:56:14 INFO Completed upload id=105104worker2 # [ 62.556634] nix-grpc-daemon[923]: time=2026-09-22T10:56:13.855Z level=INFO msg="Upload complete. (275ms)"5105lb # [ 62.561849] niks3-server[1067]: 2026/09/22 10:56:14 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5106worker2 # [ 62.562922] nix-grpc-daemon[785]: ts=2026-09-22T10:56:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=15107worker2 # [ 62.569229] nix-grpc-daemon[785]: ts=2026-09-22T10:56:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=15108worker2 # [ 62.579494] nix-daemon[873]: reaped child process 946, status = succeeded5109client # [ 62.621595] systemd[1]: dedup2.service: Deactivated successfully.5110client # [ 62.627598] systemd[1]: dedup2.service: Consumed 140ms CPU time over 3.863s wall clock time, 35.2M memory peak, 6.2K incoming IP traffic, 5.9K outgoing IP traffic.5111client # [ 62.665275] systemd[1]: dedup1.service: Deactivated successfully.5112client # [ 62.666297] systemd[1]: dedup1.service: Consumed 119ms CPU time over 4.982s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.5113client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.11 seconds)5114client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5115client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5116worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5117worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)5118worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5119worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)5120worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5121worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5122worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5123worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5124(finished: subtest: two clients wanting the same drv build it once, in 5.96 seconds)5125subtest: --timeout and --max-silent-time reach the builder5126worker2 # [ 63.790537] nix-grpc-daemon[785]: ts=2026-09-22T10:56:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550405127worker2 # [ 63.802159] nix-grpc-daemon[785]: ts=2026-09-22T10:56:15Z level=debug event=want client=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05128worker2 # [ 63.806219] nix-grpc-daemon[785]: ts=2026-09-22T10:56:15Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=15129worker1 # [ 63.846581] nix-grpc-daemon[784]: ts=2026-09-22T10:56:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325130worker1 # [ 63.859270] nix-grpc-daemon[784]: ts=2026-09-22T10:56:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478325131worker1 # [ 63.873089] nix-grpc-daemon[784]: ts=2026-09-22T10:56:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478325132worker1 # [ 63.881672] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5133worker1 # [ 64.045320] nix-daemon[853]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)5134worker1 # [ 64.058957] nix-daemon[853]: reaped child process 1055, status = succeeded5135worker1 # [ 64.208214] nix-daemon[853]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)5136worker1 # [ 64.219726] nix-daemon[853]: reaped child process 1060, status = succeeded5137lb # [ 64.238951] niks3-server[1067]: 2026/09/22 10:56:16 INFO Received uploads request method=POST path=/api/pending_closures5138worker1 # [ 64.244318] nix-grpc-daemon[865]: time=2026-09-22T10:56:16.361Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5139worker1 # [ 64.245634] nix-grpc-daemon[865]: time=2026-09-22T10:56:16.361Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5140lb # [ 64.270853] niks3-server[1067]: 2026/09/22 10:56:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5141worker1 # [ 64.273655] nix-grpc-daemon[865]: time=2026-09-22T10:56:16.391Z level=INFO msg="Uploading 1 narinfos"5142lb # [ 64.277602] niks3-server[1067]: 2026/09/22 10:56:16 INFO Signed narinfos id=11 count=15143lb # [ 64.288965] niks3-server[1067]: 2026/09/22 10:56:16 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5144lb # [ 64.350490] niks3-server[1067]: 2026/09/22 10:56:16 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5145lb # [ 64.358170] niks3-server[1067]: 2026/09/22 10:56:16 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5146worker1 # [ 64.369579] nix-grpc-daemon[865]: time=2026-09-22T10:56:16.487Z level=INFO msg="Upload complete. (474ms)"5147lb # [ 64.373567] niks3-server[1067]: 2026/09/22 10:56:16 INFO Completed upload id=115148worker1 # [ 64.372069] nix-grpc-daemon[784]: ts=2026-09-22T10:56:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_in=4245149worker1 # [ 64.375186] nix-grpc-daemon[784]: ts=2026-09-22T10:56:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325150worker1 # [ 64.381586] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5151lb # [ 64.383513] niks3-server[1067]: 2026/09/22 10:56:16 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5152worker1 # [ 64.384080] nix-daemon[853]: reaped child process 1050, status = succeeded5153worker1 # [ 64.388867] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5154lb # [ 66.047142] envoy[910]: conn peer=[2001:db8:1::1]:41922 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4169 tx=1806 ms=22125155lb # [ 66.048757] envoy[910]: conn peer=[2001:db8:1::1]:41948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1576 tx=1373 ms=35745156lb # [ 66.050208] envoy[910]: conn peer=[2001:db8:1::1]:41938 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3011 tx=1451 ms=47385157worker1 # [ 70.602977] nix-grpc-daemon[784]: ts=2026-09-22T10:56:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=6 outputs=05158worker1 # [ 70.625617] nix-daemon[853]: reaped child process 1062, status = succeeded5159worker1 # [ 70.628148] nix-daemon[853]: reaped child process 1064, status = succeeded5160worker2 # [ 70.846825] nix-grpc-daemon[785]: ts=2026-09-22T10:56:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550405161worker2 # [ 70.857161] nix-grpc-daemon[785]: ts=2026-09-22T10:56:22Z level=debug event=want client=7 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05162worker2 # [ 70.863437] nix-grpc-daemon[785]: ts=2026-09-22T10:56:22Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=15163worker2 # [ 70.869461] nix-grpc-daemon[785]: ts=2026-09-22T10:56:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405164worker2 # [ 70.882162] nix-grpc-daemon[785]: ts=2026-09-22T10:56:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550405165worker2 # [ 70.893661] nix-grpc-daemon[785]: ts=2026-09-22T10:56:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550405166worker2 # [ 70.905375] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5167worker2 # [ 71.082771] nix-daemon[873]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)5168worker2 # [ 71.096203] nix-daemon[873]: reaped child process 1003, status = succeeded5169worker2 # [ 71.243814] nix-daemon[873]: accepted connection from pid 1005, user nix-grpc-daemon (trusted)5170worker2 # [ 71.254582] nix-daemon[873]: reaped child process 1008, status = succeeded5171lb # [ 71.271337] niks3-server[1067]: 2026/09/22 10:56:23 INFO Received uploads request method=POST path=/api/pending_closures5172worker2 # [ 71.276164] nix-grpc-daemon[923]: time=2026-09-22T10:56:22.575Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5173worker2 # [ 71.277873] nix-grpc-daemon[923]: time=2026-09-22T10:56:22.575Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5174lb # [ 71.303944] niks3-server[1067]: 2026/09/22 10:56:23 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5175lb # [ 71.311688] niks3-server[1067]: 2026/09/22 10:56:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5176worker2 # [ 71.315027] nix-grpc-daemon[923]: time=2026-09-22T10:56:22.614Z level=INFO msg="Uploading 1 narinfos"5177lb # [ 71.319236] niks3-server[1067]: 2026/09/22 10:56:23 INFO Signed narinfos id=12 count=15178lb # [ 71.326251] niks3-server[1067]: 2026/09/22 10:56:23 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5179lb # [ 71.337768] niks3-server[1067]: 2026/09/22 10:56:23 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5180lb # [ 71.344560] niks3-server[1067]: 2026/09/22 10:56:23 INFO Completed upload id=125181worker2 # [ 71.340986] nix-grpc-daemon[923]: time=2026-09-22T10:56:22.640Z level=INFO msg="Upload complete. (424ms)"5182lb # [ 71.347510] niks3-server[1067]: 2026/09/22 10:56:23 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5183worker2 # [ 71.344727] nix-grpc-daemon[785]: ts=2026-09-22T10:56:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_in=4245184worker2 # [ 71.350331] nix-grpc-daemon[785]: ts=2026-09-22T10:56:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405185worker2 # [ 71.359886] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5186worker2 # [ 71.368657] nix-daemon[873]: reaped child process 998, status = succeeded5187worker2 # [ 71.371590] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5188lb # [ 76.048365] envoy[910]: conn peer=[2001:db8:1::1]:32934 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2983 tx=972 ms=68255189worker2 # [ 77.383227] nix-grpc-daemon[785]: ts=2026-09-22T10:56:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=7 outputs=05190worker2 # [ 77.394670] nix-daemon[873]: reaped child process 1012, status = succeeded5191worker2 # [ 77.395617] nix-daemon[873]: reaped child process 1010, status = succeeded5192(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.89 seconds)5193subtest: the first client leaving does not fail the second5194client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share5195client # [ 77.507575] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share.5196client # Running as unit: share1.service; invocation ID: 5ab07be6bde84268921ea95db0b9d3bb5197client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share, in 0.03 seconds)5198client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share5199worker1 # [ 77.691687] nix-grpc-daemon[784]: ts=2026-09-22T10:56:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478325200client # [ 77.709867] nix[978]: don't know how to build these paths:5201client # [ 77.711329] nix[978]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5202worker2 # [ 77.703836] nix-grpc-daemon[785]: ts=2026-09-22T10:56:29Z level=debug event=want client=8 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05203worker2 # [ 77.705803] nix-grpc-daemon[785]: ts=2026-09-22T10:56:29Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=15204worker1 # [ 77.707850] nix-grpc-daemon[784]: ts=2026-09-22T10:56:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325205worker1 # [ 77.718315] nix-grpc-daemon[784]: ts=2026-09-22T10:56:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478325206client # [ 77.739882] nix[978]: copying 1 paths to 'lb:50051'...5207client # [ 77.740694] nix[978]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5208worker1 # [ 77.727817] nix-grpc-daemon[784]: ts=2026-09-22T10:56:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478325209worker1 # [ 77.739091] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5210worker1 # [ 77.882827] nix-daemon[853]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5211worker1 # [ 77.895100] nix-daemon[853]: reaped child process 1090, status = succeeded5212worker1 # [ 78.033521] nix-daemon[853]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)5213worker1 # [ 78.045477] nix-daemon[853]: reaped child process 1095, status = succeeded5214lb # [ 78.065089] niks3-server[1067]: 2026/09/22 10:56:30 INFO Received uploads request method=POST path=/api/pending_closures5215worker1 # [ 78.067556] nix-grpc-daemon[865]: time=2026-09-22T10:56:30.184Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5216worker1 # [ 78.069087] nix-grpc-daemon[865]: time=2026-09-22T10:56:30.186Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5217lb # [ 78.093920] niks3-server[1067]: 2026/09/22 10:56:30 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5218lb # [ 78.106142] niks3-server[1067]: 2026/09/22 10:56:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5219worker1 # [ 78.109603] nix-grpc-daemon[865]: time=2026-09-22T10:56:30.226Z level=INFO msg="Uploading 1 narinfos"5220lb # [ 78.113487] niks3-server[1067]: 2026/09/22 10:56:30 INFO Signed narinfos id=13 count=15221lb # [ 78.120940] niks3-server[1067]: 2026/09/22 10:56:30 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5222lb # [ 78.132611] niks3-server[1067]: 2026/09/22 10:56:30 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5223lb # [ 78.179827] niks3-server[1067]: 2026/09/22 10:56:30 INFO Completed upload id=135224worker1 # [ 78.176005] nix-grpc-daemon[865]: time=2026-09-22T10:56:30.293Z level=INFO msg="Upload complete. (425ms)"5225lb # [ 78.182201] niks3-server[1067]: 2026/09/22 10:56:30 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5226worker1 # [ 78.180730] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_in=4325227worker1 # [ 78.187093] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325228worker1 # [ 78.198157] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5229worker1 # [ 78.212577] nix-daemon[853]: reaped child process 1085, status = succeeded5230client # [ 78.229111] nix[978]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5231worker1 # [ 78.218102] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5232client # [ 78.570776] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share.5233client # Running as unit: share2.service; invocation ID: 665cc26dd03c4c6f92c1f833785f33cb5234client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share, in 1.08 seconds)5235client # [ 78.774374] nix[994]: don't know how to build these paths:5236client # [ 78.775348] nix[994]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5237worker2 # [ 78.760907] nix-grpc-daemon[785]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550405238worker2 # [ 78.767121] nix-grpc-daemon[785]: ts=2026-09-22T10:56:30Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05239worker1 # [ 78.769445] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325240client # [ 78.784858] nix[994]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5241client: must succeed: sleep 2; systemctl kill -s INT share15242client # [ 81.855730] systemd[1]: share1.service: Sent signal SIGINT to main process 978 (nix) on client request.5243client # [ 81.865260] nix[978]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5244client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5245client: waiting for success: ! systemctl is-active share15246client # [ 81.873775] nix[978]: error: interrupted by the user5247client # [ 81.912609] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5248client # [ 81.914175] systemd[1]: share1.service: Failed with result 'exit-code'.5249client # [ 81.916389] systemd[1]: share1.service: Consumed 114ms CPU time over 4.408s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8.2K outgoing IP traffic.5250client: (finished: waiting for success: ! systemctl is-active share1, in 0.05 seconds)5251client: waiting for success: ! systemctl is-active share25252worker1 # [ 82.253955] nix-daemon[853]: reaped child process 1099, status = succeeded5253worker1 # [ 82.404868] nix-daemon[853]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5254worker1 # [ 82.416839] nix-daemon[853]: reaped child process 1136, status = succeeded5255lb # [ 82.436728] niks3-server[1067]: 2026/09/22 10:56:34 INFO Received uploads request method=POST path=/api/pending_closures5256worker1 # [ 82.442187] nix-grpc-daemon[865]: time=2026-09-22T10:56:34.559Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5257worker1 # [ 82.445059] nix-grpc-daemon[865]: time=2026-09-22T10:56:34.561Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5258lb # [ 82.474536] niks3-server[1067]: 2026/09/22 10:56:34 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5259lb # [ 82.490157] niks3-server[1067]: 2026/09/22 10:56:34 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5260lb # [ 82.499166] niks3-server[1067]: 2026/09/22 10:56:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5261worker1 # [ 82.500070] nix-grpc-daemon[865]: time=2026-09-22T10:56:34.617Z level=INFO msg="Uploading 1 narinfos"5262lb # [ 82.503574] niks3-server[1067]: 2026/09/22 10:56:34 INFO Signed narinfos id=14 count=15263lb # [ 82.508485] niks3-server[1067]: 2026/09/22 10:56:34 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5264lb # [ 82.518590] niks3-server[1067]: 2026/09/22 10:56:34 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5265lb # [ 82.521039] niks3-server[1067]: 2026/09/22 10:56:34 INFO Completed upload id=145266worker1 # [ 82.518771] nix-grpc-daemon[865]: time=2026-09-22T10:56:34.636Z level=INFO msg="Upload complete. (286ms)"5267lb # [ 82.523917] niks3-server[1067]: 2026/09/22 10:56:34 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5268worker1 # [ 82.521800] nix-grpc-daemon[784]: ts=2026-09-22T10:56:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=8 outputs=15269worker1 # [ 82.525137] nix-grpc-daemon[784]: ts=2026-09-22T10:56:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=3 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=8 outputs=15270worker1 # [ 82.531198] nix-daemon[853]: reaped child process 1097, status = succeeded5271client # [ 82.559118] systemd[1]: share2.service: Deactivated successfully.5272client # [ 82.561290] systemd[1]: share2.service: Consumed 115ms CPU time over 3.990s wall clock time, 35.4M memory peak, 5.8K incoming IP traffic, 5.6K outgoing IP traffic.5273client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5274client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5275client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5276(finished: subtest: the first client leaving does not fail the second, in 5.77 seconds)5277subtest: build hook: nix-daemon with builders = grpc://lb5278client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log5279client # [ 83.417770] systemd[1]: Started Nix Daemon.5280client # [ 83.576870] nix-daemon[1040]: accepted connection from pid 1037, user root (trusted)5281worker1 # [ 83.750485] nix-grpc-daemon[784]: ts=2026-09-22T10:56:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:478325282worker2 # [ 83.755619] nix-grpc-daemon[785]: ts=2026-09-22T10:56:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550405283worker2 # [ 83.765928] nix-grpc-daemon[785]: ts=2026-09-22T10:56:35Z level=debug event=want client=10 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05284worker2 # [ 83.769710] nix-grpc-daemon[785]: ts=2026-09-22T10:56:35Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15285worker2 # [ 83.772711] nix-grpc-daemon[785]: ts=2026-09-22T10:56:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405286worker2 # [ 83.781119] nix-grpc-daemon[785]: ts=2026-09-22T10:56:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550405287worker2 # [ 83.789801] nix-grpc-daemon[785]: ts=2026-09-22T10:56:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550405288worker2 # [ 83.798215] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5289worker2 # [ 83.978042] nix-daemon[873]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)5290worker2 # [ 83.997129] nix-daemon[873]: reaped child process 1060, status = succeeded5291worker2 # [ 84.180249] nix-daemon[873]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)5292worker2 # [ 84.200206] nix-daemon[873]: reaped child process 1065, status = succeeded5293lb # [ 84.219703] niks3-server[1067]: 2026/09/22 10:56:36 INFO Received uploads request method=POST path=/api/pending_closures5294worker2 # [ 84.313158] nix-grpc-daemon[923]: time=2026-09-22T10:56:35.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5295worker2 # [ 84.315488] nix-grpc-daemon[923]: time=2026-09-22T10:56:35.614Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5296lb # [ 84.331856] niks3-server[1067]: 2026/09/22 10:56:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5297lb # [ 84.334816] niks3-server[1067]: 2026/09/22 10:56:36 INFO Signed narinfos id=15 count=15298worker2 # [ 84.331230] nix-grpc-daemon[923]: time=2026-09-22T10:56:35.630Z level=INFO msg="Uploading 1 narinfos"5299lb # [ 84.340872] niks3-server[1067]: 2026/09/22 10:56:36 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5300lb # [ 84.346238] niks3-server[1067]: 2026/09/22 10:56:36 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5301lb # [ 84.349198] niks3-server[1067]: 2026/09/22 10:56:36 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5302lb # [ 84.360287] niks3-server[1067]: 2026/09/22 10:56:36 INFO Completed upload id=155303worker2 # [ 84.356408] nix-grpc-daemon[923]: time=2026-09-22T10:56:35.655Z level=INFO msg="Upload complete. (550ms)"5304worker2 # [ 84.358975] nix-grpc-daemon[785]: ts=2026-09-22T10:56:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_in=5125305lb # [ 84.365384] niks3-server[1067]: 2026/09/22 10:56:36 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5306worker2 # [ 84.362140] nix-grpc-daemon[785]: ts=2026-09-22T10:56:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405307worker2 # [ 84.370070] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5308worker2 # [ 84.374507] nix-daemon[873]: reaped child process 1055, status = succeeded5309worker2 # [ 84.380155] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5310worker2 # [ 85.399916] nix-daemon[873]: reaped child process 1069, status = succeeded5311worker2 # [ 85.562141] nix-daemon[873]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)5312worker2 # [ 85.580880] nix-daemon[873]: reaped child process 1088, status = succeeded5313lb # [ 85.596836] niks3-server[1067]: 2026/09/22 10:56:37 INFO Received uploads request method=POST path=/api/pending_closures5314worker2 # [ 85.599577] nix-grpc-daemon[923]: time=2026-09-22T10:56:36.898Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5315worker2 # [ 85.602749] nix-grpc-daemon[923]: time=2026-09-22T10:56:36.900Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5316lb # [ 85.624504] niks3-server[1067]: 2026/09/22 10:56:37 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5317lb # [ 85.628840] niks3-server[1067]: 2026/09/22 10:56:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5318lb # [ 85.634510] niks3-server[1067]: 2026/09/22 10:56:37 INFO Signed narinfos id=16 count=15319worker2 # [ 85.631273] nix-grpc-daemon[923]: time=2026-09-22T10:56:36.930Z level=INFO msg="Uploading 1 narinfos"5320lb # [ 85.637526] niks3-server[1067]: 2026/09/22 10:56:37 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5321lb # [ 85.641303] niks3-server[1067]: 2026/09/22 10:56:37 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5322lb # [ 85.648037] niks3-server[1067]: 2026/09/22 10:56:37 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5323lb # [ 85.654847] niks3-server[1067]: 2026/09/22 10:56:37 INFO Completed upload id=165324worker2 # [ 85.651389] nix-grpc-daemon[923]: time=2026-09-22T10:56:36.950Z level=INFO msg="Upload complete. (262ms)"5325lb # [ 85.656924] niks3-server[1067]: 2026/09/22 10:56:37 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5326worker2 # [ 85.655534] nix-grpc-daemon[785]: ts=2026-09-22T10:56:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=9 outputs=15327worker2 # [ 85.665787] nix-daemon[873]: reaped child process 1067, status = succeeded5328worker1 # [ 85.704374] nix-grpc-daemon[784]: ts=2026-09-22T10:56:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:478325329worker1 # [ 85.715102] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5330worker1 # [ 85.745676] nix-grpc-daemon[784]: ts=2026-09-22T10:56:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:478325331worker1 # [ 85.749699] nix-grpc-daemon[784]: ts=2026-09-22T10:56:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_out=1285332worker1 # [ 86.016693] nix-grpc-daemon[784]: ts=2026-09-22T10:56:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:478325333worker1 # [ 86.020563] nix-grpc-daemon[784]: ts=2026-09-22T10:56:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478325334worker1 # [ 86.024265] nix-grpc-daemon[865]: time=2026-09-22T10:56:38.141Z level=INFO msg="All 1 paths already cached"5335worker2 # [ 86.037984] nix-grpc-daemon[785]: ts=2026-09-22T10:56:37Z level=debug event=want client=11 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05336worker2 # [ 86.040505] nix-grpc-daemon[785]: ts=2026-09-22T10:56:37Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15337lb # [ 86.049394] envoy[910]: conn peer=[2001:db8:1::1]:32940 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=977 ms=65495338lb # [ 86.051344] envoy[910]: conn peer=[2001:db8:1::1]:59350 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=812 ms=41795339lb # [ 86.053739] envoy[910]: conn peer=[2001:db8:1::1]:59364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=37725340lb # [ 86.055719] envoy[910]: conn peer=[2001:db8:1::1]:41766 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=585341lb # [ 86.057471] envoy[910]: conn peer=[2001:db8:1::1]:41764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1850 ms=20545342worker2 # [ 86.083244] nix-grpc-daemon[785]: ts=2026-09-22T10:56:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405343worker2 # [ 86.093419] nix-grpc-daemon[785]: ts=2026-09-22T10:56:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550405344worker2 # [ 86.102438] nix-grpc-daemon[923]: time=2026-09-22T10:56:37.401Z level=INFO msg="All 1 paths already cached"5345worker2 # [ 86.107473] nix-grpc-daemon[785]: ts=2026-09-22T10:56:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550405346worker2 # [ 86.114067] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5347worker2 # [ 86.287898] nix-daemon[873]: accepted connection from pid 1094, user nix-grpc-daemon (trusted)5348worker2 # [ 86.305138] nix-daemon[873]: reaped child process 1097, status = succeeded5349worker2 # [ 86.468070] nix-daemon[873]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)5350worker2 # [ 86.482096] nix-daemon[873]: reaped child process 1102, status = succeeded5351lb # [ 86.497244] niks3-server[1067]: 2026/09/22 10:56:38 INFO Received uploads request method=POST path=/api/pending_closures5352worker2 # [ 86.503605] nix-grpc-daemon[923]: time=2026-09-22T10:56:37.802Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5353worker2 # [ 86.505152] nix-grpc-daemon[923]: time=2026-09-22T10:56:37.802Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5354lb # [ 86.529236] niks3-server[1067]: 2026/09/22 10:56:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5355lb # [ 86.539164] niks3-server[1067]: 2026/09/22 10:56:38 INFO Signed narinfos id=17 count=15356worker2 # [ 86.540261] nix-grpc-daemon[923]: time=2026-09-22T10:56:37.839Z level=INFO msg="Uploading 1 narinfos"5357lb # [ 86.553922] niks3-server[1067]: 2026/09/22 10:56:38 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5358lb # [ 86.789687] niks3-server[1067]: 2026/09/22 10:56:38 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5359lb # [ 86.804736] niks3-server[1067]: 2026/09/22 10:56:38 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5360lb # [ 86.818890] niks3-server[1067]: 2026/09/22 10:56:38 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5361lb # [ 86.822197] niks3-server[1067]: 2026/09/22 10:56:38 INFO Completed upload id=175362worker2 # [ 86.819200] nix-grpc-daemon[923]: time=2026-09-22T10:56:38.118Z level=INFO msg="Upload complete. (697ms)"5363worker2 # [ 86.822470] nix-grpc-daemon[785]: ts=2026-09-22T10:56:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_in=6245364worker2 # [ 86.827384] nix-grpc-daemon[785]: ts=2026-09-22T10:56:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405365worker2 # [ 86.831733] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5366worker2 # [ 86.841183] nix-daemon[873]: reaped child process 1092, status = succeeded5367worker2 # [ 86.845566] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5368worker2 # [ 87.459791] nix-daemon[873]: reaped child process 1106, status = succeeded5369worker2 # [ 87.610759] nix-daemon[873]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)5370worker2 # [ 87.625367] nix-daemon[873]: reaped child process 1125, status = succeeded5371lb # [ 87.643438] niks3-server[1067]: 2026/09/22 10:56:39 INFO Received uploads request method=POST path=/api/pending_closures5372worker2 # [ 87.649086] nix-grpc-daemon[923]: time=2026-09-22T10:56:38.948Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5373worker2 # [ 87.651690] nix-grpc-daemon[923]: time=2026-09-22T10:56:38.950Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5374lb # [ 87.687632] niks3-server[1067]: 2026/09/22 10:56:39 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5375lb # [ 87.698156] niks3-server[1067]: 2026/09/22 10:56:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5376lb # [ 87.703109] niks3-server[1067]: 2026/09/22 10:56:39 INFO Signed narinfos id=18 count=15377worker2 # [ 87.702663] nix-grpc-daemon[923]: time=2026-09-22T10:56:39.001Z level=INFO msg="Uploading 1 narinfos"5378lb # [ 87.708702] niks3-server[1067]: 2026/09/22 10:56:39 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5379lb # [ 87.717550] niks3-server[1067]: 2026/09/22 10:56:39 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5380lb # [ 87.726986] niks3-server[1067]: 2026/09/22 10:56:39 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5381lb # [ 87.731078] niks3-server[1067]: 2026/09/22 10:56:39 INFO Completed upload id=185382worker2 # [ 87.727537] nix-grpc-daemon[923]: time=2026-09-22T10:56:39.026Z level=INFO msg="Upload complete. (272ms)"5383lb # [ 87.733957] niks3-server[1067]: 2026/09/22 10:56:39 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5384worker2 # [ 87.730417] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=10 outputs=15385worker2 # [ 87.737255] nix-daemon[873]: reaped child process 1104, status = succeeded5386worker1 # [ 87.777890] nix-grpc-daemon[784]: ts=2026-09-22T10:56:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:478325387worker1 # [ 87.790610] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5388worker2 # [ 87.824498] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550405389worker2 # [ 87.830569] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_out=1765390worker2 # [ 88.060332] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:550405391worker1 # [ 88.062875] nix-grpc-daemon[784]: ts=2026-09-22T10:56:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478325392worker1 # [ 88.070503] nix-grpc-daemon[865]: time=2026-09-22T10:56:40.187Z level=INFO msg="All 2 paths already cached"5393worker2 # [ 88.083247] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=debug event=want client=12 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05394worker2 # [ 88.088770] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15395worker2 # [ 88.096373] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405396worker2 # [ 88.110813] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550405397worker2 # [ 88.125099] nix-grpc-daemon[923]: time=2026-09-22T10:56:39.423Z level=INFO msg="All 2 paths already cached"5398worker2 # [ 88.132953] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550405399worker2 # [ 88.142330] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5400worker2 # [ 88.309695] nix-daemon[873]: accepted connection from pid 1131, user nix-grpc-daemon (trusted)5401worker2 # [ 88.324678] nix-daemon[873]: reaped child process 1134, status = succeeded5402worker2 # [ 88.481851] nix-daemon[873]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)5403worker2 # [ 88.496676] nix-daemon[873]: reaped child process 1139, status = succeeded5404lb # [ 88.510305] niks3-server[1067]: 2026/09/22 10:56:40 INFO Received uploads request method=POST path=/api/pending_closures5405worker2 # [ 88.511931] nix-grpc-daemon[923]: time=2026-09-22T10:56:39.811Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5406worker2 # [ 88.513540] nix-grpc-daemon[923]: time=2026-09-22T10:56:39.811Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5407lb # [ 88.526351] niks3-server[1067]: 2026/09/22 10:56:40 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5408lb # [ 88.532801] niks3-server[1067]: 2026/09/22 10:56:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5409lb # [ 88.536580] niks3-server[1067]: 2026/09/22 10:56:40 INFO Signed narinfos id=19 count=15410worker2 # [ 88.535864] nix-grpc-daemon[923]: time=2026-09-22T10:56:39.834Z level=INFO msg="Uploading 1 narinfos"5411lb # [ 88.545591] niks3-server[1067]: 2026/09/22 10:56:40 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5412lb # [ 88.551787] niks3-server[1067]: 2026/09/22 10:56:40 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5413lb # [ 88.555082] niks3-server[1067]: 2026/09/22 10:56:40 INFO Completed upload id=195414worker2 # [ 88.551450] nix-grpc-daemon[923]: time=2026-09-22T10:56:39.850Z level=INFO msg="Upload complete. (395ms)"5415worker2 # [ 88.553817] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_in=7525416lb # [ 88.558305] niks3-server[1067]: 2026/09/22 10:56:40 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5417worker2 # [ 88.557541] nix-grpc-daemon[785]: ts=2026-09-22T10:56:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405418worker2 # [ 88.563183] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5419worker2 # [ 88.569221] nix-daemon[873]: reaped child process 1129, status = succeeded5420worker2 # [ 88.575477] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5421worker2 # [ 89.204856] nix-daemon[873]: reaped child process 1143, status = succeeded5422worker2 # [ 89.337893] nix-daemon[873]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5423worker2 # [ 89.353363] nix-daemon[873]: reaped child process 1162, status = succeeded5424lb # [ 89.368224] niks3-server[1067]: 2026/09/22 10:56:41 INFO Received uploads request method=POST path=/api/pending_closures5425worker2 # [ 89.374192] nix-grpc-daemon[923]: time=2026-09-22T10:56:40.673Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5426worker2 # [ 89.378268] nix-grpc-daemon[923]: time=2026-09-22T10:56:40.675Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5427lb # [ 89.416512] niks3-server[1067]: 2026/09/22 10:56:41 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5428lb # [ 89.426400] niks3-server[1067]: 2026/09/22 10:56:41 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5429lb # [ 89.437507] niks3-server[1067]: 2026/09/22 10:56:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5430lb # [ 89.443525] niks3-server[1067]: 2026/09/22 10:56:41 INFO Signed narinfos id=20 count=15431worker2 # [ 89.442844] nix-grpc-daemon[923]: time=2026-09-22T10:56:40.742Z level=INFO msg="Uploading 1 narinfos"5432lb # [ 89.446793] niks3-server[1067]: 2026/09/22 10:56:41 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5433lb # [ 89.457366] niks3-server[1067]: 2026/09/22 10:56:41 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5434lb # [ 89.461314] niks3-server[1067]: 2026/09/22 10:56:41 INFO Completed upload id=205435worker2 # [ 89.457739] nix-grpc-daemon[923]: time=2026-09-22T10:56:40.756Z level=INFO msg="Upload complete. (275ms)"5436lb # [ 89.463300] niks3-server[1067]: 2026/09/22 10:56:41 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5437worker2 # [ 89.464732] nix-grpc-daemon[785]: ts=2026-09-22T10:56:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=11 outputs=15438worker2 # [ 89.477934] nix-daemon[873]: reaped child process 1141, status = succeeded5439worker1 # [ 89.508068] nix-grpc-daemon[784]: ts=2026-09-22T10:56:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:478325440worker1 # [ 89.515825] nix-grpc-daemon[784]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5441worker1 # [ 89.533856] nix-grpc-daemon[784]: ts=2026-09-22T10:56:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:478325442worker1 # [ 89.536100] nix-grpc-daemon[784]: ts=2026-09-22T10:56:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_out=2325443client # [ 89.662427] nix-daemon[1040]: reaped child process 1042, status = succeeded5444client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.41 seconds)5445client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5446client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5447client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5448client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)5449client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5450client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5451(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.48 seconds)5452subtest: an input only one worker has still reaches the builder5453client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5454client # this derivation will be built:5455client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5456client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5457client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.35 seconds)5458client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5459client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.16 seconds)5460worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5461worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.21 seconds)5462worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5463worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5464client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25465client # [ 90.610929] nix-daemon[1040]: accepted connection from pid 1123, user root (trusted)5466client # this derivation will be built:5467client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5468worker1 # [ 90.768717] nix-grpc-daemon[784]: ts=2026-09-22T10:56:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:478325469client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5470worker2 # [ 90.772218] nix-grpc-daemon[785]: ts=2026-09-22T10:56:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550405471client # copying 1 paths...5472client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5473worker2 # [ 90.782824] nix-grpc-daemon[785]: ts=2026-09-22T10:56:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550405474worker2 # [ 90.789322] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5475worker2 # [ 90.948198] nix-daemon[873]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5476worker2 # [ 90.963151] nix-daemon[873]: reaped child process 1172, status = succeeded5477lb # [ 90.981964] niks3-server[1067]: 2026/09/22 10:56:42 INFO Received uploads request method=POST path=/api/pending_closures5478worker2 # [ 90.984406] nix-grpc-daemon[923]: time=2026-09-22T10:56:42.283Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5479worker2 # [ 90.986487] nix-grpc-daemon[923]: time=2026-09-22T10:56:42.285Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5480lb # [ 91.011333] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5481lb # [ 91.025921] niks3-server[1067]: 2026/09/22 10:56:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5482lb # [ 91.029556] niks3-server[1067]: 2026/09/22 10:56:43 INFO Signed narinfos id=21 count=15483worker2 # [ 91.028908] nix-grpc-daemon[923]: time=2026-09-22T10:56:42.328Z level=INFO msg="Uploading 1 narinfos"5484lb # [ 91.034500] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5485lb # [ 91.041374] niks3-server[1067]: 2026/09/22 10:56:43 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5486lb # [ 91.046244] niks3-server[1067]: 2026/09/22 10:56:43 INFO Completed upload id=215487worker2 # [ 91.042738] nix-grpc-daemon[923]: time=2026-09-22T10:56:42.342Z level=INFO msg="Upload complete. (247ms)"5488worker2 # [ 91.045162] nix-grpc-daemon[785]: ts=2026-09-22T10:56:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_in=1205489lb # [ 91.050424] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5490worker2 # [ 91.054586] nix-daemon[873]: reaped child process 1167, status = succeeded5491worker2 # [ 91.059802] nix-grpc-daemon[785]: ts=2026-09-22T10:56:42Z level=debug event=want client=13 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05492worker2 # [ 91.062164] nix-grpc-daemon[785]: ts=2026-09-22T10:56:42Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15493worker1 # [ 91.063949] nix-grpc-daemon[784]: ts=2026-09-22T10:56:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325494worker1 # [ 91.072117] nix-grpc-daemon[784]: ts=2026-09-22T10:56:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478325495client # copying 2 paths to 'lb:50051'...5496client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5497worker1 # [ 91.081867] nix-grpc-daemon[784]: ts=2026-09-22T10:56:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478325498client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5499worker1 # [ 91.086892] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5500worker1 # [ 91.218161] nix-daemon[853]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)5501worker1 # [ 91.232643] nix-daemon[853]: reaped child process 1151, status = succeeded5502worker1 # [ 91.374522] nix-daemon[853]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5503worker1 # [ 91.389071] nix-daemon[853]: reaped child process 1156, status = succeeded5504lb # [ 91.403262] niks3-server[1067]: 2026/09/22 10:56:43 INFO Received uploads request method=POST path=/api/pending_closures5505lb # [ 91.414583] niks3-server[1067]: 2026/09/22 10:56:43 INFO Received uploads request method=POST path=/api/pending_closures5506worker1 # [ 91.422280] nix-grpc-daemon[865]: time=2026-09-22T10:56:43.539Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5507worker1 # [ 91.424646] nix-grpc-daemon[865]: time=2026-09-22T10:56:43.541Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5508worker1 # [ 91.426321] nix-grpc-daemon[865]: time=2026-09-22T10:56:43.541Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5509lb # [ 91.454310] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5510lb # [ 91.464982] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5511lb # [ 91.469063] niks3-server[1067]: 2026/09/22 10:56:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5512lb # [ 91.471487] niks3-server[1067]: 2026/09/22 10:56:43 INFO Signed narinfos id=22 count=15513lb # [ 91.475910] niks3-server[1067]: 2026/09/22 10:56:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5514lb # [ 91.478382] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5515lb # [ 91.480782] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5516lb # [ 91.482287] niks3-server[1067]: 2026/09/22 10:56:43 INFO Signed narinfos id=23 count=25517worker1 # [ 91.477798] nix-grpc-daemon[865]: time=2026-09-22T10:56:43.594Z level=INFO msg="Uploading 3 narinfos"5518lb # [ 91.503754] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5519lb # [ 91.506684] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5520lb # [ 91.511078] niks3-server[1067]: 2026/09/22 10:56:43 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5521lb # [ 91.514688] niks3-server[1067]: 2026/09/22 10:56:43 INFO Completed upload id=225522lb # [ 91.515990] niks3-server[1067]: 2026/09/22 10:56:43 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5523lb # [ 91.519256] niks3-server[1067]: 2026/09/22 10:56:43 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5524lb # [ 91.522293] niks3-server[1067]: 2026/09/22 10:56:43 INFO Completed upload id=235525worker1 # [ 91.518647] nix-grpc-daemon[865]: time=2026-09-22T10:56:43.636Z level=INFO msg="Upload complete. (424ms)"5526worker1 # [ 91.524160] nix-grpc-daemon[784]: ts=2026-09-22T10:56:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=2 nar_bytes_in=9765527worker1 # [ 91.529828] nix-grpc-daemon[784]: ts=2026-09-22T10:56:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325528worker1 # [ 91.540972] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5529worker1 # [ 91.545606] nix-daemon[853]: reaped child process 1146, status = succeeded5530worker1 # [ 91.549946] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5531client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5532client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5533client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5534worker1 # [ 92.369692] nix-daemon[853]: reaped child process 1160, status = succeeded5535worker1 # [ 92.497275] nix-daemon[853]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5536worker1 # [ 92.510744] nix-daemon[853]: reaped child process 1179, status = succeeded5537lb # [ 92.525703] niks3-server[1067]: 2026/09/22 10:56:44 INFO Received uploads request method=POST path=/api/pending_closures5538worker1 # [ 92.529983] nix-grpc-daemon[865]: time=2026-09-22T10:56:44.647Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5539lb # [ 92.558714] niks3-server[1067]: 2026/09/22 10:56:44 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5540lb # [ 92.564551] niks3-server[1067]: 2026/09/22 10:56:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5541lb # [ 92.572968] niks3-server[1067]: 2026/09/22 10:56:44 INFO Signed narinfos id=24 count=15542worker1 # [ 92.573453] nix-grpc-daemon[865]: time=2026-09-22T10:56:44.690Z level=INFO msg="Uploading 1 narinfos"5543lb # [ 92.581968] niks3-server[1067]: 2026/09/22 10:56:44 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5544lb # [ 92.594728] niks3-server[1067]: 2026/09/22 10:56:44 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5545lb # [ 92.598043] niks3-server[1067]: 2026/09/22 10:56:44 INFO Completed upload id=245546worker1 # [ 92.594570] nix-grpc-daemon[865]: time=2026-09-22T10:56:44.711Z level=INFO msg="Upload complete. (235ms)"5547lb # [ 92.599907] niks3-server[1067]: 2026/09/22 10:56:44 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5548worker1 # [ 92.599482] nix-grpc-daemon[784]: ts=2026-09-22T10:56:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=12 outputs=15549worker1 # [ 92.610714] nix-daemon[853]: reaped child process 1158, status = succeeded5550client # copying 1 paths...5551worker1 # [ 92.645148] nix-grpc-daemon[784]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:478325552client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5553worker1 # [ 92.657370] nix-grpc-daemon[784]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:478325554worker1 # [ 92.659041] nix-grpc-daemon[784]: ts=2026-09-22T10:56:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_out=1205555client # [ 92.768126] nix-daemon[1040]: reaped child process 1126, status = succeeded5556client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.29 seconds)5557client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25558client # [ 92.910344] nix-daemon[1040]: accepted connection from pid 1145, user root (trusted)5559client # this derivation will be built:5560client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5561worker1 # [ 93.053232] nix-grpc-daemon[784]: ts=2026-09-22T10:56:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:478325562worker1 # [ 93.056204] nix-grpc-daemon[784]: ts=2026-09-22T10:56:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478325563client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5564worker1 # [ 93.061084] nix-grpc-daemon[865]: time=2026-09-22T10:56:45.178Z level=INFO msg="All 1 paths already cached"5565worker2 # [ 93.077310] nix-grpc-daemon[785]: ts=2026-09-22T10:56:44Z level=debug event=want client=14 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05566worker2 # [ 93.082500] nix-grpc-daemon[785]: ts=2026-09-22T10:56:44Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15567worker2 # [ 93.119793] nix-grpc-daemon[785]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405568worker2 # [ 93.131479] nix-grpc-daemon[785]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550405569client # copying 1 paths to 'lb:50051'...5570client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5571worker2 # [ 93.145626] nix-grpc-daemon[785]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550405572worker2 # [ 93.157417] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5573worker2 # [ 93.173073] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5)5574worker2 # [ 93.245965] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 2/5)5575worker2 # [ 93.319686] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 3/5)5576worker2 # [ 93.384537] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 787 ms (attempt 4/5)5577worker2 # [ 94.175701] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5578worker2 # [ 94.179938] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5579worker2 # [ 94.185230] nix-grpc-daemon[785]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5580worker2 # [ 94.341236] nix-daemon[873]: accepted connection from pid 1185, user nix-grpc-daemon (trusted)5581worker2 # [ 94.354357] nix-daemon[873]: reaped child process 1188, status = succeeded5582worker2 # [ 94.498983] nix-daemon[873]: accepted connection from pid 1190, user nix-grpc-daemon (trusted)5583worker2 # [ 94.510789] nix-daemon[873]: reaped child process 1193, status = succeeded5584lb # [ 94.526619] niks3-server[1067]: 2026/09/22 10:56:46 INFO Received uploads request method=POST path=/api/pending_closures5585worker2 # [ 94.532850] nix-grpc-daemon[923]: time=2026-09-22T10:56:45.832Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5586worker2 # [ 94.534717] nix-grpc-daemon[923]: time=2026-09-22T10:56:45.833Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5587lb # [ 94.549196] niks3-server[1067]: 2026/09/22 10:56:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5588lb # [ 94.553659] niks3-server[1067]: 2026/09/22 10:56:46 INFO Signed narinfos id=25 count=15589worker2 # [ 94.549841] nix-grpc-daemon[923]: time=2026-09-22T10:56:45.849Z level=INFO msg="Uploading 1 narinfos"5590lb # [ 94.556927] niks3-server[1067]: 2026/09/22 10:56:46 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5591lb # [ 94.559896] niks3-server[1067]: 2026/09/22 10:56:46 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5592lb # [ 94.565062] niks3-server[1067]: 2026/09/22 10:56:46 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5593lb # [ 94.569054] niks3-server[1067]: 2026/09/22 10:56:46 INFO Completed upload id=255594worker2 # [ 94.565398] nix-grpc-daemon[923]: time=2026-09-22T10:56:45.864Z level=INFO msg="Upload complete. (359ms)"5595worker2 # [ 94.567323] nix-grpc-daemon[785]: ts=2026-09-22T10:56:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=1 paths=1 nar_bytes_in=5605596lb # [ 94.572314] niks3-server[1067]: 2026/09/22 10:56:46 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5597worker2 # [ 94.571115] nix-grpc-daemon[785]: ts=2026-09-22T10:56:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550405598worker2 # [ 94.578486] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5599worker2 # [ 94.580385] nix-daemon[873]: reaped child process 1178, status = succeeded5600worker2 # [ 94.585352] nix-daemon[873]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5601client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5602client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5603client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5604worker2 # [ 95.799779] nix-daemon[873]: reaped child process 1197, status = succeeded5605worker2 # [ 95.932603] nix-daemon[873]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5606worker2 # [ 95.944539] nix-daemon[873]: reaped child process 1216, status = succeeded5607lb # [ 95.960979] niks3-server[1067]: 2026/09/22 10:56:47 INFO Received uploads request method=POST path=/api/pending_closures5608worker2 # [ 95.966054] nix-grpc-daemon[923]: time=2026-09-22T10:56:47.265Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5609lb # [ 95.985505] niks3-server[1067]: 2026/09/22 10:56:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5610lb # [ 95.989499] niks3-server[1067]: 2026/09/22 10:56:47 INFO Signed narinfos id=26 count=15611worker2 # [ 95.986028] nix-grpc-daemon[923]: time=2026-09-22T10:56:47.285Z level=INFO msg="Uploading 1 narinfos"5612lb # [ 95.993590] niks3-server[1067]: 2026/09/22 10:56:47 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5613lb # [ 95.997550] niks3-server[1067]: 2026/09/22 10:56:47 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5614lb # [ 96.004779] niks3-server[1067]: 2026/09/22 10:56:48 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5615lb # [ 96.009554] niks3-server[1067]: 2026/09/22 10:56:48 INFO Completed upload id=265616worker2 # [ 96.005955] nix-grpc-daemon[923]: time=2026-09-22T10:56:47.305Z level=INFO msg="Upload complete. (212ms)"5617lb # [ 96.012347] niks3-server[1067]: 2026/09/22 10:56:48 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5618worker2 # [ 96.009192] nix-grpc-daemon[785]: ts=2026-09-22T10:56:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=13 outputs=15619worker2 # [ 96.016366] nix-daemon[873]: reaped child process 1195, status = succeeded5620lb # [ 96.049825] envoy[910]: conn peer=[2001:db8:1::1]:41782 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3867 tx=2070 ms=18255621lb # [ 96.051626] envoy[910]: conn peer=[2001:db8:1::1]:41798 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=215622lb # [ 96.053332] envoy[910]: conn peer=[2001:db8:1::1]:41806 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=495623lb # [ 96.055916] envoy[910]: conn peer=[2001:db8:1::1]:41804 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2299 ms=15255624lb # [ 96.057780] envoy[910]: conn peer=[2001:db8:1::1]:41814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4209 tx=1828 ms=19415625lb # [ 96.059715] envoy[910]: conn peer=[2001:db8:1::1]:41828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=495626client # copying 1 paths...5627worker2 # [ 96.061083] nix-grpc-daemon[785]: ts=2026-09-22T10:56:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550405628client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5629worker2 # [ 96.076587] nix-grpc-daemon[785]: ts=2026-09-22T10:56:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550405630worker2 # [ 96.078287] nix-grpc-daemon[785]: ts=2026-09-22T10:56:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_out=1205631client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.43 seconds)5632client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25633client # [ 96.248498] nix-daemon[1040]: reaped child process 1148, status = succeeded5634client # [ 96.374866] nix-daemon[1040]: accepted connection from pid 1167, user root (trusted)5635client # this derivation will be built:5636client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5637worker2 # [ 96.528767] nix-grpc-daemon[785]: ts=2026-09-22T10:56:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:550405638client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5639worker1 # [ 96.532236] nix-grpc-daemon[784]: ts=2026-09-22T10:56:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478325640worker1 # [ 96.538815] nix-grpc-daemon[865]: time=2026-09-22T10:56:48.656Z level=INFO msg="All 1 paths already cached"5641worker2 # [ 96.548144] nix-grpc-daemon[785]: ts=2026-09-22T10:56:47Z level=debug event=want client=15 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05642worker1 # [ 96.549917] nix-grpc-daemon[784]: ts=2026-09-22T10:56:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325643worker2 # [ 96.550195] nix-grpc-daemon[785]: ts=2026-09-22T10:56:47Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15644worker1 # [ 96.556765] nix-grpc-daemon[784]: ts=2026-09-22T10:56:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:478325645worker1 # [ 96.563796] nix-grpc-daemon[865]: time=2026-09-22T10:56:48.681Z level=INFO msg="All 1 paths already cached"5646client # copying 1 paths to 'lb:50051'...5647client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5648worker1 # [ 96.567926] nix-grpc-daemon[784]: ts=2026-09-22T10:56:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:478325649worker1 # [ 96.573215] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5650worker1 # [ 96.725219] nix-daemon[853]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)5651worker1 # [ 96.740533] nix-daemon[853]: reaped child process 1186, status = succeeded5652worker1 # [ 96.891615] nix-daemon[853]: accepted connection from pid 1188, user nix-grpc-daemon (trusted)5653worker1 # [ 96.904973] nix-daemon[853]: reaped child process 1191, status = succeeded5654lb # [ 96.918579] niks3-server[1067]: 2026/09/22 10:56:48 INFO Received uploads request method=POST path=/api/pending_closures5655worker1 # [ 96.927224] nix-grpc-daemon[865]: time=2026-09-22T10:56:49.044Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5656worker1 # [ 96.928657] nix-grpc-daemon[865]: time=2026-09-22T10:56:49.044Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5657lb # [ 96.944920] niks3-server[1067]: 2026/09/22 10:56:48 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5658lb # [ 96.950523] niks3-server[1067]: 2026/09/22 10:56:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5659lb # [ 96.954207] niks3-server[1067]: 2026/09/22 10:56:48 INFO Signed narinfos id=27 count=15660worker1 # [ 96.952497] nix-grpc-daemon[865]: time=2026-09-22T10:56:49.069Z level=INFO msg="Uploading 1 narinfos"5661lb # [ 96.956763] niks3-server[1067]: 2026/09/22 10:56:48 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5662lb # [ 96.965054] niks3-server[1067]: 2026/09/22 10:56:48 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5663lb # [ 96.968734] niks3-server[1067]: 2026/09/22 10:56:48 INFO Completed upload id=275664worker1 # [ 96.964847] nix-grpc-daemon[865]: time=2026-09-22T10:56:49.082Z level=INFO msg="Upload complete. (386ms)"5665lb # [ 96.971648] niks3-server[1067]: 2026/09/22 10:56:48 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5666worker1 # [ 96.967197] nix-grpc-daemon[784]: ts=2026-09-22T10:56:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=0 paths=1 nar_bytes_in=5605667worker1 # [ 96.972341] nix-grpc-daemon[784]: ts=2026-09-22T10:56:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:478325668worker1 # [ 96.979357] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5669worker1 # [ 96.982215] nix-daemon[853]: reaped child process 1181, status = succeeded5670client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5671worker1 # [ 96.988524] nix-daemon[853]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5672client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5673client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5674worker1 # [ 98.205205] nix-daemon[853]: reaped child process 1195, status = succeeded5675worker1 # [ 98.311395] nix-daemon[853]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5676worker1 # [ 98.323330] nix-daemon[853]: reaped child process 1214, status = succeeded5677lb # [ 98.341469] niks3-server[1067]: 2026/09/22 10:56:50 INFO Received uploads request method=POST path=/api/pending_closures5678worker1 # [ 98.344555] nix-grpc-daemon[865]: time=2026-09-22T10:56:50.461Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5679lb # [ 98.359031] niks3-server[1067]: 2026/09/22 10:56:50 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5680lb # [ 98.362057] niks3-server[1067]: 2026/09/22 10:56:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5681lb # [ 98.364239] niks3-server[1067]: 2026/09/22 10:56:50 INFO Signed narinfos id=28 count=15682worker1 # [ 98.360508] nix-grpc-daemon[865]: time=2026-09-22T10:56:50.477Z level=INFO msg="Uploading 1 narinfos"5683lb # [ 98.368682] niks3-server[1067]: 2026/09/22 10:56:50 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5684lb # [ 98.371934] niks3-server[1067]: 2026/09/22 10:56:50 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5685lb # [ 98.375365] niks3-server[1067]: 2026/09/22 10:56:50 INFO Completed upload id=285686worker1 # [ 98.371590] nix-grpc-daemon[865]: time=2026-09-22T10:56:50.489Z level=INFO msg="Upload complete. (175ms)"5687lb # [ 98.376908] niks3-server[1067]: 2026/09/22 10:56:50 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5688worker1 # [ 98.374681] nix-grpc-daemon[784]: ts=2026-09-22T10:56:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47832 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=14 outputs=15689client # copying 1 paths...5690worker1 # [ 98.381318] nix-daemon[853]: reaped child process 1193, status = succeeded5691worker2 # [ 98.380811] nix-grpc-daemon[785]: ts=2026-09-22T10:56:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550405692worker2 # [ 98.387852] nix-grpc-daemon[785]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5693client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5694worker2 # [ 98.405145] nix-grpc-daemon[785]: ts=2026-09-22T10:56:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550405695worker2 # [ 98.408089] nix-grpc-daemon[785]: ts=2026-09-22T10:56:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55040 duration_s=0 paths=1 nar_bytes_out=1205696client # [ 98.510236] nix-daemon[1040]: reaped child process 1170, status = succeeded5697client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.32 seconds)5698(finished: subtest: an input only one worker has still reaches the builder, in 8.79 seconds)5699subtest: upload whose reference the worker lacks is completed from the cache5700client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5701client # this derivation will be built:5702client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5703client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5704client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.36 seconds)5705client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5706client # this derivation will be built:5707client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5708client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5709client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.37 seconds)5710client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5711client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.16 seconds)5712worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5713worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.17 seconds)5714worker2 # [ 99.615878] systemd[1]: Stopping Nix worker-protocol over gRPC...5715worker2 # [ 100.514673] nix-grpc-daemon[785]: ts=2026-09-22T10:56:51Z level=info event=signal_exit5716lb # [ 100.804981] niks3-server[1067]: 2026/09/22 10:56:52 INFO lead: released remote=192.168.1.4:437805717lb # [ 101.002710] niks3-server[1067]: 2026/09/22 10:56:53 INFO lead: acquired remote=192.168.1.3:365445718worker1 # [ 101.003937] nix-grpc-daemon[784]: ts=2026-09-22T10:56:53Z level=info event=scheduler_take_over5719worker1 # [ 101.803316] nix-grpc-daemon[784]: ts=2026-09-22T10:56:53Z level=info event=scheduler_restarting addr=lb:500515720worker2 # [ 101.805937] nix-grpc-daemon[785]: ts=2026-09-22T10:56:53Z level=info event=scheduler_restarting5721worker2 # [ 101.810941] nix-grpc-daemon[785]: ts=2026-09-22T10:56:53Z level=info event=scheduler_restarting addr=lb:500515722worker2 # [ 101.814433] nix-grpc-daemon[785]: ts=2026-09-22T10:56:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455125723worker2 # [ 101.818199] nix-grpc-daemon[785]: ts=2026-09-22T10:56:53Z level=info event=worker_gone addr=192.168.1.4:500515724worker2 # [ 101.821726] nix-grpc-daemon[785]: ts=2026-09-22T10:56:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455125725worker2 # [ 101.825371] nix-grpc-daemon[785]: ts=2026-09-22T10:56:53Z level=info event=worker_gone addr=192.168.1.3:500515726worker2 # [ 101.893201] nix-daemon[873]: reaped child process 875, status = succeeded5727worker1 # [ 102.004518] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=scheduler_connected addr=lb:500515728worker1 # [ 102.006270] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5729worker2 # [ 102.014158] nix-grpc-daemon[785]: ts=2026-09-22T10:56:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5730worker1 # [ 102.207450] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=scheduler_connected addr=lb:500515731worker1 # [ 102.211473] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5732worker1 # [ 102.411882] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=scheduler_connected addr=lb:500515733worker1 # [ 102.413678] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5734worker1 # [ 102.614660] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=scheduler_connected addr=lb:500515735worker1 # [ 102.627551] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:462285736worker1 # [ 102.632881] nix-grpc-daemon[784]: ts=2026-09-22T10:56:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05737worker2 # [ 102.670130] nix-grpc-daemon[785]: E0000 00:00:1790074613.969196 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5738worker2 # [ 102.675581] nix-grpc-daemon[785]: E0000 00:00:1790074613.969363 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5739worker2 # [ 103.656893] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5740worker2 # [ 103.661865] systemd[1]: Stopped Nix worker-protocol over gRPC.5741worker2 # [ 103.665296] systemd[1]: nix-grpc-daemon.service: Consumed 2.421s CPU time over 1min 26.433s wall clock time, 57.2M memory peak, 87.8K incoming IP traffic, 270.8K outgoing IP traffic.5742worker2 # [ 103.677208] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5743worker2 # [ 103.680886] systemd[1]: Closed Nix worker-protocol over gRPC.5744worker2 # [ 103.683477] systemd[1]: nix-grpc-daemon.socket: Consumed 304us CPU time over 1min 34.533s wall clock time, 256K memory peak, 103K incoming IP traffic, 83.6K outgoing IP traffic.5745lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5746lb: must succeed: curl -sf localhost:9901/clusters5747lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5748lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5749client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25750worker1 # [ 103.978723] nix-grpc-daemon[784]: ts=2026-09-22T10:56:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:478325751worker1 # [ 103.982585] nix-grpc-daemon[784]: ts=2026-09-22T10:56:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:478325752client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5753client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.24 seconds)5754worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5755worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5756worker2 # [ 104.066200] systemd[1]: Listening on Nix worker-protocol over gRPC.5757worker1 # [ 104.132696] systemd[1]: Stopping Nix worker-protocol over gRPC...5758worker1 # [ 104.340471] nix-grpc-daemon[784]: ts=2026-09-22T10:56:56Z level=info event=signal_exit5759lb # [ 104.648603] niks3-server[1067]: 2026/09/22 10:56:56 INFO lead: released remote=192.168.1.3:365445760worker2 # [ 104.645856] systemd[1]: Starting scheduler-token.service...5761worker2 # [ 104.735890] systemd[1]: scheduler-token.service: Deactivated successfully.5762worker2 # [ 104.738042] systemd[1]: Finished scheduler-token.service.5763worker2 # [ 104.741239] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 92ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5764worker2 # [ 104.745737] systemd[1]: Starting Nix worker-protocol over gRPC...5765worker2 # [ 104.914572] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5766worker2 # [ 104.916773] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5767worker2 # [ 104.920738] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15768worker2 # [ 104.927068] nix-grpc-daemon[1243]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5769worker2 # [ 104.928414] nix-grpc-daemon[1243]: E0000 00:00:1790074616.226380 1243 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5770worker2 # [ 104.933412] nix-grpc-daemon[1243]: E0000 00:00:1790074616.231997 1243 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5771worker2 # [ 104.938209] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5772worker2 # [ 104.942143] systemd[1]: Started Nix worker-protocol over gRPC.5773lb # [ 104.953973] niks3-server[1067]: 2026/09/22 10:56:56 INFO lead: acquired remote=192.168.1.4:344765774worker2 # [ 104.950464] nix-grpc-daemon[1243]: E0000 00:00:1790074616.249883 1283 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5775worker2 # [ 104.954181] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=scheduler_take_over5776worker2 # [ 104.964335] nix-grpc-daemon[1243]: E0000 00:00:1790074616.263647 1283 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5777worker2 # [ 104.968202] nix-grpc-daemon[1243]: E0000 00:00:1790074616.265975 1283 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5778worker2 # [ 104.979074] nix-grpc-daemon[1243]: E0000 00:00:1790074616.278352 1283 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5779worker2 # [ 104.988315] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5780worker2 # [ 105.189955] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=scheduler_connected addr=lb:500515781worker2 # [ 105.193981] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5782worker2 # [ 105.395330] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=scheduler_connected addr=lb:500515783worker2 # [ 105.399471] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5784worker2 # [ 105.600822] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=scheduler_connected addr=lb:500515785worker2 # [ 105.604904] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5786worker1 # [ 105.644762] nix-grpc-daemon[784]: ts=2026-09-22T10:56:57Z level=info event=scheduler_restarting5787worker1 # [ 105.647560] nix-grpc-daemon[784]: ts=2026-09-22T10:56:57Z level=info event=scheduler_restarting addr=lb:500515788worker1 # [ 105.649366] nix-grpc-daemon[784]: ts=2026-09-22T10:56:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:462285789worker1 # [ 105.650844] nix-grpc-daemon[784]: ts=2026-09-22T10:56:57Z level=info event=worker_gone addr=192.168.1.3:500515790worker1 # [ 105.666499] nix-daemon[853]: reaped child process 855, status = succeeded5791worker2 # [ 105.806103] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:57Z level=info event=scheduler_connected addr=lb:500515792worker2 # [ 105.810139] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5793worker1 # [ 105.851526] nix-grpc-daemon[784]: ts=2026-09-22T10:56:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5794worker2 # [ 106.012159] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:57Z level=info event=scheduler_connected addr=lb:500515795worker2 # [ 106.024156] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:57Z level=info event=oidc_jwks provider=mock keys=15796worker2 # [ 106.028731] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:470825797worker2 # [ 106.032588] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05798lb # [ 106.049677] envoy[910]: conn peer=[2001:db8:1::1]:34832 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=505799lb # [ 106.051541] envoy[910]: conn peer=[2001:db8:1::1]:34824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=1924 ms=30745800lb # [ 106.053179] envoy[910]: conn peer=[2001:db8:1::1]:34848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3746 tx=1984 ms=19305801lb # [ 106.054810] envoy[910]: conn peer=[2001:db8:1::1]:34856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=515802lb # [ 106.056356] envoy[910]: conn peer=[2001:db8:1::4]:32896 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=99674 tx=4686 ms=844335803lb # [ 106.058139] envoy[910]: conn peer=[2001:db8:1::1]:48170 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=145804lb # [ 106.059708] envoy[910]: conn peer=192.168.1.3:53184 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=26226 tx=4958 ms=884925805worker1 # [ 106.521588] nix-grpc-daemon[784]: E0000 00:00:1790074618.638872 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5806worker1 # [ 106.526480] nix-grpc-daemon[784]: E0000 00:00:1790074618.639014 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5807worker1 # [ 107.594602] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5808worker1 # [ 107.599717] systemd[1]: Stopped Nix worker-protocol over gRPC.5809worker1 # [ 107.602430] systemd[1]: nix-grpc-daemon.service: Consumed 2.695s CPU time over 1min 31.557s wall clock time, 62.3M memory peak, 94.2K incoming IP traffic, 178.8K outgoing IP traffic.5810worker1 # [ 107.614366] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5811worker1 # [ 107.617850] systemd[1]: Closed Nix worker-protocol over gRPC.5812worker1 # [ 107.620691] systemd[1]: nix-grpc-daemon.socket: Consumed 314us CPU time over 1min 38.337s wall clock time, 256K memory peak, 95.6K incoming IP traffic, 79.7K outgoing IP traffic.5813lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5814lb: must succeed: curl -sf localhost:9901/clusters5815lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5816lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5817client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25818worker2 # [ 107.911614] nix-grpc-daemon[1243]: ts=2026-09-22T10:56:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470905819worker2 # [ 107.918053] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5820worker2 # [ 107.927328] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)5821worker2 # [ 107.974041] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 2/5)5822worker2 # [ 108.017987] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 325 ms (attempt 3/5)5823worker2 # [ 108.349363] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 540 ms (attempt 4/5)5824worker2 # [ 108.895688] nix-grpc-daemon[1243]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5825worker2 # [ 108.899656] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5826client # copying 2 paths...5827client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5828client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5829worker2 # [ 108.925785] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470905830worker2 # [ 108.939969] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5831worker2 # [ 108.963522] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:00Z level=info event=niks3_push_started pid=13035832worker2 # [ 109.129094] nix-grpc-daemon[1303]: time=2026-09-22T10:57:00.427Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5833worker2 # [ 109.276127] nix-daemon[873]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5834worker2 # [ 109.290967] nix-daemon[873]: reaped child process 1319, status = succeeded5835lb # [ 109.310935] niks3-server[1067]: 2026/09/22 10:57:01 INFO Received uploads request method=POST path=/api/pending_closures5836lb # [ 109.319901] niks3-server[1067]: 2026/09/22 10:57:01 INFO Received uploads request method=POST path=/api/pending_closures5837worker2 # [ 109.324262] nix-grpc-daemon[1303]: time=2026-09-22T10:57:00.623Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5838worker2 # [ 109.327957] nix-grpc-daemon[1303]: time=2026-09-22T10:57:00.625Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5839worker2 # [ 109.329575] nix-grpc-daemon[1303]: time=2026-09-22T10:57:00.625Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5840lb # [ 109.371395] niks3-server[1067]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5841lb # [ 109.395314] niks3-server[1067]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5842lb # [ 109.401633] niks3-server[1067]: 2026/09/22 10:57:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5843lb # [ 109.404322] niks3-server[1067]: 2026/09/22 10:57:01 INFO Signed narinfos id=30 count=25844lb # [ 109.408063] niks3-server[1067]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5845lb # [ 109.410932] niks3-server[1067]: 2026/09/22 10:57:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5846lb # [ 109.414928] niks3-server[1067]: 2026/09/22 10:57:01 INFO Signed narinfos id=29 count=15847worker2 # [ 109.412116] nix-grpc-daemon[1303]: time=2026-09-22T10:57:00.710Z level=INFO msg="Uploading 3 narinfos"5848lb # [ 109.418292] niks3-server[1067]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5849lb # [ 109.433300] niks3-server[1067]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5850lb # [ 109.438171] niks3-server[1067]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5851lb # [ 109.445647] niks3-server[1067]: 2026/09/22 10:57:01 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5852lb # [ 109.448902] niks3-server[1067]: 2026/09/22 10:57:01 INFO Completed upload id=305853lb # [ 109.450533] niks3-server[1067]: 2026/09/22 10:57:01 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5854lb # [ 109.454881] niks3-server[1067]: 2026/09/22 10:57:01 INFO Completed upload id=295855worker2 # [ 109.451343] nix-grpc-daemon[1303]: time=2026-09-22T10:57:00.750Z level=INFO msg="Upload complete. (318ms)"5856worker2 # [ 109.454130] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47090 duration_s=0 paths=2 nar_bytes_in=3125857lb # [ 109.458157] niks3-server[1067]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5858worker2 # [ 109.461766] nix-daemon[873]: reaped child process 1301, status = succeeded5859client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.80 seconds)5860worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5861worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5862worker1 # [ 109.561751] systemd[1]: Listening on Nix worker-protocol over gRPC.5863worker1 # [ 109.568236] systemd[1]: Starting Nix worker-protocol over gRPC...5864worker1 # [ 109.723383] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5865worker1 # [ 109.725799] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5866worker1 # [ 109.729778] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15867worker1 # [ 109.736068] nix-grpc-daemon[1237]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5868worker1 # [ 109.739118] nix-grpc-daemon[1237]: E0000 00:00:1790074621.853442 1237 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5869worker1 # [ 109.741335] nix-grpc-daemon[1237]: E0000 00:00:1790074621.856561 1237 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5870worker1 # [ 109.748746] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5871worker1 # [ 109.752761] systemd[1]: Started Nix worker-protocol over gRPC.5872worker1 # [ 109.758085] nix-grpc-daemon[1237]: E0000 00:00:1790074621.875120 1278 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5873lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5874lb: must succeed: curl -sf localhost:9901/clusters5875worker1 # [ 109.773354] nix-grpc-daemon[1237]: E0000 00:00:1790074621.890812 1278 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5876worker1 # [ 109.778058] nix-grpc-daemon[1237]: E0000 00:00:1790074621.893145 1278 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5877worker1 # [ 109.787675] nix-grpc-daemon[1237]: E0000 00:00:1790074621.905109 1278 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5878worker1 # [ 109.798211] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:01Z level=info event=scheduler_connected addr=lb:500515879worker2 # [ 109.800920] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:470825880worker2 # [ 109.802900] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05881lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5882lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5883(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.32 seconds)5884subtest: a niks3 restart does not cost the leader its role5885lb: must succeed: curl -sf localhost:9901/clusters5886lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5887lb: must succeed: curl -sf localhost:9901/clusters5888lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5889lb # [ 109.967385] niks3-server[1067]: 2026/09/22 10:57:01 INFO Shutdown signal received, draining in-flight requests timeout=10s5890lb # [ 109.971143] systemd[1]: Stopping niks3 server...5891worker2 # [ 109.968142] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:01Z level=info event=scheduler_lock_lost status=2005892worker1 # [ 109.967755] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:02Z level=info event=scheduler_lock_lost status=2005893lb # [ 109.975786] niks3-server[1067]: 2026/09/22 10:57:01 INFO lead: released remote=192.168.1.4:344765894lb # [ 109.987293] systemd[1]: niks3.service: Deactivated successfully.5895lb # [ 109.989403] systemd[1]: Stopped niks3 server.5896lb # [ 109.990235] systemd[1]: niks3.service: Consumed 635ms CPU time over 1min 19.383s wall clock time, 21.2M memory peak, 77.1K incoming IP traffic, 112.8K outgoing IP traffic.5897lb # [ 109.999696] systemd[1]: Starting niks3 server...5898lb # [ 110.076628] niks3-server[1201]: 2026/09/22 10:57:02 goose: no migrations to run. current version: 202609200000005899lb # [ 110.085161] niks3-server[1201]: 2026/09/22 10:57:02 OK 1_commit_pending_closure.sql (6.7ms)5900lb # [ 110.089391] niks3-server[1201]: 2026/09/22 10:57:02 OK 2_object_stats_trigger.sql (3.8ms)5901lb # [ 110.090489] niks3-server[1201]: 2026/09/22 10:57:02 goose: up to current file version: 25902lb # [ 110.092146] niks3-server[1201]: 2026/09/22 10:57:02 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key5903lb # [ 110.106827] niks3-server[1201]: 2026/09/22 10:57:02 INFO Read proxy enabled — serving cache objects from S35904lb # [ 110.108218] niks3-server[1201]: 2026/09/22 10:57:02 INFO Using socket-activated listener address=0.0.0.0:57515905lb # [ 110.111430] systemd[1]: Started niks3 server.5906lb # [ 110.112123] niks3-server[1201]: 2026/09/22 10:57:02 INFO systemd watchdog enabled interval=15s5907lb # [ 110.114459] niks3-server[1201]: 2026/09/22 10:57:02 INFO Starting HTTP server address=0.0.0.0:57515908lb: waiting for unit niks3.service5909lb: (finished: waiting for unit niks3.service, in 0.04 seconds)5910lb: must succeed: curl -sf localhost:9901/clusters5911lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5912worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5913worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5914lb: must succeed: curl -sf localhost:9901/clusters5915lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5916lb: must succeed: curl -sf localhost:9901/clusters5917lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5918worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5919worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5920client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart >&25921worker2 # [ 110.640515] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470905922client # don't know how to build these paths:5923client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5924worker1 # [ 110.653171] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420625925worker1 # [ 110.664802] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)5926worker1 # [ 110.682650] nix-grpc-daemon[1237]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)5927worker1 # [ 110.784321] nix-grpc-daemon[1237]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 2/5)5928worker1 # [ 110.792786] nix-grpc-daemon[1237]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 235 ms (attempt 3/5)5929lb # [ 110.975763] niks3-server[1201]: 2026/09/22 10:57:02 INFO lead: acquired remote=192.168.1.4:345385930worker1 # [ 111.033587] nix-grpc-daemon[1237]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 4/5)5931worker1 # [ 111.217899] nix-grpc-daemon[1237]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5932worker1 # [ 111.220838] nix-grpc-daemon[1237]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5933worker2 # [ 111.267163] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:02Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05934worker2 # [ 111.271238] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:02Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15935worker2 # [ 111.310525] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470905936worker2 # [ 111.326503] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470905937client # copying 1 paths to 'lb:50051'...5938client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5939worker2 # [ 111.339840] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470905940worker2 # [ 111.349868] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5941worker2 # [ 111.551437] nix-daemon[873]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5942worker2 # [ 111.567387] nix-daemon[873]: reaped child process 1341, status = succeeded5943worker2 # [ 111.750534] nix-daemon[873]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5944worker2 # [ 111.766322] nix-daemon[873]: reaped child process 1346, status = succeeded5945lb # [ 111.783553] niks3-server[1201]: 2026/09/22 10:57:03 INFO Received uploads request method=POST path=/api/pending_closures5946worker2 # [ 111.792951] nix-grpc-daemon[1303]: time=2026-09-22T10:57:03.091Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5947worker2 # [ 111.794387] nix-grpc-daemon[1303]: time=2026-09-22T10:57:03.091Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5948lb # [ 111.826379] niks3-server[1201]: 2026/09/22 10:57:03 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5949lb # [ 111.831431] niks3-server[1201]: 2026/09/22 10:57:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5950lb # [ 111.848494] niks3-server[1201]: 2026/09/22 10:57:03 INFO Signed narinfos id=31 count=15951worker2 # [ 111.845610] nix-grpc-daemon[1303]: time=2026-09-22T10:57:03.144Z level=INFO msg="Uploading 1 narinfos"5952lb # [ 111.859085] niks3-server[1201]: 2026/09/22 10:57:03 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5953lb # [ 111.868084] niks3-server[1201]: 2026/09/22 10:57:03 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5954lb # [ 111.871127] niks3-server[1201]: 2026/09/22 10:57:03 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5955lb # [ 111.875349] niks3-server[1201]: 2026/09/22 10:57:03 INFO Completed upload id=315956worker2 # [ 111.871962] nix-grpc-daemon[1303]: time=2026-09-22T10:57:03.171Z level=INFO msg="Upload complete. (506ms)"5957worker2 # [ 111.877145] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47090 duration_s=0 paths=1 nar_bytes_in=5365958worker2 # [ 111.884356] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470905959worker2 # [ 111.895198] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5960worker2 # [ 111.904417] nix-daemon[873]: reaped child process 1336, status = succeeded5961worker2 # [ 111.909908] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5962client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5963client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5964client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5965client # farm-a> LOG-farm-a-after-niks3-restart5966worker2 # [ 113.325883] nix-daemon[873]: reaped child process 1350, status = succeeded5967worker2 # [ 113.467833] nix-daemon[873]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5968worker2 # [ 113.482138] nix-daemon[873]: reaped child process 1369, status = succeeded5969lb # [ 113.500853] niks3-server[1201]: 2026/09/22 10:57:05 INFO Received uploads request method=POST path=/api/pending_closures5970worker2 # [ 113.508538] nix-grpc-daemon[1303]: time=2026-09-22T10:57:04.807Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5971worker2 # [ 113.510128] nix-grpc-daemon[1303]: time=2026-09-22T10:57:04.809Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5972lb # [ 113.535630] niks3-server[1201]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5973lb # [ 113.551612] niks3-server[1201]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5974lb # [ 113.563763] niks3-server[1201]: 2026/09/22 10:57:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5975lb # [ 113.569622] niks3-server[1201]: 2026/09/22 10:57:05 INFO Signed narinfos id=32 count=15976worker2 # [ 113.566493] nix-grpc-daemon[1303]: time=2026-09-22T10:57:04.865Z level=INFO msg="Uploading 1 narinfos"5977lb # [ 113.573120] niks3-server[1201]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5978lb # [ 113.581830] niks3-server[1201]: 2026/09/22 10:57:05 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5979lb # [ 113.588235] niks3-server[1201]: 2026/09/22 10:57:05 INFO Completed upload id=325980worker2 # [ 113.584849] nix-grpc-daemon[1303]: time=2026-09-22T10:57:04.884Z level=INFO msg="Upload complete. (266ms)"5981lb # [ 113.590075] niks3-server[1201]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5982worker2 # [ 113.594672] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47090 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15983worker2 # [ 113.606161] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:04Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05984worker2 # [ 113.609358] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:04Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15985worker2 # [ 113.618415] nix-daemon[873]: reaped child process 1348, status = succeeded5986worker2 # [ 113.648246] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470905987worker2 # [ 113.661316] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470905988worker2 # [ 113.675443] nix-grpc-daemon[1303]: time=2026-09-22T10:57:04.974Z level=INFO msg="All 1 paths already cached"5989client # copying 1 paths to 'lb:50051'...5990client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5991worker2 # [ 113.683726] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470905992worker2 # [ 113.692487] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5993worker2 # [ 113.859581] nix-daemon[873]: accepted connection from pid 1375, user nix-grpc-daemon (trusted)5994worker2 # [ 113.873948] nix-daemon[873]: reaped child process 1378, status = succeeded5995worker2 # [ 114.039522] nix-daemon[873]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5996worker2 # [ 114.054464] nix-daemon[873]: reaped child process 1383, status = succeeded5997lb # [ 114.071920] niks3-server[1201]: 2026/09/22 10:57:06 INFO Received uploads request method=POST path=/api/pending_closures5998worker2 # [ 114.076946] nix-grpc-daemon[1303]: time=2026-09-22T10:57:05.376Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5999worker2 # [ 114.078507] nix-grpc-daemon[1303]: time=2026-09-22T10:57:05.377Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6000lb # [ 114.104500] niks3-server[1201]: 2026/09/22 10:57:06 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6001lb # [ 114.114404] niks3-server[1201]: 2026/09/22 10:57:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign6002worker2 # [ 114.117732] nix-grpc-daemon[1303]: time=2026-09-22T10:57:05.417Z level=INFO msg="Uploading 1 narinfos"6003lb # [ 114.121288] niks3-server[1201]: 2026/09/22 10:57:06 INFO Signed narinfos id=33 count=16004lb # [ 114.125576] niks3-server[1201]: 2026/09/22 10:57:06 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6005lb # [ 114.137827] niks3-server[1201]: 2026/09/22 10:57:06 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete6006lb # [ 114.143102] niks3-server[1201]: 2026/09/22 10:57:06 INFO Completed upload id=336007worker2 # [ 114.139535] nix-grpc-daemon[1303]: time=2026-09-22T10:57:05.438Z level=INFO msg="Upload complete. (433ms)"6008worker2 # [ 114.143386] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47090 duration_s=0 paths=1 nar_bytes_in=6566009lb # [ 114.149568] niks3-server[1201]: 2026/09/22 10:57:06 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6010worker2 # [ 114.148590] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470906011worker2 # [ 114.158064] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)6012worker2 # [ 114.162525] nix-daemon[873]: reaped child process 1373, status = succeeded6013worker2 # [ 114.167913] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)6014client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6015client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6016client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6017client # farm-b> LOG-farm-b-after-niks3-restart6018worker2 # [ 115.193587] nix-daemon[873]: reaped child process 1387, status = succeeded6019worker2 # [ 115.321957] nix-daemon[873]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6020worker2 # [ 115.334953] nix-daemon[873]: reaped child process 1406, status = succeeded6021lb # [ 115.349864] niks3-server[1201]: 2026/09/22 10:57:07 INFO Received uploads request method=POST path=/api/pending_closures6022worker2 # [ 115.354756] nix-grpc-daemon[1303]: time=2026-09-22T10:57:06.653Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6023worker2 # [ 115.356291] nix-grpc-daemon[1303]: time=2026-09-22T10:57:06.655Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6024lb # [ 115.373919] niks3-server[1201]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6025lb # [ 115.376887] niks3-server[1201]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6026lb # [ 115.380960] niks3-server[1201]: 2026/09/22 10:57:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign6027lb # [ 115.383471] niks3-server[1201]: 2026/09/22 10:57:07 INFO Signed narinfos id=34 count=16028worker2 # [ 115.381160] nix-grpc-daemon[1303]: time=2026-09-22T10:57:06.680Z level=INFO msg="Uploading 1 narinfos"6029lb # [ 115.387615] niks3-server[1201]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6030lb # [ 115.392752] niks3-server[1201]: 2026/09/22 10:57:07 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete6031lb # [ 115.397080] niks3-server[1201]: 2026/09/22 10:57:07 INFO Completed upload id=346032worker2 # [ 115.393371] nix-grpc-daemon[1303]: time=2026-09-22T10:57:06.692Z level=INFO msg="Upload complete. (217ms)"6033lb # [ 115.399787] niks3-server[1201]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6034worker2 # [ 115.397167] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47090 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16035worker2 # [ 115.403593] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:06Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06036worker2 # [ 115.405276] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:06Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=16037worker2 # [ 115.408527] nix-daemon[873]: reaped child process 1385, status = succeeded6038worker2 # [ 115.448297] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470906039worker2 # [ 115.458400] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470906040worker2 # [ 115.467690] nix-grpc-daemon[1303]: time=2026-09-22T10:57:06.767Z level=INFO msg="All 2 paths already cached"6041client # copying 1 paths to 'lb:50051'...6042client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6043worker2 # [ 115.474404] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470906044worker2 # [ 115.483093] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)6045worker2 # [ 115.646395] nix-daemon[873]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)6046worker2 # [ 115.659081] nix-daemon[873]: reaped child process 1415, status = succeeded6047worker2 # [ 115.829079] nix-daemon[873]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)6048worker2 # [ 115.841944] nix-daemon[873]: reaped child process 1420, status = succeeded6049lb # [ 115.860162] niks3-server[1201]: 2026/09/22 10:57:07 INFO Received uploads request method=POST path=/api/pending_closures6050worker2 # [ 115.861361] nix-grpc-daemon[1303]: time=2026-09-22T10:57:07.160Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6051worker2 # [ 115.863302] nix-grpc-daemon[1303]: time=2026-09-22T10:57:07.162Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6052lb # [ 115.876534] niks3-server[1201]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6053lb # [ 115.880308] niks3-server[1201]: 2026/09/22 10:57:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign6054lb # [ 115.882776] niks3-server[1201]: 2026/09/22 10:57:07 INFO Signed narinfos id=35 count=16055worker2 # [ 115.879289] nix-grpc-daemon[1303]: time=2026-09-22T10:57:07.178Z level=INFO msg="Uploading 1 narinfos"6056lb # [ 115.885574] niks3-server[1201]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6057lb # [ 115.890656] niks3-server[1201]: 2026/09/22 10:57:07 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete6058lb # [ 115.893059] niks3-server[1201]: 2026/09/22 10:57:07 INFO Completed upload id=356059worker2 # [ 115.889404] nix-grpc-daemon[1303]: time=2026-09-22T10:57:07.188Z level=INFO msg="Upload complete. (397ms)"6060worker2 # [ 115.891383] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47090 duration_s=0 paths=1 nar_bytes_in=7846061lb # [ 115.895658] niks3-server[1201]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6062worker2 # [ 115.894578] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470906063worker2 # [ 115.901294] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)6064worker2 # [ 115.904404] nix-daemon[873]: reaped child process 1410, status = succeeded6065client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6066client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6067worker2 # [ 115.911118] nix-daemon[873]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)6068lb # [ 116.050114] envoy[910]: conn peer=[2001:db8:1::1]:48180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1009 tx=305 ms=15586069client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6070client # farm-top> LOG-farm-top-after-niks3-restart6071worker2 # [ 116.925475] nix-daemon[873]: reaped child process 1424, status = succeeded6072worker2 # [ 117.068815] nix-daemon[873]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)6073worker2 # [ 117.081348] nix-daemon[873]: reaped child process 1443, status = succeeded6074lb # [ 117.102728] niks3-server[1201]: 2026/09/22 10:57:09 INFO Received uploads request method=POST path=/api/pending_closures6075worker2 # [ 117.107926] nix-grpc-daemon[1303]: time=2026-09-22T10:57:08.407Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6076worker2 # [ 117.110064] nix-grpc-daemon[1303]: time=2026-09-22T10:57:08.408Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6077lb # [ 117.143419] niks3-server[1201]: 2026/09/22 10:57:09 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6078lb # [ 117.149163] niks3-server[1201]: 2026/09/22 10:57:09 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6079lb # [ 117.159117] niks3-server[1201]: 2026/09/22 10:57:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6080lb # [ 117.164673] niks3-server[1201]: 2026/09/22 10:57:09 INFO Signed narinfos id=36 count=16081worker2 # [ 117.163720] nix-grpc-daemon[1303]: time=2026-09-22T10:57:08.462Z level=INFO msg="Uploading 1 narinfos"6082lb # [ 117.168886] niks3-server[1201]: 2026/09/22 10:57:09 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6083lb # [ 117.180626] niks3-server[1201]: 2026/09/22 10:57:09 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6084lb # [ 117.184967] niks3-server[1201]: 2026/09/22 10:57:09 INFO Completed upload id=366085worker2 # [ 117.183827] nix-grpc-daemon[1303]: time=2026-09-22T10:57:08.483Z level=INFO msg="Upload complete. (264ms)"6086lb # [ 117.188214] niks3-server[1201]: 2026/09/22 10:57:09 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6087worker2 # [ 117.186158] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47090 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16088worker2 # [ 117.193794] nix-daemon[873]: reaped child process 1422, status = succeeded6089client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart >&2, in 6.82 seconds)6090client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart outPath6091client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)6092(finished: subtest: a niks3 restart does not cost the leader its role, in 7.58 seconds)6093subtest: scheduler down: the other node takes the lock and keeps it6094lb: must succeed: curl -sf localhost:9901/clusters6095lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6096lb: must succeed: curl -sf localhost:9901/clusters6097lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6098lb: must succeed: curl -sf localhost:9901/clusters6099lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6100lb: must succeed: curl -sf localhost:9901/clusters6101lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6102worker2 # [ 117.655231] systemd[1]: Stopping Nix worker-protocol over gRPC...6103worker2 # [ 117.945121] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:09Z level=info event=signal_exit6104lb # [ 118.186577] niks3-server[1201]: 2026/09/22 10:57:10 INFO lead: released remote=192.168.1.4:345386105worker2 # [ 119.181940] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:10Z level=info event=scheduler_restarting6106worker2 # [ 119.184548] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:10Z level=info event=scheduler_restarting addr=lb:500516107worker1 # [ 119.183349] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:11Z level=info event=scheduler_restarting addr=lb:500516108worker2 # [ 119.186204] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:470826109worker2 # [ 119.187695] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:10Z level=info event=worker_gone addr=192.168.1.4:500516110worker2 # [ 119.188977] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:470826111worker2 # [ 119.190386] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:10Z level=info event=worker_gone addr=192.168.1.3:500516112worker2 # [ 119.199329] nix-daemon[873]: reaped child process 1295, status = succeeded6113worker1 # [ 119.382719] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:11Z level=info event=scheduler_connected addr=lb:500516114worker1 # [ 119.384638] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6115worker2 # [ 119.394270] nix-grpc-daemon[1243]: ts=2026-09-22T10:57:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6116worker1 # [ 119.585236] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:11Z level=info event=scheduler_connected addr=lb:500516117worker1 # [ 119.588113] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6118worker1 # [ 119.788556] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:11Z level=info event=scheduler_connected addr=lb:500516119worker1 # [ 119.792246] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6120worker2 # [ 119.973893] nix-grpc-daemon[1243]: E0000 00:00:1790074631.273175 1288 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6121worker2 # [ 119.979306] nix-grpc-daemon[1243]: E0000 00:00:1790074631.278382 1288 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6122worker1 # [ 119.993158] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_connected addr=lb:500516123worker1 # [ 119.996916] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6124worker1 # [ 120.197647] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_connected addr=lb:500516125worker1 # [ 120.199441] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6126worker2 # [ 120.291528] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6127worker2 # [ 120.294505] systemd[1]: Stopped Nix worker-protocol over gRPC.6128worker2 # [ 120.297891] systemd[1]: nix-grpc-daemon.service: Consumed 1.157s CPU time over 15.544s wall clock time, 35.2M memory peak, 46.1K incoming IP traffic, 82.8K outgoing IP traffic.6129worker2 # [ 120.309575] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6130worker2 # [ 120.313869] systemd[1]: Closed Nix worker-protocol over gRPC.6131worker2 # [ 120.317168] systemd[1]: nix-grpc-daemon.socket: Consumed 326us CPU time over 16.250s wall clock time, 256K memory peak, 38K incoming IP traffic, 27.8K outgoing IP traffic.6132worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6133worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)6134worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6135worker1 # [ 120.399932] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_connected addr=lb:500516136worker1 # [ 120.401675] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6137worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6138worker1 # [ 120.602606] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_connected addr=lb:500516139worker1 # [ 120.606377] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6140worker1 # [ 120.806572] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_connected addr=lb:500516141worker1 # [ 120.808276] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6142worker1 # [ 120.976210] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_take_over6143lb # [ 120.980637] niks3-server[1201]: 2026/09/22 10:57:12 INFO lead: acquired remote=192.168.1.3:420446144worker1 # [ 121.008708] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_connected addr=lb:500516145worker1 # [ 121.010617] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6146worker1 # [ 121.212801] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_connected addr=lb:500516147worker1 # [ 121.216581] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6148worker1 # [ 121.417464] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_connected addr=lb:500516149worker1 # [ 121.421344] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6150worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6151worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6152worker1 # [ 121.622329] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_connected addr=lb:500516153worker1 # [ 121.626223] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6154worker1 # [ 121.826379] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=scheduler_connected addr=lb:500516155worker1 # [ 121.829494] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:476846156worker1 # [ 121.831130] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06157worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6158worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6159client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&26160client # don't know how to build these paths:6161worker1 # [ 122.773251] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:420626162client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6163worker1 # [ 122.776386] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420626164worker1 # [ 122.794025] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:14Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06165worker1 # [ 122.796050] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:14Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=16166worker1 # [ 122.839926] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420626167worker1 # [ 122.851910] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420626168client # copying 1 paths to 'lb:50051'...6169client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6170worker1 # [ 122.865726] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420626171worker1 # [ 122.874904] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6172worker1 # [ 122.895836] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:15Z level=info event=niks3_push_started pid=13166173worker1 # [ 123.079917] nix-grpc-daemon[1316]: time=2026-09-22T10:57:15.196Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6174worker1 # [ 123.232552] nix-daemon[853]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)6175worker1 # [ 123.243551] nix-daemon[853]: reaped child process 1332, status = succeeded6176worker1 # [ 123.396504] nix-daemon[853]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6177worker1 # [ 123.408559] nix-daemon[853]: reaped child process 1337, status = succeeded6178lb # [ 123.428129] niks3-server[1201]: 2026/09/22 10:57:15 INFO Received uploads request method=POST path=/api/pending_closures6179worker1 # [ 123.429702] nix-grpc-daemon[1316]: time=2026-09-22T10:57:15.547Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6180worker1 # [ 123.432270] nix-grpc-daemon[1316]: time=2026-09-22T10:57:15.548Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6181lb # [ 123.450330] niks3-server[1201]: 2026/09/22 10:57:15 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6182lb # [ 123.454962] niks3-server[1201]: 2026/09/22 10:57:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6183lb # [ 123.457151] niks3-server[1201]: 2026/09/22 10:57:15 INFO Signed narinfos id=37 count=16184worker1 # [ 123.455064] nix-grpc-daemon[1316]: time=2026-09-22T10:57:15.572Z level=INFO msg="Uploading 1 narinfos"6185lb # [ 123.462591] niks3-server[1201]: 2026/09/22 10:57:15 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6186lb # [ 123.466685] niks3-server[1201]: 2026/09/22 10:57:15 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6187lb # [ 123.471031] niks3-server[1201]: 2026/09/22 10:57:15 INFO Completed upload id=376188worker1 # [ 123.467163] nix-grpc-daemon[1316]: time=2026-09-22T10:57:15.584Z level=INFO msg="Upload complete. (384ms)"6189worker1 # [ 123.469321] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42062 duration_s=0 paths=1 nar_bytes_in=5206190worker1 # [ 123.473130] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420626191worker1 # [ 123.480092] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6192worker1 # [ 123.482498] nix-daemon[853]: reaped child process 1314, status = succeeded6193worker1 # [ 123.487947] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6194client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6195client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6196client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6197client # farm-a> LOG-farm-a-on-standby6198worker1 # [ 124.903677] nix-daemon[853]: reaped child process 1341, status = succeeded6199worker1 # [ 125.040710] nix-daemon[853]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)6200worker1 # [ 125.052669] nix-daemon[853]: reaped child process 1360, status = succeeded6201lb # [ 125.072880] niks3-server[1201]: 2026/09/22 10:57:17 INFO Received uploads request method=POST path=/api/pending_closures6202worker1 # [ 125.082822] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6203worker1 # [ 125.084398] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.201Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6204lb # [ 125.125085] niks3-server[1201]: 2026/09/22 10:57:17 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6205lb # [ 125.127797] niks3-server[1201]: 2026/09/22 10:57:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6206lb # [ 125.131802] niks3-server[1201]: 2026/09/22 10:57:17 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6207worker1 # [ 125.128451] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.245Z level=INFO msg="Uploading 1 narinfos"6208lb # [ 125.133361] niks3-server[1201]: 2026/09/22 10:57:17 INFO Signed narinfos id=38 count=16209lb # [ 125.142310] niks3-server[1201]: 2026/09/22 10:57:17 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6210lb # [ 125.150463] niks3-server[1201]: 2026/09/22 10:57:17 INFO Completed upload id=386211worker1 # [ 125.146650] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.264Z level=INFO msg="Upload complete. (250ms)"6212lb # [ 125.153343] niks3-server[1201]: 2026/09/22 10:57:17 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6213worker1 # [ 125.150299] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42062 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16214lb # [ 125.157877] niks3-server[1201]: 2026/09/22 10:57:17 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6215worker1 # [ 125.160507] nix-daemon[853]: reaped child process 1339, status = succeeded6216worker1 # [ 125.165069] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:17Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06217worker1 # [ 125.166984] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:17Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=16218worker1 # [ 125.209733] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420626219worker1 # [ 125.221936] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420626220worker1 # [ 125.236438] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.353Z level=INFO msg="All 1 paths already cached"6221client # copying 1 paths to 'lb:50051'...6222client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6223worker1 # [ 125.244274] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420626224worker1 # [ 125.252565] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6225worker1 # [ 125.405678] nix-daemon[853]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)6226worker1 # [ 125.418076] nix-daemon[853]: reaped child process 1369, status = succeeded6227worker1 # [ 125.565167] nix-daemon[853]: accepted connection from pid 1371, user nix-grpc-daemon (trusted)6228worker1 # [ 125.577371] nix-daemon[853]: reaped child process 1374, status = succeeded6229lb # [ 125.590974] niks3-server[1201]: 2026/09/22 10:57:17 INFO Received uploads request method=POST path=/api/pending_closures6230worker1 # [ 125.595289] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.712Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6231worker1 # [ 125.596777] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.714Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6232lb # [ 125.612929] niks3-server[1201]: 2026/09/22 10:57:17 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6233lb # [ 125.617057] niks3-server[1201]: 2026/09/22 10:57:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6234lb # [ 125.620116] niks3-server[1201]: 2026/09/22 10:57:17 INFO Signed narinfos id=39 count=16235worker1 # [ 125.616163] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.733Z level=INFO msg="Uploading 1 narinfos"6236lb # [ 125.625194] niks3-server[1201]: 2026/09/22 10:57:17 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6237lb # [ 125.630201] niks3-server[1201]: 2026/09/22 10:57:17 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6238lb # [ 125.634539] niks3-server[1201]: 2026/09/22 10:57:17 INFO Completed upload id=396239worker1 # [ 125.630641] nix-grpc-daemon[1316]: time=2026-09-22T10:57:17.748Z level=INFO msg="Upload complete. (365ms)"6240lb # [ 125.636321] niks3-server[1201]: 2026/09/22 10:57:17 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6241worker1 # [ 125.632650] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42062 duration_s=0 paths=1 nar_bytes_in=6406242worker1 # [ 125.636748] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420626243worker1 # [ 125.643218] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6244worker1 # [ 125.645758] nix-daemon[853]: reaped child process 1364, status = succeeded6245client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6246client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6247worker1 # [ 125.651617] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6248lb # [ 126.050256] envoy[910]: conn peer=[2001:db8:1::1]:48194 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9700 tx=4135 ms=65596249lb # [ 126.052044] envoy[910]: conn peer=[2001:db8:1::4]:39092 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5746 tx=730 ms=144016250client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6251client # farm-b> LOG-farm-b-on-standby6252worker1 # [ 126.676669] nix-daemon[853]: reaped child process 1378, status = succeeded6253worker1 # [ 126.807759] nix-daemon[853]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6254worker1 # [ 126.820713] nix-daemon[853]: reaped child process 1397, status = succeeded6255lb # [ 126.834680] niks3-server[1201]: 2026/09/22 10:57:18 INFO Received uploads request method=POST path=/api/pending_closures6256worker1 # [ 126.840234] nix-grpc-daemon[1316]: time=2026-09-22T10:57:18.957Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6257worker1 # [ 126.842262] nix-grpc-daemon[1316]: time=2026-09-22T10:57:18.959Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6258lb # [ 126.871393] niks3-server[1201]: 2026/09/22 10:57:18 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6259lb # [ 126.877950] niks3-server[1201]: 2026/09/22 10:57:18 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6260lb # [ 126.891989] niks3-server[1201]: 2026/09/22 10:57:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6261lb # [ 126.896339] niks3-server[1201]: 2026/09/22 10:57:18 INFO Signed narinfos id=40 count=16262worker1 # [ 126.893114] nix-grpc-daemon[1316]: time=2026-09-22T10:57:19.010Z level=INFO msg="Uploading 1 narinfos"6263lb # [ 126.901868] niks3-server[1201]: 2026/09/22 10:57:18 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6264lb # [ 126.909988] niks3-server[1201]: 2026/09/22 10:57:18 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6265lb # [ 126.912899] niks3-server[1201]: 2026/09/22 10:57:18 INFO Completed upload id=406266worker1 # [ 126.909222] nix-grpc-daemon[1316]: time=2026-09-22T10:57:19.026Z level=INFO msg="Upload complete. (250ms)"6267lb # [ 126.915596] niks3-server[1201]: 2026/09/22 10:57:18 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6268worker1 # [ 126.917480] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42062 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16269worker1 # [ 126.925092] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:19Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06270worker1 # [ 126.929395] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:19Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16271worker1 # [ 126.938197] nix-daemon[853]: reaped child process 1376, status = succeeded6272worker1 # [ 126.968707] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420626273worker1 # [ 126.981331] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420626274worker1 # [ 126.994261] nix-grpc-daemon[1316]: time=2026-09-22T10:57:19.111Z level=INFO msg="All 2 paths already cached"6275client # copying 1 paths to 'lb:50051'...6276client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6277worker1 # [ 127.002396] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420626278worker1 # [ 127.010529] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6279worker1 # [ 127.150543] nix-daemon[853]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6280worker1 # [ 127.163075] nix-daemon[853]: reaped child process 1406, status = succeeded6281worker1 # [ 127.309877] nix-daemon[853]: accepted connection from pid 1408, user nix-grpc-daemon (trusted)6282worker1 # [ 127.322379] nix-daemon[853]: reaped child process 1411, status = succeeded6283lb # [ 127.337739] niks3-server[1201]: 2026/09/22 10:57:19 INFO Received uploads request method=POST path=/api/pending_closures6284worker1 # [ 127.342543] nix-grpc-daemon[1316]: time=2026-09-22T10:57:19.460Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6285worker1 # [ 127.344092] nix-grpc-daemon[1316]: time=2026-09-22T10:57:19.461Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6286lb # [ 127.368961] niks3-server[1201]: 2026/09/22 10:57:19 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6287lb # [ 127.377437] niks3-server[1201]: 2026/09/22 10:57:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6288worker1 # [ 127.381211] nix-grpc-daemon[1316]: time=2026-09-22T10:57:19.498Z level=INFO msg="Uploading 1 narinfos"6289lb # [ 127.385215] niks3-server[1201]: 2026/09/22 10:57:19 INFO Signed narinfos id=41 count=16290lb # [ 127.389604] niks3-server[1201]: 2026/09/22 10:57:19 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6291lb # [ 127.401656] niks3-server[1201]: 2026/09/22 10:57:19 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6292lb # [ 127.409354] niks3-server[1201]: 2026/09/22 10:57:19 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6293worker1 # [ 127.410686] nix-grpc-daemon[1316]: time=2026-09-22T10:57:19.528Z level=INFO msg="Upload complete. (387ms)"6294lb # [ 127.414579] niks3-server[1201]: 2026/09/22 10:57:19 INFO Completed upload id=416295worker1 # [ 127.412564] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42062 duration_s=0 paths=1 nar_bytes_in=7686296worker1 # [ 127.414991] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420626297worker1 # [ 127.420401] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6298worker1 # [ 127.425068] nix-daemon[853]: reaped child process 1401, status = succeeded6299client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6300client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6301worker1 # [ 127.429849] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6302client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6303client # farm-top> LOG-farm-top-on-standby6304worker1 # [ 128.442242] nix-daemon[853]: reaped child process 1415, status = succeeded6305worker1 # [ 128.563642] nix-daemon[853]: accepted connection from pid 1431, user nix-grpc-daemon (trusted)6306worker1 # [ 128.575487] nix-daemon[853]: reaped child process 1434, status = succeeded6307lb # [ 128.590846] niks3-server[1201]: 2026/09/22 10:57:20 INFO Received uploads request method=POST path=/api/pending_closures6308worker1 # [ 128.591621] nix-grpc-daemon[1316]: time=2026-09-22T10:57:20.708Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6309worker1 # [ 128.593099] nix-grpc-daemon[1316]: time=2026-09-22T10:57:20.710Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6310lb # [ 128.612463] niks3-server[1201]: 2026/09/22 10:57:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6311lb # [ 128.614728] niks3-server[1201]: 2026/09/22 10:57:20 INFO Signed narinfos id=42 count=16312worker1 # [ 128.611147] nix-grpc-daemon[1316]: time=2026-09-22T10:57:20.728Z level=INFO msg="Uploading 1 narinfos"6313lb # [ 128.616134] niks3-server[1201]: 2026/09/22 10:57:20 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6314lb # [ 128.620456] niks3-server[1201]: 2026/09/22 10:57:20 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6315lb # [ 128.624787] niks3-server[1201]: 2026/09/22 10:57:20 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6316lb # [ 128.628143] niks3-server[1201]: 2026/09/22 10:57:20 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6317lb # [ 128.631228] niks3-server[1201]: 2026/09/22 10:57:20 INFO Completed upload id=426318worker1 # [ 128.628550] nix-grpc-daemon[1316]: time=2026-09-22T10:57:20.746Z level=INFO msg="Upload complete. (192ms)"6319lb # [ 128.632819] niks3-server[1201]: 2026/09/22 10:57:20 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6320worker1 # [ 128.631686] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42062 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16321worker1 # [ 128.638089] nix-daemon[853]: reaped child process 1413, status = succeeded6322client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.11 seconds)6323client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath6324client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.17 seconds)6325worker2 # [ 128.912812] systemd[1]: Listening on Nix worker-protocol over gRPC.6326worker2 # [ 128.919938] systemd[1]: Starting scheduler-token.service...6327worker2 # [ 128.985280] systemd[1]: scheduler-token.service: Deactivated successfully.6328worker2 # [ 128.986507] systemd[1]: Finished scheduler-token.service.6329worker2 # [ 128.988153] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 65ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6330worker2 # [ 128.993478] systemd[1]: Starting Nix worker-protocol over gRPC...6331worker2 # [ 129.154463] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6332worker2 # [ 129.156935] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6333worker2 # [ 129.161114] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16334worker2 # [ 129.169095] nix-grpc-daemon[1464]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6335worker2 # [ 129.170416] nix-grpc-daemon[1464]: E0000 00:00:1790074640.467584 1464 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6336worker2 # [ 129.177319] nix-grpc-daemon[1464]: E0000 00:00:1790074640.472714 1464 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6337worker2 # [ 129.179932] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6338worker2 # [ 129.183684] systemd[1]: Started Nix worker-protocol over gRPC.6339worker2 # [ 129.189280] nix-grpc-daemon[1464]: E0000 00:00:1790074640.487412 1503 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6340lb: must succeed: curl -sf localhost:9901/clusters6341worker2 # [ 129.201131] nix-grpc-daemon[1464]: E0000 00:00:1790074640.500567 1503 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6342worker2 # [ 129.204157] nix-grpc-daemon[1464]: E0000 00:00:1790074640.502802 1503 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6343worker2 # [ 129.215319] nix-grpc-daemon[1464]: E0000 00:00:1790074640.514698 1503 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6344worker2 # [ 129.225691] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:20Z level=info event=scheduler_connected addr=lb:500516345worker1 # [ 129.235178] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:21Z level=info event=oidc_jwks provider=mock keys=16346worker1 # [ 129.238252] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:476846347worker1 # [ 129.242575] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6349worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6350worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6351lb: must succeed: curl -sf localhost:9901/clusters6352lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6353lb: must succeed: curl -sf localhost:9901/clusters6354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6355worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6356worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6357(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.06 seconds)6358subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6359client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs6360client # [ 129.535893] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.6361client # Running as unit: rs.service; invocation ID: 6eb58902e5a9447595f97dcae06ea7136362client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)6363worker2 # [ 129.746866] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480746364worker2 # [ 129.753176] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6365client # [ 129.772444] nix[1294]: don't know how to build these paths:6366client # [ 129.773428] nix[1294]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6367worker1 # [ 129.771781] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:21Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06368worker1 # [ 129.775762] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:21Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16369worker1 # [ 129.813459] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420626370worker1 # [ 129.826154] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:420626371client # [ 129.849299] nix[1294]: copying 1 paths to 'lb:50051'...6372worker1 # [ 129.837223] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:420626373client # [ 129.851333] nix[1294]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6374worker1 # [ 129.849663] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6375worker1 # [ 130.018172] nix-daemon[853]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6376worker1 # [ 130.030210] nix-daemon[853]: reaped child process 1450, status = succeeded6377worker1 # [ 130.177049] nix-daemon[853]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)6378worker1 # [ 130.187815] nix-daemon[853]: reaped child process 1455, status = succeeded6379lb # [ 130.205647] niks3-server[1201]: 2026/09/22 10:57:22 INFO Received uploads request method=POST path=/api/pending_closures6380worker1 # [ 130.209402] nix-grpc-daemon[1316]: time=2026-09-22T10:57:22.326Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6381worker1 # [ 130.210903] nix-grpc-daemon[1316]: time=2026-09-22T10:57:22.328Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6382lb # [ 130.232586] niks3-server[1201]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6383lb # [ 130.244616] niks3-server[1201]: 2026/09/22 10:57:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6384lb # [ 130.249458] niks3-server[1201]: 2026/09/22 10:57:22 INFO Signed narinfos id=43 count=16385worker1 # [ 130.248536] nix-grpc-daemon[1316]: time=2026-09-22T10:57:22.365Z level=INFO msg="Uploading 1 narinfos"6386lb # [ 130.255109] niks3-server[1201]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6387lb # [ 130.265106] niks3-server[1201]: 2026/09/22 10:57:22 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6388worker1 # [ 130.267377] nix-grpc-daemon[1316]: time=2026-09-22T10:57:22.384Z level=INFO msg="Upload complete. (406ms)"6389lb # [ 130.271323] niks3-server[1201]: 2026/09/22 10:57:22 INFO Completed upload id=436390lb # [ 130.276614] niks3-server[1201]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6391worker1 # [ 130.270281] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42062 duration_s=0 paths=1 nar_bytes_in=4166392worker1 # [ 130.275311] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:420626393worker1 # [ 130.284516] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6394worker1 # [ 130.289086] nix-daemon[853]: reaped child process 1445, status = succeeded6395worker1 # [ 130.294694] nix-daemon[853]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)6396client # [ 130.306394] nix[1294]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6397lb: must succeed: curl -sf localhost:9901/clusters6398lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6399lb: must succeed: curl -sf localhost:9901/clusters6400lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6401lb: must succeed: curl -sf localhost:9901/clusters6402lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6403lb: must succeed: curl -sf localhost:9901/clusters6404lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6405worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6406worker2 # [ 132.300895] systemd-journald[286]: Received client request to rotate journal, rotating.6407worker2 # [ 132.318331] systemd-journald[286]: Vacuuming done, freed 0B of archived journals from /var/log/journal/825d5723e21840e48bbdef128e6cdc7c.6408worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)6409worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6410worker1 # [ 132.236294] systemd[1]: Stopping Nix worker-protocol over gRPC...6411worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)6412client: must succeed: sleep 5.06413worker1 # [ 132.751527] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:24Z level=info event=draining6414lb # [ 136.050852] envoy[910]: conn peer=[2001:db8:1::1]:54272 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9573 tx=4052 ms=58706415client: (finished: must succeed: sleep 5.0, in 5.04 seconds)6416worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6417worker1 # [ 137.514400] nix-daemon[853]: reaped child process 1459, status = succeeded6418worker1 # [ 137.654547] nix-daemon[853]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6419worker1 # [ 137.666660] nix-daemon[853]: reaped child process 1503, status = succeeded6420lb # [ 137.687738] niks3-server[1201]: 2026/09/22 10:57:29 INFO Received uploads request method=POST path=/api/pending_closures6421worker1 # [ 137.692479] nix-grpc-daemon[1316]: time=2026-09-22T10:57:29.809Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6422worker1 # [ 137.694315] nix-grpc-daemon[1316]: time=2026-09-22T10:57:29.811Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6423lb # [ 137.732864] niks3-server[1201]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6424lb # [ 137.739089] niks3-server[1201]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6425lb # [ 137.746317] niks3-server[1201]: 2026/09/22 10:57:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6426lb # [ 137.751750] niks3-server[1201]: 2026/09/22 10:57:29 INFO Signed narinfos id=44 count=16427worker1 # [ 137.751061] nix-grpc-daemon[1316]: time=2026-09-22T10:57:29.868Z level=INFO msg="Uploading 1 narinfos"6428lb # [ 137.759095] niks3-server[1201]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6429lb # [ 137.769659] niks3-server[1201]: 2026/09/22 10:57:29 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6430lb # [ 137.772961] niks3-server[1201]: 2026/09/22 10:57:29 INFO Completed upload id=446431lb # [ 137.775056] niks3-server[1201]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6432worker1 # [ 137.770464] nix-grpc-daemon[1316]: time=2026-09-22T10:57:29.887Z level=INFO msg="Upload complete. (263ms)"6433worker1 # [ 137.776837] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42062 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16434worker1 # [ 137.788487] nix-daemon[853]: reaped child process 1457, status = succeeded6435client # [ 137.827892] systemd[1]: rs.service: Deactivated successfully.6436client # [ 137.831948] systemd[1]: rs.service: Consumed 124ms CPU time over 8.294s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6437worker1 # [ 138.754322] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:30Z level=info event=signal_exit6438lb # [ 139.211980] niks3-server[1201]: 2026/09/22 10:57:31 INFO lead: released remote=192.168.1.3:420446439worker2 # [ 140.209356] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:31Z level=info event=scheduler_restarting addr=lb:500516440worker1 # [ 140.213114] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:32Z level=info event=scheduler_restarting6441worker1 # [ 140.216366] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:476846442worker1 # [ 140.220302] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:32Z level=info event=worker_gone addr=192.168.1.4:500516443worker1 # [ 140.223483] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:476846444worker1 # [ 140.226758] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:32Z level=info event=scheduler_restarting addr=lb:500516445worker1 # [ 140.229828] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:32Z level=info event=worker_gone addr=192.168.1.3:500516446worker1 # [ 140.246473] nix-daemon[853]: reaped child process 1289, status = succeeded6447worker2 # [ 140.409127] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:31Z level=info event=scheduler_connected addr=lb:500516448worker2 # [ 140.411215] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6449worker1 # [ 140.423718] nix-grpc-daemon[1237]: ts=2026-09-22T10:57:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6450worker2 # [ 140.611824] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:31Z level=info event=scheduler_connected addr=lb:500516451worker2 # [ 140.613564] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6452worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.14 seconds)6453client: waiting for success: ! systemctl is-active rs6454client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)6455client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6456client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)6457client: must fail: journalctl -u rs | grep -q 'reconnecting'6458client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6459lb: must succeed: curl -sf localhost:9901/clusters6460worker1 # [ 140.813610] nix-grpc-daemon[1237]: E0000 00:00:1790074652.930968 1282 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6461worker2 # [ 140.814813] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516462worker1 # [ 140.815917] nix-grpc-daemon[1237]: E0000 00:00:1790074652.931100 1282 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6463lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6464worker2 # [ 140.818804] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6465worker2 # [ 141.019872] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516466worker2 # [ 141.024158] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6467worker2 # [ 141.225764] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516468worker2 # [ 141.229494] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6469worker1 # [ 141.376537] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6470worker1 # [ 141.380166] systemd[1]: Stopped Nix worker-protocol over gRPC.6471worker1 # [ 141.383568] systemd[1]: nix-grpc-daemon.service: Consumed 1.273s CPU time over 31.808s wall clock time, 35.8M memory peak, 48.6K incoming IP traffic, 103.8K outgoing IP traffic.6472worker1 # [ 141.399737] systemd[1]: Starting Nix worker-protocol over gRPC...6473worker2 # [ 141.430383] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516474worker2 # [ 141.434225] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6475worker1 # [ 141.561948] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6476worker1 # [ 141.564261] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6477worker1 # [ 141.568272] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16478worker1 # [ 141.575067] nix-grpc-daemon[1510]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6479worker1 # [ 141.576414] nix-grpc-daemon[1510]: E0000 00:00:1790074653.691937 1510 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6480worker1 # [ 141.581346] nix-grpc-daemon[1510]: E0000 00:00:1790074653.697116 1510 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6481worker1 # [ 141.583806] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6482worker1 # [ 141.585996] systemd[1]: Started Nix worker-protocol over gRPC.6483worker1 # [ 141.590052] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=scheduler_take_over6484lb # [ 141.594416] niks3-server[1201]: 2026/09/22 10:57:33 INFO lead: acquired remote=192.168.1.3:454166485worker1 # [ 141.593926] nix-grpc-daemon[1510]: E0000 00:00:1790074653.710875 1552 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6486worker1 # [ 141.605453] nix-grpc-daemon[1510]: E0000 00:00:1790074653.722917 1552 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6487worker1 # [ 141.609132] nix-grpc-daemon[1510]: E0000 00:00:1790074653.725205 1552 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6488worker1 # [ 141.619739] nix-grpc-daemon[1510]: E0000 00:00:1790074653.737218 1552 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6489worker1 # [ 141.627259] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516490worker1 # [ 141.628793] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6491worker2 # [ 141.635248] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516492worker2 # [ 141.639104] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6493lb: must succeed: curl -sf localhost:9901/clusters6494worker1 # [ 141.829443] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516495worker1 # [ 141.831205] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6496worker2 # [ 141.839575] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516497worker2 # [ 141.841240] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6498lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6499worker1 # [ 142.031526] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=scheduler_connected addr=lb:500516500worker1 # [ 142.033569] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6501worker2 # [ 142.042095] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516502worker2 # [ 142.043908] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6503worker1 # [ 142.234202] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=scheduler_connected addr=lb:500516504worker1 # [ 142.236141] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6505worker2 # [ 142.244938] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516506worker2 # [ 142.246508] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6507worker1 # [ 142.437198] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=scheduler_connected addr=lb:500516508worker1 # [ 142.444125] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:437406509worker2 # [ 142.447213] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516510worker1 # [ 142.447836] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06511worker1 # [ 142.459139] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=oidc_jwks provider=mock keys=16512worker1 # [ 142.463263] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:437406513worker1 # [ 142.467087] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06514lb: must succeed: curl -sf localhost:9901/clusters6515lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6516worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6517worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6518(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.55 seconds)6519subtest: scheduler restart mid-build: build finishes, no double build6520client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid6521client # [ 143.069626] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.6522client # Running as unit: mid.service; invocation ID: e1189d1e128d46c69274ba07968dbbc86523client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.04 seconds)6524worker1 # [ 143.254700] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:496886525worker1 # [ 143.266577] nix-daemon[853]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)6526client # [ 143.292603] nix[1329]: don't know how to build these paths:6527client # [ 143.293615] nix[1329]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6528worker1 # [ 143.288240] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:35Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06529worker1 # [ 143.293317] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:35Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16530worker1 # [ 143.337068] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496886531worker1 # [ 143.349809] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 1/5)6532worker1 # [ 143.427816] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 2/5)6533worker1 # [ 143.600793] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 201 ms (attempt 3/5)6534worker1 # [ 143.807497] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 4/5)6535worker1 # [ 143.829530] nix-grpc-daemon[1510]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6536worker1 # [ 143.832845] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6537worker1 # [ 143.851445] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496886538client # [ 143.870797] nix[1329]: copying 1 paths to 'lb:50051'...6539worker1 # [ 143.861426] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496886540client # [ 143.874321] nix[1329]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6541worker1 # [ 143.867718] nix-daemon[853]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)6542worker1 # [ 143.879559] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:35Z level=info event=niks3_push_started pid=15796543worker1 # [ 144.040830] nix-grpc-daemon[1579]: time=2026-09-22T10:57:36.157Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6544worker1 # [ 144.196820] nix-daemon[853]: accepted connection from pid 1592, user nix-grpc-daemon (trusted)6545worker1 # [ 144.211875] nix-daemon[853]: reaped child process 1599, status = succeeded6546worker1 # [ 144.363573] nix-daemon[853]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)6547worker1 # [ 144.376314] nix-daemon[853]: reaped child process 1604, status = succeeded6548lb # [ 144.396352] niks3-server[1201]: 2026/09/22 10:57:36 INFO Received uploads request method=POST path=/api/pending_closures6549worker1 # [ 144.404078] nix-grpc-daemon[1579]: time=2026-09-22T10:57:36.521Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6550worker1 # [ 144.405775] nix-grpc-daemon[1579]: time=2026-09-22T10:57:36.523Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6551lb # [ 144.435973] niks3-server[1201]: 2026/09/22 10:57:36 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6552lb # [ 144.443081] niks3-server[1201]: 2026/09/22 10:57:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6553lb # [ 144.448051] niks3-server[1201]: 2026/09/22 10:57:36 INFO Signed narinfos id=45 count=16554worker1 # [ 144.446887] nix-grpc-daemon[1579]: time=2026-09-22T10:57:36.564Z level=INFO msg="Uploading 1 narinfos"6555lb # [ 144.459660] niks3-server[1201]: 2026/09/22 10:57:36 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6556lb # [ 144.461796] niks3-server[1201]: 2026/09/22 10:57:36 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6557lb # [ 144.466750] niks3-server[1201]: 2026/09/22 10:57:36 INFO Completed upload id=456558worker1 # [ 144.462912] nix-grpc-daemon[1579]: time=2026-09-22T10:57:36.580Z level=INFO msg="Upload complete. (417ms)"6559lb # [ 144.468838] niks3-server[1201]: 2026/09/22 10:57:36 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6560worker1 # [ 144.465426] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49688 duration_s=0 paths=1 nar_bytes_in=4246561worker1 # [ 144.469400] nix-grpc-daemon[1510]: ts=2026-09-22T10:57:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496886562worker1 # [ 144.475583] nix-daemon[853]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)6563worker1 # [ 144.477931] nix-daemon[853]: reaped child process 1577, status = succeeded6564worker1 # [ 144.483084] nix-daemon[853]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)6565client # [ 144.496177] nix[1329]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6566lb # [ 146.050643] envoy[910]: conn peer=[2001:db8:1::1]:54282 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1428 ms=80416567lb # [ 146.053900] envoy[910]: conn peer=[2001:db8:1::3]:33102 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3984 tx=999 ms=306206568lb: must succeed: curl -sf localhost:9901/clusters6569lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6570lb: must succeed: curl -sf localhost:9901/clusters6571lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6572worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6573worker1 # [ 146.672379] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1510 (nix-grpc-daemon) on client request.6574worker1 # [ 146.676292] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6575lb # [ 146.687318] niks3-server[1201]: 2026/09/22 10:57:38 INFO lead: released remote=192.168.1.3:454166576worker2 # [ 146.690679] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:37Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6577client # [ 146.703672] nix[1329]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6578client # [ 146.705659] nix[1329]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6579worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)6580client: must succeed: sleep 5.06581worker1 # [ 146.701694] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6582worker1 # [ 146.703236] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6583worker1 # [ 146.706723] systemd[1]: nix-grpc-daemon.service: Consumed 369ms CPU time over 5.300s wall clock time, 33.2M memory peak, 14.8K incoming IP traffic, 21K outgoing IP traffic.6584worker1 # [ 146.711737] nix-daemon[853]: reaped child process 1606, status = succeeded6585worker1 # [ 146.713561] nix-daemon[853]: reaped child process 1569, status = succeeded6586worker1 # [ 146.722126] nix-daemon[1608]: unexpected Nix daemon error: error: interrupted by the user6587worker1 # [ 146.726786] nix-daemon[853]: reaped child process 1608, status = failed with exit code 16588worker2 # [ 146.891189] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:38Z level=info event=scheduler_connected addr=lb:500516589worker1 # [ 146.895249] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6590worker1 # [ 146.904642] systemd[1]: Starting Nix worker-protocol over gRPC...6591worker1 # [ 147.069297] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6592worker1 # [ 147.071066] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6593worker1 # [ 147.074458] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16594worker1 # [ 147.080061] nix-grpc-daemon[1645]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6595worker1 # [ 147.082093] nix-grpc-daemon[1645]: E0000 00:00:1790074659.197317 1645 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6596worker1 # [ 147.087058] nix-grpc-daemon[1645]: E0000 00:00:1790074659.202445 1645 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6597worker1 # [ 147.090116] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6598worker1 # [ 147.091821] systemd[1]: Started Nix worker-protocol over gRPC.6599worker2 # [ 147.096460] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6600lb # [ 147.103128] niks3-server[1201]: 2026/09/22 10:57:39 INFO lead: acquired remote=192.168.1.3:492066601worker1 # [ 147.101073] nix-grpc-daemon[1645]: E0000 00:00:1790074659.217750 1686 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6602worker1 # [ 147.104143] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=scheduler_take_over6603worker1 # [ 147.113757] nix-grpc-daemon[1645]: E0000 00:00:1790074659.231202 1686 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6604worker1 # [ 147.117122] nix-grpc-daemon[1645]: E0000 00:00:1790074659.233419 1686 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6605worker1 # [ 147.127506] nix-grpc-daemon[1645]: E0000 00:00:1790074659.244976 1686 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6606worker1 # [ 147.136671] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=scheduler_connected addr=lb:500516607worker1 # [ 147.138365] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:496966608worker1 # [ 147.139958] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06609worker1 # [ 147.200749] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06610worker1 # [ 147.204517] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16611worker1 # [ 147.252139] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497046612worker1 # [ 147.262160] nix-daemon[853]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6613worker1 # [ 147.276372] nix-daemon[853]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6614client # [ 147.299812] nix[1329]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6615worker1 # [ 147.288455] nix-daemon[853]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6616worker2 # [ 147.297712] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:38Z level=info event=scheduler_connected addr=lb:500516617worker1 # [ 147.302231] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=oidc_jwks provider=mock keys=16618worker1 # [ 147.303884] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:496966619worker1 # [ 147.305644] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06620client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6621worker1 # [ 151.903869] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:44Z level=info event=niks3_push_started pid=17286622worker1 # [ 151.915502] nix-daemon[853]: reaped child process 1703, status = succeeded6623client: waiting for success: ! systemctl is-active mid6624worker1 # [ 152.070678] nix-grpc-daemon[1728]: time=2026-09-22T10:57:44.187Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6625worker1 # [ 152.187271] nix-daemon[853]: accepted connection from pid 1741, user nix-grpc-daemon (trusted)6626worker1 # [ 152.199259] nix-daemon[853]: reaped child process 1744, status = succeeded6627lb # [ 152.213396] niks3-server[1201]: 2026/09/22 10:57:44 INFO Received uploads request method=POST path=/api/pending_closures6628worker1 # [ 152.214376] nix-grpc-daemon[1728]: time=2026-09-22T10:57:44.331Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6629worker1 # [ 152.215844] nix-grpc-daemon[1728]: time=2026-09-22T10:57:44.333Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6630lb # [ 152.237967] niks3-server[1201]: 2026/09/22 10:57:44 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6631worker1 # [ 152.241619] nix-grpc-daemon[1728]: time=2026-09-22T10:57:44.359Z level=INFO msg="Uploading 1 narinfos"6632lb # [ 152.245484] niks3-server[1201]: 2026/09/22 10:57:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6633lb # [ 152.246922] niks3-server[1201]: 2026/09/22 10:57:44 INFO Signed narinfos id=46 count=16634lb # [ 152.255607] niks3-server[1201]: 2026/09/22 10:57:44 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6635worker1 # [ 152.255430] nix-grpc-daemon[1728]: time=2026-09-22T10:57:44.372Z level=INFO msg="Upload complete. (181ms)"6636lb # [ 152.259668] niks3-server[1201]: 2026/09/22 10:57:44 INFO Completed upload id=466637worker1 # [ 152.257657] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49704 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16638lb # [ 152.267485] niks3-server[1201]: 2026/09/22 10:57:44 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6639worker1 # [ 152.265967] nix-daemon[853]: reaped child process 1701, status = succeeded6640lb # [ 152.272777] niks3-server[1201]: 2026/09/22 10:57:44 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6641client # [ 152.296179] systemd[1]: mid.service: Deactivated successfully.6642client # [ 152.297965] systemd[1]: mid.service: Consumed 126ms CPU time over 9.227s wall clock time, 43.4M memory peak, 8.2K incoming IP traffic, 10.2K outgoing IP traffic.6643client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6644client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6645client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6646(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.01 seconds)6647subtest: interrupting the client stops the build on the worker6648client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6649client # [ 153.069169] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6650client # Running as unit: intr.service; invocation ID: 7ad837c71ac64e2a9e16ccd0913b510d6651client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.03 seconds)6652worker1 # [ 153.233809] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:497046653client # [ 153.252368] nix[1365]: don't know how to build these paths:6654client # [ 153.253793] nix[1365]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6655worker1 # [ 153.250899] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:45Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06656worker1 # [ 153.255313] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:45Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16657worker2 # [ 153.295144] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480746658worker2 # [ 153.301296] nix-grpc-daemon[1464]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)6659worker2 # [ 153.327910] nix-grpc-daemon[1464]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 2/5)6660worker2 # [ 153.416690] nix-grpc-daemon[1464]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 3/5)6661worker2 # [ 153.501483] nix-grpc-daemon[1464]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 4/5)6662worker2 # [ 153.697539] nix-grpc-daemon[1464]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6663worker2 # [ 153.701173] nix-grpc-daemon[1464]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6664worker2 # [ 153.718177] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480746665client # [ 153.740391] nix[1365]: copying 1 paths to 'lb:50051'...6666worker2 # [ 153.728580] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480746667client # [ 153.743348] nix[1365]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6668worker2 # [ 153.742854] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6669worker2 # [ 153.761972] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:45Z level=info event=niks3_push_started pid=16126670worker2 # [ 153.921081] nix-grpc-daemon[1612]: time=2026-09-22T10:57:45.219Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6671worker2 # [ 154.077817] nix-daemon[873]: accepted connection from pid 1625, user nix-grpc-daemon (trusted)6672worker2 # [ 154.088589] nix-daemon[873]: reaped child process 1628, status = succeeded6673worker2 # [ 154.247290] nix-daemon[873]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6674worker2 # [ 154.264432] nix-daemon[873]: reaped child process 1635, status = succeeded6675lb # [ 154.283600] niks3-server[1201]: 2026/09/22 10:57:46 INFO Received uploads request method=POST path=/api/pending_closures6676worker2 # [ 154.294143] nix-grpc-daemon[1612]: time=2026-09-22T10:57:45.593Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6677worker2 # [ 154.297440] nix-grpc-daemon[1612]: time=2026-09-22T10:57:45.594Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6678lb # [ 154.330908] niks3-server[1201]: 2026/09/22 10:57:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6679lb # [ 154.336191] niks3-server[1201]: 2026/09/22 10:57:46 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6680worker2 # [ 154.338987] nix-grpc-daemon[1612]: time=2026-09-22T10:57:45.638Z level=INFO msg="Uploading 1 narinfos"6681lb # [ 154.343596] niks3-server[1201]: 2026/09/22 10:57:46 INFO Signed narinfos id=47 count=16682lb # [ 154.350525] niks3-server[1201]: 2026/09/22 10:57:46 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6683lb # [ 154.360634] niks3-server[1201]: 2026/09/22 10:57:46 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6684lb # [ 154.364948] niks3-server[1201]: 2026/09/22 10:57:46 INFO Completed upload id=476685worker2 # [ 154.362532] nix-grpc-daemon[1612]: time=2026-09-22T10:57:45.661Z level=INFO msg="Upload complete. (438ms)"6686lb # [ 154.367299] niks3-server[1201]: 2026/09/22 10:57:46 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6687worker2 # [ 154.364675] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=0 paths=1 nar_bytes_in=4246688worker2 # [ 154.367781] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480746689worker2 # [ 154.372767] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6690worker2 # [ 154.377247] nix-daemon[873]: reaped child process 1610, status = succeeded6691client # [ 154.394033] nix[1365]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6692worker2 # [ 154.382225] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6693client: must succeed: systemctl kill -s INT intr6694client # [ 155.474962] systemd[1]: intr.service: Sent signal SIGINT to main process 1365 (nix) on client request.6695worker2 # [ 155.464420] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:46Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6696client # [ 155.479790] nix[1365]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6697client # [ 155.485307] nix[1365]: error: interrupted by the user6698client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6699client # [ 155.508856] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6700client # [ 155.510312] systemd[1]: intr.service: Failed with result 'exit-code'.6701client # [ 155.512226] systemd[1]: intr.service: Consumed 109ms CPU time over 2.442s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6702worker2 # [ 155.597712] nix-daemon[873]: reaped child process 1639, status = succeeded6703worker2 # [ 155.600935] nix-daemon[1641]: unexpected Nix daemon error: error: interrupted by the user6704worker2 # [ 155.605481] nix-daemon[873]: reaped child process 1641, status = failed with exit code 16705lb # [ 156.051582] envoy[910]: conn peer=[2001:db8:1::3]:55658 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1335 tx=434 ms=50656706lb # [ 156.055733] envoy[910]: conn peer=[2001:db8:1::1]:55532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3878 tx=1808 ms=90196707lb # [ 156.059396] envoy[910]: conn peer=[2001:db8:1::1]:40528 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=827 ms=22586708(finished: subtest: interrupting the client stops the build on the worker, in 3.65 seconds)6709subtest: a deploy mid-build drains: the build finishes, the new generation takes over6710client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6711client # [ 156.720624] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6712client # Running as unit: sw.service; invocation ID: fa1126175da0478da20aa68e7767935e6713client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6714client # [ 156.920946] nix[1386]: don't know how to build these paths:6715worker2 # [ 156.907830] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480746716client # [ 156.922373] nix[1386]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6717worker1 # [ 156.917789] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:49Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06718worker1 # [ 156.923209] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:49Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16719worker1 # [ 156.966056] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497046720worker1 # [ 156.978565] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)6721worker1 # [ 157.004427] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 2/5)6722worker1 # [ 157.016519] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 283 ms (attempt 3/5)6723worker1 # [ 157.305247] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 4/5)6724worker1 # [ 157.498704] nix-grpc-daemon[1645]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6725worker1 # [ 157.500417] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6726worker1 # [ 157.512889] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:497046727client # [ 157.532119] nix[1386]: copying 1 paths to 'lb:50051'...6728worker1 # [ 157.523777] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:497046729client # [ 157.536165] nix[1386]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6730worker1 # [ 157.529774] nix-daemon[853]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6731worker1 # [ 157.680131] nix-daemon[853]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6732worker1 # [ 157.691920] nix-daemon[853]: reaped child process 1783, status = succeeded6733worker1 # [ 157.829275] nix-daemon[853]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6734worker1 # [ 157.846181] nix-daemon[853]: reaped child process 1791, status = succeeded6735lb # [ 157.861713] niks3-server[1201]: 2026/09/22 10:57:49 INFO Received uploads request method=POST path=/api/pending_closures6736worker1 # [ 157.865375] nix-grpc-daemon[1728]: time=2026-09-22T10:57:49.982Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6737worker1 # [ 157.867110] nix-grpc-daemon[1728]: time=2026-09-22T10:57:49.982Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6738lb # [ 157.889086] niks3-server[1201]: 2026/09/22 10:57:49 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6739lb # [ 157.898666] niks3-server[1201]: 2026/09/22 10:57:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6740lb # [ 157.904121] niks3-server[1201]: 2026/09/22 10:57:49 INFO Signed narinfos id=48 count=16741worker1 # [ 157.902886] nix-grpc-daemon[1728]: time=2026-09-22T10:57:50.020Z level=INFO msg="Uploading 1 narinfos"6742lb # [ 157.909866] niks3-server[1201]: 2026/09/22 10:57:49 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6743lb # [ 157.921600] niks3-server[1201]: 2026/09/22 10:57:49 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6744lb # [ 157.926712] niks3-server[1201]: 2026/09/22 10:57:49 INFO Completed upload id=486745worker1 # [ 157.922836] nix-grpc-daemon[1728]: time=2026-09-22T10:57:50.040Z level=INFO msg="Upload complete. (386ms)"6746lb # [ 157.929581] niks3-server[1201]: 2026/09/22 10:57:49 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6747worker1 # [ 157.925330] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49704 duration_s=0 paths=1 nar_bytes_in=4166748worker1 # [ 157.928071] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:497046749worker1 # [ 157.933825] nix-daemon[853]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6750worker1 # [ 157.937952] nix-daemon[853]: reaped child process 1778, status = succeeded6751client # [ 157.955145] nix[1386]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6752worker1 # [ 157.942786] nix-daemon[853]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6753worker1: must succeed: systemctl reload nix-grpc-daemon.service6754worker1 # [ 159.144576] systemd[1]: Reloading Nix worker-protocol over gRPC...6755worker1 # [ 159.181771] systemd[1]: Reloaded Nix worker-protocol over gRPC.6756worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.07 seconds)6757worker1: must succeed: pgrep -f 'read -t [9]0 x'6758worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6759client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26760client # don't know how to build these paths:6761client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6762worker2 # [ 159.428536] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480746763worker2 # [ 159.431417] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480746764worker1 # [ 159.449303] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:51Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06765worker1 # [ 159.450989] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:51Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16766worker2 # [ 159.497655] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480746767worker2 # [ 159.510182] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480746768client # copying 1 paths to 'lb:50051'...6769worker2 # [ 159.521414] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480746770client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6771worker2 # [ 159.546072] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6772worker2 # [ 159.721593] nix-daemon[873]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6773worker2 # [ 159.733691] nix-daemon[873]: reaped child process 1691, status = succeeded6774worker2 # [ 159.910208] nix-daemon[873]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6775worker2 # [ 159.921811] nix-daemon[873]: reaped child process 1696, status = succeeded6776lb # [ 159.943635] niks3-server[1201]: 2026/09/22 10:57:51 INFO Received uploads request method=POST path=/api/pending_closures6777worker2 # [ 159.951691] nix-grpc-daemon[1612]: time=2026-09-22T10:57:51.250Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6778worker2 # [ 159.953253] nix-grpc-daemon[1612]: time=2026-09-22T10:57:51.250Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6779lb # [ 159.974585] niks3-server[1201]: 2026/09/22 10:57:51 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6780lb # [ 159.985101] niks3-server[1201]: 2026/09/22 10:57:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6781lb # [ 159.992761] niks3-server[1201]: 2026/09/22 10:57:51 INFO Signed narinfos id=49 count=16782worker2 # [ 159.992507] nix-grpc-daemon[1612]: time=2026-09-22T10:57:51.291Z level=INFO msg="Uploading 1 narinfos"6783lb # [ 159.997781] niks3-server[1201]: 2026/09/22 10:57:51 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6784lb # [ 160.006856] niks3-server[1201]: 2026/09/22 10:57:52 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6785worker2 # [ 160.010301] nix-grpc-daemon[1612]: time=2026-09-22T10:57:51.309Z level=INFO msg="Upload complete. (452ms)"6786lb # [ 160.014277] niks3-server[1201]: 2026/09/22 10:57:52 INFO Completed upload id=496787lb # [ 160.018110] niks3-server[1201]: 2026/09/22 10:57:52 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6788worker2 # [ 160.013491] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=0 paths=1 nar_bytes_in=5286789worker2 # [ 160.019195] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480746790worker2 # [ 160.027470] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6791worker2 # [ 160.032961] nix-daemon[873]: reaped child process 1686, status = succeeded6792worker2 # [ 160.037820] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6793client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6794client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6795worker1 # [ 160.090649] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:52Z level=info event=draining6796client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6797client # farm-a> LOG-farm-a-during-drain6798worker2 # [ 161.863542] nix-daemon[873]: reaped child process 1700, status = succeeded6799worker2 # [ 162.019086] nix-daemon[873]: accepted connection from pid 1716, user nix-grpc-daemon (trusted)6800worker2 # [ 162.029741] nix-daemon[873]: reaped child process 1719, status = succeeded6801lb # [ 162.050310] niks3-server[1201]: 2026/09/22 10:57:54 INFO Received uploads request method=POST path=/api/pending_closures6802worker2 # [ 162.051590] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.350Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6803worker2 # [ 162.053429] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.352Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6804lb # [ 162.072249] niks3-server[1201]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6805lb # [ 162.076777] niks3-server[1201]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6806worker2 # [ 162.076582] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.375Z level=INFO msg="Uploading 1 narinfos"6807lb # [ 162.079735] niks3-server[1201]: 2026/09/22 10:57:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6808lb # [ 162.081906] niks3-server[1201]: 2026/09/22 10:57:54 INFO Signed narinfos id=50 count=16809lb # [ 162.086697] niks3-server[1201]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6810lb # [ 162.089559] niks3-server[1201]: 2026/09/22 10:57:54 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6811lb # [ 162.093654] niks3-server[1201]: 2026/09/22 10:57:54 INFO Completed upload id=506812worker2 # [ 162.091078] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.390Z level=INFO msg="Upload complete. (245ms)"6813lb # [ 162.095241] niks3-server[1201]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6814worker2 # [ 162.093392] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=2 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16815worker1 # [ 162.098617] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:54Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06816worker2 # [ 162.100388] nix-daemon[873]: reaped child process 1698, status = succeeded6817worker1 # [ 162.100431] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:54Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16818worker2 # [ 162.142490] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480746819worker2 # [ 162.152318] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480746820worker2 # [ 162.166006] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.465Z level=INFO msg="All 1 paths already cached"6821client # copying 1 paths to 'lb:50051'...6822client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6823worker2 # [ 162.172678] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480746824worker2 # [ 162.180673] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6825worker2 # [ 162.341402] nix-daemon[873]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6826worker2 # [ 162.352779] nix-daemon[873]: reaped child process 1726, status = succeeded6827worker2 # [ 162.497715] nix-daemon[873]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)6828worker2 # [ 162.510458] nix-daemon[873]: reaped child process 1731, status = succeeded6829lb # [ 162.531899] niks3-server[1201]: 2026/09/22 10:57:54 INFO Received uploads request method=POST path=/api/pending_closures6830worker2 # [ 162.536985] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.836Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6831worker2 # [ 162.538497] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.837Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6832lb # [ 162.563689] niks3-server[1201]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6833lb # [ 162.573614] niks3-server[1201]: 2026/09/22 10:57:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6834worker2 # [ 162.574748] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.874Z level=INFO msg="Uploading 1 narinfos"6835lb # [ 162.578969] niks3-server[1201]: 2026/09/22 10:57:54 INFO Signed narinfos id=51 count=16836lb # [ 162.585302] niks3-server[1201]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6837lb # [ 162.596395] niks3-server[1201]: 2026/09/22 10:57:54 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6838lb # [ 162.602663] niks3-server[1201]: 2026/09/22 10:57:54 INFO Completed upload id=516839worker2 # [ 162.599087] nix-grpc-daemon[1612]: time=2026-09-22T10:57:53.898Z level=INFO msg="Upload complete. (406ms)"6840worker2 # [ 162.601403] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=0 paths=1 nar_bytes_in=6406841lb # [ 162.607547] niks3-server[1201]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6842worker2 # [ 162.606583] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480746843worker2 # [ 162.615720] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6844worker2 # [ 162.621773] nix-daemon[873]: reaped child process 1721, status = succeeded6845client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6846worker2 # [ 162.625721] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6847client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6848client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6849client # farm-b> LOG-farm-b-during-drain6850worker2 # [ 163.439157] nix-daemon[873]: reaped child process 1735, status = succeeded6851worker2 # [ 163.573962] nix-daemon[873]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)6852worker2 # [ 163.586817] nix-daemon[873]: reaped child process 1754, status = succeeded6853lb # [ 163.607406] niks3-server[1201]: 2026/09/22 10:57:55 INFO Received uploads request method=POST path=/api/pending_closures6854worker2 # [ 163.617824] nix-grpc-daemon[1612]: time=2026-09-22T10:57:54.916Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6855worker2 # [ 163.619218] nix-grpc-daemon[1612]: time=2026-09-22T10:57:54.916Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6856lb # [ 163.651641] niks3-server[1201]: 2026/09/22 10:57:55 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6857lb # [ 163.660394] niks3-server[1201]: 2026/09/22 10:57:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6858lb # [ 163.665251] niks3-server[1201]: 2026/09/22 10:57:55 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6859lb # [ 163.671247] niks3-server[1201]: 2026/09/22 10:57:55 INFO Signed narinfos id=52 count=16860worker2 # [ 163.670657] nix-grpc-daemon[1612]: time=2026-09-22T10:57:54.969Z level=INFO msg="Uploading 1 narinfos"6861lb # [ 163.677769] niks3-server[1201]: 2026/09/22 10:57:55 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6862lb # [ 163.684766] niks3-server[1201]: 2026/09/22 10:57:55 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6863lb # [ 163.694990] niks3-server[1201]: 2026/09/22 10:57:55 INFO Completed upload id=526864worker2 # [ 163.691775] nix-grpc-daemon[1612]: time=2026-09-22T10:57:54.991Z level=INFO msg="Upload complete. (260ms)"6865lb # [ 163.701671] niks3-server[1201]: 2026/09/22 10:57:55 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6866worker1 # [ 163.702950] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:55Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06867worker2 # [ 163.699148] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16868worker1 # [ 163.704738] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:55Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16869worker2 # [ 163.712751] nix-daemon[873]: reaped child process 1733, status = succeeded6870worker2 # [ 163.746124] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480746871worker2 # [ 163.757152] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480746872worker2 # [ 163.770102] nix-grpc-daemon[1612]: time=2026-09-22T10:57:55.069Z level=INFO msg="All 2 paths already cached"6873client # copying 1 paths to 'lb:50051'...6874client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6875worker2 # [ 163.777978] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480746876worker2 # [ 163.792167] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6877worker2 # [ 163.965543] nix-daemon[873]: accepted connection from pid 1758, user nix-grpc-daemon (trusted)6878worker2 # [ 163.978811] nix-daemon[873]: reaped child process 1761, status = succeeded6879worker2 # [ 164.120713] nix-daemon[873]: accepted connection from pid 1763, user nix-grpc-daemon (trusted)6880worker2 # [ 164.133534] nix-daemon[873]: reaped child process 1766, status = succeeded6881lb # [ 164.148841] niks3-server[1201]: 2026/09/22 10:57:56 INFO Received uploads request method=POST path=/api/pending_closures6882worker2 # [ 164.154047] nix-grpc-daemon[1612]: time=2026-09-22T10:57:55.453Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6883worker2 # [ 164.155375] nix-grpc-daemon[1612]: time=2026-09-22T10:57:55.453Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6884lb # [ 164.179551] niks3-server[1201]: 2026/09/22 10:57:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6885worker2 # [ 164.182281] nix-grpc-daemon[1612]: time=2026-09-22T10:57:55.481Z level=INFO msg="Uploading 1 narinfos"6886lb # [ 164.186078] niks3-server[1201]: 2026/09/22 10:57:56 INFO Signed narinfos id=53 count=16887lb # [ 164.191675] niks3-server[1201]: 2026/09/22 10:57:56 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6888lb # [ 164.198603] niks3-server[1201]: 2026/09/22 10:57:56 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6889lb # [ 164.209645] niks3-server[1201]: 2026/09/22 10:57:56 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6890lb # [ 164.214580] niks3-server[1201]: 2026/09/22 10:57:56 INFO Completed upload id=536891worker2 # [ 164.210914] nix-grpc-daemon[1612]: time=2026-09-22T10:57:55.510Z level=INFO msg="Upload complete. (406ms)"6892worker2 # [ 164.213050] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=0 paths=1 nar_bytes_in=7686893worker2 # [ 164.216109] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480746894lb # [ 164.220127] niks3-server[1201]: 2026/09/22 10:57:56 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6895worker2 # [ 164.222541] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6896worker2 # [ 164.225708] nix-daemon[873]: reaped child process 1756, status = succeeded6897client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6898client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6899worker2 # [ 164.232117] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6900client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6901client # farm-top> LOG-farm-top-during-drain6902worker2 # [ 165.644729] nix-daemon[873]: reaped child process 1770, status = succeeded6903worker2 # [ 165.784341] nix-daemon[873]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6904worker2 # [ 165.794862] nix-daemon[873]: reaped child process 1789, status = succeeded6905lb # [ 165.817430] niks3-server[1201]: 2026/09/22 10:57:57 INFO Received uploads request method=POST path=/api/pending_closures6906worker2 # [ 165.822050] nix-grpc-daemon[1612]: time=2026-09-22T10:57:57.121Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6907worker2 # [ 165.823538] nix-grpc-daemon[1612]: time=2026-09-22T10:57:57.122Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6908lb # [ 165.854967] niks3-server[1201]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6909lb # [ 165.860230] niks3-server[1201]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6910lb # [ 165.874075] niks3-server[1201]: 2026/09/22 10:57:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6911lb # [ 165.878347] niks3-server[1201]: 2026/09/22 10:57:57 INFO Signed narinfos id=54 count=16912worker2 # [ 165.878130] nix-grpc-daemon[1612]: time=2026-09-22T10:57:57.177Z level=INFO msg="Uploading 1 narinfos"6913lb # [ 165.883938] niks3-server[1201]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6914lb # [ 165.894073] niks3-server[1201]: 2026/09/22 10:57:57 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6915worker2 # [ 165.896705] nix-grpc-daemon[1612]: time=2026-09-22T10:57:57.196Z level=INFO msg="Upload complete. (258ms)"6916lb # [ 165.900343] niks3-server[1201]: 2026/09/22 10:57:57 INFO Completed upload id=546917worker2 # [ 165.899423] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16918lb # [ 165.903910] niks3-server[1201]: 2026/09/22 10:57:57 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6919worker2 # [ 165.906069] nix-daemon[873]: reaped child process 1768, status = succeeded6920client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.71 seconds)6921client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6922lb # [ 166.051286] envoy[910]: conn peer=[2001:db8:1::1]:40532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9611 tx=4090 ms=64916923client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6924worker1: must succeed: pkill -f 'read -t [9]0 x'6925worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6926client: waiting for success: ! systemctl is-active sw6927worker1 # [ 166.375917] nix-daemon[853]: reaped child process 1796, status = succeeded6928worker1 # [ 166.515924] nix-daemon[853]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6929worker1 # [ 166.527967] nix-daemon[853]: reaped child process 1842, status = succeeded6930lb # [ 166.546506] niks3-server[1201]: 2026/09/22 10:57:58 INFO Received uploads request method=POST path=/api/pending_closures6931worker1 # [ 166.547208] nix-grpc-daemon[1728]: time=2026-09-22T10:57:58.664Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6932worker1 # [ 166.548669] nix-grpc-daemon[1728]: time=2026-09-22T10:57:58.666Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6933lb # [ 166.564721] niks3-server[1201]: 2026/09/22 10:57:58 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6934lb # [ 166.569053] niks3-server[1201]: 2026/09/22 10:57:58 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6935lb # [ 166.571270] niks3-server[1201]: 2026/09/22 10:57:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6936worker1 # [ 166.571562] nix-grpc-daemon[1728]: time=2026-09-22T10:57:58.688Z level=INFO msg="Uploading 1 narinfos"6937lb # [ 166.574480] niks3-server[1201]: 2026/09/22 10:57:58 INFO Signed narinfos id=55 count=16938lb # [ 166.580541] niks3-server[1201]: 2026/09/22 10:57:58 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6939lb # [ 166.585869] niks3-server[1201]: 2026/09/22 10:57:58 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6940lb # [ 166.588649] niks3-server[1201]: 2026/09/22 10:57:58 INFO Completed upload id=556941worker1 # [ 166.585969] nix-grpc-daemon[1728]: time=2026-09-22T10:57:58.703Z level=INFO msg="Upload complete. (225ms)"6942lb # [ 166.590510] niks3-server[1201]: 2026/09/22 10:57:58 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6943worker1 # [ 166.589312] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49704 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16944worker1 # [ 166.595947] nix-daemon[853]: reaped child process 1794, status = succeeded6945client # [ 166.627120] systemd[1]: sw.service: Deactivated successfully.6946client # [ 166.628712] systemd[1]: sw.service: Consumed 110ms CPU time over 9.908s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.2K outgoing IP traffic.6947worker1 # [ 167.092747] nix-grpc-daemon[1645]: ts=2026-09-22T10:57:59Z level=info event=signal_exit6948lb # [ 167.105709] niks3-server[1201]: 2026/09/22 10:57:59 INFO lead: released remote=192.168.1.3:492066949client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6950client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6951client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6952worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6953worker2 # [ 168.102056] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:59Z level=info event=scheduler_restarting addr=lb:500516954worker1 # [ 168.101381] nix-grpc-daemon[1645]: ts=2026-09-22T10:58:00Z level=info event=scheduler_restarting6955worker1 # [ 168.107175] nix-grpc-daemon[1645]: ts=2026-09-22T10:58:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:496966956worker1 # [ 168.108816] nix-grpc-daemon[1645]: ts=2026-09-22T10:58:00Z level=info event=worker_gone addr=192.168.1.4:500516957worker1 # [ 168.110180] nix-grpc-daemon[1645]: ts=2026-09-22T10:58:00Z level=info event=scheduler_restarting addr=lb:500516958worker1 # [ 168.111458] nix-grpc-daemon[1645]: ts=2026-09-22T10:58:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:496966959worker1 # [ 168.112905] nix-grpc-daemon[1645]: ts=2026-09-22T10:58:00Z level=info event=worker_gone addr=192.168.1.3:500516960worker1 # [ 168.121139] nix-daemon[853]: reaped child process 1699, status = succeeded6961worker2 # [ 168.302506] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:59Z level=info event=scheduler_connected addr=lb:500516962worker2 # [ 168.306555] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6963worker1 # [ 168.315674] nix-grpc-daemon[1645]: ts=2026-09-22T10:58:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6964worker1 # [ 168.419940] nix-grpc-daemon[1645]: E0000 00:00:1790074680.537183 1690 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6965worker1 # [ 168.424338] nix-grpc-daemon[1645]: E0000 00:00:1790074680.537299 1690 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6966worker2 # [ 168.507073] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:59Z level=info event=scheduler_connected addr=lb:500516967worker2 # [ 168.509075] nix-grpc-daemon[1464]: ts=2026-09-22T10:57:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6968worker2 # [ 168.709754] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_connected addr=lb:500516969worker2 # [ 168.711492] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6970worker2 # [ 168.912236] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_connected addr=lb:500516971worker2 # [ 168.913950] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6972worker2 # [ 169.115137] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_connected addr=lb:500516973worker2 # [ 169.116776] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6974worker1 # [ 169.118555] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6975worker1 # [ 169.120227] systemd[1]: nix-grpc-daemon.service: Consumed 626ms CPU time over 22.417s wall clock time, 33.4M memory peak, 25.2K incoming IP traffic, 32K outgoing IP traffic.6976worker1 # [ 169.129910] systemd[1]: Starting Nix worker-protocol over gRPC...6977worker2 # [ 169.186223] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_take_over6978lb # [ 169.190561] niks3-server[1201]: 2026/09/22 10:58:01 INFO lead: acquired remote=192.168.1.4:450306979worker1 # [ 169.282452] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6980worker1 # [ 169.284850] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6981worker1 # [ 169.288703] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16982worker1 # [ 169.296066] nix-grpc-daemon[1856]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6983worker1 # [ 169.298185] nix-grpc-daemon[1856]: E0000 00:00:1790074681.412902 1856 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6984worker1 # [ 169.303512] nix-grpc-daemon[1856]: E0000 00:00:1790074681.418069 1856 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6985worker1 # [ 169.308338] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6986worker1 # [ 169.310728] systemd[1]: Started Nix worker-protocol over gRPC.6987worker1 # [ 169.314518] nix-grpc-daemon[1856]: E0000 00:00:1790074681.432040 1898 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6988worker2 # [ 169.317457] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_connected addr=lb:500516989worker2 # [ 169.319229] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6990worker1 # [ 169.329322] nix-grpc-daemon[1856]: E0000 00:00:1790074681.445564 1898 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6991worker1 # [ 169.332139] nix-grpc-daemon[1856]: E0000 00:00:1790074681.445954 1898 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6992worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.05 seconds)6993lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6994lb: must succeed: curl -sf localhost:9901/clusters6995worker1 # [ 169.342959] nix-grpc-daemon[1856]: E0000 00:00:1790074681.460420 1898 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6996worker1 # [ 169.353960] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:01Z level=info event=scheduler_connected addr=lb:500516997worker1 # [ 169.355558] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6998lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6999worker2 # [ 169.519968] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=scheduler_connected addr=lb:500517000worker2 # [ 169.528598] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=oidc_jwks provider=mock keys=17001worker2 # [ 169.530223] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:601307002worker2 # [ 169.532165] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07003worker1 # [ 169.556279] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:01Z level=info event=scheduler_connected addr=lb:500517004worker2 # [ 169.557451] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:601307005worker2 # [ 169.559151] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07006lb: must succeed: curl -sf localhost:9901/clusters7007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7008lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.12 seconds)7009lb: must succeed: curl -sf localhost:9901/clusters7010lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7011worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7012worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7013(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.89 seconds)7014subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7015client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17016worker2 # [ 170.748250] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480747017worker1 # [ 170.755767] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047018worker1 # [ 170.761435] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7019worker1 # [ 170.770079] nix-grpc-daemon[1856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)7020worker1 # [ 170.824654] nix-grpc-daemon[1856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)7021worker1 # [ 171.025335] nix-grpc-daemon[1856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 104 ms (attempt 3/5)7022worker1 # [ 171.132103] nix-grpc-daemon[1856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 484 ms (attempt 4/5)7023worker1 # [ 171.619368] nix-grpc-daemon[1856]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7024worker1 # [ 171.623488] nix-grpc-daemon[1856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7025worker2 # [ 171.670742] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:02Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07026worker2 # [ 171.675150] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:02Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17027worker2 # [ 171.679518] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:02Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07028worker2 # [ 171.683369] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:02Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07029worker1 # [ 171.714895] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047030worker1 # [ 171.724213] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047031worker1 # [ 171.731707] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047032worker1 # [ 171.737077] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7033worker1 # [ 171.754222] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:03Z level=info event=niks3_push_started pid=19167034worker1 # [ 171.916940] nix-grpc-daemon[1916]: time=2026-09-22T10:58:04.033Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""7035worker1 # [ 172.071474] nix-daemon[853]: accepted connection from pid 1929, user nix-grpc-daemon (trusted)7036worker1 # [ 172.084145] nix-daemon[853]: reaped child process 1932, status = succeeded7037worker1 # [ 172.259959] nix-daemon[853]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)7038worker1 # [ 172.273780] nix-daemon[853]: reaped child process 1937, status = succeeded7039lb # [ 172.293670] niks3-server[1201]: 2026/09/22 10:58:04 INFO Received uploads request method=POST path=/api/pending_closures7040worker1 # [ 172.294609] nix-grpc-daemon[1916]: time=2026-09-22T10:58:04.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7041worker1 # [ 172.298180] nix-grpc-daemon[1916]: time=2026-09-22T10:58:04.413Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7042lb # [ 172.315709] niks3-server[1201]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7043lb # [ 172.318343] niks3-server[1201]: 2026/09/22 10:58:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign7044lb # [ 172.320584] niks3-server[1201]: 2026/09/22 10:58:04 INFO Signed narinfos id=56 count=17045worker1 # [ 172.318186] nix-grpc-daemon[1916]: time=2026-09-22T10:58:04.435Z level=INFO msg="Uploading 1 narinfos"7046lb # [ 172.326668] niks3-server[1201]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7047lb # [ 172.328436] niks3-server[1201]: 2026/09/22 10:58:04 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete7048lb # [ 172.333472] niks3-server[1201]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7049lb # [ 172.336505] niks3-server[1201]: 2026/09/22 10:58:04 INFO Completed upload id=567050worker1 # [ 172.332667] nix-grpc-daemon[1916]: time=2026-09-22T10:58:04.450Z level=INFO msg="Upload complete. (412ms)"7051worker1 # [ 172.335099] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=5047052worker1 # [ 172.338029] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047053worker1 # [ 172.345198] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7054worker1 # [ 172.350281] nix-daemon[853]: reaped child process 1914, status = succeeded7055worker1 # [ 172.355343] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7056worker1 # [ 173.771138] nix-daemon[853]: reaped child process 1941, status = succeeded7057worker1 # [ 173.906818] nix-daemon[853]: accepted connection from pid 1957, user nix-grpc-daemon (trusted)7058worker1 # [ 173.918928] nix-daemon[853]: reaped child process 1960, status = succeeded7059lb # [ 173.933602] niks3-server[1201]: 2026/09/22 10:58:05 INFO Received uploads request method=POST path=/api/pending_closures7060worker1 # [ 173.934327] nix-grpc-daemon[1916]: time=2026-09-22T10:58:06.051Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7061worker1 # [ 173.936521] nix-grpc-daemon[1916]: time=2026-09-22T10:58:06.053Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7062lb # [ 173.954284] niks3-server[1201]: 2026/09/22 10:58:05 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7063lb # [ 173.958581] niks3-server[1201]: 2026/09/22 10:58:05 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7064lb # [ 173.961987] niks3-server[1201]: 2026/09/22 10:58:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign7065lb # [ 173.963670] niks3-server[1201]: 2026/09/22 10:58:05 INFO Signed narinfos id=57 count=17066worker1 # [ 173.960598] nix-grpc-daemon[1916]: time=2026-09-22T10:58:06.077Z level=INFO msg="Uploading 1 narinfos"7067lb # [ 173.967952] niks3-server[1201]: 2026/09/22 10:58:05 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7068lb # [ 173.971766] niks3-server[1201]: 2026/09/22 10:58:05 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete7069lb # [ 173.974862] niks3-server[1201]: 2026/09/22 10:58:05 INFO Completed upload id=577070worker1 # [ 173.970794] nix-grpc-daemon[1916]: time=2026-09-22T10:58:06.088Z level=INFO msg="Upload complete. (208ms)"7071lb # [ 173.977598] niks3-server[1201]: 2026/09/22 10:58:05 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7072worker1 # [ 173.973260] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=17073worker2 # [ 173.975210] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:05Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17074worker2 # [ 173.980480] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:05Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07075worker1 # [ 173.980470] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047076worker1 # [ 173.983040] nix-daemon[853]: reaped child process 1939, status = succeeded7077worker1 # [ 173.987487] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047078worker1 # [ 173.992553] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047079worker1 # [ 173.996256] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7080worker1 # [ 174.121846] nix-daemon[853]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)7081worker1 # [ 174.133833] nix-daemon[853]: reaped child process 1967, status = succeeded7082worker1 # [ 174.268497] nix-daemon[853]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)7083worker1 # [ 174.280936] nix-daemon[853]: reaped child process 1972, status = succeeded7084lb # [ 174.293133] niks3-server[1201]: 2026/09/22 10:58:06 INFO Received uploads request method=POST path=/api/pending_closures7085worker1 # [ 174.295300] nix-grpc-daemon[1916]: time=2026-09-22T10:58:06.412Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7086worker1 # [ 174.296714] nix-grpc-daemon[1916]: time=2026-09-22T10:58:06.412Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7087lb # [ 174.315984] niks3-server[1201]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7088lb # [ 174.326267] niks3-server[1201]: 2026/09/22 10:58:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign7089worker1 # [ 174.329774] nix-grpc-daemon[1916]: time=2026-09-22T10:58:06.447Z level=INFO msg="Uploading 1 narinfos"7090lb # [ 174.333287] niks3-server[1201]: 2026/09/22 10:58:06 INFO Signed narinfos id=58 count=17091lb # [ 174.338147] niks3-server[1201]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7092lb # [ 174.349978] niks3-server[1201]: 2026/09/22 10:58:06 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete7093worker1 # [ 174.353116] nix-grpc-daemon[1916]: time=2026-09-22T10:58:06.470Z level=INFO msg="Upload complete. (351ms)"7094lb # [ 174.356953] niks3-server[1201]: 2026/09/22 10:58:06 INFO Completed upload id=587095worker1 # [ 174.356091] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=5047096lb # [ 174.360060] niks3-server[1201]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7097worker1 # [ 174.360401] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047098worker1 # [ 174.365164] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7099worker1 # [ 174.367464] nix-daemon[853]: reaped child process 1962, status = succeeded7100worker1 # [ 174.372767] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7101worker1 # [ 175.387314] nix-daemon[853]: reaped child process 1976, status = succeeded7102worker1 # [ 175.529515] nix-daemon[853]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)7103worker1 # [ 175.541724] nix-daemon[853]: reaped child process 1995, status = succeeded7104lb # [ 175.561704] niks3-server[1201]: 2026/09/22 10:58:07 INFO Received uploads request method=POST path=/api/pending_closures7105worker1 # [ 175.566288] nix-grpc-daemon[1916]: time=2026-09-22T10:58:07.683Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7106worker1 # [ 175.568501] nix-grpc-daemon[1916]: time=2026-09-22T10:58:07.685Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7107lb # [ 175.596589] niks3-server[1201]: 2026/09/22 10:58:07 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7108lb # [ 175.605744] niks3-server[1201]: 2026/09/22 10:58:07 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7109lb # [ 175.612477] niks3-server[1201]: 2026/09/22 10:58:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign7110lb # [ 175.619120] niks3-server[1201]: 2026/09/22 10:58:07 INFO Signed narinfos id=59 count=17111worker1 # [ 175.618368] nix-grpc-daemon[1916]: time=2026-09-22T10:58:07.735Z level=INFO msg="Uploading 1 narinfos"7112lb # [ 175.625329] niks3-server[1201]: 2026/09/22 10:58:07 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7113lb # [ 175.635790] niks3-server[1201]: 2026/09/22 10:58:07 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete7114lb # [ 175.640861] niks3-server[1201]: 2026/09/22 10:58:07 INFO Completed upload id=597115worker1 # [ 175.638635] nix-grpc-daemon[1916]: time=2026-09-22T10:58:07.756Z level=INFO msg="Upload complete. (259ms)"7116lb # [ 175.642835] niks3-server[1201]: 2026/09/22 10:58:07 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7117worker2 # [ 175.642804] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:06Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17118worker1 # [ 175.642357] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=17119worker1 # [ 175.645444] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047120worker2 # [ 175.648389] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:06Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07121worker1 # [ 175.651668] nix-daemon[853]: reaped child process 1974, status = succeeded7122worker1 # [ 175.653404] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047123worker1 # [ 175.658739] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047124worker1 # [ 175.662510] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7125worker1 # [ 175.798197] nix-daemon[853]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)7126worker1 # [ 175.810117] nix-daemon[853]: reaped child process 2002, status = succeeded7127worker1 # [ 175.972853] nix-daemon[853]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)7128worker1 # [ 175.984671] nix-daemon[853]: reaped child process 2007, status = succeeded7129lb # [ 176.003633] niks3-server[1201]: 2026/09/22 10:58:08 INFO Received uploads request method=POST path=/api/pending_closures7130worker1 # [ 176.003881] nix-grpc-daemon[1916]: time=2026-09-22T10:58:08.121Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7131worker1 # [ 176.005450] nix-grpc-daemon[1916]: time=2026-09-22T10:58:08.122Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7132lb # [ 176.019186] niks3-server[1201]: 2026/09/22 10:58:08 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7133lb # [ 176.023476] niks3-server[1201]: 2026/09/22 10:58:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign7134lb # [ 176.025358] niks3-server[1201]: 2026/09/22 10:58:08 INFO Signed narinfos id=60 count=17135worker1 # [ 176.022703] nix-grpc-daemon[1916]: time=2026-09-22T10:58:08.140Z level=INFO msg="Uploading 1 narinfos"7136lb # [ 176.027824] niks3-server[1201]: 2026/09/22 10:58:08 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7137lb # [ 176.033392] niks3-server[1201]: 2026/09/22 10:58:08 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete7138lb # [ 176.035893] niks3-server[1201]: 2026/09/22 10:58:08 INFO Completed upload id=607139worker1 # [ 176.031993] nix-grpc-daemon[1916]: time=2026-09-22T10:58:08.149Z level=INFO msg="Upload complete. (363ms)"7140worker1 # [ 176.033879] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=5047141lb # [ 176.038383] niks3-server[1201]: 2026/09/22 10:58:08 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7142worker1 # [ 176.036800] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047143worker1 # [ 176.043398] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7144worker1 # [ 176.046368] nix-daemon[853]: reaped child process 1997, status = succeeded7145lb # [ 176.051073] envoy[910]: conn peer=[2001:db8:1::1]:40530 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2936 tx=1429 ms=96937146lb # [ 176.052528] envoy[910]: conn peer=[2001:db8:1::3]:51094 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=958 tx=344 ms=211757147worker1 # [ 176.051218] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7148worker1 # [ 177.265032] nix-daemon[853]: reaped child process 2011, status = succeeded7149worker1 # [ 177.412025] nix-daemon[853]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)7150worker1 # [ 177.424290] nix-daemon[853]: reaped child process 2030, status = succeeded7151lb # [ 177.439815] niks3-server[1201]: 2026/09/22 10:58:09 INFO Received uploads request method=POST path=/api/pending_closures7152worker1 # [ 177.443210] nix-grpc-daemon[1916]: time=2026-09-22T10:58:09.560Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7153worker1 # [ 177.444739] nix-grpc-daemon[1916]: time=2026-09-22T10:58:09.562Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7154lb # [ 177.469802] niks3-server[1201]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7155lb # [ 177.475870] niks3-server[1201]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7156lb # [ 177.478147] niks3-server[1201]: 2026/09/22 10:58:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign7157lb # [ 177.480405] niks3-server[1201]: 2026/09/22 10:58:09 INFO Signed narinfos id=61 count=17158worker1 # [ 177.477906] nix-grpc-daemon[1916]: time=2026-09-22T10:58:09.595Z level=INFO msg="Uploading 1 narinfos"7159lb # [ 177.484527] niks3-server[1201]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7160lb # [ 177.488345] niks3-server[1201]: 2026/09/22 10:58:09 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete7161lb # [ 177.492366] niks3-server[1201]: 2026/09/22 10:58:09 INFO Completed upload id=617162worker1 # [ 177.488397] nix-grpc-daemon[1916]: time=2026-09-22T10:58:09.605Z level=INFO msg="Upload complete. (230ms)"7163lb # [ 177.494046] niks3-server[1201]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7164worker1 # [ 177.491269] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=17165worker1 # [ 177.496357] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047166worker2 # [ 177.494979] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:08Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17167worker1 # [ 177.500758] nix-daemon[853]: reaped child process 2009, status = succeeded7168worker1 # [ 177.504263] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047169worker2 # [ 177.505495] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:08Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07170worker1 # [ 177.510212] nix-grpc-daemon[1916]: time=2026-09-22T10:58:09.627Z level=INFO msg="All 1 paths already cached"7171worker1 # [ 177.513539] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047172worker1 # [ 177.518282] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7173worker1 # [ 177.654953] nix-daemon[853]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)7174worker1 # [ 177.667321] nix-daemon[853]: reaped child process 2037, status = succeeded7175worker1 # [ 177.811504] nix-daemon[853]: accepted connection from pid 2039, user nix-grpc-daemon (trusted)7176worker1 # [ 177.824077] nix-daemon[853]: reaped child process 2042, status = succeeded7177lb # [ 177.838797] niks3-server[1201]: 2026/09/22 10:58:09 INFO Received uploads request method=POST path=/api/pending_closures7178worker1 # [ 177.842709] nix-grpc-daemon[1916]: time=2026-09-22T10:58:09.960Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7179worker1 # [ 177.844230] nix-grpc-daemon[1916]: time=2026-09-22T10:58:09.961Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7180lb # [ 177.866193] niks3-server[1201]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7181lb # [ 177.877334] niks3-server[1201]: 2026/09/22 10:58:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7182lb # [ 177.881914] niks3-server[1201]: 2026/09/22 10:58:09 INFO Signed narinfos id=62 count=17183worker1 # [ 177.880861] nix-grpc-daemon[1916]: time=2026-09-22T10:58:09.998Z level=INFO msg="Uploading 1 narinfos"7184lb # [ 177.890189] niks3-server[1201]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7185lb # [ 177.901436] niks3-server[1201]: 2026/09/22 10:58:09 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7186worker1 # [ 177.903124] nix-grpc-daemon[1916]: time=2026-09-22T10:58:10.020Z level=INFO msg="Upload complete. (380ms)"7187lb # [ 177.906979] niks3-server[1201]: 2026/09/22 10:58:09 INFO Completed upload id=627188worker1 # [ 177.905137] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=6247189lb # [ 177.910348] niks3-server[1201]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7190worker1 # [ 177.911428] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047191worker1 # [ 177.915833] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7192worker1 # [ 177.918196] nix-daemon[853]: reaped child process 2032, status = succeeded7193worker1 # [ 177.924052] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7194worker1 # [ 179.340127] nix-daemon[853]: reaped child process 2046, status = succeeded7195worker1 # [ 179.489789] nix-daemon[853]: accepted connection from pid 2062, user nix-grpc-daemon (trusted)7196worker1 # [ 179.500856] nix-daemon[853]: reaped child process 2065, status = succeeded7197lb # [ 179.524907] niks3-server[1201]: 2026/09/22 10:58:11 INFO Received uploads request method=POST path=/api/pending_closures7198worker1 # [ 179.531862] nix-grpc-daemon[1916]: time=2026-09-22T10:58:11.649Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7199worker1 # [ 179.533605] nix-grpc-daemon[1916]: time=2026-09-22T10:58:11.650Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7200lb # [ 179.569410] niks3-server[1201]: 2026/09/22 10:58:11 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7201lb # [ 179.575470] niks3-server[1201]: 2026/09/22 10:58:11 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7202lb # [ 179.585534] niks3-server[1201]: 2026/09/22 10:58:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7203lb # [ 179.590885] niks3-server[1201]: 2026/09/22 10:58:11 INFO Signed narinfos id=63 count=17204worker1 # [ 179.590255] nix-grpc-daemon[1916]: time=2026-09-22T10:58:11.707Z level=INFO msg="Uploading 1 narinfos"7205lb # [ 179.594259] niks3-server[1201]: 2026/09/22 10:58:11 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7206lb # [ 179.605719] niks3-server[1201]: 2026/09/22 10:58:11 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7207lb # [ 179.611878] niks3-server[1201]: 2026/09/22 10:58:11 INFO Completed upload id=637208lb # [ 179.614764] niks3-server[1201]: 2026/09/22 10:58:11 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7209worker1 # [ 179.610451] nix-grpc-daemon[1916]: time=2026-09-22T10:58:11.727Z level=INFO msg="Upload complete. (278ms)"7210worker1 # [ 179.615985] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=17211worker2 # [ 179.617518] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:10Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17212worker1 # [ 179.630800] nix-daemon[853]: reaped child process 2044, status = succeeded7213worker2 # [ 179.628915] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:10Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07214worker1 # [ 179.668835] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047215worker1 # [ 179.681220] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047216worker1 # [ 179.694368] nix-grpc-daemon[1916]: time=2026-09-22T10:58:11.811Z level=INFO msg="All 1 paths already cached"7217worker1 # [ 179.701467] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047218worker1 # [ 179.709657] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7219worker1 # [ 179.889596] nix-daemon[853]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)7220worker1 # [ 179.901831] nix-daemon[853]: reaped child process 2072, status = succeeded7221worker1 # [ 180.064477] nix-daemon[853]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)7222worker1 # [ 180.076394] nix-daemon[853]: reaped child process 2077, status = succeeded7223lb # [ 180.096091] niks3-server[1201]: 2026/09/22 10:58:12 INFO Received uploads request method=POST path=/api/pending_closures7224worker1 # [ 180.099934] nix-grpc-daemon[1916]: time=2026-09-22T10:58:12.217Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7225worker1 # [ 180.101504] nix-grpc-daemon[1916]: time=2026-09-22T10:58:12.218Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7226lb # [ 180.126786] niks3-server[1201]: 2026/09/22 10:58:12 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7227lb # [ 180.137172] niks3-server[1201]: 2026/09/22 10:58:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7228lb # [ 180.142684] niks3-server[1201]: 2026/09/22 10:58:12 INFO Signed narinfos id=64 count=17229worker1 # [ 180.141915] nix-grpc-daemon[1916]: time=2026-09-22T10:58:12.259Z level=INFO msg="Uploading 1 narinfos"7230lb # [ 180.151115] niks3-server[1201]: 2026/09/22 10:58:12 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7231lb # [ 180.160613] niks3-server[1201]: 2026/09/22 10:58:12 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7232worker1 # [ 180.163062] nix-grpc-daemon[1916]: time=2026-09-22T10:58:12.280Z level=INFO msg="Upload complete. (439ms)"7233lb # [ 180.167193] niks3-server[1201]: 2026/09/22 10:58:12 INFO Completed upload id=647234lb # [ 180.170711] niks3-server[1201]: 2026/09/22 10:58:12 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7235worker1 # [ 180.165840] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=6247236worker1 # [ 180.170871] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047237worker1 # [ 180.181554] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7238worker1 # [ 180.185152] nix-daemon[853]: reaped child process 2067, status = succeeded7239worker1 # [ 180.190092] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7240worker1 # [ 181.203765] nix-daemon[853]: reaped child process 2081, status = succeeded7241worker1 # [ 181.346483] nix-daemon[853]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)7242worker1 # [ 181.362768] nix-daemon[853]: reaped child process 2100, status = succeeded7243lb # [ 181.384398] niks3-server[1201]: 2026/09/22 10:58:13 INFO Received uploads request method=POST path=/api/pending_closures7244worker1 # [ 181.389205] nix-grpc-daemon[1916]: time=2026-09-22T10:58:13.506Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7245worker1 # [ 181.394356] nix-grpc-daemon[1916]: time=2026-09-22T10:58:13.508Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7246lb # [ 181.421832] niks3-server[1201]: 2026/09/22 10:58:13 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7247lb # [ 181.433247] niks3-server[1201]: 2026/09/22 10:58:13 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7248lb # [ 181.444149] niks3-server[1201]: 2026/09/22 10:58:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7249worker1 # [ 181.445902] nix-grpc-daemon[1916]: time=2026-09-22T10:58:13.563Z level=INFO msg="Uploading 1 narinfos"7250lb # [ 181.449487] niks3-server[1201]: 2026/09/22 10:58:13 INFO Signed narinfos id=65 count=17251lb # [ 181.455407] niks3-server[1201]: 2026/09/22 10:58:13 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7252lb # [ 181.464317] niks3-server[1201]: 2026/09/22 10:58:13 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7253lb # [ 181.470085] niks3-server[1201]: 2026/09/22 10:58:13 INFO Completed upload id=657254worker1 # [ 181.466168] nix-grpc-daemon[1916]: time=2026-09-22T10:58:13.583Z level=INFO msg="Upload complete. (269ms)"7255lb # [ 181.471615] niks3-server[1201]: 2026/09/22 10:58:13 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7256worker1 # [ 181.468737] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=17257worker2 # [ 181.471168] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:12Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17258worker1 # [ 181.476187] nix-daemon[853]: reaped child process 2079, status = succeeded7259worker2 # [ 181.476866] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:12Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07260worker1 # [ 181.518505] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047261worker1 # [ 181.531324] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047262worker1 # [ 181.544656] nix-grpc-daemon[1916]: time=2026-09-22T10:58:13.662Z level=INFO msg="All 1 paths already cached"7263worker1 # [ 181.551859] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047264worker1 # [ 181.561167] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7265worker1 # [ 181.723721] nix-daemon[853]: accepted connection from pid 2104, user nix-grpc-daemon (trusted)7266worker1 # [ 181.735774] nix-daemon[853]: reaped child process 2107, status = succeeded7267worker1 # [ 181.889300] nix-daemon[853]: accepted connection from pid 2109, user nix-grpc-daemon (trusted)7268worker1 # [ 181.900974] nix-daemon[853]: reaped child process 2112, status = succeeded7269lb # [ 181.921809] niks3-server[1201]: 2026/09/22 10:58:13 INFO Received uploads request method=POST path=/api/pending_closures7270worker1 # [ 181.926071] nix-grpc-daemon[1916]: time=2026-09-22T10:58:14.043Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7271worker1 # [ 181.927500] nix-grpc-daemon[1916]: time=2026-09-22T10:58:14.043Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7272lb # [ 181.951785] niks3-server[1201]: 2026/09/22 10:58:13 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7273lb # [ 181.961913] niks3-server[1201]: 2026/09/22 10:58:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7274lb # [ 181.967236] niks3-server[1201]: 2026/09/22 10:58:13 INFO Signed narinfos id=66 count=17275worker1 # [ 181.966383] nix-grpc-daemon[1916]: time=2026-09-22T10:58:14.083Z level=INFO msg="Uploading 1 narinfos"7276lb # [ 181.974063] niks3-server[1201]: 2026/09/22 10:58:13 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7277lb # [ 181.984765] niks3-server[1201]: 2026/09/22 10:58:13 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7278worker1 # [ 181.987691] nix-grpc-daemon[1916]: time=2026-09-22T10:58:14.105Z level=INFO msg="Upload complete. (414ms)"7279lb # [ 181.991560] niks3-server[1201]: 2026/09/22 10:58:13 INFO Completed upload id=667280worker1 # [ 181.990078] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=6247281lb # [ 181.994846] niks3-server[1201]: 2026/09/22 10:58:13 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7282worker1 # [ 181.995297] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047283worker1 # [ 182.004479] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7284worker1 # [ 182.006969] nix-daemon[853]: reaped child process 2102, status = succeeded7285worker1 # [ 182.011866] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7286worker1 # [ 183.425100] nix-daemon[853]: reaped child process 2116, status = succeeded7287worker1 # [ 183.545271] nix-daemon[853]: accepted connection from pid 2132, user nix-grpc-daemon (trusted)7288worker1 # [ 183.557215] nix-daemon[853]: reaped child process 2135, status = succeeded7289lb # [ 183.574960] niks3-server[1201]: 2026/09/22 10:58:15 INFO Received uploads request method=POST path=/api/pending_closures7290worker1 # [ 183.577258] nix-grpc-daemon[1916]: time=2026-09-22T10:58:15.694Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7291worker1 # [ 183.578704] nix-grpc-daemon[1916]: time=2026-09-22T10:58:15.696Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7292lb # [ 183.613399] niks3-server[1201]: 2026/09/22 10:58:15 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7293lb # [ 183.619875] niks3-server[1201]: 2026/09/22 10:58:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7294worker1 # [ 183.625487] nix-grpc-daemon[1916]: time=2026-09-22T10:58:15.742Z level=INFO msg="Uploading 1 narinfos"7295lb # [ 183.629179] niks3-server[1201]: 2026/09/22 10:58:15 INFO Signed narinfos id=67 count=17296lb # [ 183.638342] niks3-server[1201]: 2026/09/22 10:58:15 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7297lb # [ 183.651627] niks3-server[1201]: 2026/09/22 10:58:15 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7298lb # [ 183.655839] niks3-server[1201]: 2026/09/22 10:58:15 INFO Completed upload id=677299worker1 # [ 183.651518] nix-grpc-daemon[1916]: time=2026-09-22T10:58:15.768Z level=INFO msg="Upload complete. (233ms)"7300lb # [ 183.658334] niks3-server[1201]: 2026/09/22 10:58:15 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7301worker1 # [ 183.656834] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=17302worker2 # [ 183.662948] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:14Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17303worker1 # [ 183.670323] nix-daemon[853]: reaped child process 2114, status = succeeded7304worker2 # [ 183.668611] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:14Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07305worker1 # [ 183.705238] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047306worker1 # [ 183.717906] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047307worker1 # [ 183.730885] nix-grpc-daemon[1916]: time=2026-09-22T10:58:15.848Z level=INFO msg="All 2 paths already cached"7308worker1 # [ 183.737792] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047309worker1 # [ 183.745776] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7310worker1 # [ 183.887859] nix-daemon[853]: accepted connection from pid 2139, user nix-grpc-daemon (trusted)7311worker1 # [ 183.899968] nix-daemon[853]: reaped child process 2142, status = succeeded7312worker1 # [ 184.034128] nix-daemon[853]: accepted connection from pid 2144, user nix-grpc-daemon (trusted)7313worker1 # [ 184.045853] nix-daemon[853]: reaped child process 2147, status = succeeded7314lb # [ 184.061346] niks3-server[1201]: 2026/09/22 10:58:16 INFO Received uploads request method=POST path=/api/pending_closures7315worker1 # [ 184.065735] nix-grpc-daemon[1916]: time=2026-09-22T10:58:16.183Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7316worker1 # [ 184.067553] nix-grpc-daemon[1916]: time=2026-09-22T10:58:16.184Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7317lb # [ 184.089793] niks3-server[1201]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7318lb # [ 184.100677] niks3-server[1201]: 2026/09/22 10:58:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7319lb # [ 184.105855] niks3-server[1201]: 2026/09/22 10:58:16 INFO Signed narinfos id=68 count=17320worker1 # [ 184.104899] nix-grpc-daemon[1916]: time=2026-09-22T10:58:16.222Z level=INFO msg="Uploading 1 narinfos"7321lb # [ 184.112942] niks3-server[1201]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7322lb # [ 184.123051] niks3-server[1201]: 2026/09/22 10:58:16 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7323worker1 # [ 184.126159] nix-grpc-daemon[1916]: time=2026-09-22T10:58:16.243Z level=INFO msg="Upload complete. (367ms)"7324lb # [ 184.130120] niks3-server[1201]: 2026/09/22 10:58:16 INFO Completed upload id=687325worker1 # [ 184.128382] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=7527326worker1 # [ 184.132195] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047327lb # [ 184.138790] niks3-server[1201]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7328worker1 # [ 184.141073] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7329worker1 # [ 184.145783] nix-daemon[853]: reaped child process 2137, status = succeeded7330worker1 # [ 184.150756] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7331worker1 # [ 185.564359] nix-daemon[853]: reaped child process 2151, status = succeeded7332worker1 # [ 185.704564] nix-daemon[853]: accepted connection from pid 2167, user nix-grpc-daemon (trusted)7333worker1 # [ 185.716815] nix-daemon[853]: reaped child process 2170, status = succeeded7334lb # [ 185.734823] niks3-server[1201]: 2026/09/22 10:58:17 INFO Received uploads request method=POST path=/api/pending_closures7335worker1 # [ 185.738790] nix-grpc-daemon[1916]: time=2026-09-22T10:58:17.856Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7336worker1 # [ 185.740283] nix-grpc-daemon[1916]: time=2026-09-22T10:58:17.857Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7337lb # [ 185.762474] niks3-server[1201]: 2026/09/22 10:58:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7338lb # [ 185.764534] niks3-server[1201]: 2026/09/22 10:58:17 INFO Signed narinfos id=69 count=17339worker1 # [ 185.760489] nix-grpc-daemon[1916]: time=2026-09-22T10:58:17.878Z level=INFO msg="Uploading 1 narinfos"7340lb # [ 185.769048] niks3-server[1201]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7341lb # [ 185.771464] niks3-server[1201]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7342lb # [ 185.774664] niks3-server[1201]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7343lb # [ 185.778991] niks3-server[1201]: 2026/09/22 10:58:17 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7344lb # [ 185.781912] niks3-server[1201]: 2026/09/22 10:58:17 INFO Completed upload id=697345worker1 # [ 185.778093] nix-grpc-daemon[1916]: time=2026-09-22T10:58:17.895Z level=INFO msg="Upload complete. (218ms)"7346lb # [ 185.784654] niks3-server[1201]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7347worker1 # [ 185.780464] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=17348worker2 # [ 185.783910] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:17Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17349worker1 # [ 185.787545] nix-daemon[853]: reaped child process 2149, status = succeeded7350worker1 # [ 185.827529] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047351worker1 # [ 185.837740] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047352worker1 # [ 185.849054] nix-grpc-daemon[1916]: time=2026-09-22T10:58:17.966Z level=INFO msg="All 2 paths already cached"7353worker1 # [ 185.855100] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047354worker1 # [ 185.861848] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7355worker1 # [ 185.998622] nix-daemon[853]: accepted connection from pid 2174, user nix-grpc-daemon (trusted)7356worker1 # [ 186.010876] nix-daemon[853]: reaped child process 2177, status = succeeded7357worker1 # [ 186.142522] nix-daemon[853]: accepted connection from pid 2179, user nix-grpc-daemon (trusted)7358worker1 # [ 186.155709] nix-daemon[853]: reaped child process 2182, status = succeeded7359lb # [ 186.168462] niks3-server[1201]: 2026/09/22 10:58:18 INFO Received uploads request method=POST path=/api/pending_closures7360worker1 # [ 186.170242] nix-grpc-daemon[1916]: time=2026-09-22T10:58:18.287Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7361worker1 # [ 186.171668] nix-grpc-daemon[1916]: time=2026-09-22T10:58:18.287Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7362lb # [ 186.184055] niks3-server[1201]: 2026/09/22 10:58:18 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7363lb # [ 186.187207] niks3-server[1201]: 2026/09/22 10:58:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7364lb # [ 186.190280] niks3-server[1201]: 2026/09/22 10:58:18 INFO Signed narinfos id=70 count=17365worker1 # [ 186.186589] nix-grpc-daemon[1916]: time=2026-09-22T10:58:18.303Z level=INFO msg="Uploading 1 narinfos"7366lb # [ 186.193694] niks3-server[1201]: 2026/09/22 10:58:18 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7367lb # [ 186.198026] niks3-server[1201]: 2026/09/22 10:58:18 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7368lb # [ 186.202821] niks3-server[1201]: 2026/09/22 10:58:18 INFO Completed upload id=707369worker1 # [ 186.198918] nix-grpc-daemon[1916]: time=2026-09-22T10:58:18.316Z level=INFO msg="Upload complete. (325ms)"7370worker1 # [ 186.200748] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=7527371lb # [ 186.206922] niks3-server[1201]: 2026/09/22 10:58:18 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7372worker1 # [ 186.203390] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047373worker1 # [ 186.211169] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7374worker1 # [ 186.212927] nix-daemon[853]: reaped child process 2172, status = succeeded7375worker1 # [ 186.218088] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7376worker1 # [ 187.445749] nix-daemon[853]: reaped child process 2186, status = succeeded7377worker1 # [ 187.607364] nix-daemon[853]: accepted connection from pid 2202, user nix-grpc-daemon (trusted)7378worker1 # [ 187.618429] nix-daemon[853]: reaped child process 2205, status = succeeded7379lb # [ 187.634056] niks3-server[1201]: 2026/09/22 10:58:19 INFO Received uploads request method=POST path=/api/pending_closures7380worker1 # [ 187.635238] nix-grpc-daemon[1916]: time=2026-09-22T10:58:19.752Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7381worker1 # [ 187.636714] nix-grpc-daemon[1916]: time=2026-09-22T10:58:19.754Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7382lb # [ 187.655096] niks3-server[1201]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7383lb # [ 187.658199] niks3-server[1201]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7384lb # [ 187.662640] niks3-server[1201]: 2026/09/22 10:58:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7385lb # [ 187.665039] niks3-server[1201]: 2026/09/22 10:58:19 INFO Signed narinfos id=71 count=17386worker1 # [ 187.661313] nix-grpc-daemon[1916]: time=2026-09-22T10:58:19.778Z level=INFO msg="Uploading 1 narinfos"7387lb # [ 187.668115] niks3-server[1201]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7388lb # [ 187.673097] niks3-server[1201]: 2026/09/22 10:58:19 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7389lb # [ 187.677876] niks3-server[1201]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7390lb # [ 187.680229] niks3-server[1201]: 2026/09/22 10:58:19 INFO Completed upload id=717391worker1 # [ 187.676335] nix-grpc-daemon[1916]: time=2026-09-22T10:58:19.793Z level=INFO msg="Upload complete. (250ms)"7392worker1 # [ 187.678749] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=17393worker2 # [ 187.682334] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:18Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17394worker1 # [ 187.685726] nix-daemon[853]: reaped child process 2184, status = succeeded7395worker1 # [ 187.725585] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047396worker1 # [ 187.735973] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047397worker1 # [ 187.747446] nix-grpc-daemon[1916]: time=2026-09-22T10:58:19.864Z level=INFO msg="All 2 paths already cached"7398worker1 # [ 187.753566] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604047399worker1 # [ 187.763133] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7400worker1 # [ 187.923758] nix-daemon[853]: accepted connection from pid 2209, user nix-grpc-daemon (trusted)7401worker1 # [ 187.935884] nix-daemon[853]: reaped child process 2212, status = succeeded7402worker1 # [ 188.098827] nix-daemon[853]: accepted connection from pid 2214, user nix-grpc-daemon (trusted)7403worker1 # [ 188.110805] nix-daemon[853]: reaped child process 2217, status = succeeded7404lb # [ 188.124546] niks3-server[1201]: 2026/09/22 10:58:20 INFO Received uploads request method=POST path=/api/pending_closures7405worker1 # [ 188.125212] nix-grpc-daemon[1916]: time=2026-09-22T10:58:20.242Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7406worker1 # [ 188.126656] nix-grpc-daemon[1916]: time=2026-09-22T10:58:20.244Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7407lb # [ 188.141303] niks3-server[1201]: 2026/09/22 10:58:20 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7408lb # [ 188.144359] niks3-server[1201]: 2026/09/22 10:58:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7409lb # [ 188.146073] niks3-server[1201]: 2026/09/22 10:58:20 INFO Signed narinfos id=72 count=17410worker1 # [ 188.142224] nix-grpc-daemon[1916]: time=2026-09-22T10:58:20.259Z level=INFO msg="Uploading 1 narinfos"7411lb # [ 188.150559] niks3-server[1201]: 2026/09/22 10:58:20 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7412lb # [ 188.155552] niks3-server[1201]: 2026/09/22 10:58:20 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7413lb # [ 188.158742] niks3-server[1201]: 2026/09/22 10:58:20 INFO Completed upload id=727414worker1 # [ 188.156021] nix-grpc-daemon[1916]: time=2026-09-22T10:58:20.273Z level=INFO msg="Upload complete. (383ms)"7415lb # [ 188.160453] niks3-server[1201]: 2026/09/22 10:58:20 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7416worker1 # [ 188.157724] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=0 paths=1 nar_bytes_in=7527417worker1 # [ 188.161153] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604047418worker1 # [ 188.165473] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7419worker1 # [ 188.169641] nix-daemon[853]: reaped child process 2207, status = succeeded7420worker1 # [ 188.174559] nix-daemon[853]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7421worker1 # [ 189.202361] nix-daemon[853]: reaped child process 2221, status = succeeded7422worker1 # [ 189.353897] nix-daemon[853]: accepted connection from pid 2237, user nix-grpc-daemon (trusted)7423worker1 # [ 189.366188] nix-daemon[853]: reaped child process 2240, status = succeeded7424lb # [ 189.385975] niks3-server[1201]: 2026/09/22 10:58:21 INFO Received uploads request method=POST path=/api/pending_closures7425worker1 # [ 189.391189] nix-grpc-daemon[1916]: time=2026-09-22T10:58:21.508Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7426worker1 # [ 189.392753] nix-grpc-daemon[1916]: time=2026-09-22T10:58:21.510Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7427lb # [ 189.422105] niks3-server[1201]: 2026/09/22 10:58:21 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7428lb # [ 189.435133] niks3-server[1201]: 2026/09/22 10:58:21 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7429lb # [ 189.444291] niks3-server[1201]: 2026/09/22 10:58:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7430lb # [ 189.449653] niks3-server[1201]: 2026/09/22 10:58:21 INFO Signed narinfos id=73 count=17431worker1 # [ 189.448554] nix-grpc-daemon[1916]: time=2026-09-22T10:58:21.565Z level=INFO msg="Uploading 1 narinfos"7432lb # [ 189.455339] niks3-server[1201]: 2026/09/22 10:58:21 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7433lb # [ 189.464682] niks3-server[1201]: 2026/09/22 10:58:21 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7434lb # [ 189.470235] niks3-server[1201]: 2026/09/22 10:58:21 INFO Completed upload id=737435worker1 # [ 189.466441] nix-grpc-daemon[1916]: time=2026-09-22T10:58:21.583Z level=INFO msg="Upload complete. (284ms)"7436lb # [ 189.471877] niks3-server[1201]: 2026/09/22 10:58:21 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7437worker1 # [ 189.472330] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60404 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=17438worker1 # [ 189.484573] nix-daemon[853]: reaped child process 2219, status = succeeded7439client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.99 seconds)7440worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7441worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7442client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'7443client # [ 189.655765] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7444client # Running as unit: gpu.service; invocation ID: 74001008cd4843e588ba89f2fc40b7867445client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.04 seconds)7446lb: must succeed: curl -sf localhost:9901/clusters7447lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7448lb: must succeed: curl -sf localhost:9901/clusters7449lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7450worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7451worker2 # [ 189.830491] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480747452client # [ 189.847948] nix[1466]: don't know how to build these paths:7453client # [ 189.849365] nix[1466]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7454worker1 # [ 189.838343] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604047455worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7456worker2 # [ 189.856500] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:21Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07457client # [ 189.869877] nix[1466]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker7458lb: must succeed: curl -sf localhost:9901/clusters7459lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7460lb: must succeed: curl -sf localhost:9901/clusters7461lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7462worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7463worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7464client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7465lb # [ 196.052118] envoy[910]: conn peer=[2001:db8:1::1]:45168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27128 tx=11392 ms=187787466client # [ 219.871908] nix[1466]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}7467client # [ 219.877949] nix[1466]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7468client # [ 219.901651] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7469client # [ 219.903077] systemd[1]: gpu.service: Failed with result 'exit-code'.7470client # [ 219.904943] systemd[1]: gpu.service: Consumed 134ms CPU time over 30.248s wall clock time, 37.4M memory peak, 5K incoming IP traffic, 5K outgoing IP traffic.7471client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.57 seconds)7472client: must succeed: journalctl -u gpu -o cat7473client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7474lb: must succeed: curl -sf localhost:9901/clusters7475lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7476lb: must succeed: curl -sf localhost:9901/clusters7477lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7478worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7479worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7480(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 50.25 seconds)7481subtest: low disk drains a worker and builds go to the other7482worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7483worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.97 seconds)7484worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7485worker1 # [ 222.319822] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:54Z level=info event=unhealthy reason=min_free7486worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)7487client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&27488worker1 # [ 223.106155] nix-grpc-daemon[1856]: ts=2026-09-22T10:58:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:604047489client # don't know how to build these paths:7490client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7491worker2 # [ 223.117957] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480747492worker2 # [ 223.138750] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:54Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07493worker2 # [ 223.143188] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:54Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17494worker2 # [ 223.183226] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480747495worker2 # [ 223.196221] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480747496client # copying 1 paths to 'lb:50051'...7497client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7498worker2 # [ 223.208561] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480747499worker2 # [ 223.221650] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7500worker2 # [ 223.412541] nix-daemon[873]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)7501worker2 # [ 223.427171] nix-daemon[873]: reaped child process 1826, status = succeeded7502worker2 # [ 223.620856] nix-daemon[873]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)7503worker2 # [ 223.634947] nix-daemon[873]: reaped child process 1831, status = succeeded7504lb # [ 223.659308] niks3-server[1201]: 2026/09/22 10:58:55 INFO Received uploads request method=POST path=/api/pending_closures7505worker2 # [ 223.664197] nix-grpc-daemon[1612]: time=2026-09-22T10:58:54.963Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7506worker2 # [ 223.666213] nix-grpc-daemon[1612]: time=2026-09-22T10:58:54.965Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7507lb # [ 223.688718] niks3-server[1201]: 2026/09/22 10:58:55 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7508lb # [ 223.698531] niks3-server[1201]: 2026/09/22 10:58:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7509lb # [ 223.705114] niks3-server[1201]: 2026/09/22 10:58:55 INFO Signed narinfos id=74 count=17510worker2 # [ 223.704543] nix-grpc-daemon[1612]: time=2026-09-22T10:58:55.003Z level=INFO msg="Uploading 1 narinfos"7511lb # [ 223.712347] niks3-server[1201]: 2026/09/22 10:58:55 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7512lb # [ 223.723609] niks3-server[1201]: 2026/09/22 10:58:55 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7513lb # [ 223.730205] niks3-server[1201]: 2026/09/22 10:58:55 INFO Completed upload id=747514lb # [ 223.732937] niks3-server[1201]: 2026/09/22 10:58:55 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7515worker2 # [ 223.728691] nix-grpc-daemon[1612]: time=2026-09-22T10:58:55.028Z level=INFO msg="Upload complete. (494ms)"7516worker2 # [ 223.732912] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=0 paths=1 nar_bytes_in=5127517worker2 # [ 223.738302] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480747518worker2 # [ 223.751039] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7519worker2 # [ 223.755184] nix-daemon[873]: reaped child process 1821, status = succeeded7520client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7521client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7522worker2 # [ 223.760501] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7523client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7524client # farm-a> LOG-farm-a-drain7525worker2 # [ 224.790519] nix-daemon[873]: reaped child process 1835, status = succeeded7526worker2 # [ 224.968198] nix-daemon[873]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)7527worker2 # [ 224.983469] nix-daemon[873]: reaped child process 1854, status = succeeded7528lb # [ 225.004847] niks3-server[1201]: 2026/09/22 10:58:57 INFO Received uploads request method=POST path=/api/pending_closures7529worker2 # [ 225.010037] nix-grpc-daemon[1612]: time=2026-09-22T10:58:56.309Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7530worker2 # [ 225.011668] nix-grpc-daemon[1612]: time=2026-09-22T10:58:56.311Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7531lb # [ 225.053614] niks3-server[1201]: 2026/09/22 10:58:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7532worker2 # [ 225.055440] nix-grpc-daemon[1612]: time=2026-09-22T10:58:56.354Z level=INFO msg="Uploading 1 narinfos"7533lb # [ 225.059275] niks3-server[1201]: 2026/09/22 10:58:57 INFO Signed narinfos id=75 count=17534lb # [ 225.071537] niks3-server[1201]: 2026/09/22 10:58:57 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7535lb # [ 225.129256] niks3-server[1201]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7536lb # [ 225.142576] niks3-server[1201]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7537lb # [ 225.152316] niks3-server[1201]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7538lb # [ 225.159113] niks3-server[1201]: 2026/09/22 10:58:57 INFO Completed upload id=757539worker2 # [ 225.155550] nix-grpc-daemon[1612]: time=2026-09-22T10:58:56.454Z level=INFO msg="Upload complete. (387ms)"7540lb # [ 225.162127] niks3-server[1201]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7541worker2 # [ 225.162846] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17542worker2 # [ 225.178995] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:56Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07543worker2 # [ 225.183905] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:56Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17544worker2 # [ 225.192970] nix-daemon[873]: reaped child process 1833, status = succeeded7545worker2 # [ 225.223548] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480747546worker2 # [ 225.236602] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480747547worker2 # [ 225.250193] nix-grpc-daemon[1612]: time=2026-09-22T10:58:56.549Z level=INFO msg="All 1 paths already cached"7548client # copying 1 paths to 'lb:50051'...7549client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7550worker2 # [ 225.258320] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480747551worker2 # [ 225.266734] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7552worker2 # [ 225.450686] nix-daemon[873]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)7553worker2 # [ 225.468773] nix-daemon[873]: reaped child process 1863, status = succeeded7554worker2 # [ 225.647840] nix-daemon[873]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)7555worker2 # [ 225.662593] nix-daemon[873]: reaped child process 1868, status = succeeded7556lb # [ 225.683074] niks3-server[1201]: 2026/09/22 10:58:57 INFO Received uploads request method=POST path=/api/pending_closures7557worker2 # [ 225.706430] nix-grpc-daemon[1612]: time=2026-09-22T10:58:57.005Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7558worker2 # [ 225.707843] nix-grpc-daemon[1612]: time=2026-09-22T10:58:57.005Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7559lb # [ 225.726582] niks3-server[1201]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7560lb # [ 225.737905] niks3-server[1201]: 2026/09/22 10:58:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7561lb # [ 225.743367] niks3-server[1201]: 2026/09/22 10:58:57 INFO Signed narinfos id=76 count=17562worker2 # [ 225.742842] nix-grpc-daemon[1612]: time=2026-09-22T10:58:57.042Z level=INFO msg="Uploading 1 narinfos"7563lb # [ 225.749919] niks3-server[1201]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7564lb # [ 225.762831] niks3-server[1201]: 2026/09/22 10:58:57 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7565worker2 # [ 225.763382] nix-grpc-daemon[1612]: time=2026-09-22T10:58:57.062Z level=INFO msg="Upload complete. (482ms)"7566lb # [ 225.767283] niks3-server[1201]: 2026/09/22 10:58:57 INFO Completed upload id=767567worker2 # [ 225.765594] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=0 paths=1 nar_bytes_in=6247568lb # [ 225.770940] niks3-server[1201]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7569worker2 # [ 225.771292] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480747570worker2 # [ 225.779292] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7571worker2 # [ 225.780647] nix-daemon[873]: reaped child process 1858, status = succeeded7572worker2 # [ 225.786442] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7573client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7574client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7575lb # [ 226.053325] envoy[910]: conn peer=[2001:db8:1::1]:33774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300467576client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7577client # farm-b> LOG-farm-b-drain7578worker2 # [ 226.800488] nix-daemon[873]: reaped child process 1872, status = succeeded7579worker2 # [ 226.951090] nix-daemon[873]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7580worker2 # [ 226.962103] nix-daemon[873]: reaped child process 1891, status = succeeded7581lb # [ 226.981535] niks3-server[1201]: 2026/09/22 10:58:58 INFO Received uploads request method=POST path=/api/pending_closures7582worker2 # [ 226.986705] nix-grpc-daemon[1612]: time=2026-09-22T10:58:58.285Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7583worker2 # [ 226.988919] nix-grpc-daemon[1612]: time=2026-09-22T10:58:58.287Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7584lb # [ 227.021574] niks3-server[1201]: 2026/09/22 10:58:59 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7585lb # [ 227.034726] niks3-server[1201]: 2026/09/22 10:58:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7586lb # [ 227.039609] niks3-server[1201]: 2026/09/22 10:58:59 INFO Signed narinfos id=77 count=17587worker2 # [ 227.038967] nix-grpc-daemon[1612]: time=2026-09-22T10:58:58.338Z level=INFO msg="Uploading 1 narinfos"7588lb # [ 227.047535] niks3-server[1201]: 2026/09/22 10:58:59 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7589lb # [ 227.053702] niks3-server[1201]: 2026/09/22 10:58:59 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7590lb # [ 227.063319] niks3-server[1201]: 2026/09/22 10:58:59 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7591lb # [ 227.068911] niks3-server[1201]: 2026/09/22 10:58:59 INFO Completed upload id=777592worker2 # [ 227.065331] nix-grpc-daemon[1612]: time=2026-09-22T10:58:58.364Z level=INFO msg="Upload complete. (272ms)"7593lb # [ 227.070530] niks3-server[1201]: 2026/09/22 10:58:59 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7594worker2 # [ 227.067913] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17595worker2 # [ 227.075149] nix-daemon[873]: reaped child process 1870, status = succeeded7596worker2 # [ 227.115989] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:58Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07597worker2 # [ 227.118069] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:58Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17598worker2 # [ 227.159850] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480747599worker2 # [ 227.168925] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480747600worker2 # [ 227.178205] nix-grpc-daemon[1612]: time=2026-09-22T10:58:58.477Z level=INFO msg="All 2 paths already cached"7601client # copying 1 paths to 'lb:50051'...7602client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7603worker2 # [ 227.184287] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480747604worker2 # [ 227.190624] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7605worker2 # [ 227.351381] nix-daemon[873]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7606worker2 # [ 227.364326] nix-daemon[873]: reaped child process 1900, status = succeeded7607worker2 # [ 227.515113] nix-daemon[873]: accepted connection from pid 1902, user nix-grpc-daemon (trusted)7608worker2 # [ 227.527662] nix-daemon[873]: reaped child process 1905, status = succeeded7609lb # [ 227.547191] niks3-server[1201]: 2026/09/22 10:58:59 INFO Received uploads request method=POST path=/api/pending_closures7610worker2 # [ 227.650500] nix-grpc-daemon[1612]: time=2026-09-22T10:58:58.949Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7611worker2 # [ 227.655463] nix-grpc-daemon[1612]: time=2026-09-22T10:58:58.953Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7612lb # [ 227.683816] niks3-server[1201]: 2026/09/22 10:58:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7613worker2 # [ 227.687423] nix-grpc-daemon[1612]: time=2026-09-22T10:58:58.986Z level=INFO msg="Uploading 1 narinfos"7614lb # [ 227.691991] niks3-server[1201]: 2026/09/22 10:58:59 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7615lb # [ 227.697116] niks3-server[1201]: 2026/09/22 10:58:59 INFO Signed narinfos id=78 count=17616lb # [ 227.713505] niks3-server[1201]: 2026/09/22 10:58:59 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7617lb # [ 227.719207] niks3-server[1201]: 2026/09/22 10:58:59 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7618worker2 # [ 227.723746] nix-grpc-daemon[1612]: time=2026-09-22T10:58:59.023Z level=INFO msg="Upload complete. (527ms)"7619lb # [ 227.727622] niks3-server[1201]: 2026/09/22 10:58:59 INFO Completed upload id=787620worker2 # [ 227.726224] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=0 paths=1 nar_bytes_in=7527621lb # [ 227.730826] niks3-server[1201]: 2026/09/22 10:58:59 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7622worker2 # [ 227.730133] nix-grpc-daemon[1464]: ts=2026-09-22T10:58:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480747623worker2 # [ 227.735400] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7624worker2 # [ 227.742132] nix-daemon[873]: reaped child process 1895, status = succeeded7625worker2 # [ 227.746831] nix-daemon[873]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)7626client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7627client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7628client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7629client # farm-top> LOG-farm-top-drain7630worker2 # [ 229.361965] nix-daemon[873]: reaped child process 1909, status = succeeded7631worker2 # [ 229.502725] nix-daemon[873]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)7632worker2 # [ 229.515499] nix-daemon[873]: reaped child process 1928, status = succeeded7633lb # [ 229.536190] niks3-server[1201]: 2026/09/22 10:59:01 INFO Received uploads request method=POST path=/api/pending_closures7634worker2 # [ 229.540487] nix-grpc-daemon[1612]: time=2026-09-22T10:59:00.839Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7635worker2 # [ 229.542216] nix-grpc-daemon[1612]: time=2026-09-22T10:59:00.841Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7636lb # [ 229.572533] niks3-server[1201]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7637lb # [ 229.586189] niks3-server[1201]: 2026/09/22 10:59:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7638lb # [ 229.589805] niks3-server[1201]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7639lb # [ 229.596408] niks3-server[1201]: 2026/09/22 10:59:01 INFO Signed narinfos id=79 count=17640worker2 # [ 229.596024] nix-grpc-daemon[1612]: time=2026-09-22T10:59:00.895Z level=INFO msg="Uploading 1 narinfos"7641lb # [ 229.601313] niks3-server[1201]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7642lb # [ 229.611748] niks3-server[1201]: 2026/09/22 10:59:01 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7643lb # [ 229.617808] niks3-server[1201]: 2026/09/22 10:59:01 INFO Completed upload id=797644worker2 # [ 229.614386] nix-grpc-daemon[1612]: time=2026-09-22T10:59:00.913Z level=INFO msg="Upload complete. (259ms)"7645lb # [ 229.620122] niks3-server[1201]: 2026/09/22 10:59:01 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7646worker2 # [ 229.621595] nix-grpc-daemon[1464]: ts=2026-09-22T10:59:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48074 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17647worker2 # [ 229.634382] nix-daemon[873]: reaped child process 1907, status = succeeded7648client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.78 seconds)7649client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7650client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.19 seconds)7651worker1: must succeed: rm /nix/.rw-store/fill7652worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.10 seconds)7653worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7654worker1 # [ 230.325555] nix-grpc-daemon[1856]: ts=2026-09-22T10:59:02Z level=info event=healthy reason=min_free7655worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)7656(finished: subtest: low disk drains a worker and builds go to the other, in 10.33 seconds)7657(finished: run the VM test script, in 232.00 seconds)7658test script finished in 232.22s7659cleanup7660kill QemuMachine (pid 49)7661client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7662kill QemuMachine (pid 50)7663lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7664kill QemuMachine (pid 51)7665worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7666kill QemuMachine (pid 52)7667worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7668(finished: cleanup, in 1.00 seconds)7669additionally exposed symbols:7670 client, lb, worker1, worker2,7671 vlan1,7672 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh