vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #467
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb: QEMU running (pid 50)14lb # Formatting '/build/vm-state-lb/tmp.U9tplQGCxY', fmt=raw size=107374182415client: QEMU running (pid 49)16lb # mke2fs 1.47.4 (6-Mar-2025)17client # Disk image does not exist, creating the virtualisation disk image...18lb # Discarding device blocks: 0/262144 done19worker2 # Disk image does not exist, creating the virtualisation disk image...20worker2: QEMU running (pid 51)21client # Formatting '/build/vm-state-client/tmp.vXykoLUxy6', fmt=raw size=107374182422lb # Creating filesystem with 262144 4k blocks and 65536 inodes23worker1: QEMU running (pid 52)24client # mke2fs 1.47.4 (6-Mar-2025)25lb # Filesystem UUID: f6b9a47c-f67a-4f3d-ba41-9f48e78e7e7726worker1 # Disk image does not exist, creating the virtualisation disk image...27worker2 # Formatting '/build/vm-state-worker2/tmp.9BA6kXrrT8', fmt=raw size=107374182428worker2 # mke2fs 1.47.4 (6-Mar-2025)29worker1 # Formatting '/build/vm-state-worker1/tmp.w8B03xYDYe', fmt=raw size=107374182430client # Discarding device blocks: 0/262144 done31(finished: start all VMs, in 0.20 seconds)32worker1 # mke2fs 1.47.4 (6-Mar-2025)33client # Creating filesystem with 262144 4k blocks and 65536 inodes34client: waiting for unit network-addresses-eth1.service35worker1 # Discarding device blocks: 0/262144 done36client # Filesystem UUID: eebd9fb9-44a2-4705-aa00-464edf86bfca37client # Superblock backups stored on blocks:38client # 32768, 98304, 163840, 22937639worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes40lb # Superblock backups stored on blocks:41worker2 # Discarding device blocks: 0/262144 done42worker1 # Filesystem UUID: 5022af1c-94fd-490d-9579-78e20859353343lb # 32768, 98304, 163840, 22937644worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes45worker1 # Superblock backups stored on blocks:46lb # 47worker2 # Filesystem UUID: 428465b8-190a-4615-8c65-7ba75fe7623c48worker1 # 32768, 98304, 163840, 22937649lb # Allocating group tables: 0/8 done50worker2 # Superblock backups stored on blocks:51worker1 # 52lb # Writing inode tables: 0/8 done53worker2 # 32768, 98304, 163840, 22937654worker1 # Allocating group tables: 0/8 done55lb # Creating journal (8192 blocks): done56worker2 # 57worker1 # Writing inode tables: 0/8 done58lb # Writing superblocks and filesystem accounting information: 0/8 done59worker2 # Allocating group tables: 0/8 done60worker1 # Creating journal (8192 blocks): done61lb # 62worker2 # Writing inode tables: 0/8 done63worker1 # Writing superblocks and filesystem accounting information: 0/8 done64lb # Virtualisation disk image created.65worker2 # Creating journal (8192 blocks): done66worker1 # 67client: waiting for the VM to finish booting68worker1 # Virtualisation disk image created.69client # 70worker2 # Writing superblocks and filesystem accounting information: 0/8 done71client # Allocating group tables: 0/8 done72worker2 # 73client # Writing inode tables: 0/8 done74worker2 # Virtualisation disk image created.75client # Creating journal (8192 blocks): done76client # Writing superblocks and filesystem accounting information: 0/8 done77client # 78client # Virtualisation disk image created.79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85client # 86client # 87lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0088client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089worker1 # 90worker1 # 91lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...92lb # 93lb # 94lb # 95lb # 96worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0097client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...98client # 99client # 100client # 101client # 102lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00103worker2 # 104worker2 # 105client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00106worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00107worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108worker1 # 109worker1 # 110worker1 # 111worker1 # 112worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114lb # 115lb # 116worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...117worker2 # 118worker2 # 119client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...120worker2 # 121client # 122worker2 # 123client # 124worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00125worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126worker1 # 127worker1 # 128worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker2 # 130worker2 # 131client # Booting from ROM...132lb # Booting from ROM...133worker1 # Booting from ROM...134worker2 # Booting from ROM...135client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test/init regInfo=/nix/store/ks8jzbm8bw5ap0px5d2kap4yab6b9lzn-closure-info/registration console=ttyS0,115200n8 console=tty0137client # [ 0.000000] BIOS-provided physical RAM map:138client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026144client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved145client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved146client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved147client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved148client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved149lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/dlr5gkwdzlpfiks0wm9k3cgbj9l1v40b-nixos-system-lb-test/init regInfo=/nix/store/493p756s0f7jvdvcriannaqb5380jad0-closure-info/registration console=ttyS0,115200n8 console=tty0150worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026151client # [ 0.000000] NX (Execute Disable) protection: active152lb # [ 0.000000] BIOS-provided physical RAM map:153client # [ 0.000000] APIC: Static calls initialized154lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable155client # [ 0.000000] SMBIOS 2.8 present.156lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved157client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014158worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026159client # [ 0.000000] DMI: Memory slots populated: 1/1160lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved161client # [ 0.000000] Hypervisor detected: KVM162lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable163worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w9dsmd7b0p8lwp54xcjbhm347gjly615-nixos-system-worker1-test/init regInfo=/nix/store/5svmk049wla6pwld28ygm1imv3naf003-closure-info/registration console=ttyS0,115200n8 console=tty0164client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000165worker1 # [ 0.000000] BIOS-provided physical RAM map:166lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved167client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00168worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable169client # [ 0.000000] kvm-clock: using sched offset of 454141850 cycles170lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved171worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved172lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved173worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7l0ngyns87jjp47rj5c82dw00qi8j8ik-nixos-system-worker2-test/init regInfo=/nix/store/jzyih44z3chf3qsinymzl8lnbzlhqjb8-closure-info/registration console=ttyS0,115200n8 console=tty0174client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns175worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved176worker2 # [ 0.000000] BIOS-provided physical RAM map:177lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved178client # [ 0.000005] tsc: Detected 2400.012 MHz processor179worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable180worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable181client # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000182lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved183worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved184worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved185lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved186client # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs187worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved188lb # [ 0.000000] NX (Execute Disable) protection: active189worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved190client # [ 0.000833] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT191lb # [ 0.000000] APIC: Static calls initialized192lb # [ 0.000000] SMBIOS 2.8 present.193worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved194worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable195client # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]196worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved197client # [ 0.002740] Using GB pages for direct mapping198worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved199lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014200client # [ 0.002844] RAMDISK: [mem 0x5e366000-0x5ffcffff]201worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved202lb # [ 0.000000] DMI: Memory slots populated: 1/1203worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved204client # [ 0.002852] ACPI: Early table checksum verification disabled205lb # [ 0.000000] Hypervisor detected: KVM206worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved207worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved208client # [ 0.002855] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )209lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000210worker1 # [ 0.000000] NX (Execute Disable) protection: active211worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved212worker1 # [ 0.000000] APIC: Static calls initialized213lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00214client # [ 0.002859] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)215worker1 # [ 0.000000] SMBIOS 2.8 present.216worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved217lb # [ 0.000001] kvm-clock: using sched offset of 484185859 cycles218client # [ 0.002862] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)219worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved220worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014221lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns222worker1 # [ 0.000000] DMI: Memory slots populated: 1/1223worker2 # [ 0.000000] NX (Execute Disable) protection: active224client # [ 0.002869] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)225lb # [ 0.000005] tsc: Detected 2400.012 MHz processor226worker1 # [ 0.000000] Hypervisor detected: KVM227worker2 # [ 0.000000] APIC: Static calls initialized228client # [ 0.002871] ACPI: FACS 0x000000005FFE0000 000040229worker2 # [ 0.000000] SMBIOS 2.8 present.230lb # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000231worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000232worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00233client # [ 0.002872] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)234lb # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs235worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014236worker1 # [ 0.000001] kvm-clock: using sched offset of 460352517 cycles237worker2 # [ 0.000000] DMI: Memory slots populated: 1/1238client # [ 0.002874] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)239lb # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT240worker2 # [ 0.000000] Hypervisor detected: KVM241lb # [ 0.002742] found SMP MP-table at [mem 0x000f5450-0x000f545f]242worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns243worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000244client # [ 0.002876] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)245lb # [ 0.002754] Using GB pages for direct mapping246worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor247worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00248lb # [ 0.002910] RAMDISK: [mem 0x5e367000-0x5ffcffff]249client # [ 0.002877] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)250worker1 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000251worker2 # [ 0.000000] kvm-clock: using sched offset of 462799495 cycles252lb # [ 0.002919] ACPI: Early table checksum verification disabled253client # [ 0.002878] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]254worker1 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs255lb # [ 0.002922] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )256worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns257client # [ 0.002879] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]258worker1 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT259worker2 # [ 0.000004] tsc: Detected 2400.012 MHz processor260lb # [ 0.002926] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)261client # [ 0.002880] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]262worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000263worker1 # [ 0.002735] found SMP MP-table at [mem 0x000f5450-0x000f545f]264worker1 # [ 0.002748] Using GB pages for direct mapping265lb # [ 0.002930] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)266client # [ 0.002880] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]267worker1 # [ 0.002820] RAMDISK: [mem 0x5e367000-0x5ffcffff]268worker2 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs269client # [ 0.002881] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]270lb # [ 0.002938] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)271worker1 # [ 0.002830] ACPI: Early table checksum verification disabled272worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT273lb # [ 0.002940] ACPI: FACS 0x000000005FFE0000 000040274worker1 # [ 0.002833] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )275client # [ 0.002881] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]276worker2 # [ 0.002745] found SMP MP-table at [mem 0x000f5450-0x000f545f]277worker2 # [ 0.002756] Using GB pages for direct mapping278client # [ 0.002882] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]279lb # [ 0.002941] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)280worker1 # [ 0.002838] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)281client # [ 0.003115] No NUMA configuration found282worker2 # [ 0.002889] RAMDISK: [mem 0x5e367000-0x5ffcffff]283lb # [ 0.002943] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)284worker2 # [ 0.002899] ACPI: Early table checksum verification disabled285worker1 # [ 0.002842] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)286client # [ 0.003116] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]287worker2 # [ 0.002902] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )288client # [ 0.003118] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]289lb # [ 0.002944] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)290worker1 # [ 0.002849] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)291client # [ 0.005728] Zone ranges:292worker1 # [ 0.002851] ACPI: FACS 0x000000005FFE0000 000040293worker2 # [ 0.002906] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)294client # [ 0.005729] DMA [mem 0x0000000000001000-0x0000000000ffffff]295lb # [ 0.002946] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)296client # [ 0.005730] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]297worker1 # [ 0.002852] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)298lb # [ 0.002947] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]299client # [ 0.005731] Normal empty300worker2 # [ 0.002910] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)301client # [ 0.005732] Device empty302lb # [ 0.002948] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]303worker1 # [ 0.002854] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)304worker2 # [ 0.002917] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)305client # [ 0.005733] Movable zone start for each node306client # [ 0.005733] Early memory node ranges307worker2 # [ 0.002919] ACPI: FACS 0x000000005FFE0000 000040308lb # [ 0.002949] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]309worker1 # [ 0.002855] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)310client # [ 0.005734] node 0: [mem 0x0000000000001000-0x000000000009efff]311worker2 # [ 0.002921] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)312lb # [ 0.002949] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]313worker1 # [ 0.002857] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)314client # [ 0.005735] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]315lb # [ 0.002950] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]316worker2 # [ 0.002922] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)317client # [ 0.005736] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]318worker1 # [ 0.002858] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]319lb # [ 0.002950] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]320client # [ 0.005756] On node 0, zone DMA: 1 pages in unavailable ranges321worker2 # [ 0.002924] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)322worker1 # [ 0.002859] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]323lb # [ 0.002951] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]324client # [ 0.006027] On node 0, zone DMA: 97 pages in unavailable ranges325worker1 # [ 0.002860] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]326lb # [ 0.003188] No NUMA configuration found327worker2 # [ 0.002926] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)328client # [ 0.033263] On node 0, zone DMA32: 40 pages in unavailable ranges329worker1 # [ 0.002860] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]330lb # [ 0.003189] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]331worker2 # [ 0.002927] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]332client # [ 0.033711] ACPI: PM-Timer IO Port: 0x608333lb # [ 0.003191] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]334client # [ 0.033721] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])335worker2 # [ 0.002928] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]336lb # [ 0.005629] Zone ranges:337worker1 # [ 0.002861] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]338client # [ 0.033746] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23339worker1 # [ 0.002861] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]340worker2 # [ 0.002928] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]341lb # [ 0.005630] DMA [mem 0x0000000000001000-0x0000000000ffffff]342client # [ 0.033748] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)343worker1 # [ 0.002862] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]344lb # [ 0.005631] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]345worker2 # [ 0.002929] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]346lb # [ 0.005633] Normal empty347worker1 # [ 0.003094] No NUMA configuration found348lb # [ 0.005634] Device empty349client # [ 0.033750] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)350worker2 # [ 0.002929] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]351lb # [ 0.005634] Movable zone start for each node352worker1 # [ 0.003095] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]353client # [ 0.033751] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)354lb # [ 0.005635] Early memory node ranges355worker1 # [ 0.003098] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]356worker2 # [ 0.002930] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]357client # [ 0.033752] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)358worker1 # [ 0.005911] Zone ranges:359lb # [ 0.005635] node 0: [mem 0x0000000000001000-0x000000000009efff]360worker2 # [ 0.002930] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]361worker1 # [ 0.005912] DMA [mem 0x0000000000001000-0x0000000000ffffff]362client # [ 0.033753] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)363lb # [ 0.005636] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]364worker2 # [ 0.003179] No NUMA configuration found365worker1 # [ 0.005914] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]366client # [ 0.033755] ACPI: Using ACPI (MADT) for SMP configuration information367worker2 # [ 0.003180] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]368lb # [ 0.005637] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]369worker1 # [ 0.005915] Normal empty370client # [ 0.033756] ACPI: HPET id: 0x8086a201 base: 0xfed00000371worker1 # [ 0.005916] Device empty372worker2 # [ 0.003182] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]373lb # [ 0.005658] On node 0, zone DMA: 1 pages in unavailable ranges374client # [ 0.033759] TSC deadline timer available375worker2 # [ 0.005897] Zone ranges:376worker1 # [ 0.005916] Movable zone start for each node377worker1 # [ 0.005917] Early memory node ranges378lb # [ 0.005935] On node 0, zone DMA: 97 pages in unavailable ranges379client # [ 0.033764] CPU topo: Max. logical packages: 1380worker2 # [ 0.005898] DMA [mem 0x0000000000001000-0x0000000000ffffff]381client # [ 0.033765] CPU topo: Max. logical dies: 1382worker1 # [ 0.005917] node 0: [mem 0x0000000000001000-0x000000000009efff]383lb # [ 0.034213] On node 0, zone DMA32: 40 pages in unavailable ranges384worker2 # [ 0.005899] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]385client # [ 0.033765] CPU topo: Max. dies per package: 1386lb # [ 0.034684] ACPI: PM-Timer IO Port: 0x608387worker2 # [ 0.005900] Normal empty388worker1 # [ 0.005918] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]389worker2 # [ 0.005901] Device empty390client # [ 0.033769] CPU topo: Max. threads per core: 1391lb # [ 0.034696] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])392worker2 # [ 0.005902] Movable zone start for each node393client # [ 0.033769] CPU topo: Num. cores per package: 1394worker1 # [ 0.005919] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]395lb # [ 0.034725] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23396worker2 # [ 0.005903] Early memory node ranges397client # [ 0.033770] CPU topo: Num. threads per package: 1398worker1 # [ 0.005940] On node 0, zone DMA: 1 pages in unavailable ranges399worker2 # [ 0.005903] node 0: [mem 0x0000000000001000-0x000000000009efff]400lb # [ 0.034727] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)401client # [ 0.033770] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs402worker1 # [ 0.006216] On node 0, zone DMA: 97 pages in unavailable ranges403worker2 # [ 0.005904] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]404lb # [ 0.034728] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)405worker1 # [ 0.034608] On node 0, zone DMA32: 40 pages in unavailable ranges406client # [ 0.033787] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()407worker1 # [ 0.035069] ACPI: PM-Timer IO Port: 0x608408worker2 # [ 0.005905] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]409lb # [ 0.034730] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)410client # [ 0.033818] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]411worker1 # [ 0.035080] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])412worker2 # [ 0.005925] On node 0, zone DMA: 1 pages in unavailable ranges413lb # [ 0.034731] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)414client # [ 0.033819] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]415worker2 # [ 0.006208] On node 0, zone DMA: 97 pages in unavailable ranges416worker1 # [ 0.035110] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23417lb # [ 0.034731] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)418client # [ 0.033821] [mem 0x60000000-0xafffffff] available for PCI devices419worker2 # [ 0.034440] On node 0, zone DMA32: 40 pages in unavailable ranges420worker1 # [ 0.035113] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)421lb # [ 0.034734] ACPI: Using ACPI (MADT) for SMP configuration information422client # [ 0.033822] Booting paravirtualized kernel on KVM423worker2 # [ 0.034898] ACPI: PM-Timer IO Port: 0x608424worker1 # [ 0.035114] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)425lb # [ 0.034735] ACPI: HPET id: 0x8086a201 base: 0xfed00000426worker2 # [ 0.034910] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])427lb # [ 0.034738] TSC deadline timer available428worker1 # [ 0.035115] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)429client # [ 0.033825] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns430worker2 # [ 0.034940] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23431lb # [ 0.034742] CPU topo: Max. logical packages: 1432worker1 # [ 0.035116] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)433client # [ 0.038279] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1434lb # [ 0.034743] CPU topo: Max. logical dies: 1435worker2 # [ 0.034943] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)436lb # [ 0.034743] CPU topo: Max. dies per package: 1437client # [ 0.040510] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152438worker1 # [ 0.035117] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)439worker2 # [ 0.034945] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)440lb # [ 0.034749] CPU topo: Max. threads per core: 1441client # [ 0.040553] kvm-guest: PV spinlocks disabled, single CPU442worker1 # [ 0.035119] ACPI: Using ACPI (MADT) for SMP configuration information443lb # [ 0.034750] CPU topo: Num. cores per package: 1444worker2 # [ 0.034946] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)445worker1 # [ 0.035120] ACPI: HPET id: 0x8086a201 base: 0xfed00000446lb # [ 0.034750] CPU topo: Num. threads per package: 1447worker1 # [ 0.035123] TSC deadline timer available448worker2 # [ 0.034947] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)449lb # [ 0.034750] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs450worker1 # [ 0.035127] CPU topo: Max. logical packages: 1451worker2 # [ 0.034947] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)452worker1 # [ 0.035128] CPU topo: Max. logical dies: 1453lb # [ 0.034770] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()454worker2 # [ 0.034950] ACPI: Using ACPI (MADT) for SMP configuration information455worker1 # [ 0.035129] CPU topo: Max. dies per package: 1456lb # [ 0.034803] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]457worker2 # [ 0.034951] ACPI: HPET id: 0x8086a201 base: 0xfed00000458worker1 # [ 0.035132] CPU topo: Max. threads per core: 1459client # [ 0.040555] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test/init regInfo=/nix/store/ks8jzbm8bw5ap0px5d2kap4yab6b9lzn-closure-info/registration console=ttyS0,115200n8 console=tty0460worker2 # [ 0.034954] TSC deadline timer available461worker1 # [ 0.035133] CPU topo: Num. cores per package: 1462lb # [ 0.034804] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]463worker2 # [ 0.034959] CPU topo: Max. logical packages: 1464worker1 # [ 0.035133] CPU topo: Num. threads per package: 1465worker2 # [ 0.034960] CPU topo: Max. logical dies: 1466lb # [ 0.034805] [mem 0x60000000-0xafffffff] available for PCI devices467worker1 # [ 0.035133] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs468client # [ 0.040645] Unknown kernel command line parameters "regInfo=/nix/store/ks8jzbm8bw5ap0px5d2kap4yab6b9lzn-closure-info/registration", will be passed to user space.469worker2 # [ 0.034960] CPU topo: Max. dies per package: 1470lb # [ 0.034807] Booting paravirtualized kernel on KVM471client # [ 0.040658] random: crng init done472worker2 # [ 0.034964] CPU topo: Max. threads per core: 1473worker1 # [ 0.035153] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()474worker2 # [ 0.034964] CPU topo: Num. cores per package: 1475client # [ 0.040659] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes476lb # [ 0.034810] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns477worker1 # [ 0.035187] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]478worker2 # [ 0.034965] CPU topo: Num. threads per package: 1479client # [ 0.042853] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)480lb # [ 0.039261] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1481worker1 # [ 0.035189] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]482worker2 # [ 0.034965] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs483lb # [ 0.041581] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152484worker1 # [ 0.035190] [mem 0x60000000-0xafffffff] available for PCI devices485client # [ 0.043944] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)486worker2 # [ 0.034986] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()487worker1 # [ 0.035192] Booting paravirtualized kernel on KVM488client # [ 0.043979] Fallback order for Node 0: 0489lb # [ 0.041629] kvm-guest: PV spinlocks disabled, single CPU490worker2 # [ 0.035019] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]491client # [ 0.043982] Built 1 zonelists, mobility grouping on. Total pages: 393078492client # [ 0.043983] Policy zone: DMA32493worker1 # [ 0.035195] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns494worker2 # [ 0.035020] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]495client # [ 0.047133] mem auto-init: stack:all(zero), heap alloc:on, heap free:off496worker2 # [ 0.035022] [mem 0x60000000-0xafffffff] available for PCI devices497worker1 # [ 0.039656] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1498client # [ 0.050233] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1499worker2 # [ 0.035023] Booting paravirtualized kernel on KVM500worker1 # [ 0.041945] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152501client # [ 0.053605] allocated 3145728 bytes of page_ext502worker1 # [ 0.041992] kvm-guest: PV spinlocks disabled, single CPU503lb # [ 0.041631] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/dlr5gkwdzlpfiks0wm9k3cgbj9l1v40b-nixos-system-lb-test/init regInfo=/nix/store/493p756s0f7jvdvcriannaqb5380jad0-closure-info/registration console=ttyS0,115200n8 console=tty0504worker2 # [ 0.035026] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns505client # [ 0.063402] ftrace: allocating 48731 entries in 192 pages506client # [ 0.063404] ftrace: allocated 192 pages with 2 groups507client # [ 0.064257] Dynamic Preempt: lazy508worker2 # [ 0.039495] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1509lb # [ 0.041728] Unknown kernel command line parameters "regInfo=/nix/store/493p756s0f7jvdvcriannaqb5380jad0-closure-info/registration", will be passed to user space.510client # [ 0.064421] rcu: Preemptible hierarchical RCU implementation.511worker2 # [ 0.041824] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152512lb # [ 0.041740] random: crng init done513client # [ 0.064422] rcu: RCU event tracing is enabled.514worker2 # [ 0.041871] kvm-guest: PV spinlocks disabled, single CPU515lb # [ 0.041741] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes516client # [ 0.064422] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.517worker1 # [ 0.041994] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w9dsmd7b0p8lwp54xcjbhm347gjly615-nixos-system-worker1-test/init regInfo=/nix/store/5svmk049wla6pwld28ygm1imv3naf003-closure-info/registration console=ttyS0,115200n8 console=tty0518lb # [ 0.044010] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)519client # [ 0.064424] Trampoline variant of Tasks RCU enabled.520client # [ 0.064424] Rude variant of Tasks RCU enabled.521lb # [ 0.045146] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)522client # [ 0.064425] Tracing variant of Tasks RCU enabled.523lb # [ 0.045178] Fallback order for Node 0: 0524worker1 # [ 0.042091] Unknown kernel command line parameters "regInfo=/nix/store/5svmk049wla6pwld28ygm1imv3naf003-closure-info/registration", will be passed to user space.525worker1 # [ 0.042103] random: crng init done526client # [ 0.064425] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.527lb # [ 0.045181] Built 1 zonelists, mobility grouping on. Total pages: 393078528worker2 # [ 0.041873] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7l0ngyns87jjp47rj5c82dw00qi8j8ik-nixos-system-worker2-test/init regInfo=/nix/store/jzyih44z3chf3qsinymzl8lnbzlhqjb8-closure-info/registration console=ttyS0,115200n8 console=tty0529lb # [ 0.045182] Policy zone: DMA32530worker1 # [ 0.042104] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes531client # [ 0.064426] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1532lb # [ 0.048173] mem auto-init: stack:all(zero), heap alloc:on, heap free:off533worker1 # [ 0.044409] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)534client # [ 0.064441] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.535lb # [ 0.051674] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1536worker2 # [ 0.041971] Unknown kernel command line parameters "regInfo=/nix/store/jzyih44z3chf3qsinymzl8lnbzlhqjb8-closure-info/registration", will be passed to user space.537worker1 # [ 0.045602] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)538worker2 # [ 0.041984] random: crng init done539lb # [ 0.055251] allocated 3145728 bytes of page_ext540client # [ 0.064442] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.541worker1 # [ 0.045636] Fallback order for Node 0: 0542lb # [ 0.065664] ftrace: allocating 48731 entries in 192 pages543worker2 # [ 0.041984] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes544lb # [ 0.065666] ftrace: allocated 192 pages with 2 groups545worker1 # [ 0.045639] Built 1 zonelists, mobility grouping on. Total pages: 393078546client # [ 0.064443] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.547lb # [ 0.066574] Dynamic Preempt: lazy548worker2 # [ 0.044257] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)549worker1 # [ 0.045640] Policy zone: DMA32550client # [ 0.068862] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16551worker1 # [ 0.048773] mem auto-init: stack:all(zero), heap alloc:on, heap free:off552lb # [ 0.066730] rcu: Preemptible hierarchical RCU implementation.553worker2 # [ 0.045388] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)554lb # [ 0.066731] rcu: RCU event tracing is enabled.555client # [ 0.069142] rcu: srcu_init: Setting srcu_struct sizes based on contention.556worker2 # [ 0.045420] Fallback order for Node 0: 0557worker1 # [ 0.052226] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1558lb # [ 0.066732] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.559worker1 # [ 0.055786] allocated 3145728 bytes of page_ext560worker2 # [ 0.045423] Built 1 zonelists, mobility grouping on. Total pages: 393078561client # [ 0.069149] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns562lb # [ 0.066734] Trampoline variant of Tasks RCU enabled.563worker1 # [ 0.066236] ftrace: allocating 48731 entries in 192 pages564worker2 # [ 0.045424] Policy zone: DMA32565lb # [ 0.066734] Rude variant of Tasks RCU enabled.566worker1 # [ 0.066238] ftrace: allocated 192 pages with 2 groups567worker2 # [ 0.048375] mem auto-init: stack:all(zero), heap alloc:on, heap free:off568worker1 # [ 0.067175] Dynamic Preempt: lazy569lb # [ 0.066734] Tracing variant of Tasks RCU enabled.570client # [ 0.069253] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)571client # [ 0.072781] Console: colour VGA+ 80x25572worker2 # [ 0.051807] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1573worker1 # [ 0.067339] rcu: Preemptible hierarchical RCU implementation.574lb # [ 0.066735] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.575client # [ 0.072783] printk: legacy console [tty0] enabled576worker2 # [ 0.055308] allocated 3145728 bytes of page_ext577worker1 # [ 0.067340] rcu: RCU event tracing is enabled.578client # [ 0.102295] printk: legacy console [ttyS0] enabled579worker2 # [ 0.065520] ftrace: allocating 48731 entries in 192 pages580lb # [ 0.066736] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1581worker1 # [ 0.067340] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.582worker2 # [ 0.065522] ftrace: allocated 192 pages with 2 groups583client # [ 0.198820] ACPI: Core revision 20250807584worker1 # [ 0.067342] Trampoline variant of Tasks RCU enabled.585worker2 # [ 0.066389] Dynamic Preempt: lazy586lb # [ 0.066791] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.587worker1 # [ 0.067342] Rude variant of Tasks RCU enabled.588worker2 # [ 0.066564] rcu: Preemptible hierarchical RCU implementation.589worker1 # [ 0.067343] Tracing variant of Tasks RCU enabled.590lb # [ 0.066793] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.591worker2 # [ 0.066565] rcu: RCU event tracing is enabled.592client # [ 0.199705] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns593worker1 # [ 0.067343] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.594worker2 # [ 0.066566] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.595client # [ 0.201203] APIC: Switch to symmetric I/O mode setup596lb # [ 0.066794] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.597worker2 # [ 0.066568] Trampoline variant of Tasks RCU enabled.598worker1 # [ 0.067344] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1599client # [ 0.202154] x2apic enabled600lb # [ 0.071258] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16601worker2 # [ 0.066568] Rude variant of Tasks RCU enabled.602worker2 # [ 0.066568] Tracing variant of Tasks RCU enabled.603worker1 # [ 0.067407] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.604lb # [ 0.071560] rcu: srcu_init: Setting srcu_struct sizes based on contention.605client # [ 0.202878] APIC: Switched APIC routing to: physical x2apic606worker2 # [ 0.066569] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.607worker1 # [ 0.067410] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.608lb # [ 0.071568] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns609worker2 # [ 0.066570] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1610worker1 # [ 0.067410] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.611client # [ 0.204686] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1612lb # [ 0.071679] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)613worker2 # [ 0.066587] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.614lb # [ 0.075183] Console: colour VGA+ 80x25615worker1 # [ 0.071856] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16616lb # [ 0.075186] printk: legacy console [tty0] enabled617worker2 # [ 0.066589] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.618worker1 # [ 0.072145] rcu: srcu_init: Setting srcu_struct sizes based on contention.619client # [ 0.205663] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns620lb # [ 0.104524] printk: legacy console [ttyS0] enabled621lb # [ 0.200010] ACPI: Core revision 20250807622worker2 # [ 0.066590] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623worker1 # [ 0.072154] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns624client # [ 0.207329] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)625worker2 # [ 0.070988] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16626worker1 # [ 0.072266] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)627lb # [ 0.200879] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns628worker2 # [ 0.071275] rcu: srcu_init: Setting srcu_struct sizes based on contention.629client # [ 0.208413] x86/cpu: User Mode Instruction Prevention (UMIP) activated630worker1 # [ 0.075811] Console: colour VGA+ 80x25631worker1 # [ 0.075813] printk: legacy console [tty0] enabled632lb # [ 0.202367] APIC: Switch to symmetric I/O mode setup633worker2 # [ 0.071283] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns634client # [ 0.209493] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127635worker1 # [ 0.105188] printk: legacy console [ttyS0] enabled636lb # [ 0.203348] x2apic enabled637worker1 # [ 0.200284] ACPI: Core revision 20250807638client # [ 0.210327] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0639worker2 # [ 0.071392] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)640lb # [ 0.204130] APIC: Switched APIC routing to: physical x2apic641worker2 # [ 0.074912] Console: colour VGA+ 80x25642worker2 # [ 0.074915] printk: legacy console [tty0] enabled643worker1 # [ 0.201158] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns644client # [ 0.211330] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto645worker2 # [ 0.104502] printk: legacy console [ttyS0] enabled646worker1 # [ 0.202660] APIC: Switch to symmetric I/O mode setup647lb # [ 0.205966] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1648worker2 # [ 0.199797] ACPI: Core revision 20250807649worker1 # [ 0.203620] x2apic enabled650client # [ 0.213327] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl651client # [ 0.214327] Transient Scheduler Attacks: Vulnerable: No microcode652worker1 # [ 0.204404] APIC: Switched APIC routing to: physical x2apic653worker2 # [ 0.200738] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns654lb # [ 0.206942] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns655client # [ 0.215996] Spectre V2 : Mitigation: Enhanced / Automatic IBRS656worker2 # [ 0.202232] APIC: Switch to symmetric I/O mode setup657lb # [ 0.208605] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)658worker2 # [ 0.203209] x2apic enabled659client # [ 0.216327] Speculative Return Stack Overflow: Mitigation: Safe RET660worker1 # [ 0.206186] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1661lb # [ 0.209690] x86/cpu: User Mode Instruction Prevention (UMIP) activated662worker2 # [ 0.203977] APIC: Switched APIC routing to: physical x2apic663worker1 # [ 0.207149] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns664lb # [ 0.211665] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127665worker2 # [ 0.205784] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1666worker1 # [ 0.208824] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)667lb # [ 0.212602] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0668client # [ 0.217327] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization669worker1 # [ 0.209910] x86/cpu: User Mode Instruction Prevention (UMIP) activated670worker2 # [ 0.206782] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns671client # [ 0.219334] Spectre V2 : Enabling IBPB for BPF672lb # [ 0.213607] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto673worker1 # [ 0.211013] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127674worker2 # [ 0.208474] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)675lb # [ 0.214603] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl676client # [ 0.220328] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier677worker1 # [ 0.212497] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0678worker2 # [ 0.209561] x86/cpu: User Mode Instruction Prevention (UMIP) activated679client # [ 0.221327] active return thunk: srso_alias_return_thunk680lb # [ 0.216602] Transient Scheduler Attacks: Vulnerable: No microcode681worker1 # [ 0.212826] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto682lb # [ 0.217602] Spectre V2 : Mitigation: Enhanced / Automatic IBRS683worker2 # [ 0.210639] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127684client # [ 0.222931] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'685lb # [ 0.218602] Speculative Return Stack Overflow: Mitigation: Safe RET686worker2 # [ 0.212120] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0687client # [ 0.224180] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'688worker1 # [ 0.214822] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl689client # [ 0.225026] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'690worker1 # [ 0.215822] Transient Scheduler Attacks: Vulnerable: No microcode691worker2 # [ 0.212475] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto692worker1 # [ 0.216821] Spectre V2 : Mitigation: Enhanced / Automatic IBRS693client # [ 0.226030] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'694worker2 # [ 0.214471] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl695worker1 # [ 0.217822] Speculative Return Stack Overflow: Mitigation: Safe RET696client # [ 0.227068] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'697lb # [ 0.219535] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization698worker2 # [ 0.215471] Transient Scheduler Attacks: Vulnerable: No microcode699client # [ 0.228042] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'700lb # [ 0.220611] Spectre V2 : Enabling IBPB for BPF701worker2 # [ 0.216471] Spectre V2 : Mitigation: Enhanced / Automatic IBRS702client # [ 0.229083] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'703worker2 # [ 0.217471] Speculative Return Stack Overflow: Mitigation: Safe RET704lb # [ 0.222151] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier705client # [ 0.230327] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256706lb # [ 0.222603] active return thunk: srso_alias_return_thunk707worker1 # [ 0.218821] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization708client # [ 0.231326] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64709worker1 # [ 0.220830] Spectre V2 : Enabling IBPB for BPF710lb # [ 0.224479] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'711client # [ 0.232326] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512712worker1 # [ 0.221822] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier713lb # [ 0.225601] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'714client # [ 0.233295] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024715worker2 # [ 0.218471] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization716worker1 # [ 0.222822] active return thunk: srso_alias_return_thunk717lb # [ 0.226602] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'718client # [ 0.234044] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8719worker2 # [ 0.220479] Spectre V2 : Enabling IBPB for BPF720worker1 # [ 0.223844] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'721lb # [ 0.227601] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'722client # [ 0.235043] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.723worker2 # [ 0.221472] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier724lb # [ 0.228602] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'725worker1 # [ 0.224822] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'726worker2 # [ 0.222472] active return thunk: srso_alias_return_thunk727worker1 # [ 0.225821] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'728lb # [ 0.229601] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'729worker2 # [ 0.223493] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'730worker1 # [ 0.226821] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'731lb # [ 0.230602] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'732worker2 # [ 0.224471] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'733worker1 # [ 0.227821] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'734lb # [ 0.231602] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256735worker2 # [ 0.225471] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'736worker1 # [ 0.228821] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'737lb # [ 0.232602] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64738worker2 # [ 0.226471] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'739worker1 # [ 0.229821] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'740lb # [ 0.233602] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512741worker2 # [ 0.227471] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'742worker1 # [ 0.230822] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256743lb # [ 0.234601] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024744worker2 # [ 0.228471] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'745worker1 # [ 0.232536] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64746lb # [ 0.235603] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8747worker1 # [ 0.233544] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512748worker2 # [ 0.229471] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'749lb # [ 0.236602] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.750worker1 # [ 0.234539] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024751worker2 # [ 0.231378] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256752worker1 # [ 0.234822] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8753worker2 # [ 0.232188] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64754worker2 # [ 0.233230] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512755worker1 # [ 0.235821] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.756worker2 # [ 0.234219] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757worker2 # [ 0.235215] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker2 # [ 0.236261] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759client # [ 0.270166] Freeing SMP alternatives memory: 44K760client # [ 0.270906] pid_max: default: 32768 minimum: 301761client # [ 0.271434] LSM: initializing lsm=capability,landlock,yama,bpf,ima762client # [ 0.272444] landlock: Up and running.763client # [ 0.273328] Yama: becoming mindful.764client # [ 0.274540] LSM support for eBPF active765client # [ 0.275393] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766client # [ 0.276365] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767lb # [ 0.271700] Freeing SMP alternatives memory: 44K768lb # [ 0.272491] pid_max: default: 32768 minimum: 301769client # [ 0.279094] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)770lb # [ 0.273271] LSM: initializing lsm=capability,landlock,yama,bpf,ima771client # [ 0.279843] Performance Events: Fam17h+ core perfctr, AMD PMU driver.772lb # [ 0.273707] landlock: Up and running.773client # [ 0.280331] ... version: 2774lb # [ 0.275070] Yama: becoming mindful.775worker1 # [ 0.271686] Freeing SMP alternatives memory: 44K776client # [ 0.281040] ... bit width: 48777worker1 # [ 0.272369] pid_max: default: 32768 minimum: 301778lb # [ 0.275824] LSM support for eBPF active779client # [ 0.281333] ... generic counters: 6780worker1 # [ 0.272953] LSM: initializing lsm=capability,landlock,yama,bpf,ima781lb # [ 0.276715] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)782client # [ 0.282110] ... generic bitmap: 000000000000003f783worker2 # [ 0.270909] Freeing SMP alternatives memory: 44K784worker1 # [ 0.273922] landlock: Up and running.785worker2 # [ 0.271474] pid_max: default: 32768 minimum: 301786client # [ 0.282386] ... fixed-purpose counters: 0787worker1 # [ 0.274822] Yama: becoming mindful.788lb # [ 0.277651] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)789client # [ 0.283068] ... fixed-purpose bitmap: 0000000000000000790worker1 # [ 0.275887] LSM support for eBPF active791worker2 # [ 0.272530] LSM: initializing lsm=capability,landlock,yama,bpf,ima792client # [ 0.283329] ... value mask: 0000ffffffffffff793worker2 # [ 0.273562] landlock: Up and running.794worker1 # [ 0.276631] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)795worker2 # [ 0.274160] Yama: becoming mindful.796client # [ 0.284219] ... max period: 00007fffffffffff797lb # [ 0.280293] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)798worker2 # [ 0.274685] LSM support for eBPF active799client # [ 0.284977] ... global_ctrl mask: 000000000000003f800worker1 # [ 0.277832] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)801client # [ 0.285439] signal: max sigframe size: 3376802lb # [ 0.281137] Performance Events: Fam17h+ core perfctr, AMD PMU driver.803worker2 # [ 0.275604] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)804lb # [ 0.281608] ... version: 2805client # [ 0.286202] rcu: Hierarchical SRCU implementation.806lb # [ 0.282315] ... bit width: 48807worker2 # [ 0.276512] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)808client # [ 0.286934] rcu: Max phase no-delay instances is 400.809worker1 # [ 0.280321] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)810lb # [ 0.282604] ... generic counters: 6811lb # [ 0.283309] ... generic bitmap: 000000000000003f812worker1 # [ 0.281414] Performance Events: Fam17h+ core perfctr, AMD PMU driver.813lb # [ 0.283654] ... fixed-purpose counters: 0814worker2 # [ 0.279407] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)815worker1 # [ 0.281830] ... version: 2816lb # [ 0.284366] ... fixed-purpose bitmap: 0000000000000000817worker1 # [ 0.282521] ... bit width: 48818lb # [ 0.284605] ... value mask: 0000ffffffffffff819worker2 # [ 0.280014] Performance Events: Fam17h+ core perfctr, AMD PMU driver.820worker1 # [ 0.282824] ... generic counters: 6821worker2 # [ 0.280476] ... version: 2822lb # [ 0.285476] ... max period: 00007fffffffffff823worker1 # [ 0.283504] ... generic bitmap: 000000000000003f824client # [ 0.291872] smp: Bringing up secondary CPUs ...825worker2 # [ 0.281165] ... bit width: 48826worker1 # [ 0.283824] ... fixed-purpose counters: 0827lb # [ 0.286241] ... global_ctrl mask: 000000000000003f828client # [ 0.292343] smp: Brought up 1 node, 1 CPU829worker2 # [ 0.281473] ... generic counters: 6830lb # [ 0.286730] signal: max sigframe size: 3376831worker1 # [ 0.284506] ... fixed-purpose bitmap: 0000000000000000832client # [ 0.292983] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)833worker2 # [ 0.282177] ... generic bitmap: 000000000000003f834lb # [ 0.287486] rcu: Hierarchical SRCU implementation.835worker1 # [ 0.284824] ... value mask: 0000ffffffffffff836worker2 # [ 0.282525] ... fixed-purpose counters: 0837lb # [ 0.288164] rcu: Max phase no-delay instances is 400.838worker1 # [ 0.285684] ... max period: 00007fffffffffff839worker2 # [ 0.283221] ... fixed-purpose bitmap: 0000000000000000840worker1 # [ 0.286452] ... global_ctrl mask: 000000000000003f841client # [ 0.293539] Memory: 1454568K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)842worker2 # [ 0.283474] ... value mask: 0000ffffffffffff843worker1 # [ 0.286923] signal: max sigframe size: 3376844client # [ 0.294569] devtmpfs: initialized845worker2 # [ 0.284374] ... max period: 00007fffffffffff846worker1 # [ 0.287680] rcu: Hierarchical SRCU implementation.847client # [ 0.295382] x86/mm: Memory block size: 128MB848worker2 # [ 0.285163] ... global_ctrl mask: 000000000000003f849worker1 # [ 0.288411] rcu: Max phase no-delay instances is 400.850worker2 # [ 0.285586] signal: max sigframe size: 3376851client # [ 0.297113] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)852worker2 # [ 0.286419] rcu: Hierarchical SRCU implementation.853lb # [ 0.293136] smp: Bringing up secondary CPUs ...854worker2 # [ 0.287069] rcu: Max phase no-delay instances is 400.855lb # [ 0.293618] smp: Brought up 1 node, 1 CPU856client # [ 0.298241] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).857lb # [ 0.294301] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)858client # [ 0.299409] pinctrl core: initialized pinctrl subsystem859client # [ 0.300573] PM: RTC time: 09:34:10, date: 2026-09-21860worker1 # [ 0.293452] smp: Bringing up secondary CPUs ...861lb # [ 0.294828] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)862worker1 # [ 0.293838] smp: Brought up 1 node, 1 CPU863lb # [ 0.295875] devtmpfs: initialized864worker1 # [ 0.294515] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)865worker2 # [ 0.292000] smp: Bringing up secondary CPUs ...866lb # [ 0.296663] x86/mm: Memory block size: 128MB867worker2 # [ 0.292488] smp: Brought up 1 node, 1 CPU868client # [ 0.304451] NET: Registered PF_NETLINK/PF_ROUTE protocol family869worker2 # [ 0.293143] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)870worker1 # [ 0.295033] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)871lb # [ 0.298442] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)872worker1 # [ 0.296102] devtmpfs: initialized873client # [ 0.305818] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations874worker1 # [ 0.296893] x86/mm: Memory block size: 128MB875lb # [ 0.299504] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).876worker2 # [ 0.293654] Memory: 1454588K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)877client # [ 0.306351] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations878lb # [ 0.300677] pinctrl core: initialized pinctrl subsystem879worker2 # [ 0.295118] devtmpfs: initialized880lb # [ 0.301819] PM: RTC time: 09:34:10, date: 2026-09-21881client # [ 0.307609] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations882worker1 # [ 0.298748] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)883worker2 # [ 0.295699] x86/mm: Memory block size: 128MB884client # [ 0.308340] audit: initializing netlink subsys (disabled)885worker1 # [ 0.299704] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).886client # [ 0.309331] audit: type=2000 audit(1789983250.847:1): state=initialized audit_enabled=0 res=1887worker1 # [ 0.300899] pinctrl core: initialized pinctrl subsystem888worker2 # [ 0.297502] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)889client # [ 0.310426] thermal_sys: Registered thermal governor 'fair_share'890worker1 # [ 0.301974] PM: RTC time: 09:34:10, date: 2026-09-21891lb # [ 0.305524] NET: Registered PF_NETLINK/PF_ROUTE protocol family892worker2 # [ 0.298503] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).893client # [ 0.310428] thermal_sys: Registered thermal governor 'bang_bang'894worker2 # [ 0.299556] pinctrl core: initialized pinctrl subsystem895client # [ 0.311330] thermal_sys: Registered thermal governor 'step_wise'896lb # [ 0.306840] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations897worker2 # [ 0.300701] PM: RTC time: 09:34:10, date: 2026-09-21898client # [ 0.312289] thermal_sys: Registered thermal governor 'user_space'899lb # [ 0.307629] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations900worker1 # [ 0.305719] NET: Registered PF_NETLINK/PF_ROUTE protocol family901client # [ 0.313077] thermal_sys: Registered thermal governor 'power_allocator'902client # [ 0.313341] cpuidle: using governor menu903lb # [ 0.308902] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations904worker1 # [ 0.307076] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations905lb # [ 0.309614] audit: initializing netlink subsys (disabled)906client # [ 0.315565] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5907worker2 # [ 0.304367] NET: Registered PF_NETLINK/PF_ROUTE protocol family908worker1 # [ 0.307851] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations909lb # [ 0.310606] audit: type=2000 audit(1789983250.857:1): state=initialized audit_enabled=0 res=1910client # [ 0.316560] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]911worker2 # [ 0.305679] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations912worker1 # [ 0.309124] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations913lb # [ 0.311747] thermal_sys: Registered thermal governor 'fair_share'914client # [ 0.317332] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry915worker1 # [ 0.309834] audit: initializing netlink subsys (disabled)916lb # [ 0.311749] thermal_sys: Registered thermal governor 'bang_bang'917worker2 # [ 0.306500] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations918client # [ 0.318350] PCI: Using configuration type 1 for base access919lb # [ 0.312605] thermal_sys: Registered thermal governor 'step_wise'920worker1 # [ 0.310981] thermal_sys: Registered thermal governor 'fair_share'921worker2 # [ 0.307770] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations922lb # [ 0.313508] thermal_sys: Registered thermal governor 'user_space'923client # [ 0.319417] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.924worker1 # [ 0.310990] thermal_sys: Registered thermal governor 'bang_bang'925worker2 # [ 0.308484] audit: initializing netlink subsys (disabled)926lb # [ 0.314340] thermal_sys: Registered thermal governor 'power_allocator'927worker1 # [ 0.311826] thermal_sys: Registered thermal governor 'step_wise'928lb # [ 0.314618] cpuidle: using governor menu929worker1 # [ 0.312744] thermal_sys: Registered thermal governor 'user_space'930worker2 # [ 0.309475] audit: type=2000 audit(1789983250.862:1): state=initialized audit_enabled=0 res=1931worker2 # [ 0.310646] thermal_sys: Registered thermal governor 'fair_share'932lb # [ 0.316805] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5933worker1 # [ 0.313561] audit: type=2000 audit(1789983250.861:1): state=initialized audit_enabled=0 res=1934worker2 # [ 0.310648] thermal_sys: Registered thermal governor 'bang_bang'935worker1 # [ 0.314827] thermal_sys: Registered thermal governor 'power_allocator'936lb # [ 0.317836] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]937worker1 # [ 0.314842] cpuidle: using governor menu938worker2 # [ 0.311474] thermal_sys: Registered thermal governor 'step_wise'939lb # [ 0.318607] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry940worker2 # [ 0.312446] thermal_sys: Registered thermal governor 'user_space'941worker1 # [ 0.317036] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5942lb # [ 0.319617] PCI: Using configuration type 1 for base access943worker2 # [ 0.313205] thermal_sys: Registered thermal governor 'power_allocator'944client # [ 0.326345] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages945worker2 # [ 0.313486] cpuidle: using governor menu946client # [ 0.327329] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page947worker1 # [ 0.318058] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]948lb # [ 0.320697] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.949client # [ 0.328312] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages950worker1 # [ 0.318827] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry951worker2 # [ 0.315682] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5952worker1 # [ 0.319841] PCI: Using configuration type 1 for base access953worker2 # [ 0.316719] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]954worker1 # [ 0.320945] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.955worker2 # [ 0.317477] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry956worker2 # [ 0.318486] PCI: Using configuration type 1 for base access957client # [ 0.333330] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page958worker2 # [ 0.319656] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.959lb # [ 0.327638] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages960lb # [ 0.328606] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page961worker1 # [ 0.326081] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages962worker1 # [ 0.326825] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page963worker2 # [ 0.324760] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages964worker2 # [ 0.325475] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page965lb # [ 0.333605] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages966client # [ 0.341723] ACPI: Added _OSI(Module Device)967lb # [ 0.334605] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page968worker1 # [ 0.331826] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages969client # [ 0.342331] ACPI: Added _OSI(Processor Device)970client # [ 0.343036] ACPI: Added _OSI(Processor Aggregator Device)971worker1 # [ 0.332824] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page972worker2 # [ 0.330475] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages973worker2 # [ 0.331473] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page974client # [ 0.351071] ACPI: 1 ACPI AML tables successfully acquired and loaded975lb # [ 0.345864] ACPI: Added _OSI(Module Device)976worker1 # [ 0.343227] ACPI: Added _OSI(Module Device)977lb # [ 0.346606] ACPI: Added _OSI(Processor Device)978worker1 # [ 0.343826] ACPI: Added _OSI(Processor Device)979lb # [ 0.347386] ACPI: Added _OSI(Processor Aggregator Device)980worker1 # [ 0.344557] ACPI: Added _OSI(Processor Aggregator Device)981client # [ 0.354754] ACPI: Interpreter enabled982worker2 # [ 0.341888] ACPI: Added _OSI(Module Device)983client # [ 0.355344] ACPI: PM: (supports S0 S3 S4 S5)984worker2 # [ 0.342475] ACPI: Added _OSI(Processor Device)985client # [ 0.356074] ACPI: Using IOAPIC for interrupt routing986worker2 # [ 0.343233] ACPI: Added _OSI(Processor Aggregator Device)987client # [ 0.358379] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug988lb # [ 0.353332] ACPI: 1 ACPI AML tables successfully acquired and loaded989client # [ 0.359330] PCI: Using E820 reservations for host bridge windows990worker1 # [ 0.352999] ACPI: 1 ACPI AML tables successfully acquired and loaded991client # [ 0.362475] ACPI: Enabled 2 GPEs in block 00 to 3F992lb # [ 0.357019] ACPI: Interpreter enabled993lb # [ 0.357621] ACPI: PM: (supports S0 S3 S4 S5)994lb # [ 0.358381] ACPI: Using IOAPIC for interrupt routing995worker2 # [ 0.351674] ACPI: 1 ACPI AML tables successfully acquired and loaded996worker1 # [ 0.357230] ACPI: Interpreter enabled997worker1 # [ 0.357840] ACPI: PM: (supports S0 S3 S4 S5)998worker1 # [ 0.358563] ACPI: Using IOAPIC for interrupt routing999lb # [ 0.360680] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1000worker1 # [ 0.358864] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1001lb # [ 0.361605] PCI: Using E820 reservations for host bridge windows1002worker2 # [ 0.355887] ACPI: Interpreter enabled1003worker1 # [ 0.359825] PCI: Using E820 reservations for host bridge windows1004worker2 # [ 0.356490] ACPI: PM: (supports S0 S3 S4 S5)1005worker2 # [ 0.357225] ACPI: Using IOAPIC for interrupt routing1006client # [ 0.370608] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1007lb # [ 0.364731] ACPI: Enabled 2 GPEs in block 00 to 3F1008client # [ 0.371336] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1009worker1 # [ 0.362939] ACPI: Enabled 2 GPEs in block 00 to 3F1010client # [ 0.372412] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1011worker2 # [ 0.359534] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1012client # [ 0.373462] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1013worker2 # [ 0.360474] PCI: Using E820 reservations for host bridge windows1014client # [ 0.374686] PCI host bridge to bus 0000:001015client # [ 0.375334] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1016client # [ 0.376329] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1017worker2 # [ 0.363606] ACPI: Enabled 2 GPEs in block 00 to 3F1018client # [ 0.377330] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1019lb # [ 0.372166] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1020client # [ 0.378329] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1021lb # [ 0.372611] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1022worker1 # [ 0.371564] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1023client # [ 0.379329] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1024lb # [ 0.373683] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1025worker1 # [ 0.371831] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1026client # [ 0.380329] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1027lb # [ 0.374727] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1028client # [ 0.381329] pci_bus 0000:00: root bus resource [bus 00-ff]1029lb # [ 0.375963] PCI host bridge to bus 0000:001030worker1 # [ 0.373747] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1031client # [ 0.382297] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1032lb # [ 0.376609] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1033worker2 # [ 0.371034] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1034worker1 # [ 0.374791] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1035lb # [ 0.377605] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1036client # [ 0.383743] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1037worker2 # [ 0.371480] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1038worker1 # [ 0.376002] PCI host bridge to bus 0000:001039lb # [ 0.378604] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1040worker1 # [ 0.376693] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1041worker2 # [ 0.372582] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1042lb # [ 0.379605] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1043worker1 # [ 0.377642] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1044worker2 # [ 0.373597] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1045lb # [ 0.380604] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1046worker1 # [ 0.378651] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1047worker2 # [ 0.374858] PCI host bridge to bus 0000:001048lb # [ 0.381604] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1049worker2 # [ 0.375479] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1050worker1 # [ 0.379764] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1051lb # [ 0.382605] pci_bus 0000:00: root bus resource [bus 00-ff]1052worker2 # [ 0.376473] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1053worker1 # [ 0.380712] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1054lb # [ 0.383598] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1055worker2 # [ 0.377473] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1056worker1 # [ 0.381685] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1057worker1 # [ 0.382736] pci_bus 0000:00: root bus resource [bus 00-ff]1058worker2 # [ 0.378474] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1059client # [ 0.386390] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1060lb # [ 0.385040] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1061worker1 # [ 0.383581] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1062worker2 # [ 0.379473] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1063client # [ 0.387404] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1064worker2 # [ 0.380474] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1065client # [ 0.388319] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1066worker1 # [ 0.385265] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1067worker2 # [ 0.381475] pci_bus 0000:00: root bus resource [bus 00-ff]1068client # [ 0.389678] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1069worker2 # [ 0.382474] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1070client # [ 0.391040] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1071worker2 # [ 0.383908] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1072lb # [ 0.387671] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1073client # [ 0.393348] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1074client # [ 0.394216] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1075lb # [ 0.388688] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1076client # [ 0.395060] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1077lb # [ 0.389627] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1078worker1 # [ 0.387902] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1079client # [ 0.395335] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1080worker1 # [ 0.388965] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1081lb # [ 0.390978] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1082worker1 # [ 0.389902] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1083client # [ 0.397165] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1084lb # [ 0.392348] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1085worker2 # [ 0.386540] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1086worker1 # [ 0.391171] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1087worker2 # [ 0.387486] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1088client # [ 0.400330] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1089worker2 # [ 0.388620] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1090worker1 # [ 0.392544] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1091client # [ 0.401225] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1092lb # [ 0.395650] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1093client # [ 0.402049] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1094worker2 # [ 0.389659] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1095lb # [ 0.396522] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1096lb # [ 0.397375] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1097client # [ 0.402882] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1098worker1 # [ 0.395859] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1099worker2 # [ 0.391424] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1100worker1 # [ 0.396671] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1101lb # [ 0.398458] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1102worker1 # [ 0.397538] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1103lb # [ 0.400196] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1104client # [ 0.405337] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1105worker1 # [ 0.397830] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1106worker2 # [ 0.394501] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1107client # [ 0.406151] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1108worker2 # [ 0.395336] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1109worker1 # [ 0.399393] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1110client # [ 0.407049] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1111worker2 # [ 0.396233] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1112lb # [ 0.402640] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1113worker2 # [ 0.396480] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1114lb # [ 0.403497] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1115client # [ 0.407883] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1116worker1 # [ 0.400832] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1117lb # [ 0.404364] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1118worker2 # [ 0.398037] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1119worker1 # [ 0.401654] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1120worker1 # [ 0.402577] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1121client # [ 0.410338] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1122lb # [ 0.405199] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1123client # [ 0.411201] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1124worker2 # [ 0.399512] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1125worker1 # [ 0.403426] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1126worker2 # [ 0.400369] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1127client # [ 0.412097] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1128worker2 # [ 0.401251] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1129lb # [ 0.408501] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1130client # [ 0.412886] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1131lb # [ 0.409259] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1132worker2 # [ 0.402941] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1133worker1 # [ 0.405853] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1134lb # [ 0.409627] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1135worker1 # [ 0.406752] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1136client # [ 0.414337] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1137worker1 # [ 0.407581] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1138client # [ 0.415180] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1139lb # [ 0.411184] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1140worker2 # [ 0.405508] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1141client # [ 0.416069] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1142worker1 # [ 0.408397] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1143worker2 # [ 0.406326] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144client # [ 0.416928] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1145worker2 # [ 0.407222] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1146lb # [ 0.413638] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1147worker1 # [ 0.409850] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1148lb # [ 0.414479] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1149worker2 # [ 0.408053] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150worker1 # [ 0.410705] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1151lb # [ 0.415376] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1152client # [ 0.419551] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1153worker1 # [ 0.411620] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1154client # [ 0.420335] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1155worker2 # [ 0.409504] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1156lb # [ 0.417182] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1157worker2 # [ 0.410353] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158client # [ 0.421289] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1159worker1 # [ 0.413387] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1160worker2 # [ 0.411188] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1161client # [ 0.422690] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1162lb # [ 0.419509] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1163worker1 # [ 0.414855] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1164worker2 # [ 0.412009] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1165lb # [ 0.420295] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1166worker1 # [ 0.415701] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1167client # [ 0.424345] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1168lb # [ 0.420626] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1169worker1 # [ 0.416583] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1170client # [ 0.425180] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1171worker2 # [ 0.413507] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1172client # [ 0.426054] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1173lb # [ 0.422211] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1174worker1 # [ 0.417392] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1175worker2 # [ 0.414369] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1176client # [ 0.426335] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1177worker2 # [ 0.415197] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1178client # [ 0.428071] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1179worker2 # [ 0.416072] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1180lb # [ 0.424642] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1181worker1 # [ 0.419832] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1182lb # [ 0.425532] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1183client # [ 0.430330] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1184worker1 # [ 0.421536] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1185lb # [ 0.426367] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1186client # [ 0.431317] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1187worker1 # [ 0.421846] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1188worker2 # [ 0.418482] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1189client # [ 0.432703] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1190lb # [ 0.427228] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1191worker2 # [ 0.419481] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1192worker1 # [ 0.423544] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1193worker2 # [ 0.420402] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1194lb # [ 0.428649] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1195worker1 # [ 0.425832] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1196client # [ 0.434378] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1197worker2 # [ 0.421946] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1198lb # [ 0.429505] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1199worker1 # [ 0.426657] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1200client # [ 0.435184] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1201worker1 # [ 0.427542] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1202client # [ 0.436352] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1203lb # [ 0.430290] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1204worker2 # [ 0.423482] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1205worker1 # [ 0.427831] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1206lb # [ 0.430633] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1207worker2 # [ 0.424291] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1208client # [ 0.437913] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1209worker2 # [ 0.425240] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1210worker1 # [ 0.429696] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1211lb # [ 0.432143] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1212worker2 # [ 0.426291] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1213client # [ 0.439336] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1214worker1 # [ 0.431790] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1215client # [ 0.440197] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1216lb # [ 0.433645] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1217worker2 # [ 0.428012] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1218worker1 # [ 0.432563] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1219client # [ 0.441048] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1220lb # [ 0.434581] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1221worker2 # [ 0.429434] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1222worker1 # [ 0.433360] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1223client # [ 0.441916] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1224lb # [ 0.435954] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1225worker2 # [ 0.430205] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1226client # [ 0.443357] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1227worker2 # [ 0.431011] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1228worker1 # [ 0.434832] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1229client # [ 0.444399] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1230worker1 # [ 0.435669] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1231lb # [ 0.438612] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1232worker2 # [ 0.432491] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1233lb # [ 0.440253] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1234worker1 # [ 0.436867] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1235client # [ 0.445962] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1236worker2 # [ 0.433339] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1237lb # [ 0.440654] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1238worker1 # [ 0.438398] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1239worker2 # [ 0.434496] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1240lb # [ 0.442153] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1241worker2 # [ 0.436029] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1242worker1 # [ 0.439832] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1243worker1 # [ 0.440710] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1244lb # [ 0.444360] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1245worker1 # [ 0.441597] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1246worker2 # [ 0.437492] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1247lb # [ 0.444612] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1248worker2 # [ 0.438325] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1249lb # [ 0.445566] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1250worker1 # [ 0.443300] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1251worker2 # [ 0.439243] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1252worker2 # [ 0.440893] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1253worker1 # [ 0.444488] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1254lb # [ 0.446998] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1255worker1 # [ 0.445652] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1256lb # [ 0.449415] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1257worker2 # [ 0.442148] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1258client # [ 0.446518] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1259worker1 # [ 0.447507] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1260lb # [ 0.450442] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1261worker2 # [ 0.443276] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1262client # [ 0.448975] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1263lb # [ 0.452351] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1264worker2 # [ 0.445101] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1265client # [ 0.449582] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1266client # [ 0.451170] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1267client # [ 0.452379] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1268client # [ 0.453650] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1269client # [ 0.454591] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1270client # [ 0.456346] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1271client # [ 0.457286] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1272worker1 # [ 0.448038] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1273worker1 # [ 0.450474] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1274client # [ 0.458436] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1275lb # [ 0.452800] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1276worker2 # [ 0.445694] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1277client # [ 0.459961] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1278worker1 # [ 0.451062] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1279worker2 # [ 0.448135] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1280lb # [ 0.455420] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1281worker1 # [ 0.452403] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1282worker2 # [ 0.448802] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1283lb # [ 0.456655] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1284worker1 # [ 0.453085] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1285client # [ 0.463343] ACPI: PCI: Interrupt link LNKA configured for IRQ 101286worker2 # [ 0.450074] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1287lb # [ 0.458323] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1288client # [ 0.464384] ACPI: PCI: Interrupt link LNKB configured for IRQ 101289worker1 # [ 0.454112] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1290worker2 # [ 0.451395] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1291lb # [ 0.458877] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1292client # [ 0.465425] ACPI: PCI: Interrupt link LNKC configured for IRQ 111293worker1 # [ 0.455088] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1294client # [ 0.466424] ACPI: PCI: Interrupt link LNKD configured for IRQ 111295worker2 # [ 0.452744] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1296lb # [ 0.459940] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1297client # [ 0.467417] ACPI: PCI: Interrupt link LNKE configured for IRQ 101298worker1 # [ 0.456841] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1299worker2 # [ 0.453737] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1300lb # [ 0.460858] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1301client # [ 0.468446] ACPI: PCI: Interrupt link LNKF configured for IRQ 101302worker1 # [ 0.457663] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1303client # [ 0.469450] ACPI: PCI: Interrupt link LNKG configured for IRQ 111304worker2 # [ 0.455501] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1305lb # [ 0.462656] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1306worker1 # [ 0.458845] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1307client # [ 0.470426] ACPI: PCI: Interrupt link LNKH configured for IRQ 111308worker2 # [ 0.456338] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1309client # [ 0.471366] ACPI: PCI: Interrupt link GSIA configured for IRQ 161310lb # [ 0.463494] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1311worker1 # [ 0.460841] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1312client # [ 0.472344] ACPI: PCI: Interrupt link GSIB configured for IRQ 171313worker2 # [ 0.457550] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1314lb # [ 0.464918] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1315client # [ 0.473271] ACPI: PCI: Interrupt link GSIC configured for IRQ 181316worker2 # [ 0.459406] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1317client # [ 0.474119] ACPI: PCI: Interrupt link GSID configured for IRQ 191318lb # [ 0.466411] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1319client # [ 0.475095] ACPI: PCI: Interrupt link GSIE configured for IRQ 201320worker1 # [ 0.464578] ACPI: PCI: Interrupt link LNKA configured for IRQ 101321client # [ 0.475341] ACPI: PCI: Interrupt link GSIF configured for IRQ 211322worker1 # [ 0.464932] ACPI: PCI: Interrupt link LNKB configured for IRQ 101323client # [ 0.476301] ACPI: PCI: Interrupt link GSIG configured for IRQ 221324worker2 # [ 0.463107] ACPI: PCI: Interrupt link LNKA configured for IRQ 101325worker1 # [ 0.465935] ACPI: PCI: Interrupt link LNKC configured for IRQ 111326client # [ 0.477179] ACPI: PCI: Interrupt link GSIH configured for IRQ 231327lb # [ 0.470217] ACPI: PCI: Interrupt link LNKA configured for IRQ 101328worker2 # [ 0.463586] ACPI: PCI: Interrupt link LNKB configured for IRQ 101329worker1 # [ 0.466921] ACPI: PCI: Interrupt link LNKD configured for IRQ 111330lb # [ 0.470714] ACPI: PCI: Interrupt link LNKB configured for IRQ 101331client # [ 0.479000] iommu: Default domain type: Translated1332worker1 # [ 0.467926] ACPI: PCI: Interrupt link LNKE configured for IRQ 101333worker2 # [ 0.464570] ACPI: PCI: Interrupt link LNKC configured for IRQ 111334lb # [ 0.471703] ACPI: PCI: Interrupt link LNKC configured for IRQ 111335client # [ 0.479336] iommu: DMA domain TLB invalidation policy: lazy mode1336worker2 # [ 0.465572] ACPI: PCI: Interrupt link LNKD configured for IRQ 111337worker1 # [ 0.468926] ACPI: PCI: Interrupt link LNKF configured for IRQ 101338client # [ 0.480543] ACPI: bus type USB registered1339lb # [ 0.472702] ACPI: PCI: Interrupt link LNKD configured for IRQ 111340worker2 # [ 0.466600] ACPI: PCI: Interrupt link LNKE configured for IRQ 101341worker1 # [ 0.469926] ACPI: PCI: Interrupt link LNKG configured for IRQ 111342client # [ 0.481260] usbcore: registered new interface driver usbfs1343lb # [ 0.473706] ACPI: PCI: Interrupt link LNKE configured for IRQ 101344worker1 # [ 0.470921] ACPI: PCI: Interrupt link LNKH configured for IRQ 111345worker2 # [ 0.467599] ACPI: PCI: Interrupt link LNKF configured for IRQ 101346client # [ 0.482001] usbcore: registered new interface driver hub1347worker1 # [ 0.471854] ACPI: PCI: Interrupt link GSIA configured for IRQ 161348worker2 # [ 0.468573] ACPI: PCI: Interrupt link LNKG configured for IRQ 111349lb # [ 0.474702] ACPI: PCI: Interrupt link LNKF configured for IRQ 101350client # [ 0.482340] usbcore: registered new device driver usb1351worker1 # [ 0.472762] ACPI: PCI: Interrupt link GSIB configured for IRQ 171352lb # [ 0.475704] ACPI: PCI: Interrupt link LNKG configured for IRQ 111353worker2 # [ 0.469575] ACPI: PCI: Interrupt link LNKH configured for IRQ 111354client # [ 0.484015] NetLabel: Initializing1355worker1 # [ 0.473522] ACPI: PCI: Interrupt link GSIC configured for IRQ 181356lb # [ 0.476703] ACPI: PCI: Interrupt link LNKH configured for IRQ 111357worker2 # [ 0.470511] ACPI: PCI: Interrupt link GSIA configured for IRQ 161358client # [ 0.484330] NetLabel: domain hash size = 1281359lb # [ 0.477642] ACPI: PCI: Interrupt link GSIA configured for IRQ 161360worker1 # [ 0.473837] ACPI: PCI: Interrupt link GSID configured for IRQ 191361worker2 # [ 0.471445] ACPI: PCI: Interrupt link GSIB configured for IRQ 171362client # [ 0.485041] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1363worker1 # [ 0.474779] ACPI: PCI: Interrupt link GSIE configured for IRQ 201364lb # [ 0.478553] ACPI: PCI: Interrupt link GSIB configured for IRQ 171365worker2 # [ 0.472192] ACPI: PCI: Interrupt link GSIC configured for IRQ 181366client # [ 0.485368] NetLabel: unlabeled traffic allowed by default1367worker1 # [ 0.475534] ACPI: PCI: Interrupt link GSIF configured for IRQ 211368lb # [ 0.479333] ACPI: PCI: Interrupt link GSIC configured for IRQ 181369worker2 # [ 0.472486] ACPI: PCI: Interrupt link GSID configured for IRQ 191370client # [ 0.486290] PCI: Using ACPI for IRQ routing1371lb # [ 0.479619] ACPI: PCI: Interrupt link GSID configured for IRQ 191372worker1 # [ 0.475837] ACPI: PCI: Interrupt link GSIG configured for IRQ 221373worker2 # [ 0.473395] ACPI: PCI: Interrupt link GSIE configured for IRQ 201374worker1 # [ 0.476780] ACPI: PCI: Interrupt link GSIH configured for IRQ 231375lb # [ 0.480528] ACPI: PCI: Interrupt link GSIE configured for IRQ 201376worker2 # [ 0.474164] ACPI: PCI: Interrupt link GSIF configured for IRQ 211377worker2 # [ 0.474487] ACPI: PCI: Interrupt link GSIG configured for IRQ 221378lb # [ 0.481314] ACPI: PCI: Interrupt link GSIF configured for IRQ 211379worker2 # [ 0.475450] ACPI: PCI: Interrupt link GSIH configured for IRQ 231380worker1 # [ 0.478687] iommu: Default domain type: Translated1381lb # [ 0.481619] ACPI: PCI: Interrupt link GSIG configured for IRQ 221382worker1 # [ 0.479442] iommu: DMA domain TLB invalidation policy: lazy mode1383lb # [ 0.482549] ACPI: PCI: Interrupt link GSIH configured for IRQ 231384worker2 # [ 0.477350] iommu: Default domain type: Translated1385worker1 # [ 0.480114] ACPI: bus type USB registered1386worker1 # [ 0.480849] usbcore: registered new interface driver usbfs1387worker2 # [ 0.478038] iommu: DMA domain TLB invalidation policy: lazy mode1388lb # [ 0.484508] iommu: Default domain type: Translated1389worker1 # [ 0.481725] usbcore: registered new interface driver hub1390worker2 # [ 0.478733] ACPI: bus type USB registered1391lb # [ 0.485157] iommu: DMA domain TLB invalidation policy: lazy mode1392worker1 # [ 0.482502] usbcore: registered new device driver usb1393worker2 # [ 0.479478] usbcore: registered new interface driver usbfs1394lb # [ 0.485877] ACPI: bus type USB registered1395worker2 # [ 0.480384] usbcore: registered new interface driver hub1396lb # [ 0.486591] usbcore: registered new interface driver usbfs1397worker1 # [ 0.483713] NetLabel: Initializing1398worker2 # [ 0.481210] usbcore: registered new device driver usb1399lb # [ 0.487293] usbcore: registered new interface driver hub1400worker1 # [ 0.484237] NetLabel: domain hash size = 1281401lb # [ 0.487617] usbcore: registered new device driver usb1402worker1 # [ 0.484824] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1403worker2 # [ 0.482323] NetLabel: Initializing1404worker1 # [ 0.485792] NetLabel: unlabeled traffic allowed by default1405worker2 # [ 0.482907] NetLabel: domain hash size = 1281406lb # [ 0.489374] NetLabel: Initializing1407worker1 # [ 0.486516] PCI: Using ACPI for IRQ routing1408worker2 # [ 0.483473] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1409lb # [ 0.490104] NetLabel: domain hash size = 1281410worker2 # [ 0.484443] NetLabel: unlabeled traffic allowed by default1411lb # [ 0.490607] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1412worker2 # [ 0.485278] PCI: Using ACPI for IRQ routing1413lb # [ 0.491644] NetLabel: unlabeled traffic allowed by default1414lb # [ 0.492623] PCI: Using ACPI for IRQ routing1415client # [ 0.529923] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416client # [ 0.530325] pci 0000:00:01.0: vgaarb: bridge control possible1417client # [ 0.530325] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418client # [ 0.530333] vgaarb: loaded1419client # [ 0.531071] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420client # [ 0.531329] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421client # [ 0.535412] clocksource: Switched to clocksource kvm-clock1422client # [ 0.537075] VFS: Disk quotas dquot_6.6.01423client # [ 0.539611] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1424worker1 # [ 0.530208] pci 0000:00:01.0: vgaarb: setting as boot VGA device1425client # [ 0.540975] pnp: PnP ACPI init1426worker1 # [ 0.530820] pci 0000:00:01.0: vgaarb: bridge control possible1427client # [ 0.541800] ACPI: IRQ 4 override to edge(!), high(!)1428worker2 # [ 0.529092] pci 0000:00:01.0: vgaarb: setting as boot VGA device1429worker1 # [ 0.530820] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1430client # [ 0.542799] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1431worker1 # [ 0.530830] vgaarb: loaded1432worker2 # [ 0.529470] pci 0000:00:01.0: vgaarb: bridge control possible1433lb # [ 0.537347] pci 0000:00:01.0: vgaarb: setting as boot VGA device1434client # [ 0.544344] pnp: PnP ACPI: found 6 devices1435worker1 # [ 0.531566] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01436worker2 # [ 0.529470] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1437lb # [ 0.537601] pci 0000:00:01.0: vgaarb: bridge control possible1438worker1 # [ 0.531827] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1439worker2 # [ 0.529479] vgaarb: loaded1440worker2 # [ 0.530224] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01441lb # [ 0.537601] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1442lb # [ 0.537610] vgaarb: loaded1443worker2 # [ 0.530478] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1444lb # [ 0.538346] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01445lb # [ 0.538605] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1446worker1 # [ 0.535918] clocksource: Switched to clocksource kvm-clock1447worker2 # [ 0.533581] clocksource: Switched to clocksource kvm-clock1448worker1 # [ 0.537658] VFS: Disk quotas dquot_6.6.01449client # [ 0.551956] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1450client # [ 0.553413] clocksource: Switched to clocksource acpi_pm1451client # [ 0.554402] NET: Registered PF_INET protocol family1452worker2 # [ 0.537220] VFS: Disk quotas dquot_6.6.01453worker1 # [ 0.540301] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1454lb # [ 0.544697] clocksource: Switched to clocksource kvm-clock1455worker1 # [ 0.541637] pnp: PnP ACPI init1456client # [ 0.555562] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1457worker2 # [ 0.537928] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1458worker1 # [ 0.542460] ACPI: IRQ 4 override to edge(!), high(!)1459worker2 # [ 0.539296] pnp: PnP ACPI init1460worker2 # [ 0.540181] ACPI: IRQ 4 override to edge(!), high(!)1461worker1 # [ 0.543571] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1462lb # [ 0.548354] VFS: Disk quotas dquot_6.6.01463worker1 # [ 0.545314] pnp: PnP ACPI: found 6 devices1464worker2 # [ 0.541431] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1465lb # [ 0.549300] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1466lb # [ 0.550672] pnp: PnP ACPI init1467worker2 # [ 0.543158] pnp: PnP ACPI: found 6 devices1468lb # [ 0.551489] ACPI: IRQ 4 override to edge(!), high(!)1469lb # [ 0.552496] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1470lb # [ 0.554066] pnp: PnP ACPI: found 6 devices1471worker1 # [ 0.553080] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1472worker1 # [ 0.554573] clocksource: Switched to clocksource acpi_pm1473worker2 # [ 0.550949] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1474worker1 # [ 0.555563] NET: Registered PF_INET protocol family1475worker2 # [ 0.552433] clocksource: Switched to clocksource acpi_pm1476worker2 # [ 0.553411] NET: Registered PF_INET protocol family1477worker1 # [ 0.556774] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1478worker2 # [ 0.554608] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1479lb # [ 0.561736] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1480client # [ 0.572333] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1481lb # [ 0.563181] clocksource: Switched to clocksource acpi_pm1482client # [ 0.573790] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1483lb # [ 0.564190] NET: Registered PF_INET protocol family1484client # [ 0.575068] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1485lb # [ 0.565392] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1486client # [ 0.576451] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1487client # [ 0.577725] TCP: Hash tables configured (established 16384 bind 16384)1488client # [ 0.578885] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1489client # [ 0.580213] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1490client # [ 0.581327] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1491client # [ 0.582609] NET: Registered PF_UNIX/PF_LOCAL protocol family1492client # [ 0.583616] NET: Registered PF_XDP protocol family1493client # [ 0.584479] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1494client # [ 0.585542] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1495client # [ 0.586590] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1496client # [ 0.587740] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1497worker1 # [ 0.573568] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1498client # [ 0.588873] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1499worker2 # [ 0.571529] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1500worker1 # [ 0.575122] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1501client # [ 0.589986] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1502worker1 # [ 0.576388] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1503worker2 # [ 0.573032] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1504client # [ 0.591628] ACPI: \_SB_.GSIA: Enabled at IRQ 161505worker1 # [ 0.577735] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1506worker2 # [ 0.574352] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1507lb # [ 0.582340] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1508worker1 # [ 0.578982] TCP: Hash tables configured (established 16384 bind 16384)1509client # [ 0.593797] ACPI: \_SB_.GSIB: Enabled at IRQ 171510worker2 # [ 0.575704] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1511lb # [ 0.583853] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1512worker2 # [ 0.576918] TCP: Hash tables configured (established 16384 bind 16384)1513worker1 # [ 0.580152] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1514client # [ 0.595826] ACPI: \_SB_.GSIC: Enabled at IRQ 181515worker1 # [ 0.581440] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1516lb # [ 0.585141] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1517worker2 # [ 0.578001] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1518lb # [ 0.586441] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1519worker2 # [ 0.579229] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1520worker1 # [ 0.582558] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1521client # [ 0.597836] ACPI: \_SB_.GSID: Enabled at IRQ 191522worker1 # [ 0.583758] NET: Registered PF_UNIX/PF_LOCAL protocol family1523lb # [ 0.587622] TCP: Hash tables configured (established 16384 bind 16384)1524worker2 # [ 0.580274] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1525worker1 # [ 0.584661] NET: Registered PF_XDP protocol family1526worker2 # [ 0.581476] NET: Registered PF_UNIX/PF_LOCAL protocol family1527client # [ 0.599517] PCI: CLS 0 bytes, default 641528lb # [ 0.588709] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1529worker1 # [ 0.585506] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1530worker2 # [ 0.582374] NET: Registered PF_XDP protocol family1531lb # [ 0.589934] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1532worker1 # [ 0.586492] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1533client # [ 0.600377] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1534worker2 # [ 0.583182] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1535lb # [ 0.590967] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1536worker1 # [ 0.587458] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1537client # [ 0.602031] Trying to unpack rootfs image as initramfs...1538worker2 # [ 0.584150] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1539lb # [ 0.592136] NET: Registered PF_UNIX/PF_LOCAL protocol family1540worker1 # [ 0.588500] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1541worker2 # [ 0.585113] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1542lb # [ 0.593080] NET: Registered PF_XDP protocol family1543worker1 # [ 0.589585] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1544worker2 # [ 0.586208] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1545lb # [ 0.593953] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1546worker1 # [ 0.590642] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1547worker2 # [ 0.587280] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1548lb # [ 0.594941] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1549worker2 # [ 0.588353] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1550worker1 # [ 0.592325] ACPI: \_SB_.GSIA: Enabled at IRQ 161551lb # [ 0.595949] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1552worker2 # [ 0.590052] ACPI: \_SB_.GSIA: Enabled at IRQ 161553lb # [ 0.597067] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1554lb # [ 0.598295] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1555worker1 # [ 0.594613] ACPI: \_SB_.GSIB: Enabled at IRQ 171556worker2 # [ 0.592314] ACPI: \_SB_.GSIB: Enabled at IRQ 171557lb # [ 0.599453] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1558worker1 # [ 0.596699] ACPI: \_SB_.GSIC: Enabled at IRQ 181559lb # [ 0.601196] ACPI: \_SB_.GSIA: Enabled at IRQ 161560worker2 # [ 0.594466] ACPI: \_SB_.GSIC: Enabled at IRQ 181561worker1 # [ 0.598833] ACPI: \_SB_.GSID: Enabled at IRQ 191562worker2 # [ 0.596492] ACPI: \_SB_.GSID: Enabled at IRQ 191563lb # [ 0.603869] ACPI: \_SB_.GSIB: Enabled at IRQ 171564worker1 # [ 0.600615] PCI: CLS 0 bytes, default 641565worker2 # [ 0.598226] PCI: CLS 0 bytes, default 641566lb # [ 0.605801] ACPI: \_SB_.GSIC: Enabled at IRQ 181567worker1 # [ 0.601585] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1568worker1 # [ 0.603308] Trying to unpack rootfs image as initramfs...1569worker2 # [ 0.599123] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1570lb # [ 0.607688] ACPI: \_SB_.GSID: Enabled at IRQ 191571worker2 # [ 0.600894] Trying to unpack rootfs image as initramfs...1572lb # [ 0.609324] PCI: CLS 0 bytes, default 641573lb # [ 0.610243] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574lb # [ 0.612052] Trying to unpack rootfs image as initramfs...1575client # [ 0.647007] Initialise system trusted keyrings1576client # [ 0.650250] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.645365] Initialise system trusted keyrings1578worker1 # [ 0.649122] Initialise system trusted keyrings1579worker1 # [ 0.650044] workingset: timestamp_bits=40 max_order=19 bucket_order=01580worker2 # [ 0.648255] workingset: timestamp_bits=40 max_order=19 bucket_order=01581lb # [ 0.657975] Initialise system trusted keyrings1582lb # [ 0.658985] workingset: timestamp_bits=40 max_order=19 bucket_order=01583client # [ 0.671392] Key type asymmetric registered1584client # [ 0.672055] Asymmetric key parser 'x509' registered1585client # [ 0.672905] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1586client # [ 0.676340] io scheduler mq-deadline registered1587client # [ 0.677107] io scheduler kyber registered1588client # [ 0.681386] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1589client # [ 0.682667] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1590client # [ 0.687999] Linux agpgart interface v0.1031591worker1 # [ 0.673771] Key type asymmetric registered1592client # [ 0.688758] ACPI: bus type drm_connector registered1593worker1 # [ 0.674502] Asymmetric key parser 'x509' registered1594worker2 # [ 0.671326] Key type asymmetric registered1595worker2 # [ 0.672061] Asymmetric key parser 'x509' registered1596worker1 # [ 0.675339] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1597worker2 # [ 0.672967] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1598client # [ 0.691659] usbcore: registered new interface driver usbserial_generic1599lb # [ 0.682397] Key type asymmetric registered1600client # [ 0.692759] usbserial: USB Serial support registered for generic1601worker2 # [ 0.676357] io scheduler mq-deadline registered1602worker1 # [ 0.679806] io scheduler mq-deadline registered1603worker1 # [ 0.680625] io scheduler kyber registered1604worker2 # [ 0.677148] io scheduler kyber registered1605lb # [ 0.685100] Asymmetric key parser 'x509' registered1606lb # [ 0.686025] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1607client # [ 0.696215] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1608client # [ 0.696215] Please enable it if your BIOS has the CPPC option.1609worker1 # [ 0.684235] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1610client # [ 0.698584] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1611worker2 # [ 0.681360] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1612lb # [ 0.689200] io scheduler mq-deadline registered1613lb # [ 0.689958] io scheduler kyber registered1614worker1 # [ 0.685501] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1615worker2 # [ 0.682671] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1616client # [ 0.702377] drop_monitor: Initializing network drop monitor service1617client # [ 0.703550] NET: Registered PF_INET6 protocol family1618worker1 # [ 0.689471] Linux agpgart interface v0.1031619lb # [ 0.693574] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1620worker1 # [ 0.690234] ACPI: bus type drm_connector registered1621worker2 # [ 0.688015] Linux agpgart interface v0.1031622lb # [ 0.694853] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1623client # [ 0.706661] Segment Routing with IPv61624worker2 # [ 0.688807] ACPI: bus type drm_connector registered1625worker1 # [ 0.693172] usbcore: registered new interface driver usbserial_generic1626worker1 # [ 0.694333] usbserial: USB Serial support registered for generic1627client # [ 0.709227] In-situ OAM (IOAM) with IPv61628client # [ 0.710306] IPI shorthand broadcast: enabled1629lb # [ 0.700128] Linux agpgart interface v0.1031630lb # [ 0.700901] ACPI: bus type drm_connector registered1631worker2 # [ 0.693387] usbcore: registered new interface driver usbserial_generic1632worker2 # [ 0.694487] usbserial: USB Serial support registered for generic1633worker1 # [ 0.697698] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1634worker1 # [ 0.697698] Please enable it if your BIOS has the CPPC option.1635worker1 # [ 0.700043] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1636lb # [ 0.704202] usbcore: registered new interface driver usbserial_generic1637worker2 # [ 0.697224] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1638lb # [ 0.705256] usbserial: USB Serial support registered for generic1639worker2 # [ 0.697224] Please enable it if your BIOS has the CPPC option.1640worker2 # [ 0.699448] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1641worker1 # [ 0.703872] drop_monitor: Initializing network drop monitor service1642client # [ 0.718600] sched_clock: Marking stable (584013606, 134176939)->(788923505, -70732960)1643lb # [ 0.708063] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1644worker1 # [ 0.705079] NET: Registered PF_INET6 protocol family1645lb # [ 0.708063] Please enable it if your BIOS has the CPPC option.1646worker2 # [ 0.702384] drop_monitor: Initializing network drop monitor service1647lb # [ 0.710266] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1648worker2 # [ 0.703571] NET: Registered PF_INET6 protocol family1649worker1 # [ 0.708266] Segment Routing with IPv61650client # [ 0.723402] registered taskstats version 11651lb # [ 0.713216] drop_monitor: Initializing network drop monitor service1652client # [ 0.724375] Loading compiled-in X.509 certificates1653lb # [ 0.714420] NET: Registered PF_INET6 protocol family1654worker1 # [ 0.710717] In-situ OAM (IOAM) with IPv61655worker2 # [ 0.708320] Segment Routing with IPv61656worker2 # [ 0.709013] In-situ OAM (IOAM) with IPv61657worker1 # [ 0.712839] IPI shorthand broadcast: enabled1658lb # [ 0.719490] Segment Routing with IPv61659worker2 # [ 0.712502] IPI shorthand broadcast: enabled1660lb # [ 0.720173] In-situ OAM (IOAM) with IPv61661lb # [ 0.723337] IPI shorthand broadcast: enabled1662worker1 # [ 0.721162] sched_clock: Marking stable (588013735, 132666817)->(795117144, -74436592)1663worker2 # [ 0.718843] sched_clock: Marking stable (585014846, 133187824)->(792832666, -74629996)1664worker1 # [ 0.724773] registered taskstats version 11665worker1 # [ 0.725774] Loading compiled-in X.509 certificates1666worker2 # [ 0.724316] registered taskstats version 11667lb # [ 0.731430] sched_clock: Marking stable (598015658, 133032894)->(804808816, -73760264)1668worker2 # [ 0.725301] Loading compiled-in X.509 certificates1669client # [ 0.743207] Demotion targets for Node 0: null1670lb # [ 0.733132] registered taskstats version 11671client # [ 0.744078] Key type .fscrypt registered1672client # [ 0.744800] Key type fscrypt-provisioning registered1673lb # [ 0.736114] Loading compiled-in X.509 certificates1674client # [ 0.748323] ima: No TPM chip found, activating TPM-bypass!1675client # [ 0.749269] ima: Allocated hash algorithm: sha11676client # [ 0.750084] ima: No architecture policies found1677client # [ 0.754205] PM: Magic number: 10:579:5761678client # [ 0.755827] RAS: Correctable Errors collector initialized.1679worker1 # [ 0.744698] Demotion targets for Node 0: null1680worker1 # [ 0.745605] Key type .fscrypt registered1681worker2 # [ 0.743219] Demotion targets for Node 0: null1682worker2 # [ 0.744127] Key type .fscrypt registered1683worker1 # [ 0.747698] Key type fscrypt-provisioning registered1684worker2 # [ 0.744853] Key type fscrypt-provisioning registered1685lb # [ 0.752611] Demotion targets for Node 0: null1686worker1 # [ 0.748654] ima: No TPM chip found, activating TPM-bypass!1687worker2 # [ 0.745820] ima: No TPM chip found, activating TPM-bypass!1688client # [ 0.763827] clk: Disabling unused clocks1689lb # [ 0.753526] Key type .fscrypt registered1690worker1 # [ 0.749567] ima: Allocated hash algorithm: sha11691worker1 # [ 0.751715] ima: No architecture policies found1692lb # [ 0.756067] Key type fscrypt-provisioning registered1693worker2 # [ 0.749219] ima: Allocated hash algorithm: sha11694lb # [ 0.757147] ima: No TPM chip found, activating TPM-bypass!1695worker2 # [ 0.750153] ima: No architecture policies found1696worker1 # [ 0.753701] PM: Magic number: 10:579:5761697client # [ 0.768210] PM: genpd: Disabling unused power domains1698worker1 # [ 0.755363] RAS: Correctable Errors collector initialized.1699lb # [ 0.760063] ima: Allocated hash algorithm: sha11700lb # [ 0.760881] ima: No architecture policies found1701worker2 # [ 0.754219] PM: Magic number: 10:579:5761702lb # [ 0.763070] PM: Magic number: 10:579:5761703worker2 # [ 0.755912] RAS: Correctable Errors collector initialized.1704lb # [ 0.764750] RAS: Correctable Errors collector initialized.1705worker1 # [ 0.766388] clk: Disabling unused clocks1706worker2 # [ 0.763892] clk: Disabling unused clocks1707lb # [ 0.773696] clk: Disabling unused clocks1708worker1 # [ 0.769700] PM: genpd: Disabling unused power domains1709worker2 # [ 0.768226] PM: genpd: Disabling unused power domains1710lb # [ 0.778065] PM: genpd: Disabling unused power domains1711client # [ 0.932969] Freeing initrd memory: 29096K1712client # [ 0.936111] Freeing unused decrypted memory: 2028K1713client # [ 0.938816] Freeing unused kernel image (initmem) memory: 3644K1714client # [ 0.939947] Write protecting the kernel read-only data: 32768k1715client # [ 0.941859] Freeing unused kernel image (text/rodata gap) memory: 1208K1716client # [ 0.943477] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker2 # [ 0.927679] Freeing initrd memory: 29092K1718worker1 # [ 0.931276] Freeing initrd memory: 29092K1719worker2 # [ 0.930904] Freeing unused decrypted memory: 2028K1720worker1 # [ 0.934552] Freeing unused decrypted memory: 2028K1721lb # [ 0.939446] Freeing initrd memory: 29092K1722worker2 # [ 0.933629] Freeing unused kernel image (initmem) memory: 3644K1723worker1 # [ 0.937272] Freeing unused kernel image (initmem) memory: 3644K1724worker2 # [ 0.934696] Write protecting the kernel read-only data: 32768k1725worker1 # [ 0.938342] Write protecting the kernel read-only data: 32768k1726lb # [ 0.942661] Freeing unused decrypted memory: 2028K1727worker2 # [ 0.936593] Freeing unused kernel image (text/rodata gap) memory: 1208K1728worker1 # [ 0.940229] Freeing unused kernel image (text/rodata gap) memory: 1208K1729lb # [ 0.945387] Freeing unused kernel image (initmem) memory: 3644K1730worker2 # [ 0.938135] Freeing unused kernel image (rodata/data gap) memory: 752K1731worker1 # [ 0.941799] Freeing unused kernel image (rodata/data gap) memory: 752K1732lb # [ 0.946460] Write protecting the kernel read-only data: 32768k1733lb # [ 0.948388] Freeing unused kernel image (text/rodata gap) memory: 1208K1734lb # [ 0.949966] Freeing unused kernel image (rodata/data gap) memory: 752K1735client # [ 0.994840] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736client # [ 0.995931] Run /init as init process1737client # [ 1.006529] systemd[1]: Inserted module 'autofs4'1738worker2 # [ 0.989221] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739worker1 # [ 0.992780] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740worker2 # [ 0.990298] Run /init as init process1741worker1 # [ 0.993835] Run /init as init process1742lb # [ 1.001073] x86/mm: Checked W+X mappings: passed, no W+X pages found.1743lb # [ 1.002172] Run /init as init process1744worker2 # [ 1.000631] systemd[1]: Inserted module 'autofs4'1745worker1 # [ 1.004150] systemd[1]: Inserted module 'autofs4'1746client # [ 1.022647] fuse: init (API version 7.45)1747lb # [ 1.012449] systemd[1]: Inserted module 'autofs4'1748client # [ 1.028560] ACPI: \_SB_.GSIG: Enabled at IRQ 221749client # [ 1.030939] ACPI: \_SB_.GSIH: Enabled at IRQ 231750client # [ 1.033723] ACPI: \_SB_.GSIE: Enabled at IRQ 201751worker2 # [ 1.017142] fuse: init (API version 7.45)1752worker1 # [ 1.020607] fuse: init (API version 7.45)1753client # [ 1.035944] ACPI: \_SB_.GSIF: Enabled at IRQ 211754lb # [ 1.028900] fuse: init (API version 7.45)1755worker2 # [ 1.023126] ACPI: \_SB_.GSIG: Enabled at IRQ 221756worker1 # [ 1.026575] ACPI: \_SB_.GSIG: Enabled at IRQ 221757worker2 # [ 1.025648] ACPI: \_SB_.GSIH: Enabled at IRQ 231758worker1 # [ 1.029079] ACPI: \_SB_.GSIH: Enabled at IRQ 231759lb # [ 1.034810] ACPI: \_SB_.GSIG: Enabled at IRQ 221760worker2 # [ 1.028588] ACPI: \_SB_.GSIE: Enabled at IRQ 201761worker1 # [ 1.032646] ACPI: \_SB_.GSIE: Enabled at IRQ 201762lb # [ 1.037473] ACPI: \_SB_.GSIH: Enabled at IRQ 231763worker2 # [ 1.030827] ACPI: \_SB_.GSIF: Enabled at IRQ 211764worker1 # [ 1.035718] ACPI: \_SB_.GSIF: Enabled at IRQ 211765lb # [ 1.041407] ACPI: \_SB_.GSIE: Enabled at IRQ 201766lb # [ 1.043896] ACPI: \_SB_.GSIF: Enabled at IRQ 211767client # [ 1.067335] systemd[1]: Successfully made /usr/ read-only.1768worker2 # [ 1.058009] systemd[1]: Successfully made /usr/ read-only.1769worker1 # [ 1.066863] systemd[1]: Successfully made /usr/ read-only.1770lb # [ 1.074177] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.402889] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.408012] systemd[1]: Detected virtualization kvm.1773client # [ 1.408912] systemd[1]: Detected architecture x86-64.1774client # [ 1.409817] systemd[1]: Running in initrd.1775client # [ 1.411067] systemd[1]: Initializing machine ID from random generator.1776client # [ 1.412402] systemd[1]: Hostname set to <client>.1777worker2 # [ 1.393496] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker2 # [ 1.398541] systemd[1]: Detected virtualization kvm.1779worker2 # [ 1.399382] systemd[1]: Detected architecture x86-64.1780worker2 # [ 1.400270] systemd[1]: Running in initrd.1781worker2 # [ 1.401400] systemd[1]: Initializing machine ID from random generator.1782worker2 # [ 1.402579] systemd[1]: Hostname set to <worker2>.1783lb # [ 1.409830] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784lb # [ 1.414969] systemd[1]: Detected virtualization kvm.1785lb # [ 1.415869] systemd[1]: Detected architecture x86-64.1786lb # [ 1.416786] systemd[1]: Running in initrd.1787worker1 # [ 1.403489] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1788lb # [ 1.417989] systemd[1]: Initializing machine ID from random generator.1789lb # [ 1.419281] systemd[1]: Hostname set to <lb>.1790worker1 # [ 1.415271] systemd[1]: Detected virtualization kvm.1791worker1 # [ 1.417379] systemd[1]: Detected architecture x86-64.1792worker1 # [ 1.419480] systemd[1]: Running in initrd.1793worker1 # [ 1.422097] systemd[1]: Initializing machine ID from random generator.1794worker1 # [ 1.424959] systemd[1]: Hostname set to <worker1>.1795client # [ 1.606827] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker2 # [ 1.591601] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797lb # [ 1.629365] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798client # [ 1.647691] systemd[1]: Queued start job for default target Initrd Default Target.1799worker2 # [ 1.630932] systemd[1]: Queued start job for default target Initrd Default Target.1800client # [ 1.651609] systemd[1]: Created slice Slice /system/modprobe.1801client # [ 1.652813] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1802worker2 # [ 1.635637] systemd[1]: Created slice Slice /system/modprobe.1803client # [ 1.654239] systemd[1]: Expecting device /dev/disk/by-label/nixos...1804worker2 # [ 1.636765] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1805client # [ 1.655350] systemd[1]: Reached target Path Units.1806client # [ 1.656215] systemd[1]: Reached target Slice Units.1807worker2 # [ 1.638243] systemd[1]: Expecting device /dev/disk/by-label/nixos...1808client # [ 1.657055] systemd[1]: Reached target Swaps.1809worker2 # [ 1.639327] systemd[1]: Reached target Path Units.1810client # [ 1.657878] systemd[1]: Reached target Timer Units.1811worker2 # [ 1.640263] systemd[1]: Reached target Slice Units.1812worker2 # [ 1.641123] systemd[1]: Reached target Swaps.1813client # [ 1.658903] systemd[1]: Listening on D-Bus System Message Bus Socket.1814worker2 # [ 1.641936] systemd[1]: Reached target Timer Units.1815client # [ 1.660131] systemd[1]: Listening on Journal Socket (/dev/log).1816worker2 # [ 1.642954] systemd[1]: Listening on D-Bus System Message Bus Socket.1817client # [ 1.661395] systemd[1]: Listening on Journal Sockets.1818worker2 # [ 1.644170] systemd[1]: Listening on Journal Socket (/dev/log).1819client # [ 1.662366] systemd[1]: Listening on udev Control Socket.1820worker1 # [ 1.646426] systemd[1]: bpf-restrict-fs: LSM BPF program attached1821worker2 # [ 1.645335] systemd[1]: Listening on Journal Sockets.1822client # [ 1.663361] systemd[1]: Listening on udev Kernel Socket.1823worker2 # [ 1.646335] systemd[1]: Listening on udev Control Socket.1824client # [ 1.664307] systemd[1]: Reached target Socket Units.1825worker2 # [ 1.647347] systemd[1]: Listening on udev Kernel Socket.1826worker2 # [ 1.648324] systemd[1]: Reached target Socket Units.1827client # [ 1.666131] systemd[1]: Starting Create List of Static Device Nodes...1828worker2 # [ 1.650137] systemd[1]: Starting Create List of Static Device Nodes...1829client # [ 1.670048] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1830worker2 # [ 1.654081] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1831client # [ 1.676284] systemd[1]: Starting Load Kernel Module configfs...1832worker2 # [ 1.660889] systemd[1]: Starting Load Kernel Module configfs...1833client # [ 1.690898] systemd[1]: Starting Journal Service...1834worker2 # [ 1.676626] systemd[1]: Starting Journal Service...1835lb # [ 1.684957] systemd[1]: Queued start job for default target Initrd Default Target.1836lb # [ 1.689450] systemd[1]: Created slice Slice /system/modprobe.1837lb # [ 1.690685] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1838client # [ 1.702307] systemd[1]: Starting Load Kernel Modules...1839lb # [ 1.692136] systemd[1]: Expecting device /dev/disk/by-label/nixos...1840lb # [ 1.693267] systemd[1]: Reached target Path Units.1841lb # [ 1.694190] systemd[1]: Reached target Slice Units.1842lb # [ 1.695144] systemd[1]: Reached target Swaps.1843lb # [ 1.695967] systemd[1]: Reached target Timer Units.1844lb # [ 1.697016] systemd[1]: Listening on D-Bus System Message Bus Socket.1845worker2 # [ 1.690091] systemd[1]: Starting Load Kernel Modules...1846lb # [ 1.698266] systemd[1]: Listening on Journal Socket (/dev/log).1847lb # [ 1.699448] systemd[1]: Listening on Journal Sockets.1848lb # [ 1.700467] systemd[1]: Listening on udev Control Socket.1849lb # [ 1.701530] systemd[1]: Listening on udev Kernel Socket.1850lb # [ 1.702494] systemd[1]: Reached target Socket Units.1851lb # [ 1.704358] systemd[1]: Starting Create List of Static Device Nodes...1852worker1 # [ 1.702551] systemd[1]: Queued start job for default target Initrd Default Target.1853lb # [ 1.708119] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1854client # [ 1.709294] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1855worker1 # [ 1.707106] systemd[1]: Created slice Slice /system/modprobe.1856worker1 # [ 1.708230] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1857worker1 # [ 1.709778] systemd[1]: Expecting device /dev/disk/by-label/nixos...1858worker2 # [ 1.695529] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1859worker1 # [ 1.710923] systemd[1]: Reached target Path Units.1860worker1 # [ 1.711792] systemd[1]: Reached target Slice Units.1861worker1 # [ 1.712715] systemd[1]: Reached target Swaps.1862lb # [ 1.717246] systemd[1]: Starting Load Kernel Module configfs...1863worker1 # [ 1.713527] systemd[1]: Reached target Timer Units.1864client # [ 1.729148] netfs: FS-Cache loaded1865worker1 # [ 1.714543] systemd[1]: Listening on D-Bus System Message Bus Socket.1866worker2 # [ 1.711687] netfs: FS-Cache loaded1867worker1 # [ 1.715785] systemd[1]: Listening on Journal Socket (/dev/log).1868worker1 # [ 1.716983] systemd[1]: Listening on Journal Sockets.1869worker1 # [ 1.718020] systemd[1]: Listening on udev Control Socket.1870worker1 # [ 1.719069] systemd[1]: Listening on udev Kernel Socket.1871client # [ 1.734040] 9pnet: Installing 9P2000 support1872worker1 # [ 1.720034] systemd[1]: Reached target Socket Units.1873worker2 # [ 1.718759] 9pnet: Installing 9P2000 support1874worker1 # [ 1.721947] systemd[1]: Starting Create List of Static Device Nodes...1875worker1 # [ 1.725763] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1876lb # [ 1.731240] systemd[1]: Starting Journal Service...1877client # [ 1.744019] systemd[1]: Starting Coldplug All udev Devices...1878worker2 # [ 1.730276] systemd[1]: Starting Coldplug All udev Devices...1879worker1 # [ 1.734750] systemd[1]: Starting Load Kernel Module configfs...1880lb # [ 1.747166] systemd[1]: Starting Load Kernel Modules...1881worker2 # [ 1.740787] systemd-journald[67]: Collecting audit messages is disabled.1882client # [ 1.759335] systemd[1]: Finished Create List of Static Device Nodes.1883worker2 # [ 1.744302] systemd[1]: Finished Create List of Static Device Nodes.1884worker1 # [ 1.749755] systemd[1]: Starting Journal Service...1885worker2 # [ 1.749137] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1886client # [ 1.767274] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1887client # [ 1.772685] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1888lb # [ 1.752138] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1889worker2 # [ 1.757611] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1890worker1 # [ 1.762029] systemd[1]: Starting Load Kernel Modules...1891client # [ 1.777617] systemd[1]: modprobe@configfs.service: Deactivated successfully.1892lb # [ 1.769112] netfs: FS-Cache loaded1893client # [ 1.779166] systemd-journald[67]: Collecting audit messages is disabled.1894worker2 # [ 1.763633] systemd[1]: modprobe@configfs.service: Deactivated successfully.1895lb # [ 1.773976] 9pnet: Installing 9P2000 support1896client # [ 1.784595] systemd[1]: Finished Load Kernel Module configfs.1897worker2 # [ 1.769598] systemd[1]: Finished Load Kernel Module configfs.1898client # [ 1.790658] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1899worker2 # [ 1.775627] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1900worker1 # [ 1.768768] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1901lb # [ 1.785125] systemd[1]: Starting Coldplug All udev Devices...1902client # [ 1.794693] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1903worker2 # [ 1.777589] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1904worker1 # [ 1.786280] netfs: FS-Cache loaded1905worker2 # [ 1.784276] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1906client # [ 1.802277] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1907client # [ 1.803576] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1908worker2 # [ 1.789274] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1909worker1 # [ 1.793701] 9pnet: Installing 9P2000 support1910lb # [ 1.801148] systemd[1]: Finished Create List of Static Device Nodes.1911lb # [ 1.806699] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1912worker1 # [ 1.805770] systemd[1]: Starting Coldplug All udev Devices...1913lb # [ 1.814526] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1914lb # [ 1.819018] systemd-journald[67]: Collecting audit messages is disabled.1915lb # [ 1.821552] systemd[1]: modprobe@configfs.service: Deactivated successfully.1916worker2 # [ 1.814700] systemd[1]: Finished Load Kernel Modules.1917client # [ 1.833387] systemd[1]: Finished Load Kernel Modules.1918worker1 # [ 1.819292] systemd-journald[67]: Collecting audit messages is disabled.1919worker1 # [ 1.821068] systemd[1]: Finished Create List of Static Device Nodes.1920lb # [ 1.829444] systemd[1]: Finished Load Kernel Module configfs.1921worker2 # [ 1.822401] systemd[1]: Starting Apply Kernel Variables...1922client # [ 1.840403] systemd[1]: Starting Apply Kernel Variables...1923worker1 # [ 1.828420] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1924lb # [ 1.833590] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1925lb # [ 1.838579] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1926worker1 # [ 1.835758] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1927worker2 # [ 1.833315] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1928client # [ 1.852341] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1929worker1 # [ 1.842110] systemd[1]: modprobe@configfs.service: Deactivated successfully.1930lb # [ 1.846317] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1931lb # [ 1.847801] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1932worker2 # [ 1.841390] systemd[1]: Starting Create Static Device Nodes in /dev...1933client # [ 1.861396] systemd[1]: Starting Create Static Device Nodes in /dev...1934worker1 # [ 1.850350] systemd[1]: Finished Load Kernel Module configfs.1935worker1 # [ 1.854823] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1936worker1 # [ 1.857292] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1937worker2 # [ 1.726216] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.860707] systemd[1]: Finished Apply Kernel Variables.1938worker2 # 1939worker2 # [ 1.728490] systemd-modules-load[68]: Inserted module 'virtio_balloon'1940client # [ 1.881319] systemd[1]: Finished Apply Kernel Variables.1941worker1 # [ 1.866715] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1942worker2 # [ 1.731326] systemd-modules-load[68]: Inserted module 'virtio_gpu'1943worker2 # [ 1.866391] systemd[1]: Started Journal Service.1944worker1 # [ 1.870009] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1945lb # [ 1.878635] systemd[1]: Finished Load Kernel Modules.1946client # [ 1.895282] systemd[1]: Started Journal Service.1947worker2 # [ 1.745191] systemd[1]: Finished Create Static Device Nodes in /dev.1948client # [ 1.762240] systemd-modules-load[68]: Inserted module 'dm_mod'1949lb # [ 1.887258] systemd[1]: Starting Apply Kernel Variables...1950client # [ 1.764473] systemd-modules-load[68]: Inserted module 'virtio_balloon'1951worker2 # [ 1.748085] systemd[1]: Reached target Preparation for Local File Systems.1952worker2 # [ 1.749073] systemd[1]: Reached target Local File Systems.1953client # [ 1.767312] systemd-modules-load[68]: Inserted module 'virtio_gpu'1954client # [ 1.768374] systemd[1]: Finished Create Static Device Nodes in /dev.1955worker2 # [ 1.752985] systemd[1]: Starting Create System Files and Directories...1956client # [ 1.772229] systemd[1]: Reached target Preparation for Local File Systems.1957client # [ 1.774337] systemd[1]: Reached target Local File Systems.1958lb # [ 1.899715] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1959worker2 # [ 1.760071] systemd[1]: Starting Rule-based Manager for Device Events and Files...1960client # [ 1.778125] systemd[1]: Starting Create System Files and Directories...1961worker1 # [ 1.900271] systemd[1]: Finished Load Kernel Modules.1962client # [ 1.783187] systemd[1]: Starting Rule-based Manager for Device Events and Files...1963lb # [ 1.911526] systemd[1]: Starting Create Static Device Nodes in /dev...1964worker1 # [ 1.907909] systemd[1]: Starting Apply Kernel Variables...1965worker2 # [ 1.782083] systemd[1]: Finished Create System Files and Directories.1966worker1 # [ 1.920301] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1967lb # [ 1.928640] systemd[1]: Finished Apply Kernel Variables.1968worker1 # [ 1.929893] systemd[1]: Starting Create Static Device Nodes in /dev...1969client # [ 1.815165] systemd[1]: Finished Create System Files and Directories.1970lb # [ 1.940147] systemd[1]: Started Journal Service.1971lb # [ 1.809243] systemd-modules-load[68]: Inserted module 'dm_mod'1972lb # [ 1.811385] systemd-modules-load[68]: Inserted module 'virtio_balloon'1973worker2 # [ 1.805257] systemd-udevd[83]: Using default interface naming scheme 'v261'.1974worker1 # [ 1.943043] systemd[1]: Started Journal Service.1975lb # [ 1.815318] systemd-modules-load[68]: Inserted module 'virtio_gpu'1976worker1 # [ 1.808824] systemd-modules-load[68]: Inserted module 'dm_mod'1977worker1 # [ 1.812966] systemd-modules-load[68]: Inserted module 'virtio_balloon'1978worker1 # [ 1.814042] systemd-modules-load[68]: Inserted module 'virtio_gpu'1979worker1 # [ 1.820900] systemd[1]: Finished Apply Kernel Variables.1980lb # [ 1.828153] systemd[1]: Finished Create Static Device Nodes in /dev.1981lb # [ 1.829183] systemd[1]: Reached target Preparation for Local File Systems.1982lb # [ 1.830168] systemd[1]: Reached target Local File Systems.1983client # [ 1.839327] systemd-udevd[81]: Using default interface naming scheme 'v261'.1984lb # [ 1.833946] systemd[1]: Starting Create System Files and Directories...1985worker2 # [ 1.830070] systemd[1]: Started Rule-based Manager for Device Events and Files.1986lb # [ 1.839143] systemd[1]: Starting Rule-based Manager for Device Events and Files...1987worker1 # [ 1.836197] systemd[1]: Finished Create Static Device Nodes in /dev.1988worker1 # [ 1.839083] systemd[1]: Reached target Preparation for Local File Systems.1989worker1 # [ 1.840050] systemd[1]: Reached target Local File Systems.1990worker1 # [ 1.843064] systemd[1]: Starting Create System Files and Directories...1991worker1 # [ 1.848141] systemd[1]: Starting Rule-based Manager for Device Events and Files...1992client # [ 1.864345] systemd[1]: Started Rule-based Manager for Device Events and Files.1993lb # [ 1.868208] systemd[1]: Finished Create System Files and Directories.1994worker1 # [ 1.875451] systemd[1]: Finished Create System Files and Directories.1995lb # [ 1.891835] systemd-udevd[83]: Using default interface naming scheme 'v261'.1996worker2 # [ 1.888682] systemd[1]: Finished Coldplug All udev Devices.1997worker2 # [ 1.889627] systemd[1]: Reached target System Initialization.1998worker2 # [ 1.890410] systemd[1]: Reached target Basic System.1999worker1 # [ 1.897282] systemd-udevd[83]: Using default interface naming scheme 'v261'.2000client # [ 1.911986] systemd[1]: Finished Coldplug All udev Devices.
loading 2,234 hidden lines…
4235worker2 # [ 22.958569] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker1 # [ 23.023062] nix-grpc-daemon[785]: ts=2026-09-21T09:34:33Z level=info event=scheduler_connected addr=lb:500514237worker1 # [ 23.026990] nix-grpc-daemon[785]: ts=2026-09-21T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker2 # [ 23.159867] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514239worker2 # [ 23.164299] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker1 # [ 23.227937] nix-grpc-daemon[785]: ts=2026-09-21T09:34:33Z level=info event=scheduler_connected addr=lb:500514241worker1 # [ 23.231900] nix-grpc-daemon[785]: ts=2026-09-21T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker2 # [ 23.365377] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514243worker2 # [ 23.369732] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker1 # [ 23.433109] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514245worker1 # [ 23.437537] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker2 # [ 23.570859] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514247worker2 # [ 23.575344] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker1 # [ 23.638430] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514249worker1 # [ 23.642342] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker2 # [ 23.776465] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 23.780812] nix-grpc-daemon[786]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker1 # [ 23.844195] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514253worker1 # [ 23.848093] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4254worker2 # [ 23.982436] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514255worker2 # [ 23.986625] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker1 # [ 24.049390] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514257worker1 # [ 24.053367] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker2 # [ 24.187681] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514259worker2 # [ 24.191989] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker1 # [ 24.254250] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_connected addr=lb:500514261worker1 # [ 24.258162] nix-grpc-daemon[785]: ts=2026-09-21T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker2 # [ 24.393420] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514263worker2 # [ 24.397734] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker1 # [ 24.459282] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514265worker1 # [ 24.463255] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4266worker2 # [ 24.598895] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514267worker2 # [ 24.603355] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4268worker1 # [ 24.664129] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514269worker1 # [ 24.668266] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4270worker2 # [ 24.804467] nix-grpc-daemon[786]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514271worker2 # [ 24.808823] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker1 # [ 24.870787] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514273worker1 # [ 24.873884] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4274worker2 # [ 25.011197] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514275worker2 # [ 25.014692] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4276worker1 # [ 25.071670] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514277worker1 # [ 25.075532] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker2 # [ 25.212214] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514279worker2 # [ 25.216688] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4280worker1 # [ 25.276722] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_connected addr=lb:500514281worker1 # [ 25.280674] nix-grpc-daemon[785]: ts=2026-09-21T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker2 # [ 25.417882] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514283worker2 # [ 25.422323] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284worker1 # [ 25.481686] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514285worker1 # [ 25.485697] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4286worker2 # [ 25.623602] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514287worker2 # [ 25.627974] nix-grpc-daemon[786]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4288worker1 # [ 25.686848] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514289worker1 # [ 25.690913] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290worker2 # [ 25.829216] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514291worker2 # [ 25.833632] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4292worker1 # [ 25.892740] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514293worker1 # [ 25.896499] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4294worker2 # [ 26.036158] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514295worker2 # [ 26.039782] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4296worker1 # [ 26.094740] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514297worker1 # [ 26.098734] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4298worker2 # [ 26.237239] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514299worker2 # [ 26.241613] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4300worker1 # [ 26.299766] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_connected addr=lb:500514301worker1 # [ 26.304918] nix-grpc-daemon[785]: ts=2026-09-21T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4302worker2 # [ 26.442701] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514303worker2 # [ 26.447116] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4304worker1 # [ 26.505847] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514305worker1 # [ 26.510194] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4306worker2 # [ 26.648230] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514307worker2 # [ 26.652644] nix-grpc-daemon[786]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4308worker1 # [ 26.710727] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514309worker1 # [ 26.715132] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310worker2 # [ 26.853792] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514311worker2 # [ 26.858201] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4312worker1 # [ 26.917393] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514313worker1 # [ 26.920520] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4314worker2 # [ 27.060376] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514315worker2 # [ 27.063304] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4316worker1 # [ 27.118491] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514317worker1 # [ 27.122500] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4318worker2 # [ 27.261772] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514319worker2 # [ 27.266220] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4320worker1 # [ 27.323309] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_connected addr=lb:500514321worker1 # [ 27.327075] nix-grpc-daemon[785]: ts=2026-09-21T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4322worker2 # [ 27.467280] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514323worker2 # [ 27.471679] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4324worker1 # [ 27.528058] nix-grpc-daemon[785]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514325worker1 # [ 27.532154] nix-grpc-daemon[785]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4326worker2 # [ 27.672977] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514327worker2 # [ 27.677425] nix-grpc-daemon[786]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4328worker1 # [ 27.733254] nix-grpc-daemon[785]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514329worker1 # [ 27.737252] nix-grpc-daemon[785]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4330worker2 # [ 27.878340] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514331worker2 # [ 27.882691] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4332worker1 # [ 27.939731] nix-grpc-daemon[785]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514333worker1 # [ 27.942818] nix-grpc-daemon[785]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4334worker2 # [ 28.084803] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514335worker2 # [ 28.089144] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4336worker1 # [ 28.141169] nix-grpc-daemon[785]: ts=2026-09-21T09:34:38Z level=info event=scheduler_connected addr=lb:500514337worker1 # [ 28.145157] nix-grpc-daemon[785]: ts=2026-09-21T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4338worker2 # [ 28.290501] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514339worker2 # [ 28.294837] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4340worker1 # [ 28.345952] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514341worker1 # [ 28.349964] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4342worker2 # [ 28.496153] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514343worker2 # [ 28.500499] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4344worker1 # [ 28.550926] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514345worker1 # [ 28.554940] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4346worker2 # [ 28.701684] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514347worker2 # [ 28.705605] nix-grpc-daemon[786]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4348worker1 # [ 28.755803] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514349worker1 # [ 28.759774] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4350worker2 # [ 28.907936] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514351worker2 # [ 28.911381] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4352worker1 # [ 28.962564] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514353worker1 # [ 28.965629] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4354worker2 # [ 29.110150] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514355worker2 # [ 29.113934] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4356worker1 # [ 29.162915] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_connected addr=lb:500514357worker1 # [ 29.166990] nix-grpc-daemon[785]: ts=2026-09-21T09:34:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4358worker2 # [ 29.315454] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514359worker2 # [ 29.319288] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4360worker1 # [ 29.367751] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514361worker1 # [ 29.371564] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4362worker2 # [ 29.520507] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514363worker2 # [ 29.524412] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4364worker1 # [ 29.572452] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514365worker1 # [ 29.576409] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4366worker2 # [ 29.725700] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514367worker2 # [ 29.729575] nix-grpc-daemon[786]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4368worker1 # [ 29.777499] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514369worker1 # [ 29.781724] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4370worker2 # [ 29.930435] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514371worker2 # [ 29.934322] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4372worker1 # [ 29.982644] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514373worker1 # [ 29.985172] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4374worker2 # [ 30.136377] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514375worker2 # [ 30.140124] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4376worker1 # [ 30.185577] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_connected addr=lb:500514377worker1 # [ 30.189453] nix-grpc-daemon[785]: ts=2026-09-21T09:34:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4378worker2 # [ 30.341407] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514379worker2 # [ 30.345270] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4380worker1 # [ 30.390295] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514381worker1 # [ 30.394255] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4382worker2 # [ 30.546413] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514383worker2 # [ 30.550286] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4384worker1 # [ 30.595443] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514385worker1 # [ 30.599410] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4386worker2 # [ 30.751439] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514387worker2 # [ 30.755289] nix-grpc-daemon[786]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4388worker1 # [ 30.800458] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514389worker1 # [ 30.804538] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4390worker2 # [ 30.956191] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514391worker2 # [ 30.960082] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4392worker1 # [ 31.007177] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514393worker1 # [ 31.010377] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4394worker2 # [ 31.162583] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514395worker2 # [ 31.165670] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4396lb # [ 31.195104] rustfs-bucket-start[1043]: mb s3://farm4397lb # [ 31.200768] systemd[1]: Finished rustfs-bucket.service.4398lb # [ 31.207308] systemd[1]: Starting niks3 server...4399worker1 # [ 31.207979] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_connected addr=lb:500514400worker1 # [ 31.211989] nix-grpc-daemon[785]: ts=2026-09-21T09:34:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4401worker2 # [ 31.363830] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514402worker2 # [ 31.367641] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4403worker1 # [ 31.413171] nix-grpc-daemon[785]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514404worker1 # [ 31.416973] nix-grpc-daemon[785]: ts=2026-09-21T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4405lb # [ 31.516710] postgres[1064]: [1064] ERROR: relation "goose_db_version" does not exist at character 364406lb # [ 31.517939] postgres[1064]: [1064] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4407lb # [ 31.563806] niks3-server[1059]: 2026/09/21 09:34:42 OK 20241026095416_initial_model.sql (24.62ms)4408lb # [ 31.568552] niks3-server[1059]: 2026/09/21 09:34:42 OK 20251210153512_drop_unused_gin_index.sql (2.79ms)4409lb # [ 31.573740] niks3-server[1059]: 2026/09/21 09:34:42 OK 20251218171726_add_pins.sql (4.92ms)4410worker2 # [ 31.568939] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514411lb # [ 31.580183] niks3-server[1059]: 2026/09/21 09:34:42 OK 20260628120000_add_object_size_and_stats.sql (6.67ms)4412worker2 # [ 31.572872] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4413lb # [ 31.584966] niks3-server[1059]: 2026/09/21 09:34:42 OK 20260905000000_add_claims.sql (4.72ms)4414lb # [ 31.589602] niks3-server[1059]: 2026/09/21 09:34:42 OK 20260920000000_drop_claims.sql (4.71ms)4415lb # [ 31.590982] niks3-server[1059]: 2026/09/21 09:34:42 goose: successfully migrated database to version: 202609200000004416lb # [ 31.597458] niks3-server[1059]: 2026/09/21 09:34:42 OK 1_commit_pending_closure.sql (6.5ms)4417lb # [ 31.600378] niks3-server[1059]: 2026/09/21 09:34:42 OK 2_object_stats_trigger.sql (1.68ms)4418lb # [ 31.601590] niks3-server[1059]: 2026/09/21 09:34:42 goose: up to current file version: 24419lb # [ 31.609809] niks3-server[1059]: 2026/09/21 09:34:42 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4420worker1 # [ 31.615111] nix-grpc-daemon[785]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514421worker1 # [ 31.619119] nix-grpc-daemon[785]: ts=2026-09-21T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4422lb # [ 31.715311] niks3-server[1059]: 2026/09/21 09:34:42 INFO Read proxy enabled — serving cache objects from S34423lb # [ 31.717301] niks3-server[1059]: 2026/09/21 09:34:42 INFO Using socket-activated listener address=0.0.0.0:57514424lb # [ 31.723260] niks3-server[1059]: 2026/09/21 09:34:42 INFO systemd watchdog enabled interval=15s4425lb # [ 31.724433] systemd[1]: Started niks3 server.4426lb # [ 31.725141] systemd[1]: Reached target Multi-User System.4427lb # [ 31.727517] systemd[1]: Startup finished in 872ms (kernel) + 4.099s (initrd) + 26.755s (userspace) = 31.726s.4428lb # [ 31.729516] niks3-server[1059]: 2026/09/21 09:34:42 INFO Starting HTTP server address=0.0.0.0:57514429lb # [ 31.735225] niks3-server[1059]: 2026/09/21 09:34:42 INFO lead: acquired remote=192.168.1.4:567764430worker2 # [ 31.735156] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_take_over4431worker2 # [ 31.773943] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514432worker2 # [ 31.777771] nix-grpc-daemon[786]: ts=2026-09-21T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4433worker1 # [ 31.820576] nix-grpc-daemon[785]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514434worker1 # [ 31.824387] nix-grpc-daemon[785]: ts=2026-09-21T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4435worker2 # [ 31.978474] nix-grpc-daemon[786]: ts=2026-09-21T09:34:43Z level=info event=scheduler_connected addr=lb:500514436worker2 # [ 32.017547] nix-grpc-daemon[786]: ts=2026-09-21T09:34:43Z level=info event=oidc_jwks provider=mock keys=14437worker1 # [ 32.023354] nix-grpc-daemon[785]: ts=2026-09-21T09:34:42Z level=info event=scheduler_connected addr=lb:500514438worker2 # [ 32.023456] nix-grpc-daemon[786]: ts=2026-09-21T09:34:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:450544439worker2 # [ 32.027930] nix-grpc-daemon[786]: ts=2026-09-21T09:34:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:450544440worker2 # [ 32.032115] nix-grpc-daemon[786]: ts=2026-09-21T09:34:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04441worker2 # [ 32.038210] nix-grpc-daemon[786]: ts=2026-09-21T09:34:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04442lb: (finished: waiting for unit niks3.service, in 20.28 seconds)4443lb: waiting for TCP port 5751 on localhost4444lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4445lb: (finished: waiting for TCP port 5751 on localhost, in 0.11 seconds)4446lb: waiting for unit envoy.service4447lb: (finished: waiting for unit envoy.service, in 0.08 seconds)4448lb: waiting for TCP port 50051 on localhost4449lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4450lb # [ 32.494879] envoy[900]: conn peer=[::1]:54180 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34451lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4452worker2 # [ 32.639616] systemd[1]: Starting scheduler-token.service...4453worker2 # [ 32.722508] systemd[1]: scheduler-token.service: Deactivated successfully.4454worker2 # [ 32.724080] systemd[1]: Finished scheduler-token.service.4455worker2 # [ 32.725607] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 85ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4456lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4457lb: must succeed: curl -sf localhost:9901/clusters4458lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)4459lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.11 seconds)4460subtest: one node schedules and both builders hold a WorkerSession on it4461lb: must succeed: curl -sf localhost:9901/clusters4462lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4463worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4464worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4465lb: must succeed: curl -sf localhost:9901/clusters4466lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4467lb: must succeed: curl -sf localhost:9901/clusters4468lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4469lb: must succeed: curl -sf localhost:9901/clusters4470lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4471lb: must succeed: curl -sf localhost:9901/clusters4472lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4473worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4474worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4475worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4476worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4477lb: must succeed: curl -sf localhost:9901/clusters4478lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4479lb: must succeed: curl -sf localhost:9901/clusters4480lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4481worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4482worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4483worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4484worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4485worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4486worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4487worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4488worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4489(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.82 seconds)4490subtest: balancer auth: client cert, bearer token, nothing, unknown CN4491worker2 # [ 34.059642] nix-grpc-daemon[786]: ts=2026-09-21T09:34:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:604984492worker2 # [ 34.074565] systemd[1]: Started Nix Daemon.4493worker2 # [ 34.319275] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4494worker2 # [ 34.348504] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 1/5)4495worker2 # [ 34.445190] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 109 ms (attempt 2/5)4496worker2 # [ 34.559055] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 377 ms (attempt 3/5)4497worker2 # [ 34.941998] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 785 ms (attempt 4/5)4498worker2 # [ 35.737951] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4499worker2 # [ 35.741924] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4500worker1 # [ 35.793915] nix-grpc-daemon[785]: ts=2026-09-21T09:34:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:546664501worker1 # [ 35.808568] systemd[1]: Started Nix Daemon.4502worker1 # [ 36.069339] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4503worker1 # [ 36.094620] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)4504worker1 # [ 36.187589] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 2/5)4505worker1 # [ 36.209769] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 197 ms (attempt 3/5)4506worker1 # [ 36.412496] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 333 ms (attempt 4/5)4507worker1 # [ 36.749789] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4508worker1 # [ 36.754796] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4509worker1 # [ 36.776852] nix-grpc-daemon[785]: ts=2026-09-21T09:34:47Z level=debug event=path_info_miss path=d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test error="error: path '/nix/store/d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test' is required, but there is no substituter that can build it"4510worker1 # [ 36.790211] nix-grpc-daemon[785]: ts=2026-09-21T09:34:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:546664511worker1 # [ 36.800297] nix-grpc-daemon[785]: ts=2026-09-21T09:34:47Z level=debug event=path_info_miss path=d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test error="error: path '/nix/store/d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test' is required, but there is no substituter that can build it"4512lb # [ 36.839322] envoy[900]: conn peer=[2001:db8:1::1]:56432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=27874513client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4514client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)4515client: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4516client: (finished: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)4517worker2 # [ 37.240830] nix-grpc-daemon[786]: ts=2026-09-21T09:34:48Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:604984518worker2 # [ 37.254718] nix-grpc-daemon[786]: ts=2026-09-21T09:34:48Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:604984519worker2 # [ 37.272760] nix-grpc-daemon[786]: ts=2026-09-21T09:34:48Z level=debug event=path_info_miss path=d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test error="error: path '/nix/store/d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test' is required, but there is no substituter that can build it"4520worker1 # [ 37.294595] nix-grpc-daemon[785]: ts=2026-09-21T09:34:47Z level=info event=oidc_jwks provider=mock keys=14521worker1 # [ 37.299693] nix-grpc-daemon[785]: ts=2026-09-21T09:34:47Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:546664522worker1 # [ 37.308731] nix-grpc-daemon[785]: ts=2026-09-21T09:34:47Z level=debug event=path_info_miss path=d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test error="error: path '/nix/store/d51l6nlmnlvhh06z9qj4lgflhcwyaryj-nixos-system-client-test' is required, but there is no substituter that can build it"4523client: must succeed: rm -r /var/lib/nix-grpc-store4524client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4525worker1 # [ 37.618499] nix-grpc-daemon[785]: ts=2026-09-21T09:34:48Z level=info event=denied method=QueryMissing cn=- role=none4526worker1 # [ 37.893608] nix-grpc-daemon[785]: ts=2026-09-21T09:34:48Z level=info event=denied method=QueryMissing cn=stranger role=none4527(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.25 seconds)4528subtest: DAG build is scheduled across workers and lands in the cache4529client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24530worker1 # [ 38.191606] nix-grpc-daemon[785]: ts=2026-09-21T09:34:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:546664531client # don't know how to build these paths:4532client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4533worker1 # [ 38.211269] nix-grpc-daemon[785]: ts=2026-09-21T09:34:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664534worker2 # [ 38.247158] nix-grpc-daemon[786]: ts=2026-09-21T09:34:49Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04535worker2 # [ 38.251913] nix-grpc-daemon[786]: ts=2026-09-21T09:34:49Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14536worker1 # [ 38.304975] nix-grpc-daemon[785]: ts=2026-09-21T09:34:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664537worker1 # [ 38.330255] nix-grpc-daemon[785]: ts=2026-09-21T09:34:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664538client # copying 1 paths to 'lb:50051'...4539worker1 # [ 38.345812] nix-grpc-daemon[785]: ts=2026-09-21T09:34:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546664540client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4541worker1 # [ 38.375255] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4542worker1 # [ 38.405510] nix-grpc-daemon[785]: ts=2026-09-21T09:34:49Z level=info event=niks3_push_started pid=8644543worker1 # [ 38.757218] nix-grpc-daemon[864]: time=2026-09-21T09:34:49.408Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4544worker1 # [ 38.999102] nix-daemon[851]: accepted connection from pid 877, user nix-grpc-daemon (trusted)4545worker1 # [ 39.015068] nix-daemon[851]: reaped child process 880, status = succeeded4546worker1 # [ 39.217329] nix-daemon[851]: accepted connection from pid 882, user nix-grpc-daemon (trusted)4547worker1 # [ 39.230991] nix-daemon[851]: reaped child process 885, status = succeeded4548lb # [ 39.255679] niks3-server[1059]: 2026/09/21 09:34:50 INFO Received uploads request method=POST path=/api/pending_closures4549worker1 # [ 39.281293] nix-grpc-daemon[864]: time=2026-09-21T09:34:49.936Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4550worker1 # [ 39.283731] nix-grpc-daemon[864]: time=2026-09-21T09:34:49.939Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4551lb # [ 39.336936] niks3-server[1059]: 2026/09/21 09:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4552lb # [ 39.355952] niks3-server[1059]: 2026/09/21 09:34:50 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4553lb # [ 39.364176] niks3-server[1059]: 2026/09/21 09:34:50 INFO Signed narinfos id=1 count=14554worker1 # [ 39.362883] nix-grpc-daemon[864]: time=2026-09-21T09:34:50.018Z level=INFO msg="Uploading 1 narinfos"4555lb # [ 39.373606] niks3-server[1059]: 2026/09/21 09:34:50 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4556lb # [ 39.380050] niks3-server[1059]: 2026/09/21 09:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4557lb # [ 39.392502] niks3-server[1059]: 2026/09/21 09:34:50 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4558lb # [ 39.396823] niks3-server[1059]: 2026/09/21 09:34:50 INFO Completed upload id=14559worker1 # [ 39.393698] nix-grpc-daemon[864]: time=2026-09-21T09:34:50.049Z level=INFO msg="Upload complete. (611ms)"4560worker1 # [ 39.400304] nix-grpc-daemon[785]: ts=2026-09-21T09:34:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 paths=1 nar_bytes_in=5044561worker1 # [ 39.407128] nix-grpc-daemon[785]: ts=2026-09-21T09:34:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664562worker1 # [ 39.422997] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4563worker1 # [ 39.427450] nix-daemon[851]: reaped child process 862, status = succeeded4564client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4565worker1 # [ 39.437654] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4566client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4567client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4568client # farm-a> LOG-farm-a-t14569worker1 # [ 40.467494] nix-daemon[851]: reaped child process 889, status = succeeded4570worker1 # [ 40.660462] nix-daemon[851]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4571worker1 # [ 40.674061] nix-daemon[851]: reaped child process 909, status = succeeded4572lb # [ 40.699467] niks3-server[1059]: 2026/09/21 09:34:51 INFO Received uploads request method=POST path=/api/pending_closures4573worker1 # [ 40.707688] nix-grpc-daemon[864]: time=2026-09-21T09:34:51.363Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4574worker1 # [ 40.709388] nix-grpc-daemon[864]: time=2026-09-21T09:34:51.364Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4575lb # [ 40.747434] niks3-server[1059]: 2026/09/21 09:34:51 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4576lb # [ 40.757607] niks3-server[1059]: 2026/09/21 09:34:51 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4577lb # [ 40.766395] niks3-server[1059]: 2026/09/21 09:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4578lb # [ 40.770980] niks3-server[1059]: 2026/09/21 09:34:51 INFO Signed narinfos id=2 count=14579worker1 # [ 40.770592] nix-grpc-daemon[864]: time=2026-09-21T09:34:51.425Z level=INFO msg="Uploading 1 narinfos"4580lb # [ 40.778527] niks3-server[1059]: 2026/09/21 09:34:51 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4581lb # [ 40.787611] niks3-server[1059]: 2026/09/21 09:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4582lb # [ 40.790578] niks3-server[1059]: 2026/09/21 09:34:51 INFO Completed upload id=24583lb # [ 40.793119] niks3-server[1059]: 2026/09/21 09:34:51 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4584worker1 # [ 40.788608] nix-grpc-daemon[864]: time=2026-09-21T09:34:51.444Z level=INFO msg="Upload complete. (338ms)"4585worker1 # [ 40.796451] nix-grpc-daemon[785]: ts=2026-09-21T09:34:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14586worker1 # [ 40.807297] nix-daemon[851]: reaped child process 887, status = succeeded4587worker2 # [ 40.814831] nix-grpc-daemon[786]: ts=2026-09-21T09:34:52Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04588worker2 # [ 40.819594] nix-grpc-daemon[786]: ts=2026-09-21T09:34:52Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14589worker1 # [ 40.864664] nix-grpc-daemon[785]: ts=2026-09-21T09:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664590worker1 # [ 40.877686] nix-grpc-daemon[785]: ts=2026-09-21T09:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664591worker1 # [ 40.891719] nix-grpc-daemon[864]: time=2026-09-21T09:34:51.547Z level=INFO msg="All 1 paths already cached"4592client # copying 1 paths to 'lb:50051'...4593worker1 # [ 40.900162] nix-grpc-daemon[785]: ts=2026-09-21T09:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546664594client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4595worker1 # [ 40.919401] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4596worker1 # [ 41.134138] nix-daemon[851]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4597worker1 # [ 41.148106] nix-daemon[851]: reaped child process 916, status = succeeded4598worker1 # [ 41.350099] nix-daemon[851]: accepted connection from pid 918, user nix-grpc-daemon (trusted)4599worker1 # [ 41.361922] nix-daemon[851]: reaped child process 921, status = succeeded4600lb # [ 41.385582] niks3-server[1059]: 2026/09/21 09:34:52 INFO Received uploads request method=POST path=/api/pending_closures4601worker1 # [ 41.395285] nix-grpc-daemon[864]: time=2026-09-21T09:34:52.050Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4602worker1 # [ 41.396834] nix-grpc-daemon[864]: time=2026-09-21T09:34:52.052Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4603lb # [ 41.428662] niks3-server[1059]: 2026/09/21 09:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4604worker1 # [ 41.433047] nix-grpc-daemon[864]: time=2026-09-21T09:34:52.088Z level=INFO msg="Uploading 1 narinfos"4605lb # [ 41.436385] niks3-server[1059]: 2026/09/21 09:34:52 INFO Signed narinfos id=3 count=14606lb # [ 41.441248] niks3-server[1059]: 2026/09/21 09:34:52 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4607lb # [ 41.445685] niks3-server[1059]: 2026/09/21 09:34:52 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4608lb # [ 41.455192] niks3-server[1059]: 2026/09/21 09:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4609lb # [ 41.466437] niks3-server[1059]: 2026/09/21 09:34:52 INFO Completed upload id=34610lb # [ 41.468198] niks3-server[1059]: 2026/09/21 09:34:52 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4611worker1 # [ 41.463352] nix-grpc-daemon[864]: time=2026-09-21T09:34:52.118Z level=INFO msg="Upload complete. (528ms)"4612worker1 # [ 41.468253] nix-grpc-daemon[785]: ts=2026-09-21T09:34:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_in=6244613worker1 # [ 41.474367] nix-grpc-daemon[785]: ts=2026-09-21T09:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664614worker1 # [ 41.489994] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4615worker1 # [ 41.494685] nix-daemon[851]: reaped child process 911, status = succeeded4616worker1 # [ 41.502586] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4617client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4618client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4619client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4620client # farm-b> LOG-farm-b-t14621worker1 # [ 42.720955] nix-daemon[851]: reaped child process 925, status = succeeded4622worker1 # [ 42.893451] nix-daemon[851]: accepted connection from pid 941, user nix-grpc-daemon (trusted)4623worker1 # [ 42.906832] nix-daemon[851]: reaped child process 945, status = succeeded4624lb # [ 42.931676] niks3-server[1059]: 2026/09/21 09:34:53 INFO Received uploads request method=POST path=/api/pending_closures4625worker1 # [ 42.937976] nix-grpc-daemon[864]: time=2026-09-21T09:34:53.593Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4626worker1 # [ 42.939535] nix-grpc-daemon[864]: time=2026-09-21T09:34:53.595Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4627lb # [ 42.968138] niks3-server[1059]: 2026/09/21 09:34:53 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4628lb # [ 42.976427] niks3-server[1059]: 2026/09/21 09:34:53 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4629lb # [ 42.989130] niks3-server[1059]: 2026/09/21 09:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4630worker1 # [ 42.992424] nix-grpc-daemon[864]: time=2026-09-21T09:34:53.647Z level=INFO msg="Uploading 1 narinfos"4631lb # [ 42.995248] niks3-server[1059]: 2026/09/21 09:34:53 INFO Signed narinfos id=4 count=14632lb # [ 42.999492] niks3-server[1059]: 2026/09/21 09:34:53 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4633lb # [ 43.010249] niks3-server[1059]: 2026/09/21 09:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4634lb # [ 43.017242] niks3-server[1059]: 2026/09/21 09:34:53 INFO Completed upload id=44635lb # [ 43.018986] niks3-server[1059]: 2026/09/21 09:34:53 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4636worker1 # [ 43.014204] nix-grpc-daemon[864]: time=2026-09-21T09:34:53.669Z level=INFO msg="Upload complete. (304ms)"4637worker1 # [ 43.019880] nix-grpc-daemon[785]: ts=2026-09-21T09:34:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14638worker2 # [ 43.024214] nix-grpc-daemon[786]: ts=2026-09-21T09:34:54Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04639worker1 # [ 43.035384] nix-daemon[851]: reaped child process 923, status = succeeded4640worker2 # [ 43.028748] nix-grpc-daemon[786]: ts=2026-09-21T09:34:54Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14641worker1 # [ 43.072481] nix-grpc-daemon[785]: ts=2026-09-21T09:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664642worker1 # [ 43.085292] nix-grpc-daemon[785]: ts=2026-09-21T09:34:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664643worker1 # [ 43.098428] nix-grpc-daemon[864]: time=2026-09-21T09:34:53.754Z level=INFO msg="All 2 paths already cached"4644client # copying 1 paths to 'lb:50051'...4645worker1 # [ 43.107084] nix-grpc-daemon[785]: ts=2026-09-21T09:34:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546664646client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4647worker1 # [ 43.125367] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4648worker1 # [ 43.337367] nix-daemon[851]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4649worker1 # [ 43.351787] nix-daemon[851]: reaped child process 952, status = succeeded4650worker1 # [ 43.554661] nix-daemon[851]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4651worker1 # [ 43.568195] nix-daemon[851]: reaped child process 957, status = succeeded4652lb # [ 43.591653] niks3-server[1059]: 2026/09/21 09:34:54 INFO Received uploads request method=POST path=/api/pending_closures4653worker1 # [ 43.597500] nix-grpc-daemon[864]: time=2026-09-21T09:34:54.253Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4654worker1 # [ 43.598997] nix-grpc-daemon[864]: time=2026-09-21T09:34:54.254Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4655lb # [ 43.620591] niks3-server[1059]: 2026/09/21 09:34:54 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4656lb # [ 43.633817] niks3-server[1059]: 2026/09/21 09:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4657lb # [ 43.638586] niks3-server[1059]: 2026/09/21 09:34:54 INFO Signed narinfos id=5 count=14658worker1 # [ 43.638177] nix-grpc-daemon[864]: time=2026-09-21T09:34:54.293Z level=INFO msg="Uploading 1 narinfos"4659lb # [ 43.644938] niks3-server[1059]: 2026/09/21 09:34:54 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4660lb # [ 43.656272] niks3-server[1059]: 2026/09/21 09:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4661lb # [ 43.661963] niks3-server[1059]: 2026/09/21 09:34:54 INFO Completed upload id=54662worker1 # [ 43.660802] nix-grpc-daemon[864]: time=2026-09-21T09:34:54.316Z level=INFO msg="Upload complete. (520ms)"4663lb # [ 43.665591] niks3-server[1059]: 2026/09/21 09:34:54 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4664worker1 # [ 43.664125] nix-grpc-daemon[785]: ts=2026-09-21T09:34:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_in=7524665worker1 # [ 43.669494] nix-grpc-daemon[785]: ts=2026-09-21T09:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664666worker1 # [ 43.679264] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4667worker1 # [ 43.682416] nix-daemon[851]: reaped child process 947, status = succeeded4668worker1 # [ 43.688102] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4669client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4670client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4671client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4672client # farm-top> LOG-farm-top-t14673worker1 # [ 45.114665] nix-daemon[851]: reaped child process 961, status = succeeded4674worker1 # [ 45.312399] nix-daemon[851]: accepted connection from pid 977, user nix-grpc-daemon (trusted)4675worker1 # [ 45.325861] nix-daemon[851]: reaped child process 980, status = succeeded4676lb # [ 45.350864] niks3-server[1059]: 2026/09/21 09:34:56 INFO Received uploads request method=POST path=/api/pending_closures4677worker1 # [ 45.358081] nix-grpc-daemon[864]: time=2026-09-21T09:34:56.013Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4678worker1 # [ 45.359769] nix-grpc-daemon[864]: time=2026-09-21T09:34:56.015Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4679lb # [ 45.390535] niks3-server[1059]: 2026/09/21 09:34:56 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4680lb # [ 45.406841] niks3-server[1059]: 2026/09/21 09:34:56 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4681lb # [ 45.417646] niks3-server[1059]: 2026/09/21 09:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4682lb # [ 45.422351] niks3-server[1059]: 2026/09/21 09:34:56 INFO Signed narinfos id=6 count=14683worker1 # [ 45.421845] nix-grpc-daemon[864]: time=2026-09-21T09:34:56.077Z level=INFO msg="Uploading 1 narinfos"4684lb # [ 45.425302] niks3-server[1059]: 2026/09/21 09:34:56 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4685lb # [ 45.435729] niks3-server[1059]: 2026/09/21 09:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4686lb # [ 45.439549] niks3-server[1059]: 2026/09/21 09:34:56 INFO Completed upload id=64687worker1 # [ 45.436251] nix-grpc-daemon[864]: time=2026-09-21T09:34:56.091Z level=INFO msg="Upload complete. (339ms)"4688lb # [ 45.443104] niks3-server[1059]: 2026/09/21 09:34:56 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4689worker1 # [ 45.442559] nix-grpc-daemon[785]: ts=2026-09-21T09:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14690worker1 # [ 45.456622] nix-daemon[851]: reaped child process 959, status = succeeded4691client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.57 seconds)4692client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4693client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.24 seconds)4694client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4695client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4696client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4697client # copying 3 paths...4698client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4699client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4700client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4701client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.31 seconds)4702(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.24 seconds)4703subtest: repeat is answered Cached by the scheduler without building4704worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4705worker1 # finding garbage collector roots...4706worker1 # deleting specified paths...4707worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4708worker1 # deleting unused links...4709worker1 # note: hard linking is currently saving 0.0 KiB4710worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.29 seconds)4711worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4712worker2 # finding garbage collector roots...4713worker2 # deleting specified paths...4714worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4715worker2 # deleting unused links...4716worker2 # note: hard linking is currently saving 0.0 KiB4717worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.30 seconds)4718client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24719lb # [ 46.840288] envoy[900]: conn peer=[2001:db8:1::1]:56442 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=794720lb # [ 46.844119] envoy[900]: conn peer=[2001:db8:1::1]:56458 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114721lb # [ 46.847508] envoy[900]: conn peer=[2001:db8:1::1]:56472 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=124722lb # [ 46.851314] envoy[900]: conn peer=[2001:db8:1::1]:56478 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9524 tx=4089 ms=73004723client # don't know how to build these paths:4724client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4725worker2 # [ 47.039674] nix-grpc-daemon[786]: ts=2026-09-21T09:34:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:604984726worker2 # [ 47.045539] nix-grpc-daemon[786]: ts=2026-09-21T09:34:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604984727worker2 # [ 47.072461] nix-grpc-daemon[786]: ts=2026-09-21T09:34:58Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14728client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.35 seconds)4729client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4730worker1 # [ 47.253879] nix-grpc-daemon[785]: E0000 00:00:1789983297.908014 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4731worker1 # [ 47.284154] nix-grpc-daemon[785]: E0000 00:00:1789983297.939655 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4732worker1 # [ 47.292349] nix-grpc-daemon[785]: E0000 00:00:1789983297.945138 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4733worker1 # [ 47.315078] nix-grpc-daemon[785]: E0000 00:00:1789983297.970406 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4734client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.25 seconds)4735lb: must succeed: curl -sf localhost:9901/clusters4736lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4737lb: must succeed: curl -sf localhost:9901/clusters4738lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4739worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4740worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4741(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.50 seconds)4742subtest: inputs already in the cache are not sent to the scheduler4743lb: must succeed: curl -sf localhost:9901/clusters4744lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4745lb: must succeed: curl -sf localhost:9901/clusters4746lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4747worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4748worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4749client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24750worker2 # [ 48.103628] nix-grpc-daemon[786]: ts=2026-09-21T09:34:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:604984751client # don't know how to build these paths:4752client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4753worker1 # [ 48.117823] nix-grpc-daemon[785]: ts=2026-09-21T09:34:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664754worker1 # [ 48.127528] nix-grpc-daemon[864]: time=2026-09-21T09:34:58.782Z level=INFO msg="All 2 paths already cached"4755worker2 # [ 48.136444] nix-grpc-daemon[786]: ts=2026-09-21T09:34:59Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04756worker2 # [ 48.140964] nix-grpc-daemon[786]: ts=2026-09-21T09:34:59Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14757worker1 # [ 48.190465] nix-grpc-daemon[785]: ts=2026-09-21T09:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664758worker1 # [ 48.204149] nix-grpc-daemon[785]: ts=2026-09-21T09:34:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664759worker1 # [ 48.217171] nix-grpc-daemon[864]: time=2026-09-21T09:34:58.872Z level=INFO msg="All 2 paths already cached"4760client # copying 1 paths to 'lb:50051'...4761client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4762worker1 # [ 48.225508] nix-grpc-daemon[785]: ts=2026-09-21T09:34:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546664763worker1 # [ 48.234901] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4764worker1 # [ 48.453141] nix-daemon[851]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4765worker1 # [ 48.467174] nix-daemon[851]: reaped child process 1000, status = succeeded4766worker1 # [ 48.673615] nix-daemon[851]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4767worker1 # [ 48.687357] nix-daemon[851]: reaped child process 1005, status = succeeded4768lb # [ 48.712533] niks3-server[1059]: 2026/09/21 09:34:59 INFO Received uploads request method=POST path=/api/pending_closures4769worker1 # [ 48.717315] nix-grpc-daemon[864]: time=2026-09-21T09:34:59.372Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4770worker1 # [ 48.718914] nix-grpc-daemon[864]: time=2026-09-21T09:34:59.374Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4771lb # [ 48.743798] niks3-server[1059]: 2026/09/21 09:34:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4772worker1 # [ 48.747701] nix-grpc-daemon[864]: time=2026-09-21T09:34:59.403Z level=INFO msg="Uploading 1 narinfos"4773lb # [ 48.751944] niks3-server[1059]: 2026/09/21 09:34:59 INFO Signed narinfos id=7 count=14774lb # [ 48.757497] niks3-server[1059]: 2026/09/21 09:34:59 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4775lb # [ 48.764210] niks3-server[1059]: 2026/09/21 09:34:59 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4776lb # [ 48.774886] niks3-server[1059]: 2026/09/21 09:34:59 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4777worker1 # [ 48.776466] nix-grpc-daemon[864]: time=2026-09-21T09:34:59.432Z level=INFO msg="Upload complete. (526ms)"4778lb # [ 48.779907] niks3-server[1059]: 2026/09/21 09:34:59 INFO Completed upload id=74779worker1 # [ 48.779798] nix-grpc-daemon[785]: ts=2026-09-21T09:34:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_in=7524780lb # [ 48.785108] niks3-server[1059]: 2026/09/21 09:34:59 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4781worker1 # [ 48.785896] nix-grpc-daemon[785]: ts=2026-09-21T09:34:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664782worker1 # [ 48.796261] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4783worker1 # [ 48.799385] nix-daemon[851]: reaped child process 995, status = succeeded4784worker1 # [ 48.805096] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4785client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4786client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4787client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4788client # farm-top> LOG-farm-top-t1b4789worker1 # [ 49.819833] nix-daemon[851]: reaped child process 1009, status = succeeded4790worker1 # [ 49.993672] nix-daemon[851]: accepted connection from pid 1025, user nix-grpc-daemon (trusted)4791worker1 # [ 50.007369] nix-daemon[851]: reaped child process 1028, status = succeeded4792lb # [ 50.031248] niks3-server[1059]: 2026/09/21 09:35:00 INFO Received uploads request method=POST path=/api/pending_closures4793worker1 # [ 50.036554] nix-grpc-daemon[864]: time=2026-09-21T09:35:00.692Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4794worker1 # [ 50.039050] nix-grpc-daemon[864]: time=2026-09-21T09:35:00.693Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4795lb # [ 50.069951] niks3-server[1059]: 2026/09/21 09:35:00 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4796lb # [ 50.076079] niks3-server[1059]: 2026/09/21 09:35:01 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4797lb # [ 50.088921] niks3-server[1059]: 2026/09/21 09:35:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4798lb # [ 50.093232] niks3-server[1059]: 2026/09/21 09:35:01 INFO Signed narinfos id=8 count=14799worker1 # [ 50.093234] nix-grpc-daemon[864]: time=2026-09-21T09:35:00.748Z level=INFO msg="Uploading 1 narinfos"4800lb # [ 50.102560] niks3-server[1059]: 2026/09/21 09:35:01 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4801lb # [ 50.108933] niks3-server[1059]: 2026/09/21 09:35:01 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4802lb # [ 50.117806] niks3-server[1059]: 2026/09/21 09:35:01 INFO Completed upload id=84803worker1 # [ 50.114638] nix-grpc-daemon[864]: time=2026-09-21T09:35:00.770Z level=INFO msg="Upload complete. (302ms)"4804lb # [ 50.119640] niks3-server[1059]: 2026/09/21 09:35:01 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4805worker1 # [ 50.121338] nix-grpc-daemon[785]: ts=2026-09-21T09:35:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14806worker1 # [ 50.134323] nix-daemon[851]: reaped child process 1007, status = succeeded4807client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.32 seconds)4808client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4809client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.22 seconds)4810lb: must succeed: curl -sf localhost:9901/clusters4811lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4812lb: must succeed: curl -sf localhost:9901/clusters4813lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4814worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4815worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4816(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.90 seconds)4817subtest: two clients wanting the same drv build it once4818client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4819client # [ 50.643941] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4820client # Running as unit: dedup1.service; invocation ID: 30a2cb074555418b8326419009b2f9a44821client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.08 seconds)4822client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4823worker2 # [ 50.869524] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:604984824client # [ 50.892356] nix[884]: don't know how to build these paths:4825client # [ 50.893245] nix[884]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4826worker2 # [ 50.886312] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04827worker2 # [ 50.891799] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14828worker2 # [ 50.897174] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604984829worker2 # [ 50.912362] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604984830client # [ 50.941087] nix[884]: copying 1 paths to 'lb:50051'...4831client # [ 50.942877] nix[884]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4832worker2 # [ 50.923899] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604984833worker2 # [ 50.933979] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4834worker2 # [ 50.956980] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=info event=niks3_push_started pid=9244835worker2 # [ 51.242802] nix-grpc-daemon[924]: time=2026-09-21T09:35:02.432Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""4836worker2 # [ 51.455093] nix-daemon[871]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4837worker2 # [ 51.469787] nix-daemon[871]: reaped child process 940, status = succeeded4838worker2 # [ 51.639203] nix-daemon[871]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4839worker2 # [ 51.651424] nix-daemon[871]: reaped child process 945, status = succeeded4840lb # [ 51.675793] niks3-server[1059]: 2026/09/21 09:35:02 INFO Received uploads request method=POST path=/api/pending_closures4841worker2 # [ 51.677299] nix-grpc-daemon[924]: time=2026-09-21T09:35:02.871Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4842worker2 # [ 51.679720] nix-grpc-daemon[924]: time=2026-09-21T09:35:02.873Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4843lb # [ 51.713566] niks3-server[1059]: 2026/09/21 09:35:02 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4844client # [ 51.727577] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4845lb # [ 51.723961] niks3-server[1059]: 2026/09/21 09:35:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4846client # Running as unit: dedup2.service; invocation ID: 6a43cc3539b74c6eb61808119f37e0864847lb # [ 51.729988] niks3-server[1059]: 2026/09/21 09:35:02 INFO Signed narinfos id=9 count=14848worker2 # [ 51.727617] nix-grpc-daemon[924]: time=2026-09-21T09:35:02.921Z level=INFO msg="Uploading 1 narinfos"4849client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4850lb # [ 51.740800] niks3-server[1059]: 2026/09/21 09:35:02 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4851lb # [ 51.750860] niks3-server[1059]: 2026/09/21 09:35:02 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4852lb # [ 51.757302] niks3-server[1059]: 2026/09/21 09:35:02 INFO Completed upload id=94853worker2 # [ 51.750240] nix-grpc-daemon[924]: time=2026-09-21T09:35:02.943Z level=INFO msg="Upload complete. (482ms)"4854lb # [ 51.760850] niks3-server[1059]: 2026/09/21 09:35:02 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4855worker2 # [ 51.754564] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=0 paths=1 nar_bytes_in=4324856worker2 # [ 51.759633] nix-grpc-daemon[786]: ts=2026-09-21T09:35:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604984857worker2 # [ 51.767303] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4858worker2 # [ 51.773115] nix-daemon[871]: reaped child process 922, status = succeeded4859client # [ 51.792807] nix[884]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4860worker2 # [ 51.778140] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4861worker2 # [ 51.991496] nix-grpc-daemon[786]: ts=2026-09-21T09:35:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:604984862client # [ 52.021397] nix[900]: this derivation will be built:4863worker1 # [ 52.015642] nix-grpc-daemon[785]: ts=2026-09-21T09:35:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:546664864worker1 # [ 52.027741] nix-grpc-daemon[785]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...4865client # [ 52.059902] nix[900]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4866worker2 # [ 52.051464] nix-grpc-daemon[786]: ts=2026-09-21T09:35:03Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04867client # [ 52.074315] nix[900]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4868worker2 # [ 52.056982] nix-grpc-daemon[786]: ts=2026-09-21T09:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604984869client: must succeed: sleep 24870client: (finished: must succeed: sleep 2, in 2.06 seconds)4871client: waiting for success: ! systemctl is-active dedup1 dedup24872worker2 # [ 55.404289] nix-daemon[871]: reaped child process 949, status = succeeded4873worker2 # [ 55.589843] nix-daemon[871]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4874worker2 # [ 55.600795] nix-daemon[871]: reaped child process 984, status = succeeded4875lb # [ 55.625432] niks3-server[1059]: 2026/09/21 09:35:06 INFO Received uploads request method=POST path=/api/pending_closures4876worker2 # [ 55.626930] nix-grpc-daemon[924]: time=2026-09-21T09:35:06.820Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4877worker2 # [ 55.628526] nix-grpc-daemon[924]: time=2026-09-21T09:35:06.822Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4878lb # [ 55.663829] niks3-server[1059]: 2026/09/21 09:35:06 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4879lb # [ 55.678800] niks3-server[1059]: 2026/09/21 09:35:06 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4880lb # [ 55.688981] niks3-server[1059]: 2026/09/21 09:35:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4881lb # [ 55.693495] niks3-server[1059]: 2026/09/21 09:35:06 INFO Signed narinfos id=10 count=14882worker2 # [ 55.689737] nix-grpc-daemon[924]: time=2026-09-21T09:35:06.882Z level=INFO msg="Uploading 1 narinfos"4883lb # [ 55.698569] niks3-server[1059]: 2026/09/21 09:35:06 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4884lb # [ 55.710842] niks3-server[1059]: 2026/09/21 09:35:06 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4885lb # [ 55.713607] niks3-server[1059]: 2026/09/21 09:35:06 INFO Completed upload id=104886lb # [ 55.715811] niks3-server[1059]: 2026/09/21 09:35:06 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4887worker2 # [ 55.707854] nix-grpc-daemon[924]: time=2026-09-21T09:35:06.901Z level=INFO msg="Upload complete. (316ms)"4888worker2 # [ 55.715689] nix-grpc-daemon[786]: ts=2026-09-21T09:35:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14889worker2 # [ 55.722852] nix-grpc-daemon[786]: ts=2026-09-21T09:35:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14890worker2 # [ 55.733482] nix-daemon[871]: reaped child process 947, status = succeeded4891client # [ 55.790453] systemd[1]: dedup1.service: Deactivated successfully.4892client # [ 55.793834] systemd[1]: dedup1.service: Consumed 122ms CPU time over 5.149s wall clock time, 41.5M memory peak, 7.2K incoming IP traffic, 8.6K outgoing IP traffic.4893client # [ 55.798571] systemd[1]: dedup2.service: Deactivated successfully.4894client # [ 55.801765] systemd[1]: dedup2.service: Consumed 140ms CPU time over 4.071s wall clock time, 35.2M memory peak, 6.5K incoming IP traffic, 5.9K outgoing IP traffic.4895client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4896client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4897client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4898worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4899worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4900worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4901worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4902worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4903worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4904worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4905worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4906(finished: subtest: two clients wanting the same drv build it once, in 6.10 seconds)4907subtest: build hook: nix-daemon with builders = grpc://lb4908client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4909lb # [ 56.841951] envoy[900]: conn peer=[2001:db8:1::1]:40112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=484910lb # [ 56.845620] envoy[900]: conn peer=[2001:db8:1::1]:40126 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4156 tx=1807 ms=20234911lb # [ 56.849376] envoy[900]: conn peer=[2001:db8:1::1]:40140 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3015 tx=1469 ms=48684912lb # [ 56.852866] envoy[900]: conn peer=[2001:db8:1::1]:40156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1602 tx=1487 ms=37524913client # [ 56.915575] systemd[1]: Started Nix Daemon.4914client # [ 57.129878] nix-daemon[942]: accepted connection from pid 939, user root (trusted)4915worker2 # [ 57.351185] nix-grpc-daemon[786]: ts=2026-09-21T09:35:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:604984916worker1 # [ 57.359389] nix-grpc-daemon[785]: ts=2026-09-21T09:35:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664917worker2 # [ 57.371628] nix-grpc-daemon[786]: ts=2026-09-21T09:35:08Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04918worker2 # [ 57.375911] nix-grpc-daemon[786]: ts=2026-09-21T09:35:08Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14919worker1 # [ 57.420352] nix-grpc-daemon[785]: ts=2026-09-21T09:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664920worker1 # [ 57.433151] nix-grpc-daemon[785]: ts=2026-09-21T09:35:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664921worker1 # [ 57.444793] nix-grpc-daemon[785]: ts=2026-09-21T09:35:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546664922worker1 # [ 57.456207] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4923worker1 # [ 57.693393] nix-daemon[851]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)4924worker1 # [ 57.707818] nix-daemon[851]: reaped child process 1057, status = succeeded4925worker1 # [ 57.923668] nix-daemon[851]: accepted connection from pid 1059, user nix-grpc-daemon (trusted)4926worker1 # [ 57.936798] nix-daemon[851]: reaped child process 1062, status = succeeded4927lb # [ 57.962295] niks3-server[1059]: 2026/09/21 09:35:08 INFO Received uploads request method=POST path=/api/pending_closures4928worker1 # [ 57.967742] nix-grpc-daemon[864]: time=2026-09-21T09:35:08.622Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4929worker1 # [ 57.969173] nix-grpc-daemon[864]: time=2026-09-21T09:35:08.622Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4930lb # [ 57.996153] niks3-server[1059]: 2026/09/21 09:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4931worker1 # [ 57.999430] nix-grpc-daemon[864]: time=2026-09-21T09:35:08.655Z level=INFO msg="Uploading 1 narinfos"4932lb # [ 58.002947] niks3-server[1059]: 2026/09/21 09:35:08 INFO Signed narinfos id=11 count=14933lb # [ 58.009177] niks3-server[1059]: 2026/09/21 09:35:08 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4934lb # [ 58.016464] niks3-server[1059]: 2026/09/21 09:35:08 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4935lb # [ 58.030450] niks3-server[1059]: 2026/09/21 09:35:08 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4936lb # [ 58.036345] niks3-server[1059]: 2026/09/21 09:35:08 INFO Completed upload id=114937worker1 # [ 58.035526] nix-grpc-daemon[864]: time=2026-09-21T09:35:08.691Z level=INFO msg="Upload complete. (566ms)"4938lb # [ 58.039804] niks3-server[1059]: 2026/09/21 09:35:08 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4939worker1 # [ 58.039600] nix-grpc-daemon[785]: ts=2026-09-21T09:35:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_in=5124940worker1 # [ 58.045531] nix-grpc-daemon[785]: ts=2026-09-21T09:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664941worker1 # [ 58.055934] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4942worker1 # [ 58.065480] nix-daemon[851]: reaped child process 1052, status = succeeded4943worker1 # [ 58.071463] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4944worker1 # [ 59.298761] nix-daemon[851]: reaped child process 1066, status = succeeded4945worker1 # [ 59.467293] nix-daemon[851]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)4946worker1 # [ 59.480415] nix-daemon[851]: reaped child process 1085, status = succeeded4947lb # [ 59.503915] niks3-server[1059]: 2026/09/21 09:35:10 INFO Received uploads request method=POST path=/api/pending_closures4948worker1 # [ 59.508764] nix-grpc-daemon[864]: time=2026-09-21T09:35:10.164Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4949worker1 # [ 59.510302] nix-grpc-daemon[864]: time=2026-09-21T09:35:10.166Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4950lb # [ 59.543402] niks3-server[1059]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4951lb # [ 59.551688] niks3-server[1059]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4952lb # [ 59.563125] niks3-server[1059]: 2026/09/21 09:35:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4953worker1 # [ 59.564922] nix-grpc-daemon[864]: time=2026-09-21T09:35:10.220Z level=INFO msg="Uploading 1 narinfos"4954lb # [ 59.568124] niks3-server[1059]: 2026/09/21 09:35:10 INFO Signed narinfos id=12 count=14955lb # [ 59.573086] niks3-server[1059]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4956lb # [ 59.582544] niks3-server[1059]: 2026/09/21 09:35:10 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4957lb # [ 59.588932] niks3-server[1059]: 2026/09/21 09:35:10 INFO Completed upload id=124958lb # [ 59.590808] niks3-server[1059]: 2026/09/21 09:35:10 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4959worker1 # [ 59.587060] nix-grpc-daemon[864]: time=2026-09-21T09:35:10.242Z level=INFO msg="Upload complete. (308ms)"4960worker1 # [ 59.593552] nix-grpc-daemon[785]: ts=2026-09-21T09:35:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14961worker1 # [ 59.600389] nix-grpc-daemon[785]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:546664962worker1 # [ 59.610633] nix-daemon[851]: reaped child process 1064, status = succeeded4963worker2 # [ 59.612870] nix-grpc-daemon[786]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:604984964worker2 # [ 59.625669] nix-grpc-daemon[786]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4965worker2 # [ 59.643380] nix-grpc-daemon[786]: ts=2026-09-21T09:35:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=0 paths=1 nar_bytes_out=1284966worker1 # [ 59.946696] nix-grpc-daemon[785]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:546664967worker1 # [ 59.952556] nix-grpc-daemon[785]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664968worker1 # [ 59.957520] nix-grpc-daemon[864]: time=2026-09-21T09:35:10.612Z level=INFO msg="All 1 paths already cached"4969worker2 # [ 59.964776] nix-grpc-daemon[786]: ts=2026-09-21T09:35:11Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04970worker1 # [ 59.971388] nix-grpc-daemon[785]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664971worker2 # [ 59.969371] nix-grpc-daemon[786]: ts=2026-09-21T09:35:11Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14972worker1 # [ 59.983936] nix-grpc-daemon[785]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546664973worker1 # [ 59.996635] nix-grpc-daemon[864]: time=2026-09-21T09:35:10.652Z level=INFO msg="All 1 paths already cached"4974worker1 # [ 60.003499] nix-grpc-daemon[785]: ts=2026-09-21T09:35:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546664975worker1 # [ 60.012179] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4976worker1 # [ 60.229248] nix-daemon[851]: accepted connection from pid 1089, user nix-grpc-daemon (trusted)4977worker1 # [ 60.243761] nix-daemon[851]: reaped child process 1092, status = succeeded4978worker1 # [ 60.441787] nix-daemon[851]: accepted connection from pid 1094, user nix-grpc-daemon (trusted)4979worker1 # [ 60.455384] nix-daemon[851]: reaped child process 1097, status = succeeded4980lb # [ 60.477531] niks3-server[1059]: 2026/09/21 09:35:11 INFO Received uploads request method=POST path=/api/pending_closures4981worker1 # [ 60.483817] nix-grpc-daemon[864]: time=2026-09-21T09:35:11.139Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4982worker1 # [ 60.485320] nix-grpc-daemon[864]: time=2026-09-21T09:35:11.141Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4983lb # [ 60.513879] niks3-server[1059]: 2026/09/21 09:35:11 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4984lb # [ 60.521543] niks3-server[1059]: 2026/09/21 09:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4985lb # [ 60.526562] niks3-server[1059]: 2026/09/21 09:35:11 INFO Signed narinfos id=13 count=14986worker1 # [ 60.526941] nix-grpc-daemon[864]: time=2026-09-21T09:35:11.182Z level=INFO msg="Uploading 1 narinfos"4987lb # [ 60.535279] niks3-server[1059]: 2026/09/21 09:35:11 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4988lb # [ 60.544152] niks3-server[1059]: 2026/09/21 09:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4989lb # [ 60.551728] niks3-server[1059]: 2026/09/21 09:35:11 INFO Completed upload id=134990worker1 # [ 60.548507] nix-grpc-daemon[864]: time=2026-09-21T09:35:11.204Z level=INFO msg="Upload complete. (522ms)"4991lb # [ 60.555483] niks3-server[1059]: 2026/09/21 09:35:11 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4992worker1 # [ 60.552503] nix-grpc-daemon[785]: ts=2026-09-21T09:35:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_in=6244993worker1 # [ 60.559145] nix-grpc-daemon[785]: ts=2026-09-21T09:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546664994worker1 # [ 60.569199] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4995worker1 # [ 60.572609] nix-daemon[851]: reaped child process 1087, status = succeeded4996worker1 # [ 60.579224] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4997worker1 # [ 61.794108] nix-daemon[851]: reaped child process 1101, status = succeeded4998worker1 # [ 61.940548] nix-daemon[851]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)4999worker1 # [ 61.953547] nix-daemon[851]: reaped child process 1120, status = succeeded5000lb # [ 61.974321] niks3-server[1059]: 2026/09/21 09:35:12 INFO Received uploads request method=POST path=/api/pending_closures5001worker1 # [ 61.978466] nix-grpc-daemon[864]: time=2026-09-21T09:35:12.633Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5002worker1 # [ 61.979944] nix-grpc-daemon[864]: time=2026-09-21T09:35:12.635Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5003lb # [ 62.013201] niks3-server[1059]: 2026/09/21 09:35:12 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5004lb # [ 62.024360] niks3-server[1059]: 2026/09/21 09:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5005worker1 # [ 62.029235] nix-grpc-daemon[864]: time=2026-09-21T09:35:12.684Z level=INFO msg="Uploading 1 narinfos"5006lb # [ 62.032512] niks3-server[1059]: 2026/09/21 09:35:12 INFO Signed narinfos id=14 count=15007lb # [ 62.036875] niks3-server[1059]: 2026/09/21 09:35:12 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5008lb # [ 62.041109] niks3-server[1059]: 2026/09/21 09:35:12 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5009lb # [ 62.051263] niks3-server[1059]: 2026/09/21 09:35:12 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5010lb # [ 62.054396] niks3-server[1059]: 2026/09/21 09:35:12 INFO Completed upload id=145011lb # [ 62.056559] niks3-server[1059]: 2026/09/21 09:35:12 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5012worker1 # [ 62.052523] nix-grpc-daemon[864]: time=2026-09-21T09:35:12.708Z level=INFO msg="Upload complete. (267ms)"5013worker1 # [ 62.059317] nix-grpc-daemon[785]: ts=2026-09-21T09:35:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=15014worker2 # [ 62.059325] nix-grpc-daemon[786]: ts=2026-09-21T09:35:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:604985015worker1 # [ 62.071987] nix-daemon[851]: reaped child process 1099, status = succeeded5016worker2 # [ 62.068638] nix-grpc-daemon[786]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5017worker1 # [ 62.100492] nix-grpc-daemon[785]: ts=2026-09-21T09:35:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:546665018worker1 # [ 62.106119] nix-grpc-daemon[785]: ts=2026-09-21T09:35:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_out=1765019worker2 # [ 62.365104] nix-grpc-daemon[786]: ts=2026-09-21T09:35:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:604985020worker2 # [ 62.372353] nix-grpc-daemon[786]: ts=2026-09-21T09:35:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604985021worker2 # [ 62.383797] nix-grpc-daemon[924]: time=2026-09-21T09:35:13.576Z level=INFO msg="All 2 paths already cached"5022worker2 # [ 62.398127] nix-grpc-daemon[786]: ts=2026-09-21T09:35:13Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05023worker2 # [ 62.402528] nix-grpc-daemon[786]: ts=2026-09-21T09:35:13Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15024worker1 # [ 62.446849] nix-grpc-daemon[785]: ts=2026-09-21T09:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546665025worker1 # [ 62.460849] nix-grpc-daemon[785]: ts=2026-09-21T09:35:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546665026worker1 # [ 62.473466] nix-grpc-daemon[864]: time=2026-09-21T09:35:13.129Z level=INFO msg="All 2 paths already cached"5027worker1 # [ 62.480649] nix-grpc-daemon[785]: ts=2026-09-21T09:35:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546665028worker1 # [ 62.489365] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5029worker1 # [ 62.687660] nix-daemon[851]: accepted connection from pid 1124, user nix-grpc-daemon (trusted)5030worker1 # [ 62.701218] nix-daemon[851]: reaped child process 1127, status = succeeded5031worker1 # [ 62.900693] nix-daemon[851]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)5032worker1 # [ 62.913573] nix-daemon[851]: reaped child process 1132, status = succeeded5033lb # [ 62.936703] niks3-server[1059]: 2026/09/21 09:35:13 INFO Received uploads request method=POST path=/api/pending_closures5034worker1 # [ 63.016338] nix-grpc-daemon[864]: time=2026-09-21T09:35:13.671Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5035worker1 # [ 63.018852] nix-grpc-daemon[864]: time=2026-09-21T09:35:13.673Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5036lb # [ 63.043366] niks3-server[1059]: 2026/09/21 09:35:13 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5037lb # [ 63.050467] niks3-server[1059]: 2026/09/21 09:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5038lb # [ 63.058130] niks3-server[1059]: 2026/09/21 09:35:13 INFO Signed narinfos id=15 count=15039worker1 # [ 63.055143] nix-grpc-daemon[864]: time=2026-09-21T09:35:13.710Z level=INFO msg="Uploading 1 narinfos"5040lb # [ 63.067556] niks3-server[1059]: 2026/09/21 09:35:13 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5041lb # [ 63.075163] niks3-server[1059]: 2026/09/21 09:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5042lb # [ 63.081339] niks3-server[1059]: 2026/09/21 09:35:14 INFO Completed upload id=155043worker1 # [ 63.078095] nix-grpc-daemon[864]: time=2026-09-21T09:35:13.733Z level=INFO msg="Upload complete. (574ms)"5044lb # [ 63.084842] niks3-server[1059]: 2026/09/21 09:35:14 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5045worker1 # [ 63.083647] nix-grpc-daemon[785]: ts=2026-09-21T09:35:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_in=7525046worker1 # [ 63.091221] nix-grpc-daemon[785]: ts=2026-09-21T09:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546665047worker1 # [ 63.100824] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5048worker1 # [ 63.112238] nix-daemon[851]: reaped child process 1122, status = succeeded5049worker1 # [ 63.122706] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5050worker1 # [ 64.753970] nix-daemon[851]: reaped child process 1136, status = succeeded5051worker1 # [ 64.940681] nix-daemon[851]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)5052worker1 # [ 64.954092] nix-daemon[851]: reaped child process 1155, status = succeeded5053lb # [ 64.977373] niks3-server[1059]: 2026/09/21 09:35:15 INFO Received uploads request method=POST path=/api/pending_closures5054worker1 # [ 64.983424] nix-grpc-daemon[864]: time=2026-09-21T09:35:15.638Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5055worker1 # [ 64.986060] nix-grpc-daemon[864]: time=2026-09-21T09:35:15.640Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5056lb # [ 65.022151] niks3-server[1059]: 2026/09/21 09:35:15 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5057lb # [ 65.034893] niks3-server[1059]: 2026/09/21 09:35:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5058lb # [ 65.040503] niks3-server[1059]: 2026/09/21 09:35:15 INFO Signed narinfos id=16 count=15059worker1 # [ 65.040301] nix-grpc-daemon[864]: time=2026-09-21T09:35:15.695Z level=INFO msg="Uploading 1 narinfos"5060lb # [ 65.048358] niks3-server[1059]: 2026/09/21 09:35:15 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5061lb # [ 65.056832] niks3-server[1059]: 2026/09/21 09:35:15 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5062lb # [ 65.063353] niks3-server[1059]: 2026/09/21 09:35:15 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5063lb # [ 65.066564] niks3-server[1059]: 2026/09/21 09:35:15 INFO Completed upload id=165064lb # [ 65.068326] niks3-server[1059]: 2026/09/21 09:35:15 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5065worker1 # [ 65.063484] nix-grpc-daemon[864]: time=2026-09-21T09:35:15.719Z level=INFO msg="Upload complete. (328ms)"5066worker1 # [ 65.070275] nix-grpc-daemon[785]: ts=2026-09-21T09:35:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15067worker1 # [ 65.078154] nix-grpc-daemon[785]: ts=2026-09-21T09:35:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:546665068worker1 # [ 65.087534] nix-daemon[851]: reaped child process 1134, status = succeeded5069worker2 # [ 65.087571] nix-grpc-daemon[786]: ts=2026-09-21T09:35:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:604985070worker2 # [ 65.098613] nix-grpc-daemon[786]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5071worker2 # [ 65.112133] nix-grpc-daemon[786]: ts=2026-09-21T09:35:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=0 paths=1 nar_bytes_out=2325072client # [ 65.236727] nix-daemon[942]: reaped child process 944, status = succeeded5073client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.57 seconds)5074client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5075client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5076client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5077client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5078client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5079client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5080(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.67 seconds)5081subtest: an input only one worker has still reaches the builder5082client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5083client # this derivation will be built:5084client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5085client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5086client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.46 seconds)5087client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5088client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)5089worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5090worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.26 seconds)5091worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5092worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5093client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25094client # [ 66.557326] nix-daemon[942]: accepted connection from pid 1025, user root (trusted)5095client # this derivation will be built:5096client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5097worker2 # [ 66.776745] nix-grpc-daemon[786]: ts=2026-09-21T09:35:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:604985098client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5099worker1 # [ 66.783745] nix-grpc-daemon[785]: ts=2026-09-21T09:35:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546665100lb # [ 66.842639] envoy[900]: conn peer=[2001:db8:1::1]:57034 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=334 ms=855101lb # [ 66.846453] envoy[900]: conn peer=[2001:db8:1::1]:57030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3446 tx=1755 ms=23525102lb # [ 66.850066] envoy[900]: conn peer=[2001:db8:1::1]:57036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=2069 ms=21715103lb # [ 66.853503] envoy[900]: conn peer=[2001:db8:1::1]:57044 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=185104lb # [ 66.856864] envoy[900]: conn peer=[2001:db8:1::1]:57058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4329 tx=2186 ms=28005105lb # [ 66.860990] envoy[900]: conn peer=[2001:db8:1::1]:54040 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=775106worker1 # [ 67.000761] nix-daemon[851]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5107worker1 # [ 67.012562] nix-daemon[851]: reaped child process 1165, status = succeeded5108lb # [ 67.036885] niks3-server[1059]: 2026/09/21 09:35:17 INFO Received uploads request method=POST path=/api/pending_closures5109worker1 # [ 67.236755] nix-grpc-daemon[864]: time=2026-09-21T09:35:17.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5110worker1 # [ 67.241141] nix-grpc-daemon[864]: time=2026-09-21T09:35:17.891Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5111lb # [ 67.261323] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5112lb # [ 67.267977] niks3-server[1059]: 2026/09/21 09:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5113lb # [ 67.273658] niks3-server[1059]: 2026/09/21 09:35:18 INFO Signed narinfos id=17 count=15114worker1 # [ 67.273677] nix-grpc-daemon[864]: time=2026-09-21T09:35:17.929Z level=INFO msg="Uploading 1 narinfos"5115lb # [ 67.283382] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5116lb # [ 67.291314] niks3-server[1059]: 2026/09/21 09:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5117lb # [ 67.300176] niks3-server[1059]: 2026/09/21 09:35:18 INFO Completed upload id=175118worker1 # [ 67.299366] nix-grpc-daemon[864]: time=2026-09-21T09:35:17.955Z level=INFO msg="Upload complete. (511ms)"5119lb # [ 67.305436] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5120worker2 # [ 67.318179] nix-grpc-daemon[786]: ts=2026-09-21T09:35:18Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05121worker2 # [ 67.323891] nix-grpc-daemon[786]: ts=2026-09-21T09:35:18Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15122worker2 # [ 67.363357] nix-grpc-daemon[786]: ts=2026-09-21T09:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604985123worker2 # [ 67.376453] nix-grpc-daemon[786]: ts=2026-09-21T09:35:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604985124client # copying 2 paths to 'lb:50051'...5125client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5126worker2 # [ 67.392676] nix-grpc-daemon[786]: ts=2026-09-21T09:35:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604985127client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5128worker2 # [ 67.407857] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5129worker2 # [ 67.603439] nix-daemon[871]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)5130worker2 # [ 67.616583] nix-daemon[871]: reaped child process 1014, status = succeeded5131worker2 # [ 67.774308] nix-daemon[871]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)5132worker2 # [ 67.785960] nix-daemon[871]: reaped child process 1019, status = succeeded5133lb # [ 67.809295] niks3-server[1059]: 2026/09/21 09:35:18 INFO Received uploads request method=POST path=/api/pending_closures5134lb # [ 67.818955] niks3-server[1059]: 2026/09/21 09:35:18 INFO Received uploads request method=POST path=/api/pending_closures5135worker2 # [ 67.819783] nix-grpc-daemon[924]: time=2026-09-21T09:35:19.013Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5136worker2 # [ 67.821404] nix-grpc-daemon[924]: time=2026-09-21T09:35:19.015Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5137worker2 # [ 67.823734] nix-grpc-daemon[924]: time=2026-09-21T09:35:19.015Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5138lb # [ 67.858491] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5139lb # [ 67.870179] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5140lb # [ 67.876109] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5141lb # [ 67.882973] niks3-server[1059]: 2026/09/21 09:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5142lb # [ 67.886309] niks3-server[1059]: 2026/09/21 09:35:18 INFO Signed narinfos id=18 count=15143lb # [ 67.887587] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5144lb # [ 67.889427] niks3-server[1059]: 2026/09/21 09:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5145lb # [ 67.891315] niks3-server[1059]: 2026/09/21 09:35:18 INFO Signed narinfos id=19 count=25146worker2 # [ 67.886285] nix-grpc-daemon[924]: time=2026-09-21T09:35:19.079Z level=INFO msg="Uploading 3 narinfos"5147lb # [ 67.902836] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5148lb # [ 67.907180] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5149lb # [ 67.911603] niks3-server[1059]: 2026/09/21 09:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5150lb # [ 67.915085] niks3-server[1059]: 2026/09/21 09:35:18 INFO Completed upload id=195151lb # [ 67.916370] niks3-server[1059]: 2026/09/21 09:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5152lb # [ 67.918564] niks3-server[1059]: 2026/09/21 09:35:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5153lb # [ 67.921225] niks3-server[1059]: 2026/09/21 09:35:18 INFO Completed upload id=185154worker2 # [ 67.915336] nix-grpc-daemon[924]: time=2026-09-21T09:35:19.108Z level=INFO msg="Upload complete. (492ms)"5155worker2 # [ 67.920634] nix-grpc-daemon[786]: ts=2026-09-21T09:35:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=0 paths=2 nar_bytes_in=9765156worker2 # [ 67.927338] nix-grpc-daemon[786]: ts=2026-09-21T09:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604985157worker2 # [ 67.937749] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5158worker2 # [ 67.946376] nix-daemon[871]: reaped child process 1009, status = succeeded5159worker2 # [ 67.956921] nix-grpc-daemon[786]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5160client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5161worker2 # [ 67.968613] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5162client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5163client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5164worker2 # [ 68.993930] nix-daemon[871]: reaped child process 1024, status = succeeded5165worker2 # [ 69.164294] nix-daemon[871]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)5166worker2 # [ 69.175364] nix-daemon[871]: reaped child process 1043, status = succeeded5167lb # [ 69.197780] niks3-server[1059]: 2026/09/21 09:35:20 INFO Received uploads request method=POST path=/api/pending_closures5168worker2 # [ 69.199453] nix-grpc-daemon[924]: time=2026-09-21T09:35:20.392Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5169lb # [ 69.229259] niks3-server[1059]: 2026/09/21 09:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5170worker2 # [ 69.229600] nix-grpc-daemon[924]: time=2026-09-21T09:35:20.423Z level=INFO msg="Uploading 1 narinfos"5171lb # [ 69.237353] niks3-server[1059]: 2026/09/21 09:35:20 INFO Signed narinfos id=20 count=15172lb # [ 69.245369] niks3-server[1059]: 2026/09/21 09:35:20 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5173lb # [ 69.256286] niks3-server[1059]: 2026/09/21 09:35:20 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5174lb # [ 69.260896] niks3-server[1059]: 2026/09/21 09:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5175lb # [ 69.268341] niks3-server[1059]: 2026/09/21 09:35:20 INFO Completed upload id=205176worker2 # [ 69.260871] nix-grpc-daemon[924]: time=2026-09-21T09:35:20.454Z level=INFO msg="Upload complete. (283ms)"5177lb # [ 69.271155] niks3-server[1059]: 2026/09/21 09:35:20 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5178worker2 # [ 69.265498] nix-grpc-daemon[786]: ts=2026-09-21T09:35:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15179worker2 # [ 69.277641] nix-daemon[871]: reaped child process 1021, status = succeeded5180client # copying 1 paths...5181worker1 # [ 69.319109] nix-grpc-daemon[785]: ts=2026-09-21T09:35:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:546665182worker1 # [ 69.335268] nix-grpc-daemon[785]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5183client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5184worker2 # [ 69.365366] nix-grpc-daemon[786]: ts=2026-09-21T09:35:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:604985185worker2 # [ 69.370437] nix-grpc-daemon[786]: ts=2026-09-21T09:35:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=0 paths=1 nar_bytes_out=1205186client # [ 69.448408] nix-daemon[942]: reaped child process 1028, status = succeeded5187client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.14 seconds)5188client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25189client # [ 69.649989] nix-daemon[942]: accepted connection from pid 1047, user root (trusted)5190client # this derivation will be built:5191client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5192worker1 # [ 69.829149] nix-grpc-daemon[785]: ts=2026-09-21T09:35:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:546665193client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5194worker1 # [ 69.836739] nix-grpc-daemon[785]: ts=2026-09-21T09:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546665195worker1 # [ 69.847631] nix-grpc-daemon[864]: time=2026-09-21T09:35:20.502Z level=INFO msg="All 1 paths already cached"5196worker2 # [ 69.859909] nix-grpc-daemon[786]: ts=2026-09-21T09:35:21Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05197worker1 # [ 69.866446] nix-grpc-daemon[785]: ts=2026-09-21T09:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546665198worker2 # [ 69.864674] nix-grpc-daemon[786]: ts=2026-09-21T09:35:21Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15199worker1 # [ 69.878904] nix-grpc-daemon[785]: ts=2026-09-21T09:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546665200client # copying 1 paths to 'lb:50051'...5201client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5202worker1 # [ 69.896372] nix-grpc-daemon[785]: ts=2026-09-21T09:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546665203worker1 # [ 69.905996] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5204worker1 # [ 69.925273] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)5205worker1 # [ 69.947693] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 2/5)5206worker1 # [ 69.972438] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 334 ms (attempt 3/5)5207worker1 # [ 70.312336] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 314 ms (attempt 4/5)5208worker1 # [ 70.633756] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5209worker1 # [ 70.637409] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5210worker1 # [ 70.643310] nix-grpc-daemon[785]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5211worker1 # [ 70.870385] nix-daemon[851]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)5212worker1 # [ 70.885241] nix-daemon[851]: reaped child process 1178, status = succeeded5213worker1 # [ 71.074865] nix-daemon[851]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)5214worker1 # [ 71.088501] nix-daemon[851]: reaped child process 1183, status = succeeded5215lb # [ 71.112416] niks3-server[1059]: 2026/09/21 09:35:22 INFO Received uploads request method=POST path=/api/pending_closures5216worker1 # [ 71.118409] nix-grpc-daemon[864]: time=2026-09-21T09:35:21.773Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5217worker1 # [ 71.120359] nix-grpc-daemon[864]: time=2026-09-21T09:35:21.775Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5218lb # [ 71.147377] niks3-server[1059]: 2026/09/21 09:35:22 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5219lb # [ 71.161868] niks3-server[1059]: 2026/09/21 09:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5220worker1 # [ 71.164095] nix-grpc-daemon[864]: time=2026-09-21T09:35:21.819Z level=INFO msg="Uploading 1 narinfos"5221lb # [ 71.167148] niks3-server[1059]: 2026/09/21 09:35:22 INFO Signed narinfos id=21 count=15222lb # [ 71.173273] niks3-server[1059]: 2026/09/21 09:35:22 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5223lb # [ 71.181600] niks3-server[1059]: 2026/09/21 09:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5224lb # [ 71.184707] niks3-server[1059]: 2026/09/21 09:35:22 INFO Completed upload id=215225worker1 # [ 71.181593] nix-grpc-daemon[864]: time=2026-09-21T09:35:21.837Z level=INFO msg="Upload complete. (516ms)"5226lb # [ 71.187721] niks3-server[1059]: 2026/09/21 09:35:22 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5227worker1 # [ 71.185913] nix-grpc-daemon[785]: ts=2026-09-21T09:35:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 paths=1 nar_bytes_in=5605228worker1 # [ 71.192067] nix-grpc-daemon[785]: ts=2026-09-21T09:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546665229worker1 # [ 71.203116] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5230worker1 # [ 71.210126] nix-daemon[851]: reaped child process 1168, status = succeeded5231worker1 # [ 71.213598] nix-daemon[851]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5232client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5233client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5234client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5235worker1 # [ 72.226906] nix-daemon[851]: reaped child process 1187, status = succeeded5236worker1 # [ 72.418177] nix-daemon[851]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5237worker1 # [ 72.431219] nix-daemon[851]: reaped child process 1206, status = succeeded5238lb # [ 72.457141] niks3-server[1059]: 2026/09/21 09:35:23 INFO Received uploads request method=POST path=/api/pending_closures5239worker1 # [ 72.461756] nix-grpc-daemon[864]: time=2026-09-21T09:35:23.117Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5240lb # [ 72.485837] niks3-server[1059]: 2026/09/21 09:35:23 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5241lb # [ 72.494850] niks3-server[1059]: 2026/09/21 09:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5242lb # [ 72.501899] niks3-server[1059]: 2026/09/21 09:35:23 INFO Signed narinfos id=22 count=15243worker1 # [ 72.501409] nix-grpc-daemon[864]: time=2026-09-21T09:35:23.157Z level=INFO msg="Uploading 1 narinfos"5244lb # [ 72.509140] niks3-server[1059]: 2026/09/21 09:35:23 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5245lb # [ 72.522529] niks3-server[1059]: 2026/09/21 09:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5246lb # [ 72.526980] niks3-server[1059]: 2026/09/21 09:35:23 INFO Completed upload id=225247worker1 # [ 72.526075] nix-grpc-daemon[864]: time=2026-09-21T09:35:23.181Z level=INFO msg="Upload complete. (304ms)"5248lb # [ 72.530125] niks3-server[1059]: 2026/09/21 09:35:23 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5249worker1 # [ 72.531294] nix-grpc-daemon[785]: ts=2026-09-21T09:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15250client # copying 1 paths...5251worker2 # [ 72.533681] nix-grpc-daemon[786]: ts=2026-09-21T09:35:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:604985252worker1 # [ 72.543786] nix-daemon[851]: reaped child process 1185, status = succeeded5253worker2 # [ 72.546577] nix-grpc-daemon[786]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5254client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5255worker1 # [ 72.584155] nix-grpc-daemon[785]: ts=2026-09-21T09:35:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:546665256worker1 # [ 72.588964] nix-grpc-daemon[785]: ts=2026-09-21T09:35:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_out=1205257client # [ 72.663865] nix-daemon[942]: reaped child process 1050, status = succeeded5258client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.21 seconds)5259client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25260client # [ 72.847689] nix-daemon[942]: accepted connection from pid 1069, user root (trusted)5261client # this derivation will be built:5262client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5263worker2 # [ 73.028814] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:604985264client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5265worker2 # [ 73.035187] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604985266worker2 # [ 73.040654] nix-grpc-daemon[924]: time=2026-09-21T09:35:24.233Z level=INFO msg="All 1 paths already cached"5267worker2 # [ 73.055080] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05268worker2 # [ 73.060808] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15269worker2 # [ 73.099270] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604985270worker2 # [ 73.112910] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604985271worker2 # [ 73.126292] nix-grpc-daemon[924]: time=2026-09-21T09:35:24.319Z level=INFO msg="All 1 paths already cached"5272client # copying 1 paths to 'lb:50051'...5273client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5274worker2 # [ 73.135121] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604985275worker2 # [ 73.146927] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5276worker2 # [ 73.327774] nix-daemon[871]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)5277worker2 # [ 73.340947] nix-daemon[871]: reaped child process 1055, status = succeeded5278worker2 # [ 73.501698] nix-daemon[871]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)5279worker2 # [ 73.512408] nix-daemon[871]: reaped child process 1060, status = succeeded5280lb # [ 73.535904] niks3-server[1059]: 2026/09/21 09:35:24 INFO Received uploads request method=POST path=/api/pending_closures5281worker2 # [ 73.536577] nix-grpc-daemon[924]: time=2026-09-21T09:35:24.730Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5282worker2 # [ 73.538040] nix-grpc-daemon[924]: time=2026-09-21T09:35:24.731Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5283lb # [ 73.566425] niks3-server[1059]: 2026/09/21 09:35:24 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5284lb # [ 73.576268] niks3-server[1059]: 2026/09/21 09:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5285lb # [ 73.583423] niks3-server[1059]: 2026/09/21 09:35:24 INFO Signed narinfos id=23 count=15286worker2 # [ 73.579155] nix-grpc-daemon[924]: time=2026-09-21T09:35:24.772Z level=INFO msg="Uploading 1 narinfos"5287lb # [ 73.586629] niks3-server[1059]: 2026/09/21 09:35:24 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5288lb # [ 73.600124] niks3-server[1059]: 2026/09/21 09:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5289lb # [ 73.606601] niks3-server[1059]: 2026/09/21 09:35:24 INFO Completed upload id=235290worker2 # [ 73.599093] nix-grpc-daemon[924]: time=2026-09-21T09:35:24.792Z level=INFO msg="Upload complete. (439ms)"5291lb # [ 73.609393] niks3-server[1059]: 2026/09/21 09:35:24 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5292worker2 # [ 73.602811] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=0 paths=1 nar_bytes_in=5605293worker2 # [ 73.609091] nix-grpc-daemon[786]: ts=2026-09-21T09:35:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604985294worker2 # [ 73.619539] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5295worker2 # [ 73.622037] nix-daemon[871]: reaped child process 1050, status = succeeded5296worker2 # [ 73.627160] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5297client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5298client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5299client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5300worker2 # [ 74.652245] nix-daemon[871]: reaped child process 1064, status = succeeded5301worker2 # [ 74.813274] nix-daemon[871]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)5302worker2 # [ 74.825079] nix-daemon[871]: reaped child process 1083, status = succeeded5303lb # [ 74.850308] niks3-server[1059]: 2026/09/21 09:35:25 INFO Received uploads request method=POST path=/api/pending_closures5304worker2 # [ 74.850693] nix-grpc-daemon[924]: time=2026-09-21T09:35:26.044Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5305lb # [ 74.882161] niks3-server[1059]: 2026/09/21 09:35:25 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5306lb # [ 74.885780] niks3-server[1059]: 2026/09/21 09:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5307lb # [ 74.890659] niks3-server[1059]: 2026/09/21 09:35:25 INFO Signed narinfos id=24 count=15308worker2 # [ 74.886512] nix-grpc-daemon[924]: time=2026-09-21T09:35:26.080Z level=INFO msg="Uploading 1 narinfos"5309lb # [ 74.905924] niks3-server[1059]: 2026/09/21 09:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5310lb # [ 74.911264] niks3-server[1059]: 2026/09/21 09:35:25 INFO Completed upload id=245311worker2 # [ 74.906213] nix-grpc-daemon[924]: time=2026-09-21T09:35:26.099Z level=INFO msg="Upload complete. (272ms)"5312lb # [ 74.914669] niks3-server[1059]: 2026/09/21 09:35:25 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5313worker2 # [ 74.909523] nix-grpc-daemon[786]: ts=2026-09-21T09:35:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60498 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15314lb # [ 74.920056] niks3-server[1059]: 2026/09/21 09:35:25 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5315client # copying 1 paths...5316worker2 # [ 74.921524] nix-grpc-daemon[786]: ts=2026-09-21T09:35:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:604985317worker2 # [ 74.923655] nix-daemon[871]: reaped child process 1062, status = succeeded5318client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5319worker1 # [ 74.946583] nix-grpc-daemon[785]: ts=2026-09-21T09:35:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:546665320worker1 # [ 74.958061] nix-grpc-daemon[785]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5321worker1 # [ 74.970590] nix-grpc-daemon[785]: ts=2026-09-21T09:35:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54666 duration_s=0 paths=1 nar_bytes_out=1205322client # [ 75.091545] nix-daemon[942]: reaped child process 1072, status = succeeded5323client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.43 seconds)5324(finished: subtest: an input only one worker has still reaches the builder, in 9.75 seconds)5325subtest: upload whose reference the worker lacks is completed from the cache5326client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5327client # this derivation will be built:5328client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5329client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5330client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.40 seconds)5331client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5332client # this derivation will be built:5333client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5334client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5335client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.39 seconds)5336client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5337client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.19 seconds)5338worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5339worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.27 seconds)5340worker2 # [ 76.398447] systemd[1]: Stopping Nix worker-protocol over gRPC...5341lb # [ 76.842992] envoy[900]: conn peer=[2001:db8:1::1]:54052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3898 tx=1975 ms=26075342lb # [ 76.847566] envoy[900]: conn peer=[2001:db8:1::1]:54058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175343lb # [ 76.851051] envoy[900]: conn peer=[2001:db8:1::1]:54068 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2013 ms=27725344lb # [ 76.855640] envoy[900]: conn peer=[2001:db8:1::1]:54076 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=205345lb # [ 76.859085] envoy[900]: conn peer=[2001:db8:1::1]:58100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=795346lb # [ 76.864250] envoy[900]: conn peer=[2001:db8:1::1]:58086 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3746 tx=1899 ms=19925347worker2 # [ 76.929918] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=signal_exit5348worker2 # [ 77.186129] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=scheduler_yield5349worker2 # [ 77.190796] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=scheduler_restarting addr=lb:500515350lb # [ 77.200425] niks3-server[1059]: 2026/09/21 09:35:28 INFO lead: released remote=192.168.1.4:567765351worker2 # [ 77.198552] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:450545352worker1 # [ 77.203873] nix-grpc-daemon[785]: ts=2026-09-21T09:35:27Z level=info event=scheduler_restarting addr=lb:500515353worker2 # [ 77.203353] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=worker_gone addr=192.168.1.4:500515354worker2 # [ 77.206446] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:450545355worker2 # [ 77.209647] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=worker_gone addr=192.168.1.3:500515356worker2 # [ 77.391533] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=scheduler_connected addr=lb:500515357worker2 # [ 77.395446] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5358worker1 # [ 77.404112] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_connected addr=lb:500515359worker1 # [ 77.408150] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5360worker2 # [ 77.597345] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=scheduler_connected addr=lb:500515361worker2 # [ 77.601413] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5362worker1 # [ 77.608947] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_connected addr=lb:500515363worker1 # [ 77.612996] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker2 # [ 77.802852] nix-grpc-daemon[786]: ts=2026-09-21T09:35:28Z level=info event=scheduler_connected addr=lb:500515365worker2 # [ 77.806667] nix-grpc-daemon[786]: ts=2026-09-21T09:35:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366worker1 # [ 77.814073] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_connected addr=lb:500515367worker1 # [ 77.818242] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5368worker2 # [ 78.008632] nix-grpc-daemon[786]: ts=2026-09-21T09:35:29Z level=info event=scheduler_connected addr=lb:500515369worker2 # [ 78.012361] nix-grpc-daemon[786]: ts=2026-09-21T09:35:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5370worker1 # [ 78.019229] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_connected addr=lb:500515371worker1 # [ 78.023068] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5372worker2 # [ 78.190585] nix-grpc-daemon[786]: ts=2026-09-21T09:35:29Z level=info event=scheduler_restarting5373worker1 # [ 78.225565] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_connected addr=lb:500515374worker1 # [ 78.228838] nix-grpc-daemon[785]: ts=2026-09-21T09:35:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375worker2 # [ 78.228881] nix-grpc-daemon[786]: ts=2026-09-21T09:35:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5376worker2 # [ 78.233318] nix-daemon[871]: reaped child process 873, status = succeeded5377worker1 # [ 78.426457] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_connected addr=lb:500515378worker1 # [ 78.430411] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5379worker1 # [ 78.630751] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_connected addr=lb:500515380worker1 # [ 78.632542] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5381worker2 # [ 78.719591] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5382worker2 # [ 78.724647] systemd[1]: Stopped Nix worker-protocol over gRPC.5383worker2 # [ 78.727577] systemd[1]: nix-grpc-daemon.service: Consumed 1.565s CPU time over 1min 1.100s wall clock time, 53.2M memory peak, 59.6K incoming IP traffic, 187.7K outgoing IP traffic.5384worker2 # [ 78.740297] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5385worker2 # [ 78.744051] systemd[1]: Closed Nix worker-protocol over gRPC.5386worker2 # [ 78.747294] systemd[1]: nix-grpc-daemon.socket: Consumed 298us CPU time over 1min 8.485s wall clock time, 256K memory peak, 74.2K incoming IP traffic, 62.7K outgoing IP traffic.5387lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5388lb: must succeed: curl -sf localhost:9901/clusters5389worker1 # [ 78.833660] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_connected addr=lb:500515390worker1 # [ 78.837647] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5391lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5392lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5393client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25394worker1 # [ 79.038636] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_connected addr=lb:500515395worker1 # [ 79.042683] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5396worker1 # [ 79.081248] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:546665397worker1 # [ 79.088548] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:546665398client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5399client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.28 seconds)5400worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5401worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5402worker2 # [ 79.210846] systemd[1]: Listening on Nix worker-protocol over gRPC.5403worker1 # [ 79.246289] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_connected addr=lb:500515404worker1 # [ 79.249739] nix-grpc-daemon[785]: ts=2026-09-21T09:35:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5405worker1 # [ 79.293546] systemd[1]: Stopping Nix worker-protocol over gRPC...5406worker1 # [ 79.447093] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_connected addr=lb:500515407worker1 # [ 79.451155] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5408worker1 # [ 79.652380] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_connected addr=lb:500515409worker1 # [ 79.656349] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5410worker1 # [ 79.857194] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_connected addr=lb:500515411worker1 # [ 79.861162] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker1 # [ 80.062116] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_connected addr=lb:500515413worker1 # [ 80.066080] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5414worker2 # [ 80.091668] systemd[1]: Starting scheduler-token.service...5415worker2 # [ 80.200731] systemd[1]: scheduler-token.service: Deactivated successfully.5416worker2 # [ 80.202965] systemd[1]: Finished scheduler-token.service.5417worker2 # [ 80.205137] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 111ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5418worker2 # [ 80.209702] systemd[1]: Starting Nix worker-protocol over gRPC...5419worker1 # [ 80.247061] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=signal_exit5420worker1 # [ 80.266938] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_connected addr=lb:500515421worker1 # [ 80.270669] nix-grpc-daemon[785]: ts=2026-09-21T09:35:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5422worker2 # [ 80.422104] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5423worker2 # [ 80.424084] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5424worker2 # [ 80.427950] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15425worker2 # [ 80.437070] nix-grpc-daemon[1107]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5426worker2 # [ 80.438579] nix-grpc-daemon[1107]: E0000 00:00:1789983331.629849 1107 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5427worker2 # [ 80.443067] nix-grpc-daemon[1107]: E0000 00:00:1789983331.635420 1107 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5428worker2 # [ 80.445371] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5429worker2 # [ 80.451592] systemd[1]: Started Nix worker-protocol over gRPC.5430worker2 # [ 80.459234] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=scheduler_take_over5431lb # [ 80.467499] niks3-server[1059]: 2026/09/21 09:35:31 INFO lead: acquired remote=192.168.1.4:503625432worker2 # [ 80.461061] nix-grpc-daemon[1107]: E0000 00:00:1789983331.654797 1148 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5433worker1 # [ 80.471589] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_connected addr=lb:500515434worker1 # [ 80.475548] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker2 # [ 80.474349] nix-grpc-daemon[1107]: E0000 00:00:1789983331.668114 1148 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5436worker2 # [ 80.478140] nix-grpc-daemon[1107]: E0000 00:00:1789983331.670338 1148 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5437worker2 # [ 80.489971] nix-grpc-daemon[1107]: E0000 00:00:1789983331.683681 1148 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5438worker2 # [ 80.499744] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=scheduler_connected addr=lb:500515439worker2 # [ 80.501370] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5440worker1 # [ 80.676434] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_connected addr=lb:500515441worker1 # [ 80.680375] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442worker2 # [ 80.702826] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=scheduler_connected addr=lb:500515443worker2 # [ 80.706932] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5444worker1 # [ 80.881212] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_connected addr=lb:500515445worker1 # [ 80.885169] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5446worker2 # [ 80.908156] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=scheduler_connected addr=lb:500515447worker2 # [ 80.912418] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5448worker1 # [ 81.085973] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_connected addr=lb:500515449worker1 # [ 81.089971] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450worker2 # [ 81.113578] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=scheduler_connected addr=lb:500515451worker2 # [ 81.117566] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5452worker1 # [ 81.291766] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_connected addr=lb:500515453worker1 # [ 81.295618] nix-grpc-daemon[785]: ts=2026-09-21T09:35:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5454worker2 # [ 81.318835] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=scheduler_connected addr=lb:500515455worker2 # [ 81.323111] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5456worker1 # [ 81.496588] nix-grpc-daemon[785]: ts=2026-09-21T09:35:32Z level=info event=scheduler_connected addr=lb:500515457worker2 # [ 81.498461] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:518645458worker2 # [ 81.503827] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05459worker1 # [ 81.520443] nix-grpc-daemon[785]: ts=2026-09-21T09:35:32Z level=info event=scheduler_restarting5460worker2 # [ 81.523982] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=scheduler_connected addr=lb:500515461worker2 # [ 81.529217] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:518645462worker2 # [ 81.534534] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=worker_gone addr=192.168.1.3:500515463worker2 # [ 81.541158] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=oidc_jwks provider=mock keys=15464worker1 # [ 81.545352] nix-grpc-daemon[785]: ts=2026-09-21T09:35:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5465worker2 # [ 81.545241] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:518645466worker2 # [ 81.549494] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05467worker1 # [ 81.562362] nix-daemon[851]: reaped child process 853, status = succeeded5468worker1 # [ 82.426609] nix-grpc-daemon[785]: E0000 00:00:1789983333.081287 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5469worker1 # [ 82.431566] nix-grpc-daemon[785]: E0000 00:00:1789983333.081429 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5470worker1 # [ 82.658643] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5471worker1 # [ 82.663288] systemd[1]: Stopped Nix worker-protocol over gRPC.5472worker1 # [ 82.665368] systemd[1]: nix-grpc-daemon.service: Consumed 2.943s CPU time over 1min 5.826s wall clock time, 62.1M memory peak, 90.5K incoming IP traffic, 172.6K outgoing IP traffic.5473worker1 # [ 82.677067] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5474worker1 # [ 82.680529] systemd[1]: Closed Nix worker-protocol over gRPC.5475worker1 # [ 82.682789] systemd[1]: nix-grpc-daemon.socket: Consumed 328us CPU time over 1min 12.732s wall clock time, 256K memory peak, 81.2K incoming IP traffic, 67K outgoing IP traffic.5476lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5477lb: must succeed: curl -sf localhost:9901/clusters5478lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5479lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.09 seconds)5480client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25481worker2 # [ 83.023990] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400145482worker2 # [ 83.036813] nix-daemon[871]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5483worker2 # [ 83.055422] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)5484worker2 # [ 83.091427] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 116 ms (attempt 2/5)5485worker2 # [ 83.214116] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 3/5)5486worker2 # [ 83.239148] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 188 ms (attempt 4/5)5487worker2 # [ 83.433918] nix-grpc-daemon[1107]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5488worker2 # [ 83.438290] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5489client # copying 2 paths...5490client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5491client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5492worker2 # [ 83.464686] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400145493worker2 # [ 83.478173] nix-daemon[871]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5494worker2 # [ 83.503485] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:34Z level=info event=niks3_push_started pid=11675495worker2 # [ 83.748823] nix-grpc-daemon[1167]: time=2026-09-21T09:35:34.941Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5496worker2 # [ 83.951094] nix-daemon[871]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)5497worker2 # [ 83.962848] nix-daemon[871]: reaped child process 1183, status = succeeded5498lb # [ 83.991570] niks3-server[1059]: 2026/09/21 09:35:34 INFO Received uploads request method=POST path=/api/pending_closures5499lb # [ 84.000629] niks3-server[1059]: 2026/09/21 09:35:34 INFO Received uploads request method=POST path=/api/pending_closures5500worker2 # [ 84.000282] nix-grpc-daemon[1167]: time=2026-09-21T09:35:35.193Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5501worker2 # [ 84.001899] nix-grpc-daemon[1167]: time=2026-09-21T09:35:35.195Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5502worker2 # [ 84.003887] nix-grpc-daemon[1167]: time=2026-09-21T09:35:35.197Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5503lb # [ 84.042634] niks3-server[1059]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5504lb # [ 84.065219] niks3-server[1059]: 2026/09/21 09:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5505lb # [ 84.068759] niks3-server[1059]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5506lb # [ 84.071485] niks3-server[1059]: 2026/09/21 09:35:34 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5507lb # [ 84.075966] niks3-server[1059]: 2026/09/21 09:35:35 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5508lb # [ 84.080498] niks3-server[1059]: 2026/09/21 09:35:35 INFO Signed narinfos id=25 count=15509lb # [ 84.082183] niks3-server[1059]: 2026/09/21 09:35:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5510lb # [ 84.084188] niks3-server[1059]: 2026/09/21 09:35:35 INFO Signed narinfos id=26 count=25511worker2 # [ 84.079143] nix-grpc-daemon[1167]: time=2026-09-21T09:35:35.272Z level=INFO msg="Uploading 3 narinfos"5512lb # [ 84.104106] niks3-server[1059]: 2026/09/21 09:35:35 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5513lb # [ 84.109299] niks3-server[1059]: 2026/09/21 09:35:35 INFO Completed upload id=255514lb # [ 84.111322] niks3-server[1059]: 2026/09/21 09:35:35 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5515lb # [ 84.114047] niks3-server[1059]: 2026/09/21 09:35:35 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5516lb # [ 84.115861] niks3-server[1059]: 2026/09/21 09:35:35 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5517lb # [ 84.119419] niks3-server[1059]: 2026/09/21 09:35:35 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5518lb # [ 84.121984] niks3-server[1059]: 2026/09/21 09:35:35 INFO Completed upload id=265519worker2 # [ 84.115053] nix-grpc-daemon[1167]: time=2026-09-21T09:35:35.308Z level=INFO msg="Upload complete. (362ms)"5520worker2 # [ 84.122168] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40014 duration_s=0 paths=2 nar_bytes_in=3125521worker2 # [ 84.135721] nix-daemon[871]: reaped child process 1165, status = succeeded5522client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.39 seconds)5523worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5524worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5525worker1 # [ 84.272840] systemd[1]: Listening on Nix worker-protocol over gRPC.5526worker1 # [ 84.280244] systemd[1]: Starting Nix worker-protocol over gRPC...5527worker1 # [ 84.493790] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5528worker1 # [ 84.496269] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5529worker1 # [ 84.500381] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15530worker1 # [ 84.509068] nix-grpc-daemon[1230]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5531worker1 # [ 84.510458] nix-grpc-daemon[1230]: E0000 00:00:1789983335.163776 1230 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5532worker1 # [ 84.515912] nix-grpc-daemon[1230]: E0000 00:00:1789983335.169047 1230 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5533worker1 # [ 84.519753] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5534worker1 # [ 84.523200] systemd[1]: Started Nix worker-protocol over gRPC.5535lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5536lb: must succeed: curl -sf localhost:9901/clusters5537worker1 # [ 84.530078] nix-grpc-daemon[1230]: E0000 00:00:1789983335.185152 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5538worker1 # [ 84.541595] nix-grpc-daemon[1230]: E0000 00:00:1789983335.197272 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5539worker1 # [ 84.545149] nix-grpc-daemon[1230]: E0000 00:00:1789983335.199479 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5540worker1 # [ 84.555795] nix-grpc-daemon[1230]: E0000 00:00:1789983335.211467 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5541worker1 # [ 84.566531] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:35Z level=info event=scheduler_connected addr=lb:500515542worker2 # [ 84.564334] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:518645543worker2 # [ 84.569343] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05544lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5545lb: must succeed: curl -sf localhost:9901/clusters5546lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5547lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.17 seconds)5548(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.61 seconds)5549subtest: scheduler down: the other node takes the lock and keeps it5550lb: must succeed: curl -sf localhost:9901/clusters5551lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5552lb: must succeed: curl -sf localhost:9901/clusters5553lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5554lb: must succeed: curl -sf localhost:9901/clusters5555lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5556lb: must succeed: curl -sf localhost:9901/clusters5557lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5558worker2 # [ 85.994105] systemd[1]: Stopping Nix worker-protocol over gRPC...5559worker2 # [ 86.451588] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:37Z level=info event=signal_exit5560worker2 # [ 86.661646] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:37Z level=info event=scheduler_yield5561lb # [ 86.672845] niks3-server[1059]: 2026/09/21 09:35:37 INFO lead: released remote=192.168.1.4:503625562worker1 # [ 86.671315] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:37Z level=info event=scheduler_restarting addr=lb:500515563worker2 # [ 86.669734] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:518645564worker2 # [ 86.675945] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:37Z level=info event=worker_gone addr=192.168.1.3:500515565worker2 # [ 86.679294] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:37Z level=info event=scheduler_restarting addr=lb:500515566worker2 # [ 86.682490] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:518645567worker2 # [ 86.686104] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:37Z level=info event=worker_gone addr=192.168.1.4:500515568lb # [ 86.843117] envoy[900]: conn peer=[2001:db8:1::4]:54156 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=61206 tx=2872 ms=602285569lb # [ 86.846974] envoy[900]: conn peer=[2001:db8:1::1]:58102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215570lb # [ 86.850484] envoy[900]: conn peer=192.168.1.3:35108 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=19243 tx=3682 ms=632735571lb # [ 86.854077] envoy[900]: conn peer=[2001:db8:1::1]:44912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1009 tx=282 ms=11115572worker1 # [ 86.871715] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:37Z level=info event=scheduler_connected addr=lb:500515573worker1 # [ 86.875989] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5574worker2 # [ 86.875936] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515575worker2 # [ 86.879910] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5576worker1 # [ 87.076807] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:37Z level=info event=scheduler_connected addr=lb:500515577worker1 # [ 87.080845] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5578worker2 # [ 87.081284] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515579worker2 # [ 87.085213] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5580worker1 # [ 87.281901] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:37Z level=info event=scheduler_connected addr=lb:500515581worker1 # [ 87.286067] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5582worker2 # [ 87.286365] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515583worker2 # [ 87.290396] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5584worker1 # [ 87.487365] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515585worker1 # [ 87.491792] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5586worker2 # [ 87.492476] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515587worker2 # [ 87.495673] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5588worker2 # [ 87.666327] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=scheduler_restarting5589worker2 # [ 87.689636] nix-daemon[871]: reaped child process 1159, status = succeeded5590worker1 # [ 87.693503] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515591worker1 # [ 87.697814] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5592worker2 # [ 87.704457] nix-grpc-daemon[1107]: ts=2026-09-21T09:35:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5593worker1 # [ 87.898823] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515594worker1 # [ 87.903424] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5595worker1 # [ 88.104415] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515596worker1 # [ 88.109622] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5597worker1 # [ 88.310800] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_connected addr=lb:500515598worker1 # [ 88.315377] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5599worker2 # [ 88.468680] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5600worker2 # [ 88.474365] systemd[1]: Stopped Nix worker-protocol over gRPC.5601worker2 # [ 88.476937] systemd[1]: nix-grpc-daemon.service: Consumed 393ms CPU time over 8.257s wall clock time, 33.1M memory peak, 20.2K incoming IP traffic, 31.5K outgoing IP traffic.5602worker2 # [ 88.488594] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5603worker2 # [ 88.492061] systemd[1]: Closed Nix worker-protocol over gRPC.5604worker2 # [ 88.494639] systemd[1]: nix-grpc-daemon.socket: Consumed 298us CPU time over 9.285s wall clock time, 256K memory peak, 20.7K incoming IP traffic, 15.4K outgoing IP traffic.5605worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5606worker1 # [ 88.517907] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_connected addr=lb:500515607worker1 # [ 88.521622] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5608worker1 # [ 88.719869] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_connected addr=lb:500515609worker1 # [ 88.724298] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5610worker1 # [ 88.924981] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_connected addr=lb:500515611worker1 # [ 88.929633] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5612worker1 # [ 89.130704] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_connected addr=lb:500515613worker1 # [ 89.135301] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5614worker1 # [ 89.336400] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_connected addr=lb:500515615worker1 # [ 89.340971] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5616lb # [ 89.532477] niks3-server[1059]: 2026/09/21 09:35:40 INFO lead: acquired remote=192.168.1.3:514985617worker1 # [ 89.529547] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:40Z level=info event=scheduler_take_over5618worker1 # [ 89.541711] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:40Z level=info event=scheduler_connected addr=lb:500515619worker1 # [ 89.546094] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5620worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.16 seconds)5621worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5622worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5623worker1 # [ 89.746728] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:40Z level=info event=scheduler_connected addr=lb:500515624worker1 # [ 89.750966] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:587105625worker1 # [ 89.753636] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05626worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5627worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5628client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25629worker1 # [ 91.096961] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587245630worker1 # [ 91.109823] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5631client # don't know how to build these paths:5632client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5633worker1 # [ 91.126941] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587245634worker1 # [ 91.141070] nix-grpc-daemon[1230]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)5635worker1 # [ 91.191697] nix-grpc-daemon[1230]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 2/5)5636worker1 # [ 91.211670] nix-grpc-daemon[1230]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 359 ms (attempt 3/5)5637worker1 # [ 91.578266] nix-grpc-daemon[1230]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 712 ms (attempt 4/5)5638worker1 # [ 92.291897] nix-grpc-daemon[1230]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5639worker1 # [ 92.297459] nix-grpc-daemon[1230]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5640worker1 # [ 92.330793] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:42Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05641worker1 # [ 92.336904] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:42Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15642worker1 # [ 92.387731] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587245643worker1 # [ 92.401626] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587245644client # copying 1 paths to 'lb:50051'...5645client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5646worker1 # [ 92.414760] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587245647worker1 # [ 92.426378] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5648worker1 # [ 92.446370] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:43Z level=info event=niks3_push_started pid=13105649worker1 # [ 92.689288] nix-grpc-daemon[1310]: time=2026-09-21T09:35:43.344Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5650worker1 # [ 92.897281] nix-daemon[851]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5651worker1 # [ 92.910527] nix-daemon[851]: reaped child process 1326, status = succeeded5652worker1 # [ 93.125456] nix-daemon[851]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5653worker1 # [ 93.138598] nix-daemon[851]: reaped child process 1331, status = succeeded5654lb # [ 93.165319] niks3-server[1059]: 2026/09/21 09:35:44 INFO Received uploads request method=POST path=/api/pending_closures5655worker1 # [ 93.170117] nix-grpc-daemon[1310]: time=2026-09-21T09:35:43.825Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5656worker1 # [ 93.171774] nix-grpc-daemon[1310]: time=2026-09-21T09:35:43.827Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5657lb # [ 93.207290] niks3-server[1059]: 2026/09/21 09:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5658lb # [ 93.212867] niks3-server[1059]: 2026/09/21 09:35:44 INFO Signed narinfos id=27 count=15659worker1 # [ 93.213287] nix-grpc-daemon[1310]: time=2026-09-21T09:35:43.868Z level=INFO msg="Uploading 1 narinfos"5660lb # [ 93.219648] niks3-server[1059]: 2026/09/21 09:35:44 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5661lb # [ 93.229982] niks3-server[1059]: 2026/09/21 09:35:44 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5662lb # [ 93.238590] niks3-server[1059]: 2026/09/21 09:35:44 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5663lb # [ 93.246694] niks3-server[1059]: 2026/09/21 09:35:44 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5664lb # [ 93.251080] niks3-server[1059]: 2026/09/21 09:35:44 INFO Completed upload id=275665worker1 # [ 93.247894] nix-grpc-daemon[1310]: time=2026-09-21T09:35:43.903Z level=INFO msg="Upload complete. (554ms)"5666worker1 # [ 93.251663] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58724 duration_s=0 paths=1 nar_bytes_in=5205667worker1 # [ 93.257637] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587245668worker1 # [ 93.266156] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5669worker1 # [ 93.275039] nix-daemon[851]: reaped child process 1308, status = succeeded5670worker1 # [ 93.280149] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5671client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5672client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5673client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5674client # farm-a> LOG-farm-a-on-standby5675worker1 # [ 94.508555] nix-daemon[851]: reaped child process 1335, status = succeeded5676worker1 # [ 94.718254] nix-daemon[851]: accepted connection from pid 1351, user nix-grpc-daemon (trusted)5677worker1 # [ 94.731428] nix-daemon[851]: reaped child process 1354, status = succeeded5678lb # [ 94.759524] niks3-server[1059]: 2026/09/21 09:35:45 INFO Received uploads request method=POST path=/api/pending_closures5679worker1 # [ 94.766172] nix-grpc-daemon[1310]: time=2026-09-21T09:35:45.421Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5680worker1 # [ 94.767702] nix-grpc-daemon[1310]: time=2026-09-21T09:35:45.423Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5681lb # [ 94.806075] niks3-server[1059]: 2026/09/21 09:35:45 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5682lb # [ 94.810390] niks3-server[1059]: 2026/09/21 09:35:45 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5683lb # [ 94.818954] niks3-server[1059]: 2026/09/21 09:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5684lb # [ 94.824408] niks3-server[1059]: 2026/09/21 09:35:45 INFO Signed narinfos id=28 count=15685worker1 # [ 94.824152] nix-grpc-daemon[1310]: time=2026-09-21T09:35:45.479Z level=INFO msg="Uploading 1 narinfos"5686lb # [ 94.827421] niks3-server[1059]: 2026/09/21 09:35:45 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5687lb # [ 94.835063] niks3-server[1059]: 2026/09/21 09:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5688lb # [ 94.839212] niks3-server[1059]: 2026/09/21 09:35:45 INFO Completed upload id=285689lb # [ 94.841094] niks3-server[1059]: 2026/09/21 09:35:45 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5690worker1 # [ 94.837289] nix-grpc-daemon[1310]: time=2026-09-21T09:35:45.492Z level=INFO msg="Upload complete. (348ms)"5691worker1 # [ 94.845566] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58724 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15692worker1 # [ 94.861602] nix-daemon[851]: reaped child process 1333, status = succeeded5693worker1 # [ 94.867234] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:45Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05694worker1 # [ 94.869383] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:45Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15695worker1 # [ 94.913116] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587245696worker1 # [ 94.926414] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587245697worker1 # [ 94.939659] nix-grpc-daemon[1310]: time=2026-09-21T09:35:45.595Z level=INFO msg="All 1 paths already cached"5698client # copying 1 paths to 'lb:50051'...5699client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5700worker1 # [ 94.948451] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587245701worker1 # [ 94.959230] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5702worker1 # [ 95.170695] nix-daemon[851]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5703worker1 # [ 95.184364] nix-daemon[851]: reaped child process 1363, status = succeeded5704worker1 # [ 95.368150] nix-daemon[851]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5705worker1 # [ 95.381334] nix-daemon[851]: reaped child process 1368, status = succeeded5706lb # [ 95.404735] niks3-server[1059]: 2026/09/21 09:35:46 INFO Received uploads request method=POST path=/api/pending_closures5707worker1 # [ 95.411609] nix-grpc-daemon[1310]: time=2026-09-21T09:35:46.066Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5708worker1 # [ 95.413071] nix-grpc-daemon[1310]: time=2026-09-21T09:35:46.067Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5709lb # [ 95.431963] niks3-server[1059]: 2026/09/21 09:35:46 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5710lb # [ 95.444786] niks3-server[1059]: 2026/09/21 09:35:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5711worker1 # [ 95.446281] nix-grpc-daemon[1310]: time=2026-09-21T09:35:46.101Z level=INFO msg="Uploading 1 narinfos"5712lb # [ 95.451128] niks3-server[1059]: 2026/09/21 09:35:46 INFO Signed narinfos id=29 count=15713lb # [ 95.456359] niks3-server[1059]: 2026/09/21 09:35:46 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5714lb # [ 95.467333] niks3-server[1059]: 2026/09/21 09:35:46 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5715lb # [ 95.474546] niks3-server[1059]: 2026/09/21 09:35:46 INFO Completed upload id=295716worker1 # [ 95.470960] nix-grpc-daemon[1310]: time=2026-09-21T09:35:46.126Z level=INFO msg="Upload complete. (499ms)"5717lb # [ 95.477722] niks3-server[1059]: 2026/09/21 09:35:46 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5718worker1 # [ 95.474723] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58724 duration_s=0 paths=1 nar_bytes_in=6405719worker1 # [ 95.480958] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587245720worker1 # [ 95.492364] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5721worker1 # [ 95.494415] nix-daemon[851]: reaped child process 1358, status = succeeded5722worker1 # [ 95.499589] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5723client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5724client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5725client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5726client # farm-b> LOG-farm-b-on-standby5727worker1 # [ 96.528530] nix-daemon[851]: reaped child process 1372, status = succeeded5728worker1 # [ 96.702534] nix-daemon[851]: accepted connection from pid 1388, user nix-grpc-daemon (trusted)5729worker1 # [ 96.715555] nix-daemon[851]: reaped child process 1391, status = succeeded5730lb # [ 96.735686] niks3-server[1059]: 2026/09/21 09:35:47 INFO Received uploads request method=POST path=/api/pending_closures5731worker1 # [ 96.743157] nix-grpc-daemon[1310]: time=2026-09-21T09:35:47.398Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5732worker1 # [ 96.744638] nix-grpc-daemon[1310]: time=2026-09-21T09:35:47.400Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5733lb # [ 96.776570] niks3-server[1059]: 2026/09/21 09:35:47 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5734lb # [ 96.785227] niks3-server[1059]: 2026/09/21 09:35:47 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5735lb # [ 96.795073] niks3-server[1059]: 2026/09/21 09:35:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5736lb # [ 96.799149] niks3-server[1059]: 2026/09/21 09:35:47 INFO Signed narinfos id=30 count=15737worker1 # [ 96.799411] nix-grpc-daemon[1310]: time=2026-09-21T09:35:47.454Z level=INFO msg="Uploading 1 narinfos"5738lb # [ 96.805084] niks3-server[1059]: 2026/09/21 09:35:47 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5739lb # [ 96.817554] niks3-server[1059]: 2026/09/21 09:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5740lb # [ 96.820931] niks3-server[1059]: 2026/09/21 09:35:47 INFO Completed upload id=305741worker1 # [ 96.817846] nix-grpc-daemon[1310]: time=2026-09-21T09:35:47.473Z level=INFO msg="Upload complete. (306ms)"5742lb # [ 96.823240] niks3-server[1059]: 2026/09/21 09:35:47 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5743worker1 # [ 96.827341] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58724 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15744worker1 # [ 96.837301] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:47Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05745lb # [ 96.842792] envoy[900]: conn peer=[2001:db8:1::4]:46716 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8603 tx=512 ms=71995746worker1 # [ 96.840333] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:47Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15747worker1 # [ 96.849924] nix-daemon[851]: reaped child process 1370, status = succeeded5748worker1 # [ 96.881526] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587245749worker1 # [ 96.894457] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587245750worker1 # [ 96.907428] nix-grpc-daemon[1310]: time=2026-09-21T09:35:47.563Z level=INFO msg="All 2 paths already cached"5751client # copying 1 paths to 'lb:50051'...5752client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5753worker1 # [ 96.915654] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587245754worker1 # [ 96.923882] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5755worker1 # [ 97.106434] nix-daemon[851]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)5756worker1 # [ 97.119633] nix-daemon[851]: reaped child process 1400, status = succeeded5757worker1 # [ 97.288781] nix-daemon[851]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)5758worker1 # [ 97.301763] nix-daemon[851]: reaped child process 1405, status = succeeded5759lb # [ 97.321066] niks3-server[1059]: 2026/09/21 09:35:48 INFO Received uploads request method=POST path=/api/pending_closures5760worker1 # [ 97.326410] nix-grpc-daemon[1310]: time=2026-09-21T09:35:47.982Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5761worker1 # [ 97.327965] nix-grpc-daemon[1310]: time=2026-09-21T09:35:47.983Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5762lb # [ 97.351635] niks3-server[1059]: 2026/09/21 09:35:48 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5763lb # [ 97.362635] niks3-server[1059]: 2026/09/21 09:35:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5764lb # [ 97.367256] niks3-server[1059]: 2026/09/21 09:35:48 INFO Signed narinfos id=31 count=15765worker1 # [ 97.367212] nix-grpc-daemon[1310]: time=2026-09-21T09:35:48.022Z level=INFO msg="Uploading 1 narinfos"5766lb # [ 97.374548] niks3-server[1059]: 2026/09/21 09:35:48 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5767lb # [ 97.387458] niks3-server[1059]: 2026/09/21 09:35:48 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5768lb # [ 97.392314] niks3-server[1059]: 2026/09/21 09:35:48 INFO Completed upload id=315769lb # [ 97.396044] niks3-server[1059]: 2026/09/21 09:35:48 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5770worker1 # [ 97.391303] nix-grpc-daemon[1310]: time=2026-09-21T09:35:48.046Z level=INFO msg="Upload complete. (454ms)"5771worker1 # [ 97.395771] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58724 duration_s=0 paths=1 nar_bytes_in=7685772worker1 # [ 97.403350] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587245773worker1 # [ 97.412134] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5774worker1 # [ 97.420276] nix-daemon[851]: reaped child process 1395, status = succeeded5775client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5776worker1 # [ 97.426158] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5777client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5778client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5779client # farm-top> LOG-farm-top-on-standby5780worker1 # [ 99.054233] nix-daemon[851]: reaped child process 1409, status = succeeded5781worker1 # [ 99.263100] nix-daemon[851]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)5782worker1 # [ 99.275574] nix-daemon[851]: reaped child process 1428, status = succeeded5783lb # [ 99.300991] niks3-server[1059]: 2026/09/21 09:35:50 INFO Received uploads request method=POST path=/api/pending_closures5784worker1 # [ 99.309335] nix-grpc-daemon[1310]: time=2026-09-21T09:35:49.964Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5785worker1 # [ 99.313165] nix-grpc-daemon[1310]: time=2026-09-21T09:35:49.966Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5786lb # [ 99.333482] niks3-server[1059]: 2026/09/21 09:35:50 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5787lb # [ 99.338131] niks3-server[1059]: 2026/09/21 09:35:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5788lb # [ 99.341524] niks3-server[1059]: 2026/09/21 09:35:50 INFO Signed narinfos id=32 count=15789worker1 # [ 99.338479] nix-grpc-daemon[1310]: time=2026-09-21T09:35:49.994Z level=INFO msg="Uploading 1 narinfos"5790lb # [ 99.344246] niks3-server[1059]: 2026/09/21 09:35:50 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5791lb # [ 99.348836] niks3-server[1059]: 2026/09/21 09:35:50 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5792lb # [ 99.353061] niks3-server[1059]: 2026/09/21 09:35:50 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5793lb # [ 99.357114] niks3-server[1059]: 2026/09/21 09:35:50 INFO Completed upload id=325794worker1 # [ 99.353756] nix-grpc-daemon[1310]: time=2026-09-21T09:35:50.009Z level=INFO msg="Upload complete. (319ms)"5795lb # [ 99.358962] niks3-server[1059]: 2026/09/21 09:35:50 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5796worker1 # [ 99.356361] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58724 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15797worker1 # [ 99.364519] nix-daemon[851]: reaped child process 1407, status = succeeded5798client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 8.60 seconds)5799client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5800client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.25 seconds)5801worker2 # [ 99.725872] systemd[1]: Listening on Nix worker-protocol over gRPC.5802worker2 # [ 99.733326] systemd[1]: Starting scheduler-token.service...5803worker2 # [ 99.812063] systemd[1]: scheduler-token.service: Deactivated successfully.5804worker2 # [ 99.813709] systemd[1]: Finished scheduler-token.service.5805worker2 # [ 99.816271] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5806worker2 # [ 99.820778] systemd[1]: Starting Nix worker-protocol over gRPC...5807worker2 # [ 99.982905] nix-grpc-daemon[1207]: ts=2026-09-21T09:35:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5808worker2 # [ 99.985409] nix-grpc-daemon[1207]: ts=2026-09-21T09:35:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5809worker2 # [ 99.989381] nix-grpc-daemon[1207]: ts=2026-09-21T09:35:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15810worker2 # [ 99.997067] nix-grpc-daemon[1207]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5811worker2 # [ 99.998410] nix-grpc-daemon[1207]: E0000 00:00:1789983351.190400 1207 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5812worker2 # [ 100.003956] nix-grpc-daemon[1207]: E0000 00:00:1789983351.195662 1207 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5813worker2 # [ 100.008522] nix-grpc-daemon[1207]: ts=2026-09-21T09:35:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5814worker2 # [ 100.011427] systemd[1]: Started Nix worker-protocol over gRPC.5815lb: must succeed: curl -sf localhost:9901/clusters5816worker2 # [ 100.017080] nix-grpc-daemon[1207]: E0000 00:00:1789983351.210382 1249 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5817worker2 # [ 100.029312] nix-grpc-daemon[1207]: E0000 00:00:1789983351.223012 1249 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5818worker2 # [ 100.034059] nix-grpc-daemon[1207]: E0000 00:00:1789983351.225392 1249 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5819worker2 # [ 100.044298] nix-grpc-daemon[1207]: E0000 00:00:1789983351.238001 1249 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5820worker2 # [ 100.056212] nix-grpc-daemon[1207]: ts=2026-09-21T09:35:51Z level=info event=scheduler_connected addr=lb:500515821worker1 # [ 100.070199] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:50Z level=info event=oidc_jwks provider=mock keys=15822worker1 # [ 100.074530] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:50Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:587105823worker1 # [ 100.078446] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05824lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5825worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5826worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5827lb: must succeed: curl -sf localhost:9901/clusters5828lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5829lb: must succeed: curl -sf localhost:9901/clusters5830lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5831worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5832worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5833(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.69 seconds)5834subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5835client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5836client # [ 100.460858] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5837client # Running as unit: rs.service; invocation ID: 2980e754e74f470e85bcf04cf715b1ed5838client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5839worker2 # [ 100.697898] nix-grpc-daemon[1207]: ts=2026-09-21T09:35:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:345985840worker2 # [ 100.710633] nix-daemon[871]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)5841client # [ 100.741290] nix[1172]: don't know how to build these paths:5842client # [ 100.742280] nix[1172]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5843worker1 # [ 100.738933] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:51Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05844worker1 # [ 100.743435] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:51Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15845worker1 # [ 100.782175] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587245846worker1 # [ 100.795845] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587245847client # [ 100.819214] nix[1172]: copying 1 paths to 'lb:50051'...5848worker1 # [ 100.807705] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587245849client # [ 100.822899] nix[1172]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5850worker1 # [ 100.819247] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5851worker1 # [ 101.041191] nix-daemon[851]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5852worker1 # [ 101.054443] nix-daemon[851]: reaped child process 1444, status = succeeded5853worker1 # [ 101.267613] nix-daemon[851]: accepted connection from pid 1446, user nix-grpc-daemon (trusted)5854worker1 # [ 101.280873] nix-daemon[851]: reaped child process 1449, status = succeeded5855lb # [ 101.306326] niks3-server[1059]: 2026/09/21 09:35:52 INFO Received uploads request method=POST path=/api/pending_closures5856worker1 # [ 101.312190] nix-grpc-daemon[1310]: time=2026-09-21T09:35:51.967Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5857worker1 # [ 101.313719] nix-grpc-daemon[1310]: time=2026-09-21T09:35:51.967Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5858lb # [ 101.335894] niks3-server[1059]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5859lb # [ 101.344643] niks3-server[1059]: 2026/09/21 09:35:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5860worker1 # [ 101.346565] nix-grpc-daemon[1310]: time=2026-09-21T09:35:52.002Z level=INFO msg="Uploading 1 narinfos"5861lb # [ 101.350574] niks3-server[1059]: 2026/09/21 09:35:52 INFO Signed narinfos id=33 count=15862lb # [ 101.355430] niks3-server[1059]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5863lb # [ 101.365935] niks3-server[1059]: 2026/09/21 09:35:52 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5864lb # [ 101.373619] niks3-server[1059]: 2026/09/21 09:35:52 INFO Completed upload id=335865worker1 # [ 101.370411] nix-grpc-daemon[1310]: time=2026-09-21T09:35:52.026Z level=INFO msg="Upload complete. (538ms)"5866lb # [ 101.377053] niks3-server[1059]: 2026/09/21 09:35:52 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5867worker1 # [ 101.373979] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58724 duration_s=0 paths=1 nar_bytes_in=4165868worker1 # [ 101.381135] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587245869worker1 # [ 101.390820] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5870worker1 # [ 101.393243] nix-daemon[851]: reaped child process 1439, status = succeeded5871worker1 # [ 101.398939] nix-daemon[851]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5872client # [ 101.411491] nix[1172]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5873lb: must succeed: curl -sf localhost:9901/clusters5874lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5875lb: must succeed: curl -sf localhost:9901/clusters5876lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5877lb: must succeed: curl -sf localhost:9901/clusters5878lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5879lb: must succeed: curl -sf localhost:9901/clusters5880lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5881worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5882worker2 # [ 103.484934] systemd-journald[287]: Received client request to rotate journal, rotating.5883worker2 # [ 103.504521] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/8bdfad3fbfad47f88ed8af8d3f2ffd3c.5884worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5885worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5886worker1 # [ 103.436654] systemd[1]: Stopping Nix worker-protocol over gRPC...5887worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5888client: must succeed: sleep 5.05889worker1 # [ 103.522442] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:54Z level=info event=draining5890lb # [ 106.843539] envoy[900]: conn peer=[2001:db8:1::1]:44926 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9650 tx=4156 ms=82825891client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5892worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5893worker1 # [ 108.826908] nix-daemon[851]: reaped child process 1453, status = succeeded5894worker1 # [ 109.025856] nix-daemon[851]: accepted connection from pid 1494, user nix-grpc-daemon (trusted)5895worker1 # [ 109.039194] nix-daemon[851]: reaped child process 1497, status = succeeded5896lb # [ 109.067781] niks3-server[1059]: 2026/09/21 09:35:59 INFO Received uploads request method=POST path=/api/pending_closures5897worker1 # [ 109.073409] nix-grpc-daemon[1310]: time=2026-09-21T09:35:59.728Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5898worker1 # [ 109.075244] nix-grpc-daemon[1310]: time=2026-09-21T09:35:59.730Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5899lb # [ 109.108391] niks3-server[1059]: 2026/09/21 09:36:00 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5900lb # [ 109.124913] niks3-server[1059]: 2026/09/21 09:36:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5901lb # [ 109.130977] niks3-server[1059]: 2026/09/21 09:36:00 INFO Signed narinfos id=34 count=15902worker1 # [ 109.132056] nix-grpc-daemon[1310]: time=2026-09-21T09:35:59.787Z level=INFO msg="Uploading 1 narinfos"5903lb # [ 109.136375] niks3-server[1059]: 2026/09/21 09:36:00 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5904lb # [ 109.141860] niks3-server[1059]: 2026/09/21 09:36:00 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5905lb # [ 109.150129] niks3-server[1059]: 2026/09/21 09:36:00 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5906lb # [ 109.153387] niks3-server[1059]: 2026/09/21 09:36:00 INFO Completed upload id=345907lb # [ 109.155695] niks3-server[1059]: 2026/09/21 09:36:00 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5908worker1 # [ 109.151347] nix-grpc-daemon[1310]: time=2026-09-21T09:35:59.807Z level=INFO msg="Upload complete. (338ms)"5909worker1 # [ 109.160625] nix-grpc-daemon[1230]: ts=2026-09-21T09:35:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58724 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15910worker1 # [ 109.174152] nix-daemon[851]: reaped child process 1451, status = succeeded5911client # [ 109.213466] systemd[1]: rs.service: Deactivated successfully.5912client # [ 109.217174] systemd[1]: rs.service: Consumed 138ms CPU time over 8.755s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5913worker1 # [ 109.524941] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=signal_exit5914worker1 # [ 109.529915] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=scheduler_yield5915lb # [ 109.534296] niks3-server[1059]: 2026/09/21 09:36:00 INFO lead: released remote=192.168.1.3:514985916worker1 # [ 109.535661] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=scheduler_restarting addr=lb:500515917worker2 # [ 109.533757] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:00Z level=info event=scheduler_restarting addr=lb:500515918worker1 # [ 109.540987] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:587105919worker1 # [ 109.544964] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=worker_gone addr=192.168.1.3:500515920worker1 # [ 109.548777] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:587105921worker1 # [ 109.552577] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=worker_gone addr=192.168.1.4:500515922worker2 # [ 109.734087] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:00Z level=info event=scheduler_connected addr=lb:500515923worker1 # [ 109.740604] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=scheduler_connected addr=lb:500515924worker2 # [ 109.738729] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5925worker1 # [ 109.745123] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5926worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.11 seconds)5927client: waiting for success: ! systemctl is-active rs5928worker2 # [ 109.939973] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_connected addr=lb:500515929worker1 # [ 109.945977] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=scheduler_connected addr=lb:500515930worker2 # [ 109.944691] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5931worker1 # [ 109.950609] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5932client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5933client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5934client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5935client: must fail: journalctl -u rs | grep -q 'reconnecting'5936lb # [ 110.021829] niks3-server[1059]: 2026/09/21 09:36:00 INFO lead: acquired remote=192.168.1.4:356805937worker2 # [ 110.014912] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_take_over5938client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5939lb: must succeed: curl -sf localhost:9901/clusters5940lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5941worker1 # [ 110.148105] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=scheduler_connected addr=lb:500515942worker2 # [ 110.145955] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_connected addr=lb:500515943worker1 # [ 110.153100] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5944worker2 # [ 110.150295] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5945worker2 # [ 110.348403] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_connected addr=lb:500515946worker1 # [ 110.355453] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:01Z level=info event=scheduler_connected addr=lb:500515947worker2 # [ 110.352953] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5948worker1 # [ 110.359312] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5949worker1 # [ 110.530203] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:01Z level=info event=scheduler_restarting5950worker2 # [ 110.555409] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_connected addr=lb:500515951worker2 # [ 110.558868] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5952worker1 # [ 110.570895] nix-daemon[851]: reaped child process 1300, status = succeeded5953worker1 # [ 110.573349] nix-grpc-daemon[1230]: ts=2026-09-21T09:36:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5954worker1 # [ 110.691152] nix-grpc-daemon[1230]: E0000 00:00:1789983361.346386 1275 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5955worker1 # [ 110.696877] nix-grpc-daemon[1230]: E0000 00:00:1789983361.346588 1275 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5956worker2 # [ 110.756298] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_connected addr=lb:500515957worker2 # [ 110.760635] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5958worker2 # [ 110.961807] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=scheduler_connected addr=lb:500515959worker2 # [ 110.966241] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5960lb: must succeed: curl -sf localhost:9901/clusters5961worker2 # [ 111.167555] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=scheduler_connected addr=lb:500515962worker2 # [ 111.171442] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5963lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5964worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5965worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5966worker2 # [ 111.372412] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=scheduler_connected addr=lb:500515967worker2 # [ 111.376295] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5968worker2 # [ 111.578125] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=scheduler_connected addr=lb:500515969worker2 # [ 111.592183] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=oidc_jwks provider=mock keys=15970worker2 # [ 111.595796] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:594525971worker2 # [ 111.599656] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05972worker1 # [ 112.230666] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5973worker1 # [ 112.233092] systemd[1]: Stopped Nix worker-protocol over gRPC.5974worker1 # [ 112.235090] systemd[1]: nix-grpc-daemon.service: Consumed 1.398s CPU time over 27.948s wall clock time, 36M memory peak, 48.6K incoming IP traffic, 95.4K outgoing IP traffic.5975worker1 # [ 112.250986] systemd[1]: Starting Nix worker-protocol over gRPC...5976lb: must succeed: curl -sf localhost:9901/clusters5977lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5978worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5979worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5980worker1 # [ 112.472925] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5981worker1 # [ 112.475233] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5982worker1 # [ 112.479773] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15983worker1 # [ 112.488071] nix-grpc-daemon[1504]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5984worker1 # [ 112.489651] nix-grpc-daemon[1504]: E0000 00:00:1789983363.143520 1504 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5985worker1 # [ 112.495738] nix-grpc-daemon[1504]: E0000 00:00:1789983363.149374 1504 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5986worker1 # [ 112.498970] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5987worker1 # [ 112.500874] systemd[1]: Started Nix worker-protocol over gRPC.5988worker1 # [ 112.508069] nix-grpc-daemon[1504]: E0000 00:00:1789983363.163544 1545 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5989worker1 # [ 112.520100] nix-grpc-daemon[1504]: E0000 00:00:1789983363.175835 1545 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5990worker1 # [ 112.525057] nix-grpc-daemon[1504]: E0000 00:00:1789983363.178458 1545 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5991worker1 # [ 112.535218] nix-grpc-daemon[1504]: E0000 00:00:1789983363.190892 1545 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5992worker1 # [ 112.545069] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:03Z level=info event=scheduler_connected addr=lb:500515993worker2 # [ 112.543078] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:03Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:594525994worker2 # [ 112.547189] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05995lb: must succeed: curl -sf localhost:9901/clusters5996lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5997worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5998worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5999(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.22 seconds)6000subtest: scheduler restart mid-build: build finishes, no double build6001client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid6002client # [ 113.684461] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.6003client # Running as unit: mid.service; invocation ID: edd66da8d1d743a7b8bb84f84f917f246004client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.08 seconds)6005worker1 # [ 113.935364] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:460366006worker1 # [ 113.948209] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6007client # [ 113.974504] nix[1208]: don't know how to build these paths:6008client # [ 113.975487] nix[1208]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6009worker2 # [ 113.970456] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:05Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06010worker2 # [ 113.974517] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:05Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16011worker2 # [ 114.014557] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345986012worker2 # [ 114.027827] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 1/5)6013worker2 # [ 114.133063] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 2/5)6014worker2 # [ 114.205771] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 339 ms (attempt 3/5)6015worker2 # [ 114.550697] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 796 ms (attempt 4/5)6016worker2 # [ 115.353870] nix-grpc-daemon[1207]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6017worker2 # [ 115.357831] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6018worker2 # [ 115.377909] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:345986019client # [ 115.406978] nix[1208]: copying 1 paths to 'lb:50051'...6020worker2 # [ 115.390785] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:345986021client # [ 115.410402] nix[1208]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6022worker2 # [ 115.405982] nix-daemon[871]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)6023worker2 # [ 115.427574] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:06Z level=info event=niks3_push_started pid=13306024worker2 # [ 115.662878] nix-grpc-daemon[1330]: time=2026-09-21T09:36:06.855Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6025worker2 # [ 115.863139] nix-daemon[871]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)6026worker2 # [ 115.875364] nix-daemon[871]: reaped child process 1346, status = succeeded6027worker2 # [ 116.086035] nix-daemon[871]: accepted connection from pid 1348, user nix-grpc-daemon (trusted)6028worker2 # [ 116.102581] nix-daemon[871]: reaped child process 1354, status = succeeded6029lb # [ 116.125717] niks3-server[1059]: 2026/09/21 09:36:07 INFO Received uploads request method=POST path=/api/pending_closures6030worker2 # [ 116.129638] nix-grpc-daemon[1330]: time=2026-09-21T09:36:07.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6031worker2 # [ 116.131058] nix-grpc-daemon[1330]: time=2026-09-21T09:36:07.321Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6032lb # [ 116.160505] niks3-server[1059]: 2026/09/21 09:36:07 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6033lb # [ 116.174690] niks3-server[1059]: 2026/09/21 09:36:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign6034worker2 # [ 116.175796] nix-grpc-daemon[1330]: time=2026-09-21T09:36:07.369Z level=INFO msg="Uploading 1 narinfos"6035lb # [ 116.182770] niks3-server[1059]: 2026/09/21 09:36:07 INFO Signed narinfos id=35 count=16036lb # [ 116.199426] niks3-server[1059]: 2026/09/21 09:36:07 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete6037lb # [ 116.207617] niks3-server[1059]: 2026/09/21 09:36:07 INFO Completed upload id=356038worker2 # [ 116.199933] nix-grpc-daemon[1330]: time=2026-09-21T09:36:07.393Z level=INFO msg="Upload complete. (533ms)"6039worker2 # [ 116.201884] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34598 duration_s=0 paths=1 nar_bytes_in=4246040lb # [ 116.212606] niks3-server[1059]: 2026/09/21 09:36:07 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6041worker2 # [ 116.206193] nix-grpc-daemon[1207]: ts=2026-09-21T09:36:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:345986042worker2 # [ 116.208270] nix-grpc-daemon[1330]: time=2026-09-21T09:36:07.401Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"6043worker2 # [ 116.215286] nix-daemon[871]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)6044worker2 # [ 116.217559] nix-daemon[871]: reaped child process 1328, status = succeeded6045worker2 # [ 116.224106] nix-daemon[871]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)6046client # [ 116.239215] nix[1208]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6047lb # [ 116.844220] envoy[900]: conn peer=[2001:db8:1::1]:55862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1404 ms=84746048lb # [ 116.848432] envoy[900]: conn peer=[2001:db8:1::3]:32842 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5170 tx=1213 ms=259966049lb: must succeed: curl -sf localhost:9901/clusters6050lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6051lb: must succeed: curl -sf localhost:9901/clusters6052lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6053worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6054worker2 # [ 118.734546] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1207 (nix-grpc-daemon) on client request.6055worker2 # [ 118.737404] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6056lb # [ 118.755799] niks3-server[1059]: 2026/09/21 09:36:09 INFO lead: released remote=192.168.1.4:356806057worker2 # [ 118.761669] systemd[1]: Starting scheduler-token.service...6058client # [ 118.774936] nix[1208]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6059worker1 # [ 118.760757] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:09Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6060client # [ 118.779840] nix[1208]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6061worker2 # [ 118.771886] nix-daemon[871]: reaped child process 1266, status = succeeded6062worker2 # [ 118.775095] nix-daemon[871]: reaped child process 1357, status = succeeded6063worker2 # [ 118.781454] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6064worker2 # [ 118.783212] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6065worker2 # [ 118.785756] systemd[1]: nix-grpc-daemon.service: Consumed 481ms CPU time over 18.958s wall clock time, 33.2M memory peak, 17.5K incoming IP traffic, 39.4K outgoing IP traffic.6066worker2 # [ 118.794256] nix-daemon[1359]: unexpected Nix daemon error: error: interrupted by the user6067worker2 # [ 118.799503] nix-daemon[871]: reaped child process 1359, status = failed with exit code 16068worker2 # [ 118.856173] systemd[1]: scheduler-token.service: Deactivated successfully.6069worker2 # [ 118.857286] systemd[1]: Finished scheduler-token.service.6070worker2 # [ 118.858774] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 96ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6071worker2 # [ 118.861358] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6072worker2 # [ 118.866338] systemd[1]: Starting scheduler-token.service...6073worker2 # [ 118.942735] systemd[1]: scheduler-token.service: Deactivated successfully.6074worker2 # [ 118.944314] systemd[1]: Finished scheduler-token.service.6075worker2 # [ 118.945932] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 77ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6076worker2 # [ 118.951335] systemd[1]: Starting Nix worker-protocol over gRPC...6077worker1 # [ 118.961679] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:09Z level=info event=scheduler_connected addr=lb:500516078worker2 # [ 119.147275] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6079worker2 # [ 119.149406] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6080worker2 # [ 119.153382] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16081worker2 # [ 119.161065] nix-grpc-daemon[1402]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6082worker2 # [ 119.162402] nix-grpc-daemon[1402]: E0000 00:00:1789983370.354427 1402 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6083worker2 # [ 119.169450] nix-grpc-daemon[1402]: E0000 00:00:1789983370.359527 1402 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6084worker2 # [ 119.172167] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6085worker2 # [ 119.174413] systemd[1]: Started Nix worker-protocol over gRPC.6086worker2 # [ 119.181077] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=scheduler_take_over6087worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.49 seconds)6088lb # [ 119.188477] niks3-server[1059]: 2026/09/21 09:36:10 INFO lead: acquired remote=192.168.1.4:479306089client: must succeed: sleep 5.06090worker1 # [ 119.185119] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6091worker2 # [ 119.184082] nix-grpc-daemon[1402]: E0000 00:00:1789983370.377814 1443 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6092worker2 # [ 119.195881] nix-grpc-daemon[1402]: E0000 00:00:1789983370.389599 1443 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6093worker2 # [ 119.200053] nix-grpc-daemon[1402]: E0000 00:00:1789983370.391906 1443 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6094worker2 # [ 119.209978] nix-grpc-daemon[1402]: E0000 00:00:1789983370.403696 1443 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6095worker2 # [ 119.219100] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=scheduler_connected addr=lb:500516096worker2 # [ 119.224058] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=oidc_jwks provider=mock keys=16097worker2 # [ 119.225464] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:554386098worker2 # [ 119.227101] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06099worker2 # [ 119.271207] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06100worker2 # [ 119.273514] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16101worker2 # [ 119.320921] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554546102worker2 # [ 119.325933] nix-daemon[871]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)6103worker2 # [ 119.332036] nix-daemon[871]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)6104worker2 # [ 119.339079] nix-daemon[871]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)6105client # [ 119.355207] nix[1208]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6106worker2 # [ 119.383445] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:554386107worker1 # [ 119.386428] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:10Z level=info event=scheduler_connected addr=lb:500516108worker2 # [ 119.385084] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06109client: (finished: must succeed: sleep 5.0, in 5.06 seconds)6110client: waiting for success: ! systemctl is-active mid6111worker2 # [ 124.552326] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:15Z level=info event=niks3_push_started pid=14856112worker2 # [ 124.563361] nix-daemon[871]: reaped child process 1460, status = succeeded6113worker2 # [ 124.758826] nix-grpc-daemon[1485]: time=2026-09-21T09:36:15.952Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6114worker2 # [ 124.943963] nix-daemon[871]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6115worker2 # [ 124.955051] nix-daemon[871]: reaped child process 1501, status = succeeded6116lb # [ 124.981841] niks3-server[1059]: 2026/09/21 09:36:15 INFO Received uploads request method=POST path=/api/pending_closures6117worker2 # [ 124.983581] nix-grpc-daemon[1485]: time=2026-09-21T09:36:16.177Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6118worker2 # [ 124.985202] nix-grpc-daemon[1485]: time=2026-09-21T09:36:16.178Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6119lb # [ 125.028636] niks3-server[1059]: 2026/09/21 09:36:15 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6120worker2 # [ 125.030627] nix-grpc-daemon[1485]: time=2026-09-21T09:36:16.224Z level=INFO msg="Uploading 1 narinfos"6121lb # [ 125.037919] niks3-server[1059]: 2026/09/21 09:36:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6122lb # [ 125.040921] niks3-server[1059]: 2026/09/21 09:36:15 INFO Signed narinfos id=36 count=16123lb # [ 125.058232] niks3-server[1059]: 2026/09/21 09:36:15 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6124lb # [ 125.062056] niks3-server[1059]: 2026/09/21 09:36:15 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6125lb # [ 125.064058] niks3-server[1059]: 2026/09/21 09:36:15 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6126lb # [ 125.070704] niks3-server[1059]: 2026/09/21 09:36:15 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6127lb # [ 125.079734] niks3-server[1059]: 2026/09/21 09:36:16 INFO Completed upload id=366128worker2 # [ 125.072851] nix-grpc-daemon[1485]: time=2026-09-21T09:36:16.266Z level=INFO msg="Upload complete. (310ms)"6129worker2 # [ 125.079700] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55454 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16130worker2 # [ 125.092111] nix-daemon[871]: reaped child process 1458, status = succeeded6131client # [ 125.146425] systemd[1]: mid.service: Deactivated successfully.6132client # [ 125.148564] systemd[1]: mid.service: Consumed 157ms CPU time over 11.461s wall clock time, 41.4M memory peak, 8.5K incoming IP traffic, 10.4K outgoing IP traffic.6133client: (finished: waiting for success: ! systemctl is-active mid, in 1.13 seconds)6134client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6135client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6136(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 12.09 seconds)6137subtest: interrupting the client stops the build on the worker6138client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6139client # [ 125.748506] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6140client # Running as unit: intr.service; invocation ID: 3fcf8ea2097642ff961e48aded665e3a6141client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6142worker2 # [ 125.979494] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:554546143client # [ 126.003634] nix[1244]: don't know how to build these paths:6144client # [ 126.004599] nix[1244]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6145worker2 # [ 125.997847] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:17Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06146worker2 # [ 126.002462] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:17Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16147worker1 # [ 126.048417] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366148worker1 # [ 126.062087] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)6149worker1 # [ 126.094857] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 168 ms (attempt 2/5)6150worker1 # [ 126.269867] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 3/5)6151worker1 # [ 126.318529] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 615 ms (attempt 4/5)6152lb # [ 126.845175] envoy[900]: conn peer=[2001:db8:1::4]:51330 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=9471 tx=623 ms=187066153lb # [ 126.849401] envoy[900]: conn peer=[2001:db8:1::1]:57182 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3921 tx=1855 ms=111676154worker1 # [ 126.940908] nix-grpc-daemon[1504]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6155worker1 # [ 126.945852] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6156worker1 # [ 126.967603] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366157client # [ 126.991445] nix[1244]: copying 1 paths to 'lb:50051'...6158worker1 # [ 126.982896] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366159client # [ 126.995159] nix[1244]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6160worker1 # [ 126.990640] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6161worker1 # [ 127.003085] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:17Z level=info event=niks3_push_started pid=16046162worker1 # [ 127.242391] nix-grpc-daemon[1604]: time=2026-09-21T09:36:17.897Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6163worker1 # [ 127.461392] nix-daemon[851]: accepted connection from pid 1617, user nix-grpc-daemon (trusted)6164worker1 # [ 127.474926] nix-daemon[851]: reaped child process 1620, status = succeeded6165worker1 # [ 127.699841] nix-daemon[851]: accepted connection from pid 1622, user nix-grpc-daemon (trusted)6166worker1 # [ 127.712941] nix-daemon[851]: reaped child process 1625, status = succeeded6167lb # [ 127.742506] niks3-server[1059]: 2026/09/21 09:36:18 INFO Received uploads request method=POST path=/api/pending_closures6168worker1 # [ 127.750788] nix-grpc-daemon[1604]: time=2026-09-21T09:36:18.406Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6169worker1 # [ 127.752584] nix-grpc-daemon[1604]: time=2026-09-21T09:36:18.408Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6170lb # [ 127.782503] niks3-server[1059]: 2026/09/21 09:36:18 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6171lb # [ 127.791658] niks3-server[1059]: 2026/09/21 09:36:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6172lb # [ 127.796321] niks3-server[1059]: 2026/09/21 09:36:18 INFO Signed narinfos id=37 count=16173worker1 # [ 127.795851] nix-grpc-daemon[1604]: time=2026-09-21T09:36:18.451Z level=INFO msg="Uploading 1 narinfos"6174lb # [ 127.808724] niks3-server[1059]: 2026/09/21 09:36:18 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6175lb # [ 127.815952] niks3-server[1059]: 2026/09/21 09:36:18 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6176lb # [ 127.823977] niks3-server[1059]: 2026/09/21 09:36:18 INFO Completed upload id=376177lb # [ 127.825855] niks3-server[1059]: 2026/09/21 09:36:18 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6178worker1 # [ 127.822220] nix-grpc-daemon[1604]: time=2026-09-21T09:36:18.477Z level=INFO msg="Upload complete. (574ms)"6179worker1 # [ 127.828286] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=4246180worker1 # [ 127.833906] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366181worker1 # [ 127.845260] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6182worker1 # [ 127.849272] nix-daemon[851]: reaped child process 1602, status = succeeded6183worker1 # [ 127.854746] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6184client # [ 127.867757] nix[1244]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6185client: must succeed: systemctl kill -s INT intr6186worker1 # [ 129.408929] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:20Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6187client # [ 129.419848] systemd[1]: intr.service: Sent signal SIGINT to main process 1244 (nix) on client request.6188client # [ 129.426909] nix[1244]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6189client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6190client # [ 129.433816] nix[1244]: error: interrupted by the user6191client # [ 129.468613] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6192client # [ 129.470621] systemd[1]: intr.service: Failed with result 'exit-code'.6193client # [ 129.472342] systemd[1]: intr.service: Consumed 131ms CPU time over 3.721s wall clock time, 43.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.6194worker1 # [ 129.475383] nix-daemon[851]: reaped child process 1627, status = succeeded6195worker1 # [ 129.481252] nix-daemon[1629]: unexpected Nix daemon error: error: interrupted by the user6196worker1 # [ 129.489201] nix-daemon[851]: reaped child process 1629, status = failed with exit code 16197(finished: subtest: interrupting the client stops the build on the worker, in 3.87 seconds)6198subtest: a deploy mid-build drains: the build finishes, the new generation takes over6199client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6200client # [ 129.636900] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6201client # Running as unit: sw.service; invocation ID: 16a3c87b60d54a49b908240359defbd46202client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6203worker1 # [ 129.883228] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:460366204client # [ 129.903664] nix[1265]: don't know how to build these paths:6205client # [ 129.904645] nix[1265]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6206worker2 # [ 129.897748] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:21Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06207worker2 # [ 129.901845] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:21Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16208worker2 # [ 129.940848] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554546209worker2 # [ 129.955292] nix-grpc-daemon[1402]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 1/5)6210worker2 # [ 129.972240] nix-grpc-daemon[1402]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 2/5)6211worker2 # [ 130.148602] nix-grpc-daemon[1402]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 121 ms (attempt 3/5)6212worker2 # [ 130.269922] nix-grpc-daemon[1402]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 496 ms (attempt 4/5)6213worker2 # [ 130.772227] nix-grpc-daemon[1402]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6214worker2 # [ 130.776165] nix-grpc-daemon[1402]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6215worker2 # [ 130.795866] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:554546216client # [ 130.823155] nix[1265]: copying 1 paths to 'lb:50051'...6217client # [ 130.827159] nix[1265]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6218worker2 # [ 130.810760] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:554546219worker2 # [ 130.819927] nix-daemon[871]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)6220worker2 # [ 131.018442] nix-daemon[871]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6221worker2 # [ 131.030457] nix-daemon[871]: reaped child process 1544, status = succeeded6222worker2 # [ 131.230701] nix-daemon[871]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6223worker2 # [ 131.242608] nix-daemon[871]: reaped child process 1549, status = succeeded6224lb # [ 131.269744] niks3-server[1059]: 2026/09/21 09:36:22 INFO Received uploads request method=POST path=/api/pending_closures6225worker2 # [ 131.273600] nix-grpc-daemon[1485]: time=2026-09-21T09:36:22.466Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6226worker2 # [ 131.275063] nix-grpc-daemon[1485]: time=2026-09-21T09:36:22.466Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6227lb # [ 131.306196] niks3-server[1059]: 2026/09/21 09:36:22 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6228lb # [ 131.313586] niks3-server[1059]: 2026/09/21 09:36:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6229lb # [ 131.316174] niks3-server[1059]: 2026/09/21 09:36:22 INFO Signed narinfos id=38 count=16230worker2 # [ 131.310822] nix-grpc-daemon[1485]: time=2026-09-21T09:36:22.504Z level=INFO msg="Uploading 1 narinfos"6231lb # [ 131.324059] niks3-server[1059]: 2026/09/21 09:36:22 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6232lb # [ 131.326436] niks3-server[1059]: 2026/09/21 09:36:22 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6233lb # [ 131.331812] niks3-server[1059]: 2026/09/21 09:36:22 INFO Completed upload id=386234worker2 # [ 131.324272] nix-grpc-daemon[1485]: time=2026-09-21T09:36:22.518Z level=INFO msg="Upload complete. (490ms)"6235lb # [ 131.333450] niks3-server[1059]: 2026/09/21 09:36:22 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6236worker2 # [ 131.326424] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55454 duration_s=0 paths=1 nar_bytes_in=4166237worker2 # [ 131.331698] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554546238worker2 # [ 131.338696] nix-daemon[871]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)6239worker2 # [ 131.340962] nix-daemon[871]: reaped child process 1535, status = succeeded6240worker2 # [ 131.346449] nix-daemon[871]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)6241client # [ 131.363533] nix[1265]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6242worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26243worker2 # Checking switch inhibitors... done6244worker2 # activating the configuration...6245worker2 # [ 134.357570] nixos[1585]: switching to system configuration /nix/store/y33j1a0pq2kxhid9yk5wngb94s0rm43s-nixos-system-worker2-test6246worker2 # [ 134.360464] systemd[1]: Stopped target Local File Systems.6247worker2 # [ 134.361301] systemd[1]: Stopped target Remote File Systems.6248worker2 # setting up /etc...6249worker2 # [ 135.127679] systemd[1]: Reload requested from client PID 1585 ('.switch-to-conf') (unit backdoor.service)...6250worker2 # [ 135.129753] systemd[1]: Reloading...6251worker2 # [ 136.051308] systemd[1]: Reloading finished in 919 ms.6252worker2 # restarting sysinit-reactivation.target6253worker2 # [ 136.112771] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6254worker2 # [ 136.265339] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6255worker2 # [ 136.267072] systemd[1]: Reached target Reactivate sysinit units.6256worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6257worker2 # [ 136.273303] dbus-broker-launch[534]: Caught SIGHUP, trigger reload.6258worker2 # [ 136.275482] systemd[1]: Reloading D-Bus System Message Bus...6259worker2 # [ 136.278655] systemd[1]: Reloading Nix worker-protocol over gRPC...6260worker2 # [ 136.315426] dbus-broker-launch[534]: Looking up NSS user entry for 'systemd-timesync'...6261worker2 # [ 136.317735] dbus-broker-launch[534]: NSS returned no entry for 'systemd-timesync'6262worker2 # [ 136.318812] dbus-broker-launch[534]: Invalid user-name in /nix/store/s93qka0z548yjbxqnmj4c4b32apww6jv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6263worker2 # [ 136.329772] systemd[1]: Reloaded Nix worker-protocol over gRPC.6264worker2 # [ 136.352378] systemd[1]: Reloaded D-Bus System Message Bus.6265worker2 # [ 136.356571] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6266worker2 # [ 136.359055] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6267worker2 # [ 136.363174] systemd[1]: Reached target Local File Systems.6268worker2 # [ 136.365485] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6269worker2 # [ 136.371344] systemd[1]: Starting Create SUID/SGID Wrappers...6270worker2 # [ 136.372261] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6271worker2 # [ 136.373464] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676272worker2 # [ 136.375343] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6273worker2 # [ 136.376784] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6274worker2 # [ 136.377971] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6275worker2 # [ 136.381236] systemd[1]: Reached target Remote File Systems.6276lb # [ 136.845894] envoy[900]: conn peer=[2001:db8:1::1]:57658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=851 ms=34496277worker2 # [ 136.920755] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6278worker2 # [ 136.921863] systemd[1]: Finished Create SUID/SGID Wrappers.6279worker2 # [ 136.928442] systemd[1]: Starting linger-users.service...6280worker2 # [ 136.980166] systemd[1]: linger-users.service: Deactivated successfully.6281worker2 # [ 136.981151] systemd[1]: Finished linger-users.service.6282worker2 # [ 137.171428] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:28Z level=info event=draining6283worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6284worker2 # [ 137.234897] nixos[1585]: finished switching to system configuration /nix/store/y33j1a0pq2kxhid9yk5wngb94s0rm43s-nixos-system-worker2-test6285worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.88 seconds)6286worker2: must succeed: pgrep -f 'read -t [9]0 x'6287worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)6288client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26289worker1 # [ 137.598893] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:460366290client # don't know how to build these paths:6291client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6292worker1 # [ 137.608376] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366293worker2 # [ 137.626863] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:28Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06294worker2 # [ 137.628993] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:28Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16295worker1 # [ 137.683720] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366296worker1 # [ 137.697890] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366297client # copying 1 paths to 'lb:50051'...6298worker1 # [ 137.712677] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366299client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6300worker1 # [ 137.731975] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6301worker1 # [ 137.968265] nix-daemon[851]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6302worker1 # [ 137.981588] nix-daemon[851]: reaped child process 1683, status = succeeded6303worker1 # [ 138.209098] nix-daemon[851]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6304worker1 # [ 138.220663] nix-daemon[851]: reaped child process 1688, status = succeeded6305lb # [ 138.248658] niks3-server[1059]: 2026/09/21 09:36:29 INFO Received uploads request method=POST path=/api/pending_closures6306worker1 # [ 138.257473] nix-grpc-daemon[1604]: time=2026-09-21T09:36:28.912Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6307worker1 # [ 138.259353] nix-grpc-daemon[1604]: time=2026-09-21T09:36:28.914Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6308lb # [ 138.284649] niks3-server[1059]: 2026/09/21 09:36:29 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6309lb # [ 138.297805] niks3-server[1059]: 2026/09/21 09:36:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6310lb # [ 138.303358] niks3-server[1059]: 2026/09/21 09:36:29 INFO Signed narinfos id=39 count=16311worker1 # [ 138.303386] nix-grpc-daemon[1604]: time=2026-09-21T09:36:28.958Z level=INFO msg="Uploading 1 narinfos"6312lb # [ 138.317133] niks3-server[1059]: 2026/09/21 09:36:29 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6313lb # [ 138.327811] niks3-server[1059]: 2026/09/21 09:36:29 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6314lb # [ 138.331888] niks3-server[1059]: 2026/09/21 09:36:29 INFO Completed upload id=396315lb # [ 138.333713] niks3-server[1059]: 2026/09/21 09:36:29 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6316worker1 # [ 138.328834] nix-grpc-daemon[1604]: time=2026-09-21T09:36:28.984Z level=INFO msg="Upload complete. (581ms)"6317worker1 # [ 138.335163] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=5286318worker1 # [ 138.341555] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366319worker1 # [ 138.352498] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6320worker1 # [ 138.361612] nix-daemon[851]: reaped child process 1678, status = succeeded6321worker1 # [ 138.367277] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6322client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6323client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6324client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6325client # farm-a> LOG-farm-a-during-drain6326worker1 # [ 139.794857] nix-daemon[851]: reaped child process 1692, status = succeeded6327worker1 # [ 140.008321] nix-daemon[851]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6328worker1 # [ 140.021805] nix-daemon[851]: reaped child process 1711, status = succeeded6329lb # [ 140.048913] niks3-server[1059]: 2026/09/21 09:36:30 INFO Received uploads request method=POST path=/api/pending_closures6330worker1 # [ 140.056402] nix-grpc-daemon[1604]: time=2026-09-21T09:36:30.710Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6331worker1 # [ 140.058766] nix-grpc-daemon[1604]: time=2026-09-21T09:36:30.710Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6332lb # [ 140.082176] niks3-server[1059]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6333lb # [ 140.098293] niks3-server[1059]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6334lb # [ 140.107960] niks3-server[1059]: 2026/09/21 09:36:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6335lb # [ 140.113066] niks3-server[1059]: 2026/09/21 09:36:31 INFO Signed narinfos id=40 count=16336worker1 # [ 140.112613] nix-grpc-daemon[1604]: time=2026-09-21T09:36:30.767Z level=INFO msg="Uploading 1 narinfos"6337lb # [ 140.118650] niks3-server[1059]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6338lb # [ 140.128718] niks3-server[1059]: 2026/09/21 09:36:31 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6339lb # [ 140.131598] niks3-server[1059]: 2026/09/21 09:36:31 INFO Completed upload id=406340lb # [ 140.133586] niks3-server[1059]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6341worker1 # [ 140.129559] nix-grpc-daemon[1604]: time=2026-09-21T09:36:30.785Z level=INFO msg="Upload complete. (353ms)"6342worker2 # [ 140.136520] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:31Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06343worker2 # [ 140.138193] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:31Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16344worker1 # [ 140.135504] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16345worker1 # [ 140.151120] nix-daemon[851]: reaped child process 1690, status = succeeded6346worker1 # [ 140.185143] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366347worker1 # [ 140.199144] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366348worker1 # [ 140.213481] nix-grpc-daemon[1604]: time=2026-09-21T09:36:30.869Z level=INFO msg="All 1 paths already cached"6349client # copying 1 paths to 'lb:50051'...6350worker1 # [ 140.222472] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366351client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6352worker1 # [ 140.242178] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6353worker1 # [ 140.469099] nix-daemon[851]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6354worker1 # [ 140.480855] nix-daemon[851]: reaped child process 1718, status = succeeded6355worker1 # [ 140.708527] nix-daemon[851]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6356worker1 # [ 140.721433] nix-daemon[851]: reaped child process 1723, status = succeeded6357lb # [ 140.749612] niks3-server[1059]: 2026/09/21 09:36:31 INFO Received uploads request method=POST path=/api/pending_closures6358worker1 # [ 140.754573] nix-grpc-daemon[1604]: time=2026-09-21T09:36:31.410Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6359worker1 # [ 140.756866] nix-grpc-daemon[1604]: time=2026-09-21T09:36:31.412Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6360lb # [ 140.780378] niks3-server[1059]: 2026/09/21 09:36:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6361lb # [ 140.786366] niks3-server[1059]: 2026/09/21 09:36:31 INFO Signed narinfos id=41 count=16362worker1 # [ 140.786482] nix-grpc-daemon[1604]: time=2026-09-21T09:36:31.442Z level=INFO msg="Uploading 1 narinfos"6363lb # [ 140.795259] niks3-server[1059]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6364lb # [ 140.803411] niks3-server[1059]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6365lb # [ 140.812600] niks3-server[1059]: 2026/09/21 09:36:31 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6366worker1 # [ 140.815495] nix-grpc-daemon[1604]: time=2026-09-21T09:36:31.471Z level=INFO msg="Upload complete. (559ms)"6367lb # [ 140.818942] niks3-server[1059]: 2026/09/21 09:36:31 INFO Completed upload id=416368worker1 # [ 140.817834] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=6406369lb # [ 140.821970] niks3-server[1059]: 2026/09/21 09:36:31 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6370worker1 # [ 140.821516] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366371worker1 # [ 140.829508] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6372worker1 # [ 140.832116] nix-daemon[851]: reaped child process 1713, status = succeeded6373worker1 # [ 140.838058] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6374client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6375client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6376client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6377client # farm-b> LOG-farm-b-during-drain6378worker1 # [ 141.864993] nix-daemon[851]: reaped child process 1727, status = succeeded6379worker1 # [ 142.073401] nix-daemon[851]: accepted connection from pid 1743, user nix-grpc-daemon (trusted)6380worker1 # [ 142.086385] nix-daemon[851]: reaped child process 1746, status = succeeded6381lb # [ 142.113745] niks3-server[1059]: 2026/09/21 09:36:33 INFO Received uploads request method=POST path=/api/pending_closures6382worker1 # [ 142.121079] nix-grpc-daemon[1604]: time=2026-09-21T09:36:32.776Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6383worker1 # [ 142.123063] nix-grpc-daemon[1604]: time=2026-09-21T09:36:32.778Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6384lb # [ 142.153866] niks3-server[1059]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6385lb # [ 142.158526] niks3-server[1059]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6386lb # [ 142.168368] niks3-server[1059]: 2026/09/21 09:36:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6387lb # [ 142.172320] niks3-server[1059]: 2026/09/21 09:36:33 INFO Signed narinfos id=42 count=16388worker1 # [ 142.171382] nix-grpc-daemon[1604]: time=2026-09-21T09:36:32.826Z level=INFO msg="Uploading 1 narinfos"6389lb # [ 142.179588] niks3-server[1059]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6390lb # [ 142.184080] niks3-server[1059]: 2026/09/21 09:36:33 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6391lb # [ 142.187206] niks3-server[1059]: 2026/09/21 09:36:33 INFO Completed upload id=426392worker1 # [ 142.184115] nix-grpc-daemon[1604]: time=2026-09-21T09:36:32.839Z level=INFO msg="Upload complete. (338ms)"6393lb # [ 142.190278] niks3-server[1059]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6394worker1 # [ 142.190243] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16395worker2 # [ 142.191451] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:33Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06396worker2 # [ 142.194206] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:33Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16397worker1 # [ 142.202755] nix-daemon[851]: reaped child process 1725, status = succeeded6398worker1 # [ 142.240192] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366399worker1 # [ 142.258564] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366400worker1 # [ 142.271420] nix-grpc-daemon[1604]: time=2026-09-21T09:36:32.927Z level=INFO msg="All 2 paths already cached"6401client # copying 1 paths to 'lb:50051'...6402worker1 # [ 142.280388] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366403client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6404worker1 # [ 142.300173] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6405worker1 # [ 142.527488] nix-daemon[851]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6406worker1 # [ 142.540776] nix-daemon[851]: reaped child process 1753, status = succeeded6407worker1 # [ 142.766489] nix-daemon[851]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)6408worker1 # [ 142.779407] nix-daemon[851]: reaped child process 1758, status = succeeded6409lb # [ 142.806727] niks3-server[1059]: 2026/09/21 09:36:33 INFO Received uploads request method=POST path=/api/pending_closures6410worker1 # [ 142.813706] nix-grpc-daemon[1604]: time=2026-09-21T09:36:33.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6411worker1 # [ 142.815250] nix-grpc-daemon[1604]: time=2026-09-21T09:36:33.470Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6412lb # [ 142.840645] niks3-server[1059]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6413lb # [ 142.850747] niks3-server[1059]: 2026/09/21 09:36:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6414lb # [ 142.856363] niks3-server[1059]: 2026/09/21 09:36:33 INFO Signed narinfos id=43 count=16415worker1 # [ 142.855986] nix-grpc-daemon[1604]: time=2026-09-21T09:36:33.511Z level=INFO msg="Uploading 1 narinfos"6416lb # [ 142.862925] niks3-server[1059]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6417lb # [ 142.875933] niks3-server[1059]: 2026/09/21 09:36:33 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6418lb # [ 142.880908] niks3-server[1059]: 2026/09/21 09:36:33 INFO Completed upload id=436419worker1 # [ 142.877238] nix-grpc-daemon[1604]: time=2026-09-21T09:36:33.532Z level=INFO msg="Upload complete. (562ms)"6420worker1 # [ 142.880819] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=7686421lb # [ 142.885406] niks3-server[1059]: 2026/09/21 09:36:33 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6422worker1 # [ 142.886507] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366423worker1 # [ 142.895952] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6424worker1 # [ 142.902187] nix-daemon[851]: reaped child process 1748, status = succeeded6425worker1 # [ 142.907486] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6426client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6427client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6428client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6429client # farm-top> LOG-farm-top-during-drain6430worker1 # [ 143.728305] nix-daemon[851]: reaped child process 1762, status = succeeded6431worker1 # [ 143.940815] nix-daemon[851]: accepted connection from pid 1778, user nix-grpc-daemon (trusted)6432worker1 # [ 143.952503] nix-daemon[851]: reaped child process 1781, status = succeeded6433lb # [ 143.981461] niks3-server[1059]: 2026/09/21 09:36:34 INFO Received uploads request method=POST path=/api/pending_closures6434worker1 # [ 143.988104] nix-grpc-daemon[1604]: time=2026-09-21T09:36:34.643Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6435worker1 # [ 143.991059] nix-grpc-daemon[1604]: time=2026-09-21T09:36:34.645Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6436lb # [ 144.013608] niks3-server[1059]: 2026/09/21 09:36:34 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6437lb # [ 144.029241] niks3-server[1059]: 2026/09/21 09:36:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6438lb # [ 144.037430] niks3-server[1059]: 2026/09/21 09:36:34 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6439worker1 # [ 144.039402] nix-grpc-daemon[1604]: time=2026-09-21T09:36:34.694Z level=INFO msg="Uploading 1 narinfos"6440lb # [ 144.042574] niks3-server[1059]: 2026/09/21 09:36:34 INFO Signed narinfos id=44 count=16441lb # [ 144.047461] niks3-server[1059]: 2026/09/21 09:36:34 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6442lb # [ 144.056144] niks3-server[1059]: 2026/09/21 09:36:34 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6443lb # [ 144.061937] niks3-server[1059]: 2026/09/21 09:36:34 INFO Completed upload id=446444worker1 # [ 144.058784] nix-grpc-daemon[1604]: time=2026-09-21T09:36:34.714Z level=INFO msg="Upload complete. (344ms)"6445lb # [ 144.063832] niks3-server[1059]: 2026/09/21 09:36:34 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6446worker1 # [ 144.065146] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16447worker1 # [ 144.078830] nix-daemon[851]: reaped child process 1760, status = succeeded6448client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.81 seconds)6449client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6450client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.26 seconds)6451worker2: must succeed: pkill -f 'read -t [9]0 x'6452worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6453client: waiting for success: ! systemctl is-active sw6454worker2 # [ 144.583234] nix-daemon[871]: reaped child process 1553, status = succeeded6455worker2 # [ 144.784895] nix-daemon[871]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6456worker2 # [ 144.796939] nix-daemon[871]: reaped child process 1734, status = succeeded6457lb # [ 144.827275] niks3-server[1059]: 2026/09/21 09:36:35 INFO Received uploads request method=POST path=/api/pending_closures6458worker2 # [ 144.829230] nix-grpc-daemon[1485]: time=2026-09-21T09:36:36.022Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6459worker2 # [ 144.830742] nix-grpc-daemon[1485]: time=2026-09-21T09:36:36.024Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6460lb # [ 144.872848] niks3-server[1059]: 2026/09/21 09:36:35 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6461lb # [ 144.877522] niks3-server[1059]: 2026/09/21 09:36:35 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6462lb # [ 144.883590] niks3-server[1059]: 2026/09/21 09:36:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6463lb # [ 144.888359] niks3-server[1059]: 2026/09/21 09:36:35 INFO Signed narinfos id=45 count=16464worker2 # [ 144.884044] nix-grpc-daemon[1485]: time=2026-09-21T09:36:36.077Z level=INFO msg="Uploading 1 narinfos"6465lb # [ 144.891092] niks3-server[1059]: 2026/09/21 09:36:35 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6466lb # [ 144.902491] niks3-server[1059]: 2026/09/21 09:36:35 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6467lb # [ 144.907248] niks3-server[1059]: 2026/09/21 09:36:35 INFO Completed upload id=456468lb # [ 144.909056] niks3-server[1059]: 2026/09/21 09:36:35 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6469worker2 # [ 144.901364] nix-grpc-daemon[1485]: time=2026-09-21T09:36:36.095Z level=INFO msg="Upload complete. (332ms)"6470worker2 # [ 144.908317] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55454 duration_s=13 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16471worker2 # [ 144.919160] nix-daemon[871]: reaped child process 1551, status = succeeded6472client # [ 144.969908] systemd[1]: sw.service: Deactivated successfully.6473client # [ 144.972312] systemd[1]: sw.service: Consumed 145ms CPU time over 15.332s wall clock time, 43.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.6474worker2 # [ 145.177431] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=signal_exit6475worker2 # [ 145.402776] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=scheduler_yield6476worker1 # [ 145.410755] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_restarting addr=lb:500516477lb # [ 145.413550] niks3-server[1059]: 2026/09/21 09:36:36 INFO lead: released remote=192.168.1.4:479306478worker2 # [ 145.409908] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:554386479worker2 # [ 145.414289] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=worker_gone addr=192.168.1.3:500516480worker2 # [ 145.418308] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=scheduler_restarting addr=lb:500516481worker2 # [ 145.421337] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:554386482worker2 # [ 145.424577] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=worker_gone addr=192.168.1.4:500516483client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)6484client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6485client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6486worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6487worker1 # [ 145.610666] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_connected addr=lb:500516488worker1 # [ 145.615232] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6489worker2 # [ 145.615693] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=scheduler_connected addr=lb:500516490worker2 # [ 145.620754] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6491worker2 # [ 145.682174] systemd[1]: Starting scheduler-token.service...6492worker2 # [ 145.766539] systemd[1]: scheduler-token.service: Deactivated successfully.6493worker2 # [ 145.768259] systemd[1]: Finished scheduler-token.service.6494worker2 # [ 145.769852] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 86ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6495worker1 # [ 145.816069] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_connected addr=lb:500516496worker1 # [ 145.820471] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6497worker2 # [ 145.821556] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:37Z level=info event=scheduler_connected addr=lb:500516498worker2 # [ 145.823319] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6499worker1 # [ 146.021578] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_connected addr=lb:500516500worker1 # [ 146.025999] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6501worker2 # [ 146.025993] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:37Z level=info event=scheduler_connected addr=lb:500516502worker2 # [ 146.029406] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6503worker1 # [ 146.227267] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_connected addr=lb:500516504worker2 # [ 146.226980] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:37Z level=info event=scheduler_connected addr=lb:500516505worker1 # [ 146.231679] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6506worker2 # [ 146.230923] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6507worker2 # [ 146.407294] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:37Z level=info event=scheduler_restarting6508worker1 # [ 146.433442] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_connected addr=lb:500516509worker1 # [ 146.437203] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6510worker2 # [ 146.444236] nix-daemon[871]: reaped child process 1456, status = succeeded6511worker2 # [ 146.446605] nix-grpc-daemon[1402]: ts=2026-09-21T09:36:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6512worker2 # [ 146.628520] nix-grpc-daemon[1402]: E0000 00:00:1789983397.822028 1447 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6513worker1 # [ 146.638108] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_connected addr=lb:500516514worker2 # [ 146.633336] nix-grpc-daemon[1402]: E0000 00:00:1789983397.822178 1447 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6515worker1 # [ 146.642220] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6516worker1 # [ 146.842812] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_connected addr=lb:500516517lb # [ 146.847397] envoy[900]: conn peer=[2001:db8:1::1]:35054 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=4089 ms=64866518worker1 # [ 146.847206] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6519lb # [ 146.851425] envoy[900]: conn peer=[2001:db8:1::1]:57672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1448 ms=150376520lb # [ 146.854987] envoy[900]: conn peer=[2001:db8:1::4]:41902 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4876 tx=545 ms=272186521worker1 # [ 147.048281] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_connected addr=lb:500516522worker1 # [ 147.052681] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6523worker2 # [ 147.216887] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6524worker2 # [ 147.220438] systemd[1]: nix-grpc-daemon.service: Consumed 670ms CPU time over 28.435s wall clock time, 33.3M memory peak, 25.9K incoming IP traffic, 36.8K outgoing IP traffic.6525worker2 # [ 147.236955] systemd[1]: Starting Nix worker-protocol over gRPC...6526worker1 # [ 147.253801] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_connected addr=lb:500516527worker1 # [ 147.258352] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6528worker2 # [ 147.446459] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6529worker2 # [ 147.448361] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6530worker2 # [ 147.452354] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16531worker1 # [ 147.460442] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_connected addr=lb:500516532worker2 # [ 147.460065] nix-grpc-daemon[1751]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6533worker2 # [ 147.462094] nix-grpc-daemon[1751]: E0000 00:00:1789983398.653503 1751 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6534worker1 # [ 147.464695] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6535worker2 # [ 147.466836] nix-grpc-daemon[1751]: E0000 00:00:1789983398.658653 1751 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6536worker2 # [ 147.471797] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6537worker2 # [ 147.474536] systemd[1]: Started Nix worker-protocol over gRPC.6538worker2 # [ 147.476926] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=scheduler_take_over6539lb # [ 147.484847] niks3-server[1059]: 2026/09/21 09:36:38 INFO lead: acquired remote=192.168.1.4:491366540worker2 # [ 147.481307] nix-grpc-daemon[1751]: E0000 00:00:1789983398.675016 1792 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6541worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.90 seconds)6542lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6543lb: must succeed: curl -sf localhost:9901/clusters6544worker2 # [ 147.495046] nix-grpc-daemon[1751]: E0000 00:00:1789983398.687707 1792 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6545worker2 # [ 147.497162] nix-grpc-daemon[1751]: E0000 00:00:1789983398.687899 1792 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6546worker2 # [ 147.508060] nix-grpc-daemon[1751]: E0000 00:00:1789983398.701770 1792 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6547worker2 # [ 147.519505] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=scheduler_connected addr=lb:500516548worker2 # [ 147.521174] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6549lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6550worker1 # [ 147.666098] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_connected addr=lb:500516551worker1 # [ 147.670585] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6552worker2 # [ 147.722684] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=scheduler_connected addr=lb:500516553worker2 # [ 147.726558] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6554worker1 # [ 147.871821] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_connected addr=lb:500516555worker1 # [ 147.876311] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6556worker2 # [ 147.927785] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=scheduler_connected addr=lb:500516557worker2 # [ 147.931734] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6558worker1 # [ 148.077134] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_connected addr=lb:500516559worker1 # [ 148.081675] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6560worker2 # [ 148.133180] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=scheduler_connected addr=lb:500516561worker2 # [ 148.137117] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6562worker1 # [ 148.282843] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:38Z level=info event=scheduler_connected addr=lb:500516563worker2 # [ 148.284526] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:340266564worker2 # [ 148.288977] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06565worker2 # [ 148.338119] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=scheduler_connected addr=lb:500516566worker2 # [ 148.348141] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=oidc_jwks provider=mock keys=16567worker2 # [ 148.351913] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:340266568worker2 # [ 148.355781] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06569lb: must succeed: curl -sf localhost:9901/clusters6570lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6571lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.17 seconds)6572lb: must succeed: curl -sf localhost:9901/clusters6573lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6574worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6575worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6576(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 19.24 seconds)6577subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6578client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16579worker2 # [ 149.091339] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340426580worker2 # [ 149.102845] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)6581worker2 # [ 149.116989] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340426582worker2 # [ 149.129427] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)6583worker2 # [ 149.153095] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 2/5)6584worker2 # [ 149.194124] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 358 ms (attempt 3/5)6585worker2 # [ 149.558667] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 647 ms (attempt 4/5)6586worker2 # [ 150.212349] nix-grpc-daemon[1751]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6587worker2 # [ 150.216600] nix-grpc-daemon[1751]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6588worker2 # [ 150.265960] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:41Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06589worker2 # [ 150.270219] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:41Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16590worker2 # [ 150.274508] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:41Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06591worker2 # [ 150.278296] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:41Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06592worker1 # [ 150.316509] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366593worker1 # [ 150.330207] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366594worker1 # [ 150.343732] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366595worker1 # [ 150.355964] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6596worker1 # [ 150.590711] nix-daemon[851]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6597worker1 # [ 150.604621] nix-daemon[851]: reaped child process 1788, status = succeeded6598worker1 # [ 150.828405] nix-daemon[851]: accepted connection from pid 1790, user nix-grpc-daemon (trusted)6599worker1 # [ 150.841587] nix-daemon[851]: reaped child process 1793, status = succeeded6600lb # [ 150.869842] niks3-server[1059]: 2026/09/21 09:36:41 INFO Received uploads request method=POST path=/api/pending_closures6601worker1 # [ 150.875688] nix-grpc-daemon[1604]: time=2026-09-21T09:36:41.531Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6602worker1 # [ 150.877577] nix-grpc-daemon[1604]: time=2026-09-21T09:36:41.532Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6603lb # [ 150.901603] niks3-server[1059]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6604lb # [ 150.911208] niks3-server[1059]: 2026/09/21 09:36:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6605lb # [ 150.916236] niks3-server[1059]: 2026/09/21 09:36:41 INFO Signed narinfos id=46 count=16606worker1 # [ 150.915869] nix-grpc-daemon[1604]: time=2026-09-21T09:36:41.571Z level=INFO msg="Uploading 1 narinfos"6607lb # [ 150.923325] niks3-server[1059]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6608lb # [ 150.932924] niks3-server[1059]: 2026/09/21 09:36:41 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6609worker1 # [ 150.936509] nix-grpc-daemon[1604]: time=2026-09-21T09:36:41.592Z level=INFO msg="Upload complete. (567ms)"6610lb # [ 150.940099] niks3-server[1059]: 2026/09/21 09:36:41 INFO Completed upload id=466611worker1 # [ 150.938740] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=5046612lb # [ 150.943381] niks3-server[1059]: 2026/09/21 09:36:41 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6613worker1 # [ 150.943793] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366614worker1 # [ 150.955066] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6615worker1 # [ 150.959033] nix-daemon[851]: reaped child process 1783, status = succeeded6616worker1 # [ 150.964259] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6617worker1 # [ 152.191939] nix-daemon[851]: reaped child process 1797, status = succeeded6618worker1 # [ 152.404101] nix-daemon[851]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)6619worker1 # [ 152.415790] nix-daemon[851]: reaped child process 1816, status = succeeded6620lb # [ 152.443891] niks3-server[1059]: 2026/09/21 09:36:43 INFO Received uploads request method=POST path=/api/pending_closures6621worker1 # [ 152.449409] nix-grpc-daemon[1604]: time=2026-09-21T09:36:43.104Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6622worker1 # [ 152.451140] nix-grpc-daemon[1604]: time=2026-09-21T09:36:43.106Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6623lb # [ 152.483625] niks3-server[1059]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6624lb # [ 152.488553] niks3-server[1059]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6625lb # [ 152.502318] niks3-server[1059]: 2026/09/21 09:36:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6626lb # [ 152.506889] niks3-server[1059]: 2026/09/21 09:36:43 INFO Signed narinfos id=47 count=16627worker1 # [ 152.506869] nix-grpc-daemon[1604]: time=2026-09-21T09:36:43.162Z level=INFO msg="Uploading 1 narinfos"6628lb # [ 152.515978] niks3-server[1059]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6629lb # [ 152.523816] niks3-server[1059]: 2026/09/21 09:36:43 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6630lb # [ 152.529249] niks3-server[1059]: 2026/09/21 09:36:43 INFO Completed upload id=476631worker1 # [ 152.526143] nix-grpc-daemon[1604]: time=2026-09-21T09:36:43.181Z level=INFO msg="Upload complete. (353ms)"6632lb # [ 152.532065] niks3-server[1059]: 2026/09/21 09:36:43 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6633worker1 # [ 152.531788] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16634worker2 # [ 152.535311] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:43Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16635worker2 # [ 152.539785] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:43Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06636worker1 # [ 152.546133] nix-daemon[851]: reaped child process 1795, status = succeeded6637worker1 # [ 152.582930] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366638worker1 # [ 152.596061] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366639worker1 # [ 152.607619] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366640worker1 # [ 152.616403] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6641worker1 # [ 152.842548] nix-daemon[851]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6642worker1 # [ 152.855602] nix-daemon[851]: reaped child process 1823, status = succeeded6643worker1 # [ 153.081103] nix-daemon[851]: accepted connection from pid 1825, user nix-grpc-daemon (trusted)6644worker1 # [ 153.094156] nix-daemon[851]: reaped child process 1828, status = succeeded6645lb # [ 153.120214] niks3-server[1059]: 2026/09/21 09:36:44 INFO Received uploads request method=POST path=/api/pending_closures6646worker1 # [ 153.124842] nix-grpc-daemon[1604]: time=2026-09-21T09:36:43.780Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6647worker1 # [ 153.126691] nix-grpc-daemon[1604]: time=2026-09-21T09:36:43.782Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6648lb # [ 153.147814] niks3-server[1059]: 2026/09/21 09:36:44 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6649lb # [ 153.158682] niks3-server[1059]: 2026/09/21 09:36:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6650lb # [ 153.164054] niks3-server[1059]: 2026/09/21 09:36:44 INFO Signed narinfos id=48 count=16651worker1 # [ 153.163699] nix-grpc-daemon[1604]: time=2026-09-21T09:36:43.819Z level=INFO msg="Uploading 1 narinfos"6652lb # [ 153.170620] niks3-server[1059]: 2026/09/21 09:36:44 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6653lb # [ 153.183095] niks3-server[1059]: 2026/09/21 09:36:44 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6654lb # [ 153.187788] niks3-server[1059]: 2026/09/21 09:36:44 INFO Completed upload id=486655worker1 # [ 153.186790] nix-grpc-daemon[1604]: time=2026-09-21T09:36:43.842Z level=INFO msg="Upload complete. (556ms)"6656lb # [ 153.190971] niks3-server[1059]: 2026/09/21 09:36:44 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6657worker1 # [ 153.190637] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=5046658worker1 # [ 153.195491] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366659worker1 # [ 153.209252] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6660worker1 # [ 153.210832] nix-daemon[851]: reaped child process 1818, status = succeeded6661worker1 # [ 153.216129] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6662worker1 # [ 154.028159] nix-daemon[851]: reaped child process 1832, status = succeeded6663worker1 # [ 154.235527] nix-daemon[851]: accepted connection from pid 1848, user nix-grpc-daemon (trusted)6664worker1 # [ 154.248672] nix-daemon[851]: reaped child process 1851, status = succeeded6665lb # [ 154.276703] niks3-server[1059]: 2026/09/21 09:36:45 INFO Received uploads request method=POST path=/api/pending_closures6666worker1 # [ 154.282818] nix-grpc-daemon[1604]: time=2026-09-21T09:36:44.938Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6667worker1 # [ 154.286077] nix-grpc-daemon[1604]: time=2026-09-21T09:36:44.940Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6668lb # [ 154.322967] niks3-server[1059]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6669lb # [ 154.327918] niks3-server[1059]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6670lb # [ 154.341734] niks3-server[1059]: 2026/09/21 09:36:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6671lb # [ 154.346972] niks3-server[1059]: 2026/09/21 09:36:45 INFO Signed narinfos id=49 count=16672worker1 # [ 154.346343] nix-grpc-daemon[1604]: time=2026-09-21T09:36:45.001Z level=INFO msg="Uploading 1 narinfos"6673lb # [ 154.349915] niks3-server[1059]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6674lb # [ 154.359313] niks3-server[1059]: 2026/09/21 09:36:45 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6675lb # [ 154.363662] niks3-server[1059]: 2026/09/21 09:36:45 INFO Completed upload id=496676lb # [ 154.365525] niks3-server[1059]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6677worker1 # [ 154.361747] nix-grpc-daemon[1604]: time=2026-09-21T09:36:45.017Z level=INFO msg="Upload complete. (339ms)"6678worker2 # [ 154.366666] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:45Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16679worker1 # [ 154.368697] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16680worker2 # [ 154.375365] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:45Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06681worker1 # [ 154.382376] nix-daemon[851]: reaped child process 1830, status = succeeded6682worker1 # [ 154.414724] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366683worker1 # [ 154.428512] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366684worker1 # [ 154.440595] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366685worker1 # [ 154.449417] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6686worker1 # [ 154.682098] nix-daemon[851]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6687worker1 # [ 154.693636] nix-daemon[851]: reaped child process 1858, status = succeeded6688worker1 # [ 154.922415] nix-daemon[851]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)6689worker1 # [ 154.935362] nix-daemon[851]: reaped child process 1863, status = succeeded6690lb # [ 154.961376] niks3-server[1059]: 2026/09/21 09:36:45 INFO Received uploads request method=POST path=/api/pending_closures6691worker1 # [ 154.966843] nix-grpc-daemon[1604]: time=2026-09-21T09:36:45.622Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6692worker1 # [ 154.968808] nix-grpc-daemon[1604]: time=2026-09-21T09:36:45.624Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6693lb # [ 154.988844] niks3-server[1059]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6694lb # [ 155.001404] niks3-server[1059]: 2026/09/21 09:36:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6695lb # [ 155.006586] niks3-server[1059]: 2026/09/21 09:36:45 INFO Signed narinfos id=50 count=16696worker1 # [ 155.006289] nix-grpc-daemon[1604]: time=2026-09-21T09:36:45.661Z level=INFO msg="Uploading 1 narinfos"6697lb # [ 155.013628] niks3-server[1059]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6698lb # [ 155.024867] niks3-server[1059]: 2026/09/21 09:36:45 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6699lb # [ 155.032330] niks3-server[1059]: 2026/09/21 09:36:45 INFO Completed upload id=506700worker1 # [ 155.028970] nix-grpc-daemon[1604]: time=2026-09-21T09:36:45.684Z level=INFO msg="Upload complete. (565ms)"6701lb # [ 155.034924] niks3-server[1059]: 2026/09/21 09:36:45 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6702worker1 # [ 155.032356] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=5046703worker1 # [ 155.037981] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366704worker1 # [ 155.047269] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6705worker1 # [ 155.052479] nix-daemon[851]: reaped child process 1853, status = succeeded6706worker1 # [ 155.058601] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6707worker1 # [ 156.284680] nix-daemon[851]: reaped child process 1867, status = succeeded6708worker1 # [ 156.498673] nix-daemon[851]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6709worker1 # [ 156.511586] nix-daemon[851]: reaped child process 1886, status = succeeded6710lb # [ 156.539709] niks3-server[1059]: 2026/09/21 09:36:47 INFO Received uploads request method=POST path=/api/pending_closures6711worker1 # [ 156.546258] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6712worker1 # [ 156.547799] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.200Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6713lb # [ 156.576285] niks3-server[1059]: 2026/09/21 09:36:47 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6714lb # [ 156.582669] niks3-server[1059]: 2026/09/21 09:36:47 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6715lb # [ 156.594070] niks3-server[1059]: 2026/09/21 09:36:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6716worker1 # [ 156.596696] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.252Z level=INFO msg="Uploading 1 narinfos"6717lb # [ 156.599939] niks3-server[1059]: 2026/09/21 09:36:47 INFO Signed narinfos id=51 count=16718lb # [ 156.603493] niks3-server[1059]: 2026/09/21 09:36:47 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6719lb # [ 156.613829] niks3-server[1059]: 2026/09/21 09:36:47 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6720lb # [ 156.618235] niks3-server[1059]: 2026/09/21 09:36:47 INFO Completed upload id=516721worker1 # [ 156.616376] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.272Z level=INFO msg="Upload complete. (350ms)"6722lb # [ 156.621294] niks3-server[1059]: 2026/09/21 09:36:47 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6723worker2 # [ 156.618932] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:47Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16724worker1 # [ 156.621929] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16725worker1 # [ 156.630742] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366726worker2 # [ 156.628970] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:47Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06727worker1 # [ 156.643315] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366728worker1 # [ 156.646128] nix-daemon[851]: reaped child process 1865, status = succeeded6729worker1 # [ 156.650290] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.306Z level=INFO msg="All 1 paths already cached"6730worker1 # [ 156.654376] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366731worker1 # [ 156.659846] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6732worker1 # [ 156.870153] nix-daemon[851]: accepted connection from pid 1890, user nix-grpc-daemon (trusted)6733worker1 # [ 156.883247] nix-daemon[851]: reaped child process 1893, status = succeeded6734worker1 # [ 157.105122] nix-daemon[851]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6735worker1 # [ 157.118301] nix-daemon[851]: reaped child process 1898, status = succeeded6736lb # [ 157.144447] niks3-server[1059]: 2026/09/21 09:36:48 INFO Received uploads request method=POST path=/api/pending_closures6737worker1 # [ 157.149441] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.805Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6738worker1 # [ 157.151279] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.806Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6739lb # [ 157.176455] niks3-server[1059]: 2026/09/21 09:36:48 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6740lb # [ 157.187372] niks3-server[1059]: 2026/09/21 09:36:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6741lb # [ 157.192242] niks3-server[1059]: 2026/09/21 09:36:48 INFO Signed narinfos id=52 count=16742worker1 # [ 157.191786] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.847Z level=INFO msg="Uploading 1 narinfos"6743lb # [ 157.199069] niks3-server[1059]: 2026/09/21 09:36:48 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6744lb # [ 157.210559] niks3-server[1059]: 2026/09/21 09:36:48 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6745lb # [ 157.215366] niks3-server[1059]: 2026/09/21 09:36:48 INFO Completed upload id=526746worker1 # [ 157.212102] nix-grpc-daemon[1604]: time=2026-09-21T09:36:47.867Z level=INFO msg="Upload complete. (547ms)"6747lb # [ 157.220688] niks3-server[1059]: 2026/09/21 09:36:48 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6748worker1 # [ 157.216312] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=6246749worker1 # [ 157.222179] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366750worker1 # [ 157.232137] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6751worker1 # [ 157.237904] nix-daemon[851]: reaped child process 1888, status = succeeded6752worker1 # [ 157.242807] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6753worker1 # [ 158.870388] nix-daemon[851]: reaped child process 1902, status = succeeded6754worker1 # [ 159.084548] nix-daemon[851]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)6755worker1 # [ 159.097976] nix-daemon[851]: reaped child process 1921, status = succeeded6756lb # [ 159.125508] niks3-server[1059]: 2026/09/21 09:36:50 INFO Received uploads request method=POST path=/api/pending_closures6757worker1 # [ 159.130934] nix-grpc-daemon[1604]: time=2026-09-21T09:36:49.786Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6758worker1 # [ 159.134990] nix-grpc-daemon[1604]: time=2026-09-21T09:36:49.788Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6759lb # [ 159.160736] niks3-server[1059]: 2026/09/21 09:36:50 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6760lb # [ 159.174972] niks3-server[1059]: 2026/09/21 09:36:50 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6761lb # [ 159.187198] niks3-server[1059]: 2026/09/21 09:36:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6762worker1 # [ 159.188359] nix-grpc-daemon[1604]: time=2026-09-21T09:36:49.843Z level=INFO msg="Uploading 1 narinfos"6763lb # [ 159.191672] niks3-server[1059]: 2026/09/21 09:36:50 INFO Signed narinfos id=53 count=16764lb # [ 159.196660] niks3-server[1059]: 2026/09/21 09:36:50 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6765lb # [ 159.205543] niks3-server[1059]: 2026/09/21 09:36:50 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6766lb # [ 159.211742] niks3-server[1059]: 2026/09/21 09:36:50 INFO Completed upload id=536767lb # [ 159.213623] niks3-server[1059]: 2026/09/21 09:36:50 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6768worker1 # [ 159.208384] nix-grpc-daemon[1604]: time=2026-09-21T09:36:49.864Z level=INFO msg="Upload complete. (356ms)"6769worker2 # [ 159.211623] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:50Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16770worker1 # [ 159.215570] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16771worker1 # [ 159.224541] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366772worker2 # [ 159.221498] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:50Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06773worker1 # [ 159.236512] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366774worker1 # [ 159.242849] nix-daemon[851]: reaped child process 1900, status = succeeded6775worker1 # [ 159.247806] nix-grpc-daemon[1604]: time=2026-09-21T09:36:49.903Z level=INFO msg="All 1 paths already cached"6776worker1 # [ 159.252428] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366777worker1 # [ 159.256720] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6778worker1 # [ 159.468525] nix-daemon[851]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)6779worker1 # [ 159.482175] nix-daemon[851]: reaped child process 1928, status = succeeded6780worker1 # [ 159.705209] nix-daemon[851]: accepted connection from pid 1930, user nix-grpc-daemon (trusted)6781worker1 # [ 159.718318] nix-daemon[851]: reaped child process 1933, status = succeeded6782lb # [ 159.745497] niks3-server[1059]: 2026/09/21 09:36:50 INFO Received uploads request method=POST path=/api/pending_closures6783worker1 # [ 159.756451] nix-grpc-daemon[1604]: time=2026-09-21T09:36:50.411Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6784worker1 # [ 159.758105] nix-grpc-daemon[1604]: time=2026-09-21T09:36:50.411Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6785lb # [ 159.777785] niks3-server[1059]: 2026/09/21 09:36:50 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6786lb # [ 159.789437] niks3-server[1059]: 2026/09/21 09:36:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6787lb # [ 159.794706] niks3-server[1059]: 2026/09/21 09:36:50 INFO Signed narinfos id=54 count=16788worker1 # [ 159.793934] nix-grpc-daemon[1604]: time=2026-09-21T09:36:50.449Z level=INFO msg="Uploading 1 narinfos"6789lb # [ 159.801894] niks3-server[1059]: 2026/09/21 09:36:50 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6790lb # [ 159.811327] niks3-server[1059]: 2026/09/21 09:36:50 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6791lb # [ 159.817991] niks3-server[1059]: 2026/09/21 09:36:50 INFO Completed upload id=546792worker1 # [ 159.814496] nix-grpc-daemon[1604]: time=2026-09-21T09:36:50.470Z level=INFO msg="Upload complete. (551ms)"6793lb # [ 159.821086] niks3-server[1059]: 2026/09/21 09:36:50 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6794worker1 # [ 159.819138] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=6246795worker1 # [ 159.824520] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366796worker1 # [ 159.833357] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6797worker1 # [ 159.840203] nix-daemon[851]: reaped child process 1923, status = succeeded6798worker1 # [ 159.845543] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6799worker1 # [ 161.058715] nix-daemon[851]: reaped child process 1937, status = succeeded6800worker1 # [ 161.260497] nix-daemon[851]: accepted connection from pid 1953, user nix-grpc-daemon (trusted)6801worker1 # [ 161.273726] nix-daemon[851]: reaped child process 1956, status = succeeded6802lb # [ 161.300898] niks3-server[1059]: 2026/09/21 09:36:52 INFO Received uploads request method=POST path=/api/pending_closures6803worker1 # [ 161.307331] nix-grpc-daemon[1604]: time=2026-09-21T09:36:51.962Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6804worker1 # [ 161.308750] nix-grpc-daemon[1604]: time=2026-09-21T09:36:51.962Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6805lb # [ 161.336614] niks3-server[1059]: 2026/09/21 09:36:52 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6806lb # [ 161.344394] niks3-server[1059]: 2026/09/21 09:36:52 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6807lb # [ 161.353535] niks3-server[1059]: 2026/09/21 09:36:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6808lb # [ 161.357233] niks3-server[1059]: 2026/09/21 09:36:52 INFO Signed narinfos id=55 count=16809worker1 # [ 161.356546] nix-grpc-daemon[1604]: time=2026-09-21T09:36:52.012Z level=INFO msg="Uploading 1 narinfos"6810lb # [ 161.366323] niks3-server[1059]: 2026/09/21 09:36:52 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6811lb # [ 161.373745] niks3-server[1059]: 2026/09/21 09:36:52 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6812lb # [ 161.377620] niks3-server[1059]: 2026/09/21 09:36:52 INFO Completed upload id=556813worker1 # [ 161.374381] nix-grpc-daemon[1604]: time=2026-09-21T09:36:52.030Z level=INFO msg="Upload complete. (321ms)"6814lb # [ 161.379288] niks3-server[1059]: 2026/09/21 09:36:52 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6815worker1 # [ 161.380800] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16816worker2 # [ 161.376701] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:52Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16817worker1 # [ 161.386530] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366818worker2 # [ 161.386755] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:52Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06819worker1 # [ 161.398313] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366820worker1 # [ 161.401167] nix-daemon[851]: reaped child process 1935, status = succeeded6821worker1 # [ 161.405221] nix-grpc-daemon[1604]: time=2026-09-21T09:36:52.060Z level=INFO msg="All 1 paths already cached"6822worker1 # [ 161.409280] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366823worker1 # [ 161.413871] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6824worker1 # [ 161.615283] nix-daemon[851]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)6825worker1 # [ 161.628624] nix-daemon[851]: reaped child process 1963, status = succeeded6826worker1 # [ 161.839878] nix-daemon[851]: accepted connection from pid 1965, user nix-grpc-daemon (trusted)6827worker1 # [ 161.852989] nix-daemon[851]: reaped child process 1968, status = succeeded6828lb # [ 161.879155] niks3-server[1059]: 2026/09/21 09:36:52 INFO Received uploads request method=POST path=/api/pending_closures6829worker1 # [ 161.884439] nix-grpc-daemon[1604]: time=2026-09-21T09:36:52.539Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6830worker1 # [ 161.886105] nix-grpc-daemon[1604]: time=2026-09-21T09:36:52.539Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6831lb # [ 161.905235] niks3-server[1059]: 2026/09/21 09:36:52 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6832lb # [ 161.917362] niks3-server[1059]: 2026/09/21 09:36:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6833worker1 # [ 161.918846] nix-grpc-daemon[1604]: time=2026-09-21T09:36:52.574Z level=INFO msg="Uploading 1 narinfos"6834lb # [ 161.923380] niks3-server[1059]: 2026/09/21 09:36:52 INFO Signed narinfos id=56 count=16835lb # [ 161.929306] niks3-server[1059]: 2026/09/21 09:36:52 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6836lb # [ 161.940325] niks3-server[1059]: 2026/09/21 09:36:52 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6837worker1 # [ 161.943512] nix-grpc-daemon[1604]: time=2026-09-21T09:36:52.599Z level=INFO msg="Upload complete. (523ms)"6838lb # [ 161.946846] niks3-server[1059]: 2026/09/21 09:36:52 INFO Completed upload id=566839worker1 # [ 161.945914] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=6246840lb # [ 161.950432] niks3-server[1059]: 2026/09/21 09:36:52 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6841worker1 # [ 161.951440] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366842worker1 # [ 161.963327] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6843worker1 # [ 161.965246] nix-daemon[851]: reaped child process 1958, status = succeeded6844worker1 # [ 161.970584] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6845worker1 # [ 162.783938] nix-daemon[851]: reaped child process 1972, status = succeeded6846worker1 # [ 162.979880] nix-daemon[851]: accepted connection from pid 1988, user nix-grpc-daemon (trusted)6847worker1 # [ 162.993292] nix-daemon[851]: reaped child process 1991, status = succeeded6848lb # [ 163.021392] niks3-server[1059]: 2026/09/21 09:36:53 INFO Received uploads request method=POST path=/api/pending_closures6849worker1 # [ 163.029486] nix-grpc-daemon[1604]: time=2026-09-21T09:36:53.684Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6850worker1 # [ 163.031298] nix-grpc-daemon[1604]: time=2026-09-21T09:36:53.687Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6851lb # [ 163.063175] niks3-server[1059]: 2026/09/21 09:36:53 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6852lb # [ 163.069910] niks3-server[1059]: 2026/09/21 09:36:53 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6853lb # [ 163.077048] niks3-server[1059]: 2026/09/21 09:36:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6854lb # [ 163.082261] niks3-server[1059]: 2026/09/21 09:36:54 INFO Signed narinfos id=57 count=16855worker1 # [ 163.081213] nix-grpc-daemon[1604]: time=2026-09-21T09:36:53.736Z level=INFO msg="Uploading 1 narinfos"6856lb # [ 163.088569] niks3-server[1059]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6857lb # [ 163.094432] niks3-server[1059]: 2026/09/21 09:36:54 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6858lb # [ 163.098159] niks3-server[1059]: 2026/09/21 09:36:54 INFO Completed upload id=576859lb # [ 163.099906] niks3-server[1059]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6860worker1 # [ 163.096054] nix-grpc-daemon[1604]: time=2026-09-21T09:36:53.751Z level=INFO msg="Upload complete. (318ms)"6861worker1 # [ 163.103918] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16862worker2 # [ 163.099859] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:54Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16863worker1 # [ 163.109918] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366864worker2 # [ 163.110192] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:54Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06865worker1 # [ 163.120526] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366866worker1 # [ 163.125298] nix-daemon[851]: reaped child process 1970, status = succeeded6867worker1 # [ 163.127596] nix-grpc-daemon[1604]: time=2026-09-21T09:36:53.783Z level=INFO msg="All 2 paths already cached"6868worker1 # [ 163.131837] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366869worker1 # [ 163.137323] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6870worker1 # [ 163.347472] nix-daemon[851]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)6871worker1 # [ 163.360855] nix-daemon[851]: reaped child process 1998, status = succeeded6872worker1 # [ 163.584507] nix-daemon[851]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)6873worker1 # [ 163.597902] nix-daemon[851]: reaped child process 2003, status = succeeded6874lb # [ 163.624078] niks3-server[1059]: 2026/09/21 09:36:54 INFO Received uploads request method=POST path=/api/pending_closures6875worker1 # [ 163.630298] nix-grpc-daemon[1604]: time=2026-09-21T09:36:54.286Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6876worker1 # [ 163.632115] nix-grpc-daemon[1604]: time=2026-09-21T09:36:54.287Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6877lb # [ 163.663987] niks3-server[1059]: 2026/09/21 09:36:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6878lb # [ 163.666491] niks3-server[1059]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6879lb # [ 163.672098] niks3-server[1059]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6880lb # [ 163.677341] niks3-server[1059]: 2026/09/21 09:36:54 INFO Signed narinfos id=58 count=16881worker1 # [ 163.674052] nix-grpc-daemon[1604]: time=2026-09-21T09:36:54.329Z level=INFO msg="Uploading 1 narinfos"6882lb # [ 163.684467] niks3-server[1059]: 2026/09/21 09:36:54 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6883lb # [ 163.687478] niks3-server[1059]: 2026/09/21 09:36:54 INFO Completed upload id=586884worker1 # [ 163.685150] nix-grpc-daemon[1604]: time=2026-09-21T09:36:54.340Z level=INFO msg="Upload complete. (542ms)"6885lb # [ 163.691086] niks3-server[1059]: 2026/09/21 09:36:54 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6886worker1 # [ 163.688839] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=7526887worker1 # [ 163.694607] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366888worker1 # [ 163.702291] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6889worker1 # [ 163.708333] nix-daemon[851]: reaped child process 1993, status = succeeded6890worker1 # [ 163.714897] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6891worker1 # [ 165.139977] nix-daemon[851]: reaped child process 2007, status = succeeded6892worker1 # [ 165.347626] nix-daemon[851]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)6893worker1 # [ 165.361127] nix-daemon[851]: reaped child process 2026, status = succeeded6894lb # [ 165.388607] niks3-server[1059]: 2026/09/21 09:36:56 INFO Received uploads request method=POST path=/api/pending_closures6895worker1 # [ 165.394355] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.049Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6896worker1 # [ 165.397271] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.051Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6897lb # [ 165.425564] niks3-server[1059]: 2026/09/21 09:36:56 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6898lb # [ 165.437441] niks3-server[1059]: 2026/09/21 09:36:56 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6899lb # [ 165.446066] niks3-server[1059]: 2026/09/21 09:36:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6900worker1 # [ 165.447392] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.102Z level=INFO msg="Uploading 1 narinfos"6901lb # [ 165.451660] niks3-server[1059]: 2026/09/21 09:36:56 INFO Signed narinfos id=59 count=16902lb # [ 165.464175] niks3-server[1059]: 2026/09/21 09:36:56 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6903lb # [ 165.466651] niks3-server[1059]: 2026/09/21 09:36:56 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6904lb # [ 165.469402] niks3-server[1059]: 2026/09/21 09:36:56 INFO Completed upload id=596905worker1 # [ 165.466228] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.121Z level=INFO msg="Upload complete. (342ms)"6906lb # [ 165.472225] niks3-server[1059]: 2026/09/21 09:36:56 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6907worker2 # [ 165.468836] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:56Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16908worker1 # [ 165.473917] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16909worker1 # [ 165.487647] nix-daemon[851]: reaped child process 2005, status = succeeded6910worker1 # [ 165.516545] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366911worker1 # [ 165.537008] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366912worker1 # [ 165.552686] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.208Z level=INFO msg="All 2 paths already cached"6913worker1 # [ 165.561316] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366914worker1 # [ 165.570973] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6915worker1 # [ 165.796436] nix-daemon[851]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)6916worker1 # [ 165.810626] nix-daemon[851]: reaped child process 2033, status = succeeded6917worker1 # [ 166.031890] nix-daemon[851]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)6918worker1 # [ 166.045314] nix-daemon[851]: reaped child process 2038, status = succeeded6919lb # [ 166.072389] niks3-server[1059]: 2026/09/21 09:36:56 INFO Received uploads request method=POST path=/api/pending_closures6920worker1 # [ 166.077969] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.733Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6921worker1 # [ 166.079618] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.733Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6922lb # [ 166.099149] niks3-server[1059]: 2026/09/21 09:36:57 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6923lb # [ 166.110457] niks3-server[1059]: 2026/09/21 09:36:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6924worker1 # [ 166.114421] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.769Z level=INFO msg="Uploading 1 narinfos"6925lb # [ 166.117638] niks3-server[1059]: 2026/09/21 09:36:57 INFO Signed narinfos id=60 count=16926lb # [ 166.121960] niks3-server[1059]: 2026/09/21 09:36:57 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6927lb # [ 166.134384] niks3-server[1059]: 2026/09/21 09:36:57 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6928lb # [ 166.141486] niks3-server[1059]: 2026/09/21 09:36:57 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6929lb # [ 166.146973] niks3-server[1059]: 2026/09/21 09:36:57 INFO Completed upload id=606930worker1 # [ 166.143586] nix-grpc-daemon[1604]: time=2026-09-21T09:36:56.799Z level=INFO msg="Upload complete. (558ms)"6931worker1 # [ 166.147739] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=7526932worker1 # [ 166.154285] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366933worker1 # [ 166.165592] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6934worker1 # [ 166.169803] nix-daemon[851]: reaped child process 2028, status = succeeded6935worker1 # [ 166.175323] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6936worker1 # [ 167.389324] nix-daemon[851]: reaped child process 2042, status = succeeded6937worker1 # [ 167.593320] nix-daemon[851]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)6938worker1 # [ 167.606810] nix-daemon[851]: reaped child process 2061, status = succeeded6939lb # [ 167.633507] niks3-server[1059]: 2026/09/21 09:36:58 INFO Received uploads request method=POST path=/api/pending_closures6940worker1 # [ 167.641228] nix-grpc-daemon[1604]: time=2026-09-21T09:36:58.296Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6941worker1 # [ 167.644058] nix-grpc-daemon[1604]: time=2026-09-21T09:36:58.298Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6942lb # [ 167.673550] niks3-server[1059]: 2026/09/21 09:36:58 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6943lb # [ 167.678188] niks3-server[1059]: 2026/09/21 09:36:58 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6944lb # [ 167.691899] niks3-server[1059]: 2026/09/21 09:36:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6945lb # [ 167.695819] niks3-server[1059]: 2026/09/21 09:36:58 INFO Signed narinfos id=61 count=16946worker1 # [ 167.695081] nix-grpc-daemon[1604]: time=2026-09-21T09:36:58.350Z level=INFO msg="Uploading 1 narinfos"6947lb # [ 167.701631] niks3-server[1059]: 2026/09/21 09:36:58 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6948lb # [ 167.709192] niks3-server[1059]: 2026/09/21 09:36:58 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6949lb # [ 167.713273] niks3-server[1059]: 2026/09/21 09:36:58 INFO Completed upload id=616950lb # [ 167.715144] niks3-server[1059]: 2026/09/21 09:36:58 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6951worker1 # [ 167.710090] nix-grpc-daemon[1604]: time=2026-09-21T09:36:58.365Z level=INFO msg="Upload complete. (327ms)"6952worker2 # [ 167.713225] nix-grpc-daemon[1751]: ts=2026-09-21T09:36:58Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16953worker1 # [ 167.715834] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16954worker1 # [ 167.731115] nix-daemon[851]: reaped child process 2040, status = succeeded6955worker1 # [ 167.760526] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366956worker1 # [ 167.774583] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460366957worker1 # [ 167.789481] nix-grpc-daemon[1604]: time=2026-09-21T09:36:58.445Z level=INFO msg="All 2 paths already cached"6958worker1 # [ 167.797693] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460366959worker1 # [ 167.807394] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6960worker1 # [ 168.029429] nix-daemon[851]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)6961worker1 # [ 168.042957] nix-daemon[851]: reaped child process 2068, status = succeeded6962worker1 # [ 168.264494] nix-daemon[851]: accepted connection from pid 2070, user nix-grpc-daemon (trusted)6963worker1 # [ 168.277524] nix-daemon[851]: reaped child process 2073, status = succeeded6964lb # [ 168.303908] niks3-server[1059]: 2026/09/21 09:36:59 INFO Received uploads request method=POST path=/api/pending_closures6965worker1 # [ 168.310099] nix-grpc-daemon[1604]: time=2026-09-21T09:36:58.965Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6966worker1 # [ 168.311712] nix-grpc-daemon[1604]: time=2026-09-21T09:36:58.965Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6967lb # [ 168.332446] niks3-server[1059]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6968lb # [ 168.342379] niks3-server[1059]: 2026/09/21 09:36:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6969worker1 # [ 168.344352] nix-grpc-daemon[1604]: time=2026-09-21T09:36:58.999Z level=INFO msg="Uploading 1 narinfos"6970lb # [ 168.349592] niks3-server[1059]: 2026/09/21 09:36:59 INFO Signed narinfos id=62 count=16971lb # [ 168.355333] niks3-server[1059]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6972lb # [ 168.365057] niks3-server[1059]: 2026/09/21 09:36:59 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6973lb # [ 168.372083] niks3-server[1059]: 2026/09/21 09:36:59 INFO Completed upload id=626974worker1 # [ 168.368623] nix-grpc-daemon[1604]: time=2026-09-21T09:36:59.024Z level=INFO msg="Upload complete. (547ms)"6975lb # [ 168.375724] niks3-server[1059]: 2026/09/21 09:36:59 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6976worker1 # [ 168.371875] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=0 paths=1 nar_bytes_in=7526977worker1 # [ 168.377200] nix-grpc-daemon[1504]: ts=2026-09-21T09:36:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460366978worker1 # [ 168.387921] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6979worker1 # [ 168.394175] nix-daemon[851]: reaped child process 2063, status = succeeded6980worker1 # [ 168.397605] nix-daemon[851]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6981worker1 # [ 169.412636] nix-daemon[851]: reaped child process 2077, status = succeeded6982worker1 # [ 169.609195] nix-daemon[851]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)6983worker1 # [ 169.622503] nix-daemon[851]: reaped child process 2096, status = succeeded6984lb # [ 169.650519] niks3-server[1059]: 2026/09/21 09:37:00 INFO Received uploads request method=POST path=/api/pending_closures6985worker1 # [ 169.658601] nix-grpc-daemon[1604]: time=2026-09-21T09:37:00.314Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6986worker1 # [ 169.660493] nix-grpc-daemon[1604]: time=2026-09-21T09:37:00.316Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6987lb # [ 169.686767] niks3-server[1059]: 2026/09/21 09:37:00 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6988lb # [ 169.699647] niks3-server[1059]: 2026/09/21 09:37:00 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6989lb # [ 169.708896] niks3-server[1059]: 2026/09/21 09:37:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6990worker1 # [ 169.710356] nix-grpc-daemon[1604]: time=2026-09-21T09:37:00.365Z level=INFO msg="Uploading 1 narinfos"6991lb # [ 169.715227] niks3-server[1059]: 2026/09/21 09:37:00 INFO Signed narinfos id=63 count=16992lb # [ 169.719870] niks3-server[1059]: 2026/09/21 09:37:00 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6993lb # [ 169.728607] niks3-server[1059]: 2026/09/21 09:37:00 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6994lb # [ 169.735166] niks3-server[1059]: 2026/09/21 09:37:00 INFO Completed upload id=636995lb # [ 169.736952] niks3-server[1059]: 2026/09/21 09:37:00 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6996worker1 # [ 169.732092] nix-grpc-daemon[1604]: time=2026-09-21T09:37:00.387Z level=INFO msg="Upload complete. (327ms)"6997worker1 # [ 169.739279] nix-grpc-daemon[1504]: ts=2026-09-21T09:37:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46036 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16998worker1 # [ 169.752758] nix-daemon[851]: reaped child process 2075, status = succeeded6999client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.00 seconds)7000worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7001worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7002client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'7003client # [ 169.956243] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7004client # Running as unit: gpu.service; invocation ID: 158dc5a17e49496f8de9d75718ac8af57005client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.07 seconds)7006lb: must succeed: curl -sf localhost:9901/clusters7007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7008lb: must succeed: curl -sf localhost:9901/clusters7009lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7010worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7011worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7012worker2 # [ 170.221592] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340427013client # [ 170.244720] nix[1345]: don't know how to build these paths:7014client # [ 170.245728] nix[1345]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7015worker1 # [ 170.233938] nix-grpc-daemon[1504]: ts=2026-09-21T09:37:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460367016client # [ 170.270310] nix[1345]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker7017worker2 # [ 170.252800] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:01Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07018lb: must succeed: curl -sf localhost:9901/clusters7019lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)7020lb: must succeed: curl -sf localhost:9901/clusters7021lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7022worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7023worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7024client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7025lb # [ 176.848210] envoy[900]: conn peer=[2001:db8:1::1]:46656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27171 tx=11433 ms=206687026client # [ 200.271670] nix[1345]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}7027client # [ 200.280823] nix[1345]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7028client # [ 200.325915] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7029client # [ 200.328378] systemd[1]: gpu.service: Failed with result 'exit-code'.7030client # [ 200.331752] systemd[1]: gpu.service: Consumed 147ms CPU time over 30.369s wall clock time, 37.6M memory peak, 5.1K incoming IP traffic, 4.9K outgoing IP traffic.7031client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.04 seconds)7032client: must succeed: journalctl -u gpu -o cat7033client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7034lb: must succeed: curl -sf localhost:9901/clusters7035lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7036lb: must succeed: curl -sf localhost:9901/clusters7037lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7038worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7039worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)7040(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.92 seconds)7041subtest: low disk drains a worker and builds go to the other7042worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7043worker1 # [ 201.537360] nix-grpc-daemon[1504]: ts=2026-09-21T09:37:32Z level=info event=unhealthy reason=min_free7044worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.94 seconds)7045worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7046worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.06 seconds)7047client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&27048client # don't know how to build these paths:7049client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7050worker2 # [ 201.985945] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340427051worker2 # [ 201.992157] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340427052worker2 # [ 202.014839] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:33Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07053worker2 # [ 202.019814] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:33Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17054worker2 # [ 202.059947] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340427055worker2 # [ 202.072766] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340427056client # copying 1 paths to 'lb:50051'...7057client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7058worker2 # [ 202.085100] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340427059worker2 # [ 202.097427] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7060worker2 # [ 202.117454] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:33Z level=info event=niks3_push_started pid=18417061worker2 # [ 202.372167] nix-grpc-daemon[1841]: time=2026-09-21T09:37:33.565Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""7062worker2 # [ 202.583078] nix-daemon[871]: accepted connection from pid 1854, user nix-grpc-daemon (trusted)7063worker2 # [ 202.594876] nix-daemon[871]: reaped child process 1857, status = succeeded7064worker2 # [ 202.809353] nix-daemon[871]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)7065worker2 # [ 202.821183] nix-daemon[871]: reaped child process 1862, status = succeeded7066lb # [ 202.851799] niks3-server[1059]: 2026/09/21 09:37:33 INFO Received uploads request method=POST path=/api/pending_closures7067worker2 # [ 202.853956] nix-grpc-daemon[1841]: time=2026-09-21T09:37:34.047Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7068worker2 # [ 202.855526] nix-grpc-daemon[1841]: time=2026-09-21T09:37:34.049Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7069lb # [ 202.885880] niks3-server[1059]: 2026/09/21 09:37:33 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7070lb # [ 202.896304] niks3-server[1059]: 2026/09/21 09:37:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7071lb # [ 202.901503] niks3-server[1059]: 2026/09/21 09:37:33 INFO Signed narinfos id=64 count=17072worker2 # [ 202.897104] nix-grpc-daemon[1841]: time=2026-09-21T09:37:34.090Z level=INFO msg="Uploading 1 narinfos"7073lb # [ 202.911665] niks3-server[1059]: 2026/09/21 09:37:33 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7074lb # [ 202.920669] niks3-server[1059]: 2026/09/21 09:37:33 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7075lb # [ 202.927481] niks3-server[1059]: 2026/09/21 09:37:33 INFO Completed upload id=647076worker2 # [ 202.920305] nix-grpc-daemon[1841]: time=2026-09-21T09:37:34.114Z level=INFO msg="Upload complete. (543ms)"7077lb # [ 202.931511] niks3-server[1059]: 2026/09/21 09:37:33 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7078worker2 # [ 202.923704] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34042 duration_s=0 paths=1 nar_bytes_in=5127079worker2 # [ 202.929431] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340427080worker2 # [ 202.940082] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7081worker2 # [ 202.942630] nix-daemon[871]: reaped child process 1839, status = succeeded7082client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7083client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7084worker2 # [ 202.948069] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7085client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7086client # farm-a> LOG-farm-a-drain7087worker2 # [ 204.173547] nix-daemon[871]: reaped child process 1866, status = succeeded7088worker2 # [ 204.383113] nix-daemon[871]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)7089worker2 # [ 204.394883] nix-daemon[871]: reaped child process 1885, status = succeeded7090lb # [ 204.424221] niks3-server[1059]: 2026/09/21 09:37:35 INFO Received uploads request method=POST path=/api/pending_closures7091worker2 # [ 204.429300] nix-grpc-daemon[1841]: time=2026-09-21T09:37:35.622Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7092worker2 # [ 204.430698] nix-grpc-daemon[1841]: time=2026-09-21T09:37:35.622Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7093lb # [ 204.461993] niks3-server[1059]: 2026/09/21 09:37:35 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7094lb # [ 204.473875] niks3-server[1059]: 2026/09/21 09:37:35 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7095lb # [ 204.482309] niks3-server[1059]: 2026/09/21 09:37:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7096worker2 # [ 204.479718] nix-grpc-daemon[1841]: time=2026-09-21T09:37:35.672Z level=INFO msg="Uploading 1 narinfos"7097lb # [ 204.486490] niks3-server[1059]: 2026/09/21 09:37:35 INFO Signed narinfos id=65 count=17098lb # [ 204.490900] niks3-server[1059]: 2026/09/21 09:37:35 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7099lb # [ 204.495043] niks3-server[1059]: 2026/09/21 09:37:35 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7100lb # [ 204.497795] niks3-server[1059]: 2026/09/21 09:37:35 INFO Completed upload id=657101worker2 # [ 204.490649] nix-grpc-daemon[1841]: time=2026-09-21T09:37:35.684Z level=INFO msg="Upload complete. (334ms)"7102lb # [ 204.500898] niks3-server[1059]: 2026/09/21 09:37:35 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7103worker2 # [ 204.496740] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34042 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17104worker2 # [ 204.506082] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:35Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07105worker2 # [ 204.508168] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:35Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17106worker2 # [ 204.513466] nix-daemon[871]: reaped child process 1864, status = succeeded7107worker2 # [ 204.552155] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340427108worker2 # [ 204.563595] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340427109worker2 # [ 204.575643] nix-grpc-daemon[1841]: time=2026-09-21T09:37:35.769Z level=INFO msg="All 1 paths already cached"7110client # copying 1 paths to 'lb:50051'...7111client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7112worker2 # [ 204.582564] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340427113worker2 # [ 204.591388] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7114worker2 # [ 204.808342] nix-daemon[871]: accepted connection from pid 1891, user nix-grpc-daemon (trusted)7115worker2 # [ 204.820415] nix-daemon[871]: reaped child process 1894, status = succeeded7116worker2 # [ 205.041140] nix-daemon[871]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)7117worker2 # [ 205.052928] nix-daemon[871]: reaped child process 1899, status = succeeded7118lb # [ 205.083998] niks3-server[1059]: 2026/09/21 09:37:36 INFO Received uploads request method=POST path=/api/pending_closures7119worker2 # [ 205.085432] nix-grpc-daemon[1841]: time=2026-09-21T09:37:36.279Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7120worker2 # [ 205.087466] nix-grpc-daemon[1841]: time=2026-09-21T09:37:36.280Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7121lb # [ 205.120551] niks3-server[1059]: 2026/09/21 09:37:36 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7122lb # [ 205.129366] niks3-server[1059]: 2026/09/21 09:37:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7123worker2 # [ 205.126704] nix-grpc-daemon[1841]: time=2026-09-21T09:37:36.320Z level=INFO msg="Uploading 1 narinfos"7124lb # [ 205.134062] niks3-server[1059]: 2026/09/21 09:37:36 INFO Signed narinfos id=66 count=17125lb # [ 205.138457] niks3-server[1059]: 2026/09/21 09:37:36 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7126lb # [ 205.147628] niks3-server[1059]: 2026/09/21 09:37:36 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7127lb # [ 205.150580] niks3-server[1059]: 2026/09/21 09:37:36 INFO Completed upload id=667128worker2 # [ 205.143275] nix-grpc-daemon[1841]: time=2026-09-21T09:37:36.337Z level=INFO msg="Upload complete. (540ms)"7129lb # [ 205.152339] niks3-server[1059]: 2026/09/21 09:37:36 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7130worker2 # [ 205.146528] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34042 duration_s=0 paths=1 nar_bytes_in=6247131worker2 # [ 205.152281] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340427132worker2 # [ 205.159936] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7133worker2 # [ 205.167321] nix-daemon[871]: reaped child process 1889, status = succeeded7134client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7135client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7136worker2 # [ 205.172699] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7137client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7138client # farm-b> LOG-farm-b-drain7139worker2 # [ 206.397711] nix-daemon[871]: reaped child process 1903, status = succeeded7140worker2 # [ 206.610088] nix-daemon[871]: accepted connection from pid 1919, user nix-grpc-daemon (trusted)7141worker2 # [ 206.620913] nix-daemon[871]: reaped child process 1922, status = succeeded7142lb # [ 206.652461] niks3-server[1059]: 2026/09/21 09:37:37 INFO Received uploads request method=POST path=/api/pending_closures7143worker2 # [ 206.654225] nix-grpc-daemon[1841]: time=2026-09-21T09:37:37.847Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7144worker2 # [ 206.656242] nix-grpc-daemon[1841]: time=2026-09-21T09:37:37.849Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7145lb # [ 206.691812] niks3-server[1059]: 2026/09/21 09:37:37 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7146lb # [ 206.698797] niks3-server[1059]: 2026/09/21 09:37:37 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7147lb # [ 206.706304] niks3-server[1059]: 2026/09/21 09:37:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7148lb # [ 206.710490] niks3-server[1059]: 2026/09/21 09:37:37 INFO Signed narinfos id=67 count=17149worker2 # [ 206.706386] nix-grpc-daemon[1841]: time=2026-09-21T09:37:37.899Z level=INFO msg="Uploading 1 narinfos"7150lb # [ 206.719814] niks3-server[1059]: 2026/09/21 09:37:37 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7151lb # [ 206.728260] niks3-server[1059]: 2026/09/21 09:37:37 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7152lb # [ 206.734618] niks3-server[1059]: 2026/09/21 09:37:37 INFO Completed upload id=677153worker2 # [ 206.727546] nix-grpc-daemon[1841]: time=2026-09-21T09:37:37.921Z level=INFO msg="Upload complete. (347ms)"7154lb # [ 206.737067] niks3-server[1059]: 2026/09/21 09:37:37 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7155worker2 # [ 206.736545] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34042 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17156worker2 # [ 206.746570] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:37Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07157worker2 # [ 206.749903] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:37Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17158worker2 # [ 206.758441] nix-daemon[871]: reaped child process 1901, status = succeeded7159worker2 # [ 206.790639] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340427160worker2 # [ 206.803494] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340427161worker2 # [ 206.817346] nix-grpc-daemon[1841]: time=2026-09-21T09:37:38.011Z level=INFO msg="All 2 paths already cached"7162client # copying 1 paths to 'lb:50051'...7163client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7164worker2 # [ 206.824817] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340427165worker2 # [ 206.833410] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7166lb # [ 206.849776] envoy[900]: conn peer=[2001:db8:1::1]:52156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300547167worker2 # [ 207.056078] nix-daemon[871]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)7168worker2 # [ 207.067945] nix-daemon[871]: reaped child process 1931, status = succeeded7169worker2 # [ 207.291336] nix-daemon[871]: accepted connection from pid 1933, user nix-grpc-daemon (trusted)7170worker2 # [ 207.304497] nix-daemon[871]: reaped child process 1936, status = succeeded7171lb # [ 207.334339] niks3-server[1059]: 2026/09/21 09:37:38 INFO Received uploads request method=POST path=/api/pending_closures7172worker2 # [ 207.336536] nix-grpc-daemon[1841]: time=2026-09-21T09:37:38.530Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7173worker2 # [ 207.338128] nix-grpc-daemon[1841]: time=2026-09-21T09:37:38.531Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7174lb # [ 207.372726] niks3-server[1059]: 2026/09/21 09:37:38 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7175lb # [ 207.384641] niks3-server[1059]: 2026/09/21 09:37:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7176lb # [ 207.390102] niks3-server[1059]: 2026/09/21 09:37:38 INFO Signed narinfos id=68 count=17177worker2 # [ 207.385759] nix-grpc-daemon[1841]: time=2026-09-21T09:37:38.579Z level=INFO msg="Uploading 1 narinfos"7178lb # [ 207.395842] niks3-server[1059]: 2026/09/21 09:37:38 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7179lb # [ 207.407386] niks3-server[1059]: 2026/09/21 09:37:38 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7180lb # [ 207.412523] niks3-server[1059]: 2026/09/21 09:37:38 INFO Completed upload id=687181worker2 # [ 207.407472] nix-grpc-daemon[1841]: time=2026-09-21T09:37:38.601Z level=INFO msg="Upload complete. (561ms)"7182lb # [ 207.416588] niks3-server[1059]: 2026/09/21 09:37:38 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7183worker2 # [ 207.411232] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34042 duration_s=0 paths=1 nar_bytes_in=7527184worker2 # [ 207.415939] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340427185worker2 # [ 207.426425] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7186worker2 # [ 207.433733] nix-daemon[871]: reaped child process 1926, status = succeeded7187client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7188client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7189worker2 # [ 207.439180] nix-daemon[871]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)7190client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7191client # farm-top> LOG-farm-top-drain7192worker2 # [ 208.250984] nix-daemon[871]: reaped child process 1940, status = succeeded7193worker2 # [ 208.455625] nix-daemon[871]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)7194worker2 # [ 208.466470] nix-daemon[871]: reaped child process 1959, status = succeeded7195lb # [ 208.497926] niks3-server[1059]: 2026/09/21 09:37:39 INFO Received uploads request method=POST path=/api/pending_closures7196worker2 # [ 208.499512] nix-grpc-daemon[1841]: time=2026-09-21T09:37:39.693Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7197worker2 # [ 208.501351] nix-grpc-daemon[1841]: time=2026-09-21T09:37:39.694Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7198lb # [ 208.533150] niks3-server[1059]: 2026/09/21 09:37:39 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7199lb # [ 208.543560] niks3-server[1059]: 2026/09/21 09:37:39 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7200lb # [ 208.555488] niks3-server[1059]: 2026/09/21 09:37:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7201lb # [ 208.559351] niks3-server[1059]: 2026/09/21 09:37:39 INFO Signed narinfos id=69 count=17202worker2 # [ 208.552752] nix-grpc-daemon[1841]: time=2026-09-21T09:37:39.746Z level=INFO msg="Uploading 1 narinfos"7203lb # [ 208.565214] niks3-server[1059]: 2026/09/21 09:37:39 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7204lb # [ 208.573180] niks3-server[1059]: 2026/09/21 09:37:39 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7205lb # [ 208.575927] niks3-server[1059]: 2026/09/21 09:37:39 INFO Completed upload id=697206worker2 # [ 208.568849] nix-grpc-daemon[1841]: time=2026-09-21T09:37:39.762Z level=INFO msg="Upload complete. (323ms)"7207lb # [ 208.578959] niks3-server[1059]: 2026/09/21 09:37:39 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7208worker2 # [ 208.577082] nix-grpc-daemon[1751]: ts=2026-09-21T09:37:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34042 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17209worker2 # [ 208.587521] nix-daemon[871]: reaped child process 1938, status = succeeded7210client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.90 seconds)7211client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7212client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.27 seconds)7213worker1: must succeed: rm /nix/.rw-store/fill7214worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.14 seconds)7215worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7216worker1 # [ 209.541951] nix-grpc-daemon[1504]: ts=2026-09-21T09:37:40Z level=info event=healthy reason=min_free7217worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.14 seconds)7218(finished: subtest: low disk drains a worker and builds go to the other, in 9.55 seconds)7219(finished: run the VM test script, in 211.14 seconds)7220test script finished in 211.43s7221cleanup7222kill QemuMachine (pid 49)7223client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7224kill QemuMachine (pid 50)7225lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7226kill QemuMachine (pid 52)7227worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7228kill QemuMachine (pid 51)7229worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7230(finished: cleanup, in 1.01 seconds)7231additionally exposed symbols:7232 client, lb, worker1, worker2,7233 vlan1,7234 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh