vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #482
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.ImTxGlDYok', fmt=raw size=107374182414worker2 # Disk image does not exist, creating the virtualisation disk image...15client: QEMU running (pid 49)16worker2: QEMU running (pid 52)17client # mke2fs 1.47.4 (6-Mar-2025)18worker2 # Formatting '/build/vm-state-worker2/tmp.M6jyJDzx6W', fmt=raw size=107374182419lb # Disk image does not exist, creating the virtualisation disk image...20lb: QEMU running (pid 50)21client # Discarding device blocks: 0/262144 done22worker2 # mke2fs 1.47.4 (6-Mar-2025)23client # Creating filesystem with 262144 4k blocks and 65536 inodes24worker1 # Disk image does not exist, creating the virtualisation disk image...25worker1: QEMU running (pid 51)26client # Filesystem UUID: 85d358d0-23fa-4505-a31c-bdb524336f1427worker1 # Formatting '/build/vm-state-worker1/tmp.KEowICboqq', fmt=raw size=107374182428worker2 # Discarding device blocks: 0/262144 done29lb # Formatting '/build/vm-state-lb/tmp.WB1irQTsC5', fmt=raw size=107374182430client # Superblock backups stored on blocks:31(finished: start all VMs, in 0.23 seconds)32lb # mke2fs 1.47.4 (6-Mar-2025)33client # 32768, 98304, 163840, 22937634client: waiting for unit network-addresses-eth1.service35lb # Discarding device blocks: 0/262144 done36client # 37client: waiting for the VM to finish booting38lb # Creating filesystem with 262144 4k blocks and 65536 inodes39client # Allocating group tables: 0/8 done40worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes41worker1 # mke2fs 1.47.4 (6-Mar-2025)42lb # Filesystem UUID: 5383dc27-4c93-41dd-8d70-9e94456196fb43client # Writing inode tables: 0/8 done44worker2 # Filesystem UUID: cb54f6d8-1e9e-431e-85ca-a5f7f9727a5945worker1 # Discarding device blocks: 0/262144 done46lb # Superblock backups stored on blocks:47client # Creating journal (8192 blocks): done48worker2 # Superblock backups stored on blocks:49worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes50lb # 32768, 98304, 163840, 22937651client # Writing superblocks and filesystem accounting information: 0/8 done52worker2 # 32768, 98304, 163840, 22937653worker1 # Filesystem UUID: 36d0de19-e90d-4134-bd00-38d9cef3f02554lb # 55client # 56worker2 # 57worker1 # Superblock backups stored on blocks:58lb # Allocating group tables: 0/8 done59client # Virtualisation disk image created.60worker2 # Allocating group tables: 0/8 done61worker1 # 32768, 98304, 163840, 22937662worker2 # Writing inode tables: 0/8 done63worker1 # 64worker2 # Creating journal (8192 blocks): done65worker1 # Allocating group tables: 0/8 done66worker2 # Writing superblocks and filesystem accounting information: 0/8 done67worker1 # Writing inode tables: 0/8 done68worker2 # 69worker1 # Creating journal (8192 blocks): done70worker2 # Virtualisation disk image created.71worker1 # Writing superblocks and filesystem accounting information: 0/8 done72lb # Writing inode tables: 0/8 done73worker1 # 74lb # Creating journal (8192 blocks): done75worker1 # Virtualisation disk image created.76lb # Writing superblocks and filesystem accounting information: 0/8 done77lb # 78lb # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086lb # 87lb # 88worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...89worker2 # 90worker2 # 91lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0092worker2 # 93worker2 # 94client # 95client # 96worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0097client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0098lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...99lb # 100lb # 101lb # 102lb # 103lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00104client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...105client # 106client # 107client # 108client # 109worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...110worker2 # 111worker2 # 112client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114lb # 115lb # 116client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117client # 118client # 119worker1 # 120worker1 # 121worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123worker1 # 124worker1 # 125worker1 # 126worker1 # 127worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker1 # 130worker1 # 131worker2 # Booting from ROM...132lb # Booting from ROM...133client # Booting from ROM...134worker1 # Booting from ROM...135client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test/init regInfo=/nix/store/v58y4g30w74cah80479jy4r0f6qilcdy-closure-info/registration console=ttyS0,115200n8 console=tty0137client # [ 0.000000] BIOS-provided physical RAM map:138client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026141client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable143client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved144lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026145client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved146worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fvarbidl2wqmz4dyj4zg00dwpibfzyyl-nixos-system-worker2-test/init regInfo=/nix/store/prkxh6fbnyym568nsc5698wcagbkrg1p-closure-info/registration console=ttyS0,115200n8 console=tty0147client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved148worker2 # [ 0.000000] BIOS-provided physical RAM map:149client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved150worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable151client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved152worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved153lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mvx6cn1l6frdr55gdj0p01an8nxnxzn-nixos-system-lb-test/init regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration console=ttyS0,115200n8 console=tty0154client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved155lb # [ 0.000000] BIOS-provided physical RAM map:156worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved157client # [ 0.000000] NX (Execute Disable) protection: active158lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable159client # [ 0.000000] APIC: Static calls initialized160worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable161client # [ 0.000000] SMBIOS 2.8 present.162lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved163worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved164client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014165lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved166worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved167client # [ 0.000000] DMI: Memory slots populated: 1/1168lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable169client # [ 0.000000] Hypervisor detected: KVM170worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved171client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000172lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved173worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved174client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00175lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved176worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved177client # [ 0.000000] kvm-clock: using sched offset of 468810675 cycles178lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved179worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved180lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved181client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns182worker2 # [ 0.000000] NX (Execute Disable) protection: active183worker2 # [ 0.000000] APIC: Static calls initialized184client # [ 0.000005] tsc: Detected 2400.012 MHz processor185lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved186worker2 # [ 0.000000] SMBIOS 2.8 present.187client # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000188lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved189client # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs190worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014191lb # [ 0.000000] NX (Execute Disable) protection: active192lb # [ 0.000000] APIC: Static calls initialized193worker2 # [ 0.000000] DMI: Memory slots populated: 1/1194client # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT195lb # [ 0.000000] SMBIOS 2.8 present.196worker2 # [ 0.000000] Hypervisor detected: KVM197client # [ 0.002726] found SMP MP-table at [mem 0x000f5450-0x000f545f]198worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000199client # [ 0.002736] Using GB pages for direct mapping200lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014201worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00202lb # [ 0.000000] DMI: Memory slots populated: 1/1203client # [ 0.002882] RAMDISK: [mem 0x5e366000-0x5ffcffff]204worker2 # [ 0.000001] kvm-clock: using sched offset of 469083264 cycles205lb # [ 0.000000] Hypervisor detected: KVM206client # [ 0.002889] ACPI: Early table checksum verification disabled207lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000208client # [ 0.002892] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )209lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00210worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns211worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor212lb # [ 0.000001] kvm-clock: using sched offset of 472525997 cycles213client # [ 0.002896] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)214worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000215client # [ 0.002900] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)216lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns217worker2 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs218lb # [ 0.000005] tsc: Detected 2400.012 MHz processor219client # [ 0.002907] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)220worker2 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT221lb # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000222client # [ 0.002909] ACPI: FACS 0x000000005FFE0000 000040223worker2 # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]224lb # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs225client # [ 0.002910] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)226worker2 # [ 0.002750] Using GB pages for direct mapping227worker2 # [ 0.002870] RAMDISK: [mem 0x5e367000-0x5ffcffff]228lb # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT229client # [ 0.002912] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)230worker2 # [ 0.002879] ACPI: Early table checksum verification disabled231lb # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]232lb # [ 0.002750] Using GB pages for direct mapping233worker2 # [ 0.002882] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )234client # [ 0.002914] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)235lb # [ 0.002890] RAMDISK: [mem 0x5e367000-0x5ffcffff]236client # [ 0.002915] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)237worker2 # [ 0.002886] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238lb # [ 0.002901] ACPI: Early table checksum verification disabled239lb # [ 0.002904] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )240client # [ 0.002916] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]241worker2 # [ 0.002891] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)242client # [ 0.002917] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]243lb # [ 0.002908] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)244worker2 # [ 0.002898] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)245client # [ 0.002918] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]246worker2 # [ 0.002900] ACPI: FACS 0x000000005FFE0000 000040247lb # [ 0.002912] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)248client # [ 0.002918] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]249worker2 # [ 0.002901] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)250lb # [ 0.002919] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)251client # [ 0.002919] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]252lb # [ 0.002921] ACPI: FACS 0x000000005FFE0000 000040253worker2 # [ 0.002903] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)254client # [ 0.002919] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]255lb # [ 0.002922] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)256worker2 # [ 0.002904] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)257client # [ 0.002920] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]258client # [ 0.003141] No NUMA configuration found259lb # [ 0.002924] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)260worker2 # [ 0.002906] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)261client # [ 0.003142] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]262lb # [ 0.002926] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)263worker2 # [ 0.002907] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]264client # [ 0.003144] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]265client # [ 0.005731] Zone ranges:266lb # [ 0.002927] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)267worker2 # [ 0.002908] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]268client # [ 0.005731] DMA [mem 0x0000000000001000-0x0000000000ffffff]269worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026270lb # [ 0.002928] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]271client # [ 0.005733] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]272worker2 # [ 0.002909] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]273client # [ 0.005734] Normal empty274lb # [ 0.002929] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]275client # [ 0.005735] Device empty276worker2 # [ 0.002909] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]277client # [ 0.005735] Movable zone start for each node278lb # [ 0.002930] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]279worker2 # [ 0.002910] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]280client # [ 0.005736] Early memory node ranges281lb # [ 0.002930] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]282worker2 # [ 0.002910] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]283client # [ 0.005736] node 0: [mem 0x0000000000001000-0x000000000009efff]284lb # [ 0.002931] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]285client # [ 0.005737] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]286worker2 # [ 0.002911] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]287worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pmxyh77a0n43v9c1gr64pjx9f8gnrqm0-nixos-system-worker1-test/init regInfo=/nix/store/413ximqr7f3si1lwll5arwnvb9rxmjpw-closure-info/registration console=ttyS0,115200n8 console=tty0288worker2 # [ 0.003141] No NUMA configuration found289worker1 # [ 0.000000] BIOS-provided physical RAM map:290client # [ 0.005738] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]291lb # [ 0.002931] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]292worker2 # [ 0.003142] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]293client # [ 0.005758] On node 0, zone DMA: 1 pages in unavailable ranges294lb # [ 0.002932] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]295worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable296lb # [ 0.003173] No NUMA configuration found297worker2 # [ 0.003144] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]298client # [ 0.006034] On node 0, zone DMA: 97 pages in unavailable ranges299worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved300worker2 # [ 0.005476] Zone ranges:301lb # [ 0.003174] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]302client # [ 0.032915] On node 0, zone DMA32: 40 pages in unavailable ranges303worker2 # [ 0.005477] DMA [mem 0x0000000000001000-0x0000000000ffffff]304worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved305client # [ 0.033366] ACPI: PM-Timer IO Port: 0x608306lb # [ 0.003176] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]307lb # [ 0.005557] Zone ranges:308worker2 # [ 0.005479] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]309worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable310client # [ 0.033377] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])311worker2 # [ 0.005480] Normal empty312worker2 # [ 0.005481] Device empty313lb # [ 0.005558] DMA [mem 0x0000000000001000-0x0000000000ffffff]314worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved315client # [ 0.033403] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23316worker2 # [ 0.005482] Movable zone start for each node317lb # [ 0.005559] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]318worker2 # [ 0.005483] Early memory node ranges319lb # [ 0.005561] Normal empty320worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved321client # [ 0.033405] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)322lb # [ 0.005561] Device empty323worker2 # [ 0.005483] node 0: [mem 0x0000000000001000-0x000000000009efff]324client # [ 0.033406] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)325worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved326lb # [ 0.005562] Movable zone start for each node327lb # [ 0.005563] Early memory node ranges328worker2 # [ 0.005484] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]329client # [ 0.033408] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)330worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved331lb # [ 0.005563] node 0: [mem 0x0000000000001000-0x000000000009efff]332worker2 # [ 0.005485] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]333client # [ 0.033409] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)334worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved335lb # [ 0.005564] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]336worker2 # [ 0.005506] On node 0, zone DMA: 1 pages in unavailable ranges337client # [ 0.033409] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)338worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved339lb # [ 0.005565] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]340worker2 # [ 0.005780] On node 0, zone DMA: 97 pages in unavailable ranges341worker1 # [ 0.000000] NX (Execute Disable) protection: active342client # [ 0.033412] ACPI: Using ACPI (MADT) for SMP configuration information343lb # [ 0.005586] On node 0, zone DMA: 1 pages in unavailable ranges344worker2 # [ 0.035062] On node 0, zone DMA32: 40 pages in unavailable ranges345worker1 # [ 0.000000] APIC: Static calls initialized346client # [ 0.033413] ACPI: HPET id: 0x8086a201 base: 0xfed00000347worker1 # [ 0.000000] SMBIOS 2.8 present.348worker2 # [ 0.035570] ACPI: PM-Timer IO Port: 0x608349client # [ 0.033416] TSC deadline timer available350lb # [ 0.005861] On node 0, zone DMA: 97 pages in unavailable ranges351worker2 # [ 0.035586] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])352client # [ 0.033420] CPU topo: Max. logical packages: 1353lb # [ 0.033902] On node 0, zone DMA32: 40 pages in unavailable ranges354worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014355client # [ 0.033421] CPU topo: Max. logical dies: 1356lb # [ 0.034369] ACPI: PM-Timer IO Port: 0x608357worker2 # [ 0.035622] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23358worker1 # [ 0.000000] DMI: Memory slots populated: 1/1359client # [ 0.033421] CPU topo: Max. dies per package: 1360lb # [ 0.034381] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])361worker1 # [ 0.000000] Hypervisor detected: KVM362worker2 # [ 0.035625] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)363client # [ 0.033424] CPU topo: Max. threads per core: 1364worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000365lb # [ 0.034411] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23366client # [ 0.033425] CPU topo: Num. cores per package: 1367worker2 # [ 0.035627] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)368client # [ 0.033426] CPU topo: Num. threads per package: 1369lb # [ 0.034413] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)370worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00371worker2 # [ 0.035628] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)372worker1 # [ 0.000000] kvm-clock: using sched offset of 498209788 cycles373client # [ 0.033426] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs374lb # [ 0.034415] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)375worker2 # [ 0.035629] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)376lb # [ 0.034416] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)377client # [ 0.033443] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()378worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns379worker2 # [ 0.035630] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)380lb # [ 0.034417] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)381worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor382client # [ 0.033475] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]383worker2 # [ 0.035632] ACPI: Using ACPI (MADT) for SMP configuration information384worker1 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000385lb # [ 0.034418] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)386client # [ 0.033476] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]387worker2 # [ 0.035633] ACPI: HPET id: 0x8086a201 base: 0xfed00000388worker2 # [ 0.035639] TSC deadline timer available389client # [ 0.033477] [mem 0x60000000-0xafffffff] available for PCI devices390lb # [ 0.034420] ACPI: Using ACPI (MADT) for SMP configuration information391worker1 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs392worker2 # [ 0.035643] CPU topo: Max. logical packages: 1393lb # [ 0.034421] ACPI: HPET id: 0x8086a201 base: 0xfed00000394client # [ 0.033479] Booting paravirtualized kernel on KVM395worker1 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT396worker2 # [ 0.035644] CPU topo: Max. logical dies: 1397lb # [ 0.034425] TSC deadline timer available398worker2 # [ 0.035645] CPU topo: Max. dies per package: 1399lb # [ 0.034429] CPU topo: Max. logical packages: 1400client # [ 0.033482] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns401worker1 # [ 0.002735] found SMP MP-table at [mem 0x000f5450-0x000f545f]402worker2 # [ 0.035648] CPU topo: Max. threads per core: 1403lb # [ 0.034430] CPU topo: Max. logical dies: 1404worker1 # [ 0.002746] Using GB pages for direct mapping405lb # [ 0.034431] CPU topo: Max. dies per package: 1406client # [ 0.037931] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1407worker2 # [ 0.035649] CPU topo: Num. cores per package: 1408worker1 # [ 0.002823] RAMDISK: [mem 0x5e367000-0x5ffcffff]409worker2 # [ 0.035649] CPU topo: Num. threads per package: 1410client # [ 0.040109] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152411worker1 # [ 0.002832] ACPI: Early table checksum verification disabled412lb # [ 0.034434] CPU topo: Max. threads per core: 1413worker1 # [ 0.002835] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )414client # [ 0.040152] kvm-guest: PV spinlocks disabled, single CPU415lb # [ 0.034435] CPU topo: Num. cores per package: 1416worker2 # [ 0.035649] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs417lb # [ 0.034435] CPU topo: Num. threads per package: 1418worker2 # [ 0.035675] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()419lb # [ 0.034435] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs420worker1 # [ 0.002839] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)421worker2 # [ 0.035714] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]422lb # [ 0.034455] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()423worker1 # [ 0.002844] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)424worker2 # [ 0.035715] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]425lb # [ 0.034489] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]426worker1 # [ 0.002850] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)427worker2 # [ 0.035717] [mem 0x60000000-0xafffffff] available for PCI devices428client # [ 0.040154] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test/init regInfo=/nix/store/v58y4g30w74cah80479jy4r0f6qilcdy-closure-info/registration console=ttyS0,115200n8 console=tty0429worker1 # [ 0.002853] ACPI: FACS 0x000000005FFE0000 000040430worker2 # [ 0.035719] Booting paravirtualized kernel on KVM431lb # [ 0.034491] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]432worker1 # [ 0.002854] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)433lb # [ 0.034492] [mem 0x60000000-0xafffffff] available for PCI devices434worker2 # [ 0.035723] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns435lb # [ 0.034494] Booting paravirtualized kernel on KVM436client # [ 0.040243] Unknown kernel command line parameters "regInfo=/nix/store/v58y4g30w74cah80479jy4r0f6qilcdy-closure-info/registration", will be passed to user space.437worker1 # [ 0.002856] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)438client # [ 0.040256] random: crng init done439worker2 # [ 0.040291] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1440lb # [ 0.034497] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns441worker1 # [ 0.002857] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)442client # [ 0.040257] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes443worker2 # [ 0.042584] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152444lb # [ 0.038955] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1445worker2 # [ 0.042632] kvm-guest: PV spinlocks disabled, single CPU446client # [ 0.042405] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)447worker1 # [ 0.002859] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)448lb # [ 0.041250] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152449worker1 # [ 0.002860] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]450client # [ 0.043476] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)451lb # [ 0.041298] kvm-guest: PV spinlocks disabled, single CPU452client # [ 0.043508] Fallback order for Node 0: 0453worker1 # [ 0.002861] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]454client # [ 0.043511] Built 1 zonelists, mobility grouping on. Total pages: 393078455client # [ 0.043512] Policy zone: DMA32456worker1 # [ 0.002862] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]457client # [ 0.046283] mem auto-init: stack:all(zero), heap alloc:on, heap free:off458worker1 # [ 0.002862] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]459worker2 # [ 0.042634] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fvarbidl2wqmz4dyj4zg00dwpibfzyyl-nixos-system-worker2-test/init regInfo=/nix/store/prkxh6fbnyym568nsc5698wcagbkrg1p-closure-info/registration console=ttyS0,115200n8 console=tty0460client # [ 0.049270] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1461worker1 # [ 0.002863] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]462client # [ 0.052600] allocated 3145728 bytes of page_ext463lb # [ 0.041299] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mvx6cn1l6frdr55gdj0p01an8nxnxzn-nixos-system-lb-test/init regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration console=ttyS0,115200n8 console=tty0464worker1 # [ 0.002863] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]465client # [ 0.062078] ftrace: allocating 48731 entries in 192 pages466worker2 # [ 0.042733] Unknown kernel command line parameters "regInfo=/nix/store/prkxh6fbnyym568nsc5698wcagbkrg1p-closure-info/registration", will be passed to user space.467worker2 # [ 0.042746] random: crng init done468client # [ 0.062080] ftrace: allocated 192 pages with 2 groups469worker1 # [ 0.002864] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]470client # [ 0.062939] Dynamic Preempt: lazy471worker1 # [ 0.003089] No NUMA configuration found472lb # [ 0.041394] Unknown kernel command line parameters "regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration", will be passed to user space.473worker2 # [ 0.042747] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes474client # [ 0.063119] rcu: Preemptible hierarchical RCU implementation.475lb # [ 0.041406] random: crng init done476worker1 # [ 0.003090] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]477client # [ 0.063120] rcu: RCU event tracing is enabled.478worker2 # [ 0.044989] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)479lb # [ 0.041407] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes480worker1 # [ 0.003092] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]481client # [ 0.063120] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.482worker1 # [ 0.005499] Zone ranges:483worker2 # [ 0.046128] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)484lb # [ 0.043666] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)485client # [ 0.063122] Trampoline variant of Tasks RCU enabled.486worker2 # [ 0.046166] Fallback order for Node 0: 0487worker1 # [ 0.005500] DMA [mem 0x0000000000001000-0x0000000000ffffff]488client # [ 0.063122] Rude variant of Tasks RCU enabled.489worker1 # [ 0.005502] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]490lb # [ 0.045064] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)491worker2 # [ 0.046169] Built 1 zonelists, mobility grouping on. Total pages: 393078492client # [ 0.063122] Tracing variant of Tasks RCU enabled.493worker2 # [ 0.046170] Policy zone: DMA32494lb # [ 0.045095] Fallback order for Node 0: 0495worker1 # [ 0.005503] Normal empty496worker1 # [ 0.005504] Device empty497client # [ 0.063123] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.498worker2 # [ 0.049071] mem auto-init: stack:all(zero), heap alloc:on, heap free:off499lb # [ 0.045098] Built 1 zonelists, mobility grouping on. Total pages: 393078500worker1 # [ 0.005504] Movable zone start for each node501lb # [ 0.045099] Policy zone: DMA32502worker1 # [ 0.005505] Early memory node ranges503client # [ 0.063124] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1504worker2 # [ 0.052488] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1505worker1 # [ 0.005505] node 0: [mem 0x0000000000001000-0x000000000009efff]506lb # [ 0.047708] mem auto-init: stack:all(zero), heap alloc:on, heap free:off507worker2 # [ 0.055952] allocated 3145728 bytes of page_ext508client # [ 0.063142] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.509worker1 # [ 0.005507] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]510worker2 # [ 0.066262] ftrace: allocating 48731 entries in 192 pages511lb # [ 0.051016] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1512worker2 # [ 0.066264] ftrace: allocated 192 pages with 2 groups513lb # [ 0.054519] allocated 3145728 bytes of page_ext514worker1 # [ 0.005508] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]515client # [ 0.063144] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.516worker2 # [ 0.067172] Dynamic Preempt: lazy517lb # [ 0.064789] ftrace: allocating 48731 entries in 192 pages518worker1 # [ 0.005528] On node 0, zone DMA: 1 pages in unavailable ranges519worker2 # [ 0.067305] rcu: Preemptible hierarchical RCU implementation.520lb # [ 0.064791] ftrace: allocated 192 pages with 2 groups521client # [ 0.063145] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.522lb # [ 0.065673] Dynamic Preempt: lazy523worker2 # [ 0.067305] rcu: RCU event tracing is enabled.524worker1 # [ 0.005807] On node 0, zone DMA: 97 pages in unavailable ranges525client # [ 0.067501] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16526lb # [ 0.065871] rcu: Preemptible hierarchical RCU implementation.527worker1 # [ 0.033893] On node 0, zone DMA32: 40 pages in unavailable ranges528worker2 # [ 0.067306] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.529client # [ 0.067783] rcu: srcu_init: Setting srcu_struct sizes based on contention.530lb # [ 0.065872] rcu: RCU event tracing is enabled.531worker1 # [ 0.034360] ACPI: PM-Timer IO Port: 0x608532worker2 # [ 0.067307] Trampoline variant of Tasks RCU enabled.533worker1 # [ 0.034371] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])534lb # [ 0.065872] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.535worker2 # [ 0.067308] Rude variant of Tasks RCU enabled.536client # [ 0.067790] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns537worker2 # [ 0.067308] Tracing variant of Tasks RCU enabled.538lb # [ 0.065874] Trampoline variant of Tasks RCU enabled.539worker1 # [ 0.034401] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23540lb # [ 0.065875] Rude variant of Tasks RCU enabled.541worker2 # [ 0.067309] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.542worker1 # [ 0.034404] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)543client # [ 0.067899] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)544lb # [ 0.065875] Tracing variant of Tasks RCU enabled.545client # [ 0.071421] Console: colour VGA+ 80x25546worker1 # [ 0.034406] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)547worker2 # [ 0.067310] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1548client # [ 0.071424] printk: legacy console [tty0] enabled549lb # [ 0.065876] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.550client # [ 0.100887] printk: legacy console [ttyS0] enabled551worker1 # [ 0.034407] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)552worker2 # [ 0.067368] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.553lb # [ 0.065877] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1554client # [ 0.197274] ACPI: Core revision 20250807555worker1 # [ 0.034408] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)556worker2 # [ 0.067370] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.557lb # [ 0.065895] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.558worker1 # [ 0.034408] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)559client # [ 0.198149] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns560lb # [ 0.065897] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.561worker1 # [ 0.034411] ACPI: Using ACPI (MADT) for SMP configuration information562worker2 # [ 0.067371] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.563worker1 # [ 0.034412] ACPI: HPET id: 0x8086a201 base: 0xfed00000564worker2 # [ 0.071820] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16565client # [ 0.199623] APIC: Switch to symmetric I/O mode setup566worker1 # [ 0.034415] TSC deadline timer available567lb # [ 0.065898] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.568client # [ 0.200547] x2apic enabled569worker2 # [ 0.072158] rcu: srcu_init: Setting srcu_struct sizes based on contention.570worker1 # [ 0.034420] CPU topo: Max. logical packages: 1571lb # [ 0.070708] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16572worker1 # [ 0.034420] CPU topo: Max. logical dies: 1573lb # [ 0.070979] rcu: srcu_init: Setting srcu_struct sizes based on contention.574worker2 # [ 0.072165] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns575client # [ 0.201277] APIC: Switched APIC routing to: physical x2apic576worker1 # [ 0.034421] CPU topo: Max. dies per package: 1577worker1 # [ 0.034424] CPU topo: Max. threads per core: 1578lb # [ 0.070987] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns579worker2 # [ 0.072266] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)580worker1 # [ 0.034425] CPU topo: Num. cores per package: 1581worker2 # [ 0.075808] Console: colour VGA+ 80x25582worker1 # [ 0.034425] CPU topo: Num. threads per package: 1583client # [ 0.203041] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1584lb # [ 0.071105] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)585worker2 # [ 0.075811] printk: legacy console [tty0] enabled586worker1 # [ 0.034425] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs587lb # [ 0.074607] Console: colour VGA+ 80x25588worker2 # [ 0.105479] printk: legacy console [ttyS0] enabled589lb # [ 0.074611] printk: legacy console [tty0] enabled590worker1 # [ 0.034444] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()591client # [ 0.203989] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns592worker2 # [ 0.203288] ACPI: Core revision 20250807593lb # [ 0.104099] printk: legacy console [ttyS0] enabled594worker1 # [ 0.034477] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]595lb # [ 0.200165] ACPI: Core revision 20250807596client # [ 0.205562] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)597worker1 # [ 0.034478] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]598worker2 # [ 0.204178] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns599worker1 # [ 0.034480] [mem 0x60000000-0xafffffff] available for PCI devices600client # [ 0.207645] x86/cpu: User Mode Instruction Prevention (UMIP) activated601worker2 # [ 0.205695] APIC: Switch to symmetric I/O mode setup602lb # [ 0.201054] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns603worker1 # [ 0.034481] Booting paravirtualized kernel on KVM604worker2 # [ 0.206665] x2apic enabled605client # [ 0.208681] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127606lb # [ 0.202530] APIC: Switch to symmetric I/O mode setup607worker1 # [ 0.034484] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns608lb # [ 0.203507] x2apic enabled609client # [ 0.209560] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0610worker2 # [ 0.207451] APIC: Switched APIC routing to: physical x2apic611worker1 # [ 0.038994] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1612lb # [ 0.204283] APIC: Switched APIC routing to: physical x2apic613worker1 # [ 0.041314] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152614client # [ 0.210563] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto615worker2 # [ 0.209254] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1616worker1 # [ 0.041361] kvm-guest: PV spinlocks disabled, single CPU617lb # [ 0.206080] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1618client # [ 0.211560] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl619worker2 # [ 0.210244] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns620client # [ 0.212560] Transient Scheduler Attacks: Vulnerable: No microcode621lb # [ 0.207041] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns622client # [ 0.213559] Spectre V2 : Mitigation: Enhanced / Automatic IBRS623worker2 # [ 0.211893] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)624lb # [ 0.208665] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)625client # [ 0.214559] Speculative Return Stack Overflow: Mitigation: Safe RET626worker1 # [ 0.041362] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pmxyh77a0n43v9c1gr64pjx9f8gnrqm0-nixos-system-worker1-test/init regInfo=/nix/store/413ximqr7f3si1lwll5arwnvb9rxmjpw-closure-info/registration console=ttyS0,115200n8 console=tty0627worker2 # [ 0.212981] x86/cpu: User Mode Instruction Prevention (UMIP) activated628lb # [ 0.209757] x86/cpu: User Mode Instruction Prevention (UMIP) activated629worker2 # [ 0.214068] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127630worker1 # [ 0.041458] Unknown kernel command line parameters "regInfo=/nix/store/413ximqr7f3si1lwll5arwnvb9rxmjpw-closure-info/registration", will be passed to user space.631worker1 # [ 0.041471] random: crng init done632lb # [ 0.210804] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127633worker2 # [ 0.215598] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0634worker1 # [ 0.041472] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes635lb # [ 0.211662] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0636client # [ 0.215559] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization637worker1 # [ 0.044009] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)638worker2 # [ 0.216662] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto639client # [ 0.217568] Spectre V2 : Enabling IBPB for BPF640lb # [ 0.212665] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto641worker1 # [ 0.045146] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)642worker1 # [ 0.045179] Fallback order for Node 0: 0643worker2 # [ 0.217890] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl644client # [ 0.218279] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier645lb # [ 0.214661] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl646worker1 # [ 0.045182] Built 1 zonelists, mobility grouping on. Total pages: 393078647worker2 # [ 0.218890] Transient Scheduler Attacks: Vulnerable: No microcode648worker1 # [ 0.045183] Policy zone: DMA32649client # [ 0.219560] active return thunk: srso_alias_return_thunk650lb # [ 0.215662] Transient Scheduler Attacks: Vulnerable: No microcode651worker1 # [ 0.047808] mem auto-init: stack:all(zero), heap alloc:on, heap free:off652worker2 # [ 0.220595] Spectre V2 : Mitigation: Enhanced / Automatic IBRS653client # [ 0.220393] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'654lb # [ 0.216661] Spectre V2 : Mitigation: Enhanced / Automatic IBRS655worker1 # [ 0.051103] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1656worker2 # [ 0.220890] Speculative Return Stack Overflow: Mitigation: Safe RET657worker1 # [ 0.054636] allocated 3145728 bytes of page_ext658client # [ 0.221559] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'659lb # [ 0.217661] Speculative Return Stack Overflow: Mitigation: Safe RET660worker1 # [ 0.065223] ftrace: allocating 48731 entries in 192 pages661worker1 # [ 0.065225] ftrace: allocated 192 pages with 2 groups662client # [ 0.222559] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'663worker1 # [ 0.066106] Dynamic Preempt: lazy664client # [ 0.223559] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'665worker1 # [ 0.066288] rcu: Preemptible hierarchical RCU implementation.666worker1 # [ 0.066289] rcu: RCU event tracing is enabled.667client # [ 0.224559] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'668worker2 # [ 0.221890] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization669worker1 # [ 0.066290] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.670client # [ 0.225559] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'671lb # [ 0.218661] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization672worker2 # [ 0.223897] Spectre V2 : Enabling IBPB for BPF673worker1 # [ 0.066291] Trampoline variant of Tasks RCU enabled.674worker1 # [ 0.066292] Rude variant of Tasks RCU enabled.675lb # [ 0.220669] Spectre V2 : Enabling IBPB for BPF676client # [ 0.226559] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'677worker1 # [ 0.066292] Tracing variant of Tasks RCU enabled.678worker2 # [ 0.224891] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier679worker1 # [ 0.066293] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.680client # [ 0.227560] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256681lb # [ 0.221662] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier682worker2 # [ 0.225890] active return thunk: srso_alias_return_thunk683worker1 # [ 0.066293] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1684lb # [ 0.222662] active return thunk: srso_alias_return_thunk685client # [ 0.228559] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64686worker2 # [ 0.226912] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'687worker1 # [ 0.066316] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.688client # [ 0.229559] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512689lb # [ 0.223683] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'690worker2 # [ 0.227890] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'691client # [ 0.230559] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024692worker1 # [ 0.066318] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.693lb # [ 0.224661] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'694worker2 # [ 0.229620] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'695client # [ 0.231559] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8696worker1 # [ 0.066318] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.697lb # [ 0.225661] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'698worker2 # [ 0.230615] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'699worker1 # [ 0.070808] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16700lb # [ 0.226661] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'701client # [ 0.232463] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.702worker2 # [ 0.231622] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'703worker1 # [ 0.071104] rcu: srcu_init: Setting srcu_struct sizes based on contention.704lb # [ 0.227661] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'705worker2 # [ 0.231890] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'706worker1 # [ 0.071111] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns707lb # [ 0.228661] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'708worker2 # [ 0.232890] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'709worker1 # [ 0.071225] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)710worker1 # [ 0.074838] Console: colour VGA+ 80x25711lb # [ 0.229661] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'712worker2 # [ 0.234890] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256713worker1 # [ 0.074841] printk: legacy console [tty0] enabled714lb # [ 0.230661] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256715worker1 # [ 0.105302] printk: legacy console [ttyS0] enabled716worker2 # [ 0.235890] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64717worker1 # [ 0.202664] ACPI: Core revision 20250807718lb # [ 0.231661] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64719worker2 # [ 0.236890] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512720lb # [ 0.232661] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512721worker2 # [ 0.237890] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024722worker1 # [ 0.203583] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns723lb # [ 0.233661] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024724worker2 # [ 0.238845] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8725worker1 # [ 0.205156] APIC: Switch to symmetric I/O mode setup726lb # [ 0.234661] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8727worker1 # [ 0.206153] x2apic enabled728worker2 # [ 0.239630] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.729lb # [ 0.235661] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.730worker1 # [ 0.206928] APIC: Switched APIC routing to: physical x2apic731worker1 # [ 0.208787] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1732worker1 # [ 0.209836] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns733worker1 # [ 0.211587] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)734worker1 # [ 0.212672] x86/cpu: User Mode Instruction Prevention (UMIP) activated735worker1 # [ 0.214732] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127736worker1 # [ 0.215584] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0737worker1 # [ 0.216587] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto738worker1 # [ 0.217584] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl739worker1 # [ 0.218584] Transient Scheduler Attacks: Vulnerable: No microcode740worker1 # [ 0.219583] Spectre V2 : Mitigation: Enhanced / Automatic IBRS741worker1 # [ 0.220584] Speculative Return Stack Overflow: Mitigation: Safe RET742worker1 # [ 0.221584] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization743worker1 # [ 0.223592] Spectre V2 : Enabling IBPB for BPF744worker1 # [ 0.224584] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier745worker1 # [ 0.225584] active return thunk: srso_alias_return_thunk746worker1 # [ 0.226605] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'747client # [ 0.266527] Freeing SMP alternatives memory: 44K748worker1 # [ 0.228583] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'749client # [ 0.267133] pid_max: default: 32768 minimum: 301750worker1 # [ 0.229583] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'751client # [ 0.267645] LSM: initializing lsm=capability,landlock,yama,bpf,ima752worker1 # [ 0.230583] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'753client # [ 0.268663] landlock: Up and running.754client # [ 0.269560] Yama: becoming mindful.755worker1 # [ 0.231583] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'756client # [ 0.270774] LSM support for eBPF active757worker1 # [ 0.232584] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'758client # [ 0.271565] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)759worker1 # [ 0.233584] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'760client # [ 0.272604] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)761worker1 # [ 0.234584] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256762worker1 # [ 0.235583] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64763worker1 # [ 0.236583] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512764worker2 # [ 0.274187] Freeing SMP alternatives memory: 44K765client # [ 0.275443] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)766lb # [ 0.269619] Freeing SMP alternatives memory: 44K767worker1 # [ 0.237583] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024768worker2 # [ 0.274893] pid_max: default: 32768 minimum: 301769lb # [ 0.270212] pid_max: default: 32768 minimum: 301770worker1 # [ 0.238583] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8771worker2 # [ 0.275719] LSM: initializing lsm=capability,landlock,yama,bpf,ima772client # [ 0.276142] Performance Events: Fam17h+ core perfctr, AMD PMU driver.773lb # [ 0.270765] LSM: initializing lsm=capability,landlock,yama,bpf,ima774worker2 # [ 0.276921] landlock: Up and running.775client # [ 0.276564] ... version: 2776lb # [ 0.271776] landlock: Up and running.777worker1 # [ 0.239584] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.778client # [ 0.277241] ... bit width: 48779worker2 # [ 0.277529] Yama: becoming mindful.780lb # [ 0.272662] Yama: becoming mindful.781client # [ 0.277562] ... generic counters: 6782worker2 # [ 0.278110] LSM support for eBPF active783lb # [ 0.273825] LSM support for eBPF active784client # [ 0.278238] ... generic bitmap: 000000000000003f785worker2 # [ 0.279012] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)786client # [ 0.278572] ... fixed-purpose counters: 0787lb # [ 0.274573] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)788client # [ 0.279273] ... fixed-purpose bitmap: 0000000000000000789worker2 # [ 0.279934] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)790lb # [ 0.275701] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)791client # [ 0.279562] ... value mask: 0000ffffffffffff792client # [ 0.280456] ... max period: 00007fffffffffff793client # [ 0.281264] ... global_ctrl mask: 000000000000003f794worker2 # [ 0.282659] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)795lb # [ 0.278104] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)796client # [ 0.281673] signal: max sigframe size: 3376797client # [ 0.282453] rcu: Hierarchical SRCU implementation.798worker2 # [ 0.283790] Performance Events: Fam17h+ core perfctr, AMD PMU driver.799lb # [ 0.279294] Performance Events: Fam17h+ core perfctr, AMD PMU driver.800client # [ 0.283149] rcu: Max phase no-delay instances is 400.801worker2 # [ 0.284667] ... version: 2802lb # [ 0.279666] ... version: 2803worker2 # [ 0.284893] ... bit width: 48804lb # [ 0.280375] ... bit width: 48805worker2 # [ 0.285603] ... generic counters: 6806lb # [ 0.280664] ... generic counters: 6807worker2 # [ 0.285938] ... generic bitmap: 000000000000003f808lb # [ 0.281356] ... generic bitmap: 000000000000003f809worker2 # [ 0.286860] ... fixed-purpose counters: 0810lb # [ 0.281670] ... fixed-purpose counters: 0811worker2 # [ 0.287443] ... fixed-purpose bitmap: 0000000000000000812lb # [ 0.282386] ... fixed-purpose bitmap: 0000000000000000813worker2 # [ 0.287892] ... value mask: 0000ffffffffffff814client # [ 0.288120] smp: Bringing up secondary CPUs ...815lb # [ 0.282664] ... value mask: 0000ffffffffffff816worker2 # [ 0.288768] ... max period: 00007fffffffffff817client # [ 0.288575] smp: Brought up 1 node, 1 CPU818lb # [ 0.283533] ... max period: 00007fffffffffff819worker2 # [ 0.289538] ... global_ctrl mask: 000000000000003f820client # [ 0.289218] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)821lb # [ 0.284317] ... global_ctrl mask: 000000000000003f822worker2 # [ 0.290006] signal: max sigframe size: 3376823lb # [ 0.284777] signal: max sigframe size: 3376824worker2 # [ 0.290819] rcu: Hierarchical SRCU implementation.825lb # [ 0.285561] rcu: Hierarchical SRCU implementation.826client # [ 0.289723] Memory: 1454548K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)827worker2 # [ 0.291480] rcu: Max phase no-delay instances is 400.828lb # [ 0.286260] rcu: Max phase no-delay instances is 400.829client # [ 0.290803] devtmpfs: initialized830client # [ 0.291621] x86/mm: Memory block size: 128MB831client # [ 0.293427] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)832client # [ 0.294528] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).833worker2 # [ 0.296410] smp: Bringing up secondary CPUs ...834lb # [ 0.291297] smp: Bringing up secondary CPUs ...835worker2 # [ 0.296905] smp: Brought up 1 node, 1 CPU836client # [ 0.295646] pinctrl core: initialized pinctrl subsystem837lb # [ 0.291678] smp: Brought up 1 node, 1 CPU838worker2 # [ 0.297582] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)839client # [ 0.296683] PM: RTC time: 15:07:14, date: 2026-09-21840lb # [ 0.292359] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)841worker2 # [ 0.298053] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)842lb # [ 0.292769] Memory: 1454584K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)843worker2 # [ 0.299165] devtmpfs: initialized844lb # [ 0.293940] devtmpfs: initialized845client # [ 0.300213] NET: Registered PF_NETLINK/PF_ROUTE protocol family846worker2 # [ 0.300036] x86/mm: Memory block size: 128MB847lb # [ 0.294759] x86/mm: Memory block size: 128MB848client # [ 0.301078] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations849worker2 # [ 0.301843] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)850lb # [ 0.296560] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)851client # [ 0.301585] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations852worker2 # [ 0.302805] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).853lb # [ 0.297563] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).854client # [ 0.302846] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations855lb # [ 0.298746] pinctrl core: initialized pinctrl subsystem856worker2 # [ 0.303978] pinctrl core: initialized pinctrl subsystem857client # [ 0.303571] audit: initializing netlink subsys (disabled)858worker1 # [ 0.275077] Freeing SMP alternatives memory: 44K859worker2 # [ 0.305105] PM: RTC time: 15:07:14, date: 2026-09-21860lb # [ 0.299899] PM: RTC time: 15:07:14, date: 2026-09-21861worker1 # [ 0.275587] pid_max: default: 32768 minimum: 301862client # [ 0.304751] thermal_sys: Registered thermal governor 'fair_share'863worker1 # [ 0.276716] LSM: initializing lsm=capability,landlock,yama,bpf,ima864client # [ 0.304754] thermal_sys: Registered thermal governor 'bang_bang'865worker1 # [ 0.277690] landlock: Up and running.866client # [ 0.305563] thermal_sys: Registered thermal governor 'step_wise'867worker1 # [ 0.278584] Yama: becoming mindful.868client # [ 0.306512] thermal_sys: Registered thermal governor 'user_space'869worker1 # [ 0.279411] LSM support for eBPF active870worker2 # [ 0.308751] NET: Registered PF_NETLINK/PF_ROUTE protocol family871lb # [ 0.303546] NET: Registered PF_NETLINK/PF_ROUTE protocol family872client # [ 0.307328] audit: type=2000 audit(1790003234.702:1): state=initialized audit_enabled=0 res=1873worker1 # [ 0.280153] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)874worker2 # [ 0.310083] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations875lb # [ 0.304900] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations876client # [ 0.308565] thermal_sys: Registered thermal governor 'power_allocator'877worker1 # [ 0.280629] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)878client # [ 0.308578] cpuidle: using governor menu879worker2 # [ 0.310917] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations880lb # [ 0.305705] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations881client # [ 0.310573] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5882worker2 # [ 0.312183] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations883lb # [ 0.306952] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations884worker1 # [ 0.283761] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)885worker2 # [ 0.312903] audit: initializing netlink subsys (disabled)886lb # [ 0.307674] audit: initializing netlink subsys (disabled)887client # [ 0.311768] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]888lb # [ 0.308791] thermal_sys: Registered thermal governor 'fair_share'889worker1 # [ 0.285199] Performance Events: Fam17h+ core perfctr, AMD PMU driver.890client # [ 0.312565] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry891worker2 # [ 0.313895] audit: type=2000 audit(1790003234.711:1): state=initialized audit_enabled=0 res=1892worker1 # [ 0.285593] ... version: 2893lb # [ 0.308794] thermal_sys: Registered thermal governor 'bang_bang'894client # [ 0.313566] PCI: Using configuration type 1 for base access895worker2 # [ 0.315071] thermal_sys: Registered thermal governor 'fair_share'896worker1 # [ 0.286275] ... bit width: 48897lb # [ 0.309665] thermal_sys: Registered thermal governor 'step_wise'898worker1 # [ 0.286586] ... generic counters: 6899worker2 # [ 0.315073] thermal_sys: Registered thermal governor 'bang_bang'900client # [ 0.314618] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.901lb # [ 0.310584] thermal_sys: Registered thermal governor 'user_space'902worker1 # [ 0.287287] ... generic bitmap: 000000000000003f903worker2 # [ 0.315893] thermal_sys: Registered thermal governor 'step_wise'904worker1 # [ 0.287586] ... fixed-purpose counters: 0905lb # [ 0.311379] audit: type=2000 audit(1790003234.712:1): state=initialized audit_enabled=0 res=1906worker2 # [ 0.316810] thermal_sys: Registered thermal governor 'user_space'907worker1 # [ 0.288274] ... fixed-purpose bitmap: 0000000000000000908lb # [ 0.312666] thermal_sys: Registered thermal governor 'power_allocator'909worker2 # [ 0.317620] thermal_sys: Registered thermal governor 'power_allocator'910worker1 # [ 0.288586] ... value mask: 0000ffffffffffff911lb # [ 0.312680] cpuidle: using governor menu912worker2 # [ 0.317907] cpuidle: using governor menu913worker1 # [ 0.289458] ... max period: 00007fffffffffff914lb # [ 0.314888] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5915worker1 # [ 0.290284] ... global_ctrl mask: 000000000000003f916worker2 # [ 0.320136] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5917worker1 # [ 0.290699] signal: max sigframe size: 3376918lb # [ 0.315887] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]919worker1 # [ 0.291522] rcu: Hierarchical SRCU implementation.920worker2 # [ 0.321131] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]921client # [ 0.321802] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages922worker1 # [ 0.292172] rcu: Max phase no-delay instances is 400.923lb # [ 0.316667] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry924worker2 # [ 0.321895] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry925client # [ 0.322564] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page926lb # [ 0.317676] PCI: Using configuration type 1 for base access927worker2 # [ 0.322905] PCI: Using configuration type 1 for base access928client # [ 0.323549] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages929lb # [ 0.318772] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.930worker2 # [ 0.324005] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.931worker1 # [ 0.297177] smp: Bringing up secondary CPUs ...932worker1 # [ 0.297601] smp: Brought up 1 node, 1 CPU933worker1 # [ 0.298322] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)934worker2 # [ 0.329179] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages935client # [ 0.330562] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page936worker2 # [ 0.329894] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page937worker1 # [ 0.298750] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)938lb # [ 0.325818] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages939worker1 # [ 0.299914] devtmpfs: initialized940lb # [ 0.326665] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page941worker1 # [ 0.300748] x86/mm: Memory block size: 128MB942worker1 # [ 0.302518] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)943worker2 # [ 0.334893] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages944client # [ 0.336941] ACPI: Added _OSI(Module Device)945worker2 # [ 0.335892] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page946worker1 # [ 0.303561] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).947client # [ 0.337564] ACPI: Added _OSI(Processor Device)948lb # [ 0.331665] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages949client # [ 0.338278] ACPI: Added _OSI(Processor Aggregator Device)950lb # [ 0.332664] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page951worker1 # [ 0.304965] pinctrl core: initialized pinctrl subsystem952worker1 # [ 0.305911] PM: RTC time: 15:07:14, date: 2026-09-21953worker1 # [ 0.309735] NET: Registered PF_NETLINK/PF_ROUTE protocol family954worker1 # [ 0.311143] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations955worker2 # [ 0.346295] ACPI: Added _OSI(Module Device)956client # [ 0.346759] ACPI: 1 ACPI AML tables successfully acquired and loaded957worker1 # [ 0.311612] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations958worker2 # [ 0.346894] ACPI: Added _OSI(Processor Device)959lb # [ 0.343061] ACPI: Added _OSI(Module Device)960worker2 # [ 0.347607] ACPI: Added _OSI(Processor Aggregator Device)961client # [ 0.348473] ACPI: Interpreter enabled962lb # [ 0.343665] ACPI: Added _OSI(Processor Device)963worker1 # [ 0.312886] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations964client # [ 0.348997] ACPI: PM: (supports S0 S3 S4 S5)965lb # [ 0.344394] ACPI: Added _OSI(Processor Aggregator Device)966worker1 # [ 0.313597] audit: initializing netlink subsys (disabled)967worker1 # [ 0.314775] thermal_sys: Registered thermal governor 'fair_share'968worker1 # [ 0.314777] thermal_sys: Registered thermal governor 'bang_bang'969client # [ 0.351563] ACPI: Using IOAPIC for interrupt routing970worker1 # [ 0.315587] thermal_sys: Registered thermal governor 'step_wise'971client # [ 0.352477] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug972worker1 # [ 0.316591] audit: type=2000 audit(1790003234.748:1): state=initialized audit_enabled=0 res=1973worker1 # [ 0.318589] thermal_sys: Registered thermal governor 'user_space'974worker1 # [ 0.318591] thermal_sys: Registered thermal governor 'power_allocator'975client # [ 0.355563] PCI: Using E820 reservations for host bridge windows976lb # [ 0.351215] ACPI: 1 ACPI AML tables successfully acquired and loaded977worker1 # [ 0.319610] cpuidle: using governor menu978client # [ 0.356636] ACPI: Enabled 2 GPEs in block 00 to 3F979worker2 # [ 0.356489] ACPI: 1 ACPI AML tables successfully acquired and loaded980worker1 # [ 0.321870] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5981worker1 # [ 0.322837] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]982lb # [ 0.355068] ACPI: Interpreter enabled983worker1 # [ 0.323590] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry984worker2 # [ 0.360302] ACPI: Interpreter enabled985lb # [ 0.355680] ACPI: PM: (supports S0 S3 S4 S5)986worker1 # [ 0.324598] PCI: Using configuration type 1 for base access987worker2 # [ 0.360911] ACPI: PM: (supports S0 S3 S4 S5)988lb # [ 0.356368] ACPI: Using IOAPIC for interrupt routing989worker2 # [ 0.361624] ACPI: Using IOAPIC for interrupt routing990worker1 # [ 0.325723] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.991worker2 # [ 0.361961] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug992worker2 # [ 0.362893] PCI: Using E820 reservations for host bridge windows993lb # [ 0.358713] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug994client # [ 0.365276] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])995lb # [ 0.359664] PCI: Using E820 reservations for host bridge windows996client # [ 0.366475] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]997worker2 # [ 0.366003] ACPI: Enabled 2 GPEs in block 00 to 3F998worker1 # [ 0.330871] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages999client # [ 0.367648] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1000lb # [ 0.362785] ACPI: Enabled 2 GPEs in block 00 to 3F1001worker1 # [ 0.331587] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1002client # [ 0.368684] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1003client # [ 0.369952] PCI host bridge to bus 0000:001004client # [ 0.370567] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1005client # [ 0.371562] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1006client # [ 0.372562] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1007worker1 # [ 0.336587] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1008worker1 # [ 0.337586] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1009client # [ 0.373562] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1010lb # [ 0.369953] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1011worker2 # [ 0.374729] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1012client # [ 0.374562] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1013lb # [ 0.370670] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1014client # [ 0.375562] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1015worker2 # [ 0.375652] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1016lb # [ 0.371741] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1017client # [ 0.376562] pci_bus 0000:00: root bus resource [bus 00-ff]1018worker2 # [ 0.377398] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1019lb # [ 0.372787] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1020client # [ 0.377513] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1021worker2 # [ 0.378016] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1022lb # [ 0.374048] PCI host bridge to bus 0000:001023client # [ 0.378970] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1024worker2 # [ 0.379234] PCI host bridge to bus 0000:001025lb # [ 0.374669] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1026worker2 # [ 0.379897] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1027lb # [ 0.375664] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1028worker2 # [ 0.380893] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1029lb # [ 0.376664] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1030worker2 # [ 0.381893] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1031worker1 # [ 0.347971] ACPI: Added _OSI(Module Device)1032lb # [ 0.377664] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1033worker1 # [ 0.348588] ACPI: Added _OSI(Processor Device)1034worker2 # [ 0.382893] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1035lb # [ 0.378664] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1036worker1 # [ 0.349349] ACPI: Added _OSI(Processor Aggregator Device)1037worker2 # [ 0.383893] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1038lb # [ 0.379664] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1039lb # [ 0.380669] pci_bus 0000:00: root bus resource [bus 00-ff]1040worker2 # [ 0.384893] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1041worker2 # [ 0.385893] pci_bus 0000:00: root bus resource [bus 00-ff]1042lb # [ 0.381658] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1043client # [ 0.381679] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1044client # [ 0.382575] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1045worker2 # [ 0.386921] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1046lb # [ 0.383161] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1047client # [ 0.383527] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1048worker2 # [ 0.388338] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1049client # [ 0.384487] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1050client # [ 0.386329] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1051worker1 # [ 0.358145] ACPI: 1 ACPI AML tables successfully acquired and loaded1052client # [ 0.389578] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1053client # [ 0.390454] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1054worker1 # [ 0.362016] ACPI: Interpreter enabled1055lb # [ 0.385747] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1056client # [ 0.391349] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1057worker1 # [ 0.362602] ACPI: PM: (supports S0 S3 S4 S5)1058lb # [ 0.386677] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1059client # [ 0.392379] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1060worker1 # [ 0.363323] ACPI: Using IOAPIC for interrupt routing1061worker2 # [ 0.390978] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1062lb # [ 0.387610] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1063worker2 # [ 0.391974] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1064worker1 # [ 0.363634] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1065client # [ 0.394287] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1066lb # [ 0.388836] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1067worker1 # [ 0.364587] PCI: Using E820 reservations for host bridge windows1068worker2 # [ 0.392960] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1069lb # [ 0.390389] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1070worker2 # [ 0.394218] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1071client # [ 0.396578] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1072worker1 # [ 0.367686] ACPI: Enabled 2 GPEs in block 00 to 3F1073client # [ 0.397422] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1074worker2 # [ 0.395621] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1075client # [ 0.398297] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1076lb # [ 0.393696] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1077lb # [ 0.394542] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1078client # [ 0.399105] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1079lb # [ 0.395395] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1080worker2 # [ 0.398923] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1081lb # [ 0.395670] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1082worker2 # [ 0.399811] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1083worker2 # [ 0.400680] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1084client # [ 0.401716] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1085lb # [ 0.397794] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1086worker2 # [ 0.401818] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1087client # [ 0.402569] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1088worker1 # [ 0.376625] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1089client # [ 0.403522] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1090worker2 # [ 0.403242] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1091lb # [ 0.400640] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1092worker1 # [ 0.378124] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1093client # [ 0.404941] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1094lb # [ 0.401278] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1095worker1 # [ 0.378665] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1096lb # [ 0.401686] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1097worker2 # [ 0.404923] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1098worker1 # [ 0.379708] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1099worker2 # [ 0.405781] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1100client # [ 0.406570] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1101lb # [ 0.403224] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1102client # [ 0.407371] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1103worker1 # [ 0.380929] PCI host bridge to bus 0000:001104worker2 # [ 0.406604] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1105worker1 # [ 0.381591] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1106client # [ 0.408284] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1107worker2 # [ 0.407504] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1108worker1 # [ 0.382587] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1109client # [ 0.409122] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1110worker1 # [ 0.383586] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1111lb # [ 0.406664] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1112worker1 # [ 0.384586] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1113lb # [ 0.407542] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1114client # [ 0.410578] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1115worker1 # [ 0.385586] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1116worker2 # [ 0.409900] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1117lb # [ 0.408400] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1118client # [ 0.411396] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1119worker2 # [ 0.410724] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1120worker1 # [ 0.386586] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1121client # [ 0.412294] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1122worker2 # [ 0.411623] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1123worker1 # [ 0.387587] pci_bus 0000:00: root bus resource [bus 00-ff]1124lb # [ 0.409209] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125worker1 # [ 0.388530] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1126client # [ 0.413139] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1127worker2 # [ 0.412504] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1128lb # [ 0.410696] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1129worker1 # [ 0.390077] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1130lb # [ 0.411543] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1131lb # [ 0.412399] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1132worker2 # [ 0.414861] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1133client # [ 0.415572] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1134worker2 # [ 0.415542] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1135client # [ 0.416569] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1136client # [ 0.417521] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1137lb # [ 0.413204] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1138worker2 # [ 0.415914] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1139client # [ 0.418972] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1140lb # [ 0.415671] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1141worker2 # [ 0.417433] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1142lb # [ 0.416520] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1143client # [ 0.420570] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1144lb # [ 0.417397] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1145worker2 # [ 0.418922] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1146client # [ 0.421443] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1147worker1 # [ 0.392651] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1148worker2 # [ 0.419764] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1149client # [ 0.422290] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1150lb # [ 0.418225] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1151worker1 # [ 0.393600] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1152worker2 # [ 0.420611] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1153client # [ 0.422568] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1154worker1 # [ 0.394570] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1155worker2 # [ 0.421508] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1156worker1 # [ 0.395924] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1157client # [ 0.424071] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1158lb # [ 0.420699] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1159lb # [ 0.421671] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1160worker1 # [ 0.397325] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1161client # [ 0.426539] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1162lb # [ 0.422615] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1163client # [ 0.427297] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1164worker2 # [ 0.423901] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1165lb # [ 0.424111] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1166worker2 # [ 0.425579] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1167client # [ 0.428102] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1168worker1 # [ 0.399617] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1169worker2 # [ 0.425915] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1170worker1 # [ 0.400490] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1171lb # [ 0.425672] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1172worker1 # [ 0.401381] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1173client # [ 0.429580] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1174worker2 # [ 0.427500] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1175lb # [ 0.426515] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1176client # [ 0.430383] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1177worker1 # [ 0.402453] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1178lb # [ 0.427376] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1179lb # [ 0.427670] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1180client # [ 0.431284] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1181worker2 # [ 0.428900] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1182worker1 # [ 0.404152] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1183worker2 # [ 0.429793] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1184client # [ 0.432108] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1185worker2 # [ 0.430651] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1186lb # [ 0.429675] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1187worker1 # [ 0.405619] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1188worker2 # [ 0.430899] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1189client # [ 0.433570] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1190worker1 # [ 0.406562] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1191lb # [ 0.431702] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1192client # [ 0.434413] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1193worker1 # [ 0.407309] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1194lb # [ 0.432644] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1195worker2 # [ 0.432476] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1196client # [ 0.435298] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1197lb # [ 0.434006] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1198worker1 # [ 0.408181] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1199worker2 # [ 0.433906] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1200client # [ 0.436129] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1201worker2 # [ 0.434885] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1202client # [ 0.437202] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1203lb # [ 0.436462] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1204lb # [ 0.437277] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1205worker2 # [ 0.436455] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1206worker1 # [ 0.410594] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1207client # [ 0.437765] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1208worker1 # [ 0.411459] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1209lb # [ 0.437685] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1210client # [ 0.440233] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1211worker1 # [ 0.412390] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1212worker2 # [ 0.438643] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1213lb # [ 0.440030] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1214worker1 # [ 0.414038] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1215worker2 # [ 0.438900] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1216worker2 # [ 0.439915] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1217lb # [ 0.441671] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1218lb # [ 0.442494] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1219worker1 # [ 0.416430] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1220worker2 # [ 0.441445] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1221lb # [ 0.443395] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1222worker1 # [ 0.417263] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1223lb # [ 0.444251] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1224worker1 # [ 0.417608] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1225worker2 # [ 0.442900] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1226worker2 # [ 0.443756] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1227worker1 # [ 0.419145] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1228lb # [ 0.445303] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1229worker2 # [ 0.444698] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1230lb # [ 0.445869] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1231worker1 # [ 0.420593] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1232client # [ 0.440753] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1233worker2 # [ 0.446384] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1234lb # [ 0.447347] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1235worker1 # [ 0.421478] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1236client # [ 0.443164] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1237worker2 # [ 0.447596] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1238worker1 # [ 0.422682] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1239client # [ 0.443815] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1240worker2 # [ 0.448098] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1241client # [ 0.445144] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1242worker2 # [ 0.449566] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1243worker1 # [ 0.424385] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1244client # [ 0.445816] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1245client # [ 0.446860] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1246worker1 # [ 0.427502] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1247worker1 # [ 0.428284] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1248client # [ 0.447810] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1249worker1 # [ 0.428609] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1250client # [ 0.449590] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1251worker1 # [ 0.430229] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1252client # [ 0.450482] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1253lb # [ 0.447858] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1254client # [ 0.451615] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1255lb # [ 0.450365] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1256client # [ 0.453202] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1257lb # [ 0.450904] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1258worker1 # [ 0.432618] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1259worker2 # [ 0.450084] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1260worker1 # [ 0.433594] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1261lb # [ 0.452283] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1262worker1 # [ 0.434572] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1263worker2 # [ 0.452566] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1264worker1 # [ 0.435485] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1265lb # [ 0.452940] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1266client # [ 0.457002] ACPI: PCI: Interrupt link LNKA configured for IRQ 101267worker2 # [ 0.453171] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1268client # [ 0.457669] ACPI: PCI: Interrupt link LNKB configured for IRQ 101269lb # [ 0.453951] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1270worker1 # [ 0.436908] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1271client # [ 0.458661] ACPI: PCI: Interrupt link LNKC configured for IRQ 111272worker2 # [ 0.454539] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1273lb # [ 0.454911] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1274client # [ 0.459661] ACPI: PCI: Interrupt link LNKD configured for IRQ 111275worker1 # [ 0.438623] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1276worker2 # [ 0.455162] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1277client # [ 0.460660] ACPI: PCI: Interrupt link LNKE configured for IRQ 101278worker1 # [ 0.439610] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1279lb # [ 0.456670] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1280client # [ 0.461657] ACPI: PCI: Interrupt link LNKF configured for IRQ 101281worker2 # [ 0.456213] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1282lb # [ 0.457508] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1283client # [ 0.462670] ACPI: PCI: Interrupt link LNKG configured for IRQ 111284lb # [ 0.458677] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1285worker2 # [ 0.457151] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1286worker1 # [ 0.441129] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1287client # [ 0.463665] ACPI: PCI: Interrupt link LNKH configured for IRQ 111288client # [ 0.464605] ACPI: PCI: Interrupt link GSIA configured for IRQ 161289worker2 # [ 0.458911] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1290lb # [ 0.460259] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1291client # [ 0.465584] ACPI: PCI: Interrupt link GSIB configured for IRQ 171292worker1 # [ 0.442639] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1293worker2 # [ 0.459761] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1294client # [ 0.466542] ACPI: PCI: Interrupt link GSIC configured for IRQ 181295worker1 # [ 0.443525] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1296client # [ 0.467310] ACPI: PCI: Interrupt link GSID configured for IRQ 191297worker1 # [ 0.444370] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1298worker2 # [ 0.461461] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1299client # [ 0.467574] ACPI: PCI: Interrupt link GSIE configured for IRQ 201300lb # [ 0.463643] ACPI: PCI: Interrupt link LNKA configured for IRQ 101301client # [ 0.468505] ACPI: PCI: Interrupt link GSIF configured for IRQ 211302worker2 # [ 0.462580] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1303worker1 # [ 0.446017] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1304lb # [ 0.464493] ACPI: PCI: Interrupt link LNKB configured for IRQ 101305client # [ 0.469294] ACPI: PCI: Interrupt link GSIG configured for IRQ 221306client # [ 0.469575] ACPI: PCI: Interrupt link GSIH configured for IRQ 231307lb # [ 0.465485] ACPI: PCI: Interrupt link LNKC configured for IRQ 111308lb # [ 0.466497] ACPI: PCI: Interrupt link LNKD configured for IRQ 111309worker1 # [ 0.448520] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1310client # [ 0.471571] iommu: Default domain type: Translated1311lb # [ 0.467484] ACPI: PCI: Interrupt link LNKE configured for IRQ 101312worker2 # [ 0.465949] ACPI: PCI: Interrupt link LNKA configured for IRQ 101313client # [ 0.472348] iommu: DMA domain TLB invalidation policy: lazy mode1314lb # [ 0.468513] ACPI: PCI: Interrupt link LNKF configured for IRQ 101315worker1 # [ 0.449586] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1316client # [ 0.473523] ACPI: bus type USB registered1317worker2 # [ 0.467003] ACPI: PCI: Interrupt link LNKB configured for IRQ 101318worker1 # [ 0.450579] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1319lb # [ 0.469467] ACPI: PCI: Interrupt link LNKG configured for IRQ 111320client # [ 0.474106] usbcore: registered new interface driver usbfs1321worker2 # [ 0.467995] ACPI: PCI: Interrupt link LNKC configured for IRQ 111322client # [ 0.474577] usbcore: registered new interface driver hub1323lb # [ 0.470722] ACPI: PCI: Interrupt link LNKH configured for IRQ 111324worker2 # [ 0.469001] ACPI: PCI: Interrupt link LNKD configured for IRQ 111325client # [ 0.475449] usbcore: registered new device driver usb1326worker1 # [ 0.452161] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1327lb # [ 0.471705] ACPI: PCI: Interrupt link GSIA configured for IRQ 161328worker2 # [ 0.470003] ACPI: PCI: Interrupt link LNKE configured for IRQ 101329lb # [ 0.472650] ACPI: PCI: Interrupt link GSIB configured for IRQ 171330worker1 # [ 0.453323] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1331client # [ 0.477182] NetLabel: Initializing1332worker2 # [ 0.471005] ACPI: PCI: Interrupt link LNKF configured for IRQ 101333lb # [ 0.473423] ACPI: PCI: Interrupt link GSIC configured for IRQ 181334client # [ 0.477563] NetLabel: domain hash size = 1281335worker2 # [ 0.471995] ACPI: PCI: Interrupt link LNKG configured for IRQ 111336lb # [ 0.473679] ACPI: PCI: Interrupt link GSID configured for IRQ 191337client # [ 0.478273] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1338worker1 # [ 0.453807] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1339lb # [ 0.474628] ACPI: PCI: Interrupt link GSIE configured for IRQ 201340worker2 # [ 0.472991] ACPI: PCI: Interrupt link LNKH configured for IRQ 111341client # [ 0.478598] NetLabel: unlabeled traffic allowed by default1342worker1 # [ 0.455252] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1343client # [ 0.479478] PCI: Using ACPI for IRQ routing1344lb # [ 0.475385] ACPI: PCI: Interrupt link GSIF configured for IRQ 211345worker2 # [ 0.473931] ACPI: PCI: Interrupt link GSIA configured for IRQ 161346lb # [ 0.475677] ACPI: PCI: Interrupt link GSIG configured for IRQ 221347worker2 # [ 0.474905] ACPI: PCI: Interrupt link GSIB configured for IRQ 171348lb # [ 0.476625] ACPI: PCI: Interrupt link GSIH configured for IRQ 231349worker2 # [ 0.475908] ACPI: PCI: Interrupt link GSIC configured for IRQ 181350worker2 # [ 0.476908] ACPI: PCI: Interrupt link GSID configured for IRQ 191351lb # [ 0.478540] iommu: Default domain type: Translated1352worker2 # [ 0.477908] ACPI: PCI: Interrupt link GSIE configured for IRQ 201353lb # [ 0.479259] iommu: DMA domain TLB invalidation policy: lazy mode1354worker2 # [ 0.478905] ACPI: PCI: Interrupt link GSIF configured for IRQ 211355lb # [ 0.479937] ACPI: bus type USB registered1356worker2 # [ 0.479907] ACPI: PCI: Interrupt link GSIG configured for IRQ 221357lb # [ 0.480678] usbcore: registered new interface driver usbfs1358lb # [ 0.481562] usbcore: registered new interface driver hub1359worker2 # [ 0.480905] ACPI: PCI: Interrupt link GSIH configured for IRQ 231360lb # [ 0.482301] usbcore: registered new device driver usb1361worker2 # [ 0.482945] iommu: Default domain type: Translated1362lb # [ 0.483553] NetLabel: Initializing1363worker2 # [ 0.483803] iommu: DMA domain TLB invalidation policy: lazy mode1364lb # [ 0.484094] NetLabel: domain hash size = 1281365worker2 # [ 0.484913] ACPI: bus type USB registered1366lb # [ 0.484680] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1367lb # [ 0.485613] NetLabel: unlabeled traffic allowed by default1368worker2 # [ 0.485710] usbcore: registered new interface driver usbfs1369lb # [ 0.486458] PCI: Using ACPI for IRQ routing1370worker2 # [ 0.486575] usbcore: registered new interface driver hub1371worker1 # [ 0.455783] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1372worker2 # [ 0.486903] usbcore: registered new device driver usb1373worker2 # [ 0.488707] NetLabel: Initializing1374worker1 # [ 0.459260] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1375worker2 # [ 0.489377] NetLabel: domain hash size = 1281376worker2 # [ 0.490454] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1377worker1 # [ 0.459864] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1378worker2 # [ 0.490973] NetLabel: unlabeled traffic allowed by default1379worker2 # [ 0.491904] PCI: Using ACPI for IRQ routing1380worker1 # [ 0.461617] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1381worker1 # [ 0.462863] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1382worker1 # [ 0.463920] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1383worker1 # [ 0.464886] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1384worker1 # [ 0.466643] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1385worker1 # [ 0.467552] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1386worker1 # [ 0.468750] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1387worker1 # [ 0.470616] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1388worker1 # [ 0.475867] ACPI: PCI: Interrupt link LNKA configured for IRQ 101389worker1 # [ 0.476697] ACPI: PCI: Interrupt link LNKB configured for IRQ 101390worker1 # [ 0.477697] ACPI: PCI: Interrupt link LNKC configured for IRQ 111391client # [ 0.523383] pci 0000:00:01.0: vgaarb: setting as boot VGA device1392worker1 # [ 0.478701] ACPI: PCI: Interrupt link LNKD configured for IRQ 111393client # [ 0.523558] pci 0000:00:01.0: vgaarb: bridge control possible1394worker1 # [ 0.479686] ACPI: PCI: Interrupt link LNKE configured for IRQ 101395worker1 # [ 0.480685] ACPI: PCI: Interrupt link LNKF configured for IRQ 101396client # [ 0.523558] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1397client # [ 0.523566] vgaarb: loaded1398worker1 # [ 0.481691] ACPI: PCI: Interrupt link LNKG configured for IRQ 111399client # [ 0.524319] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01400client # [ 0.524562] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1401worker1 # [ 0.482690] ACPI: PCI: Interrupt link LNKH configured for IRQ 111402worker1 # [ 0.483633] ACPI: PCI: Interrupt link GSIA configured for IRQ 161403worker1 # [ 0.484604] ACPI: PCI: Interrupt link GSIB configured for IRQ 171404worker1 # [ 0.485599] ACPI: PCI: Interrupt link GSIC configured for IRQ 181405worker1 # [ 0.486602] ACPI: PCI: Interrupt link GSID configured for IRQ 191406client # [ 0.528643] clocksource: Switched to clocksource kvm-clock1407worker1 # [ 0.487602] ACPI: PCI: Interrupt link GSIE configured for IRQ 201408worker1 # [ 0.488598] ACPI: PCI: Interrupt link GSIF configured for IRQ 211409worker1 # [ 0.489598] ACPI: PCI: Interrupt link GSIG configured for IRQ 221410client # [ 0.532114] VFS: Disk quotas dquot_6.6.01411lb # [ 0.529990] pci 0000:00:01.0: vgaarb: setting as boot VGA device1412worker1 # [ 0.490598] ACPI: PCI: Interrupt link GSIH configured for IRQ 231413lb # [ 0.530660] pci 0000:00:01.0: vgaarb: bridge control possible1414client # [ 0.532809] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1415client # [ 0.534127] pnp: PnP ACPI init1416worker1 # [ 0.492720] iommu: Default domain type: Translated1417lb # [ 0.530660] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418lb # [ 0.530669] vgaarb: loaded1419client # [ 0.534925] ACPI: IRQ 4 override to edge(!), high(!)1420worker1 # [ 0.493542] iommu: DMA domain TLB invalidation policy: lazy mode1421lb # [ 0.531380] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01422worker1 # [ 0.494665] ACPI: bus type USB registered1423client # [ 0.535888] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1424lb # [ 0.531665] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1425worker1 # [ 0.495424] usbcore: registered new interface driver usbfs1426client # [ 0.537398] pnp: PnP ACPI: found 6 devices1427worker1 # [ 0.496338] usbcore: registered new interface driver hub1428worker1 # [ 0.496598] usbcore: registered new device driver usb1429worker1 # [ 0.498386] NetLabel: Initializing1430lb # [ 0.534788] clocksource: Switched to clocksource kvm-clock1431worker2 # [ 0.537504] pci 0000:00:01.0: vgaarb: setting as boot VGA device1432worker1 # [ 0.498592] NetLabel: domain hash size = 1281433worker2 # [ 0.537889] pci 0000:00:01.0: vgaarb: bridge control possible1434worker1 # [ 0.499335] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1435worker2 # [ 0.537889] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1436worker1 # [ 0.499651] NetLabel: unlabeled traffic allowed by default1437worker2 # [ 0.537898] vgaarb: loaded1438worker1 # [ 0.500575] PCI: Using ACPI for IRQ routing1439lb # [ 0.538274] VFS: Disk quotas dquot_6.6.01440worker2 # [ 0.538733] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01441lb # [ 0.539029] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1442client # [ 0.544977] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1443worker2 # [ 0.539494] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1444lb # [ 0.540335] pnp: PnP ACPI init1445client # [ 0.546383] clocksource: Switched to clocksource acpi_pm1446lb # [ 0.541121] ACPI: IRQ 4 override to edge(!), high(!)1447client # [ 0.547375] NET: Registered PF_INET protocol family1448lb # [ 0.542093] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1449client # [ 0.548546] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1450lb # [ 0.543629] pnp: PnP ACPI: found 6 devices1451worker2 # [ 0.542987] clocksource: Switched to clocksource kvm-clock1452worker2 # [ 0.546954] VFS: Disk quotas dquot_6.6.01453worker2 # [ 0.547697] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1454worker2 # [ 0.549161] pnp: PnP ACPI init1455worker2 # [ 0.549996] ACPI: IRQ 4 override to edge(!), high(!)1456lb # [ 0.551227] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1457worker2 # [ 0.551008] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1458lb # [ 0.552639] clocksource: Switched to clocksource acpi_pm1459worker2 # [ 0.552577] pnp: PnP ACPI: found 6 devices1460lb # [ 0.553560] NET: Registered PF_INET protocol family1461lb # [ 0.554770] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1462client # [ 0.564939] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1463client # [ 0.566418] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1464worker2 # [ 0.560278] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1465client # [ 0.567748] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1466worker2 # [ 0.561801] clocksource: Switched to clocksource acpi_pm1467client # [ 0.569084] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1468worker2 # [ 0.562784] NET: Registered PF_INET protocol family1469client # [ 0.570338] TCP: Hash tables configured (established 16384 bind 16384)1470worker2 # [ 0.564006] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1471client # [ 0.571479] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1472client # [ 0.572726] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1473client # [ 0.573865] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1474client # [ 0.575098] NET: Registered PF_UNIX/PF_LOCAL protocol family1475client # [ 0.576102] NET: Registered PF_XDP protocol family1476client # [ 0.576971] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1477lb # [ 0.571454] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1478client # [ 0.578010] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1479client # [ 0.578984] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1480lb # [ 0.572965] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1481client # [ 0.580074] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1482lb # [ 0.574230] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1483client # [ 0.581179] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1484lb # [ 0.575537] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1485client # [ 0.582282] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1486lb # [ 0.576757] TCP: Hash tables configured (established 16384 bind 16384)1487client # [ 0.584247] ACPI: \_SB_.GSIA: Enabled at IRQ 161488lb # [ 0.577848] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1489lb # [ 0.579066] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1490client # [ 0.586424] ACPI: \_SB_.GSIB: Enabled at IRQ 171491lb # [ 0.580120] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1492lb # [ 0.581327] NET: Registered PF_UNIX/PF_LOCAL protocol family1493lb # [ 0.582264] NET: Registered PF_XDP protocol family1494client # [ 0.588560] ACPI: \_SB_.GSIC: Enabled at IRQ 181495worker2 # [ 0.580941] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1496worker1 # [ 0.545060] pci 0000:00:01.0: vgaarb: setting as boot VGA device1497lb # [ 0.583083] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1498worker2 # [ 0.582514] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1499worker1 # [ 0.545582] pci 0000:00:01.0: vgaarb: bridge control possible1500lb # [ 0.584051] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1501client # [ 0.590526] ACPI: \_SB_.GSID: Enabled at IRQ 191502worker2 # [ 0.583875] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1503lb # [ 0.585025] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1504worker1 # [ 0.545582] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1505worker1 # [ 0.545590] vgaarb: loaded1506client # [ 0.592219] PCI: CLS 0 bytes, default 641507lb # [ 0.586078] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1508worker2 # [ 0.585214] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1509worker1 # [ 0.546327] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01510lb # [ 0.587130] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1511worker2 # [ 0.586479] TCP: Hash tables configured (established 16384 bind 16384)1512client # [ 0.593125] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1513worker1 # [ 0.546588] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1514lb # [ 0.588172] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1515client # [ 0.594744] Trying to unpack rootfs image as initramfs...1516worker2 # [ 0.587563] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1517lb # [ 0.589990] ACPI: \_SB_.GSIA: Enabled at IRQ 161518worker2 # [ 0.588820] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1519worker2 # [ 0.589922] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1520worker1 # [ 0.549673] clocksource: Switched to clocksource kvm-clock1521worker2 # [ 0.591224] NET: Registered PF_UNIX/PF_LOCAL protocol family1522lb # [ 0.592272] ACPI: \_SB_.GSIB: Enabled at IRQ 171523worker2 # [ 0.592235] NET: Registered PF_XDP protocol family1524worker2 # [ 0.593146] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1525lb # [ 0.594740] ACPI: \_SB_.GSIC: Enabled at IRQ 181526worker2 # [ 0.594216] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1527worker1 # [ 0.553399] VFS: Disk quotas dquot_6.6.01528worker2 # [ 0.595314] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1529worker1 # [ 0.554149] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1530lb # [ 0.596732] ACPI: \_SB_.GSID: Enabled at IRQ 191531worker1 # [ 0.555564] pnp: PnP ACPI init1532worker2 # [ 0.596464] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1533lb # [ 0.598431] PCI: CLS 0 bytes, default 641534worker1 # [ 0.556380] ACPI: IRQ 4 override to edge(!), high(!)1535worker2 # [ 0.597610] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1536worker2 # [ 0.598719] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1537worker1 # [ 0.557455] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1538lb # [ 0.599395] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1539worker1 # [ 0.558981] pnp: PnP ACPI: found 6 devices1540lb # [ 0.601098] Trying to unpack rootfs image as initramfs...1541worker2 # [ 0.600442] ACPI: \_SB_.GSIA: Enabled at IRQ 161542worker2 # [ 0.602716] ACPI: \_SB_.GSIB: Enabled at IRQ 171543worker2 # [ 0.605085] ACPI: \_SB_.GSIC: Enabled at IRQ 181544worker2 # [ 0.607124] ACPI: \_SB_.GSID: Enabled at IRQ 191545worker2 # [ 0.608865] PCI: CLS 0 bytes, default 641546worker1 # [ 0.566766] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1547worker1 # [ 0.568272] clocksource: Switched to clocksource acpi_pm1548worker2 # [ 0.609855] Trying to unpack rootfs image as initramfs...1549worker1 # [ 0.569278] NET: Registered PF_INET protocol family1550worker1 # [ 0.570549] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1551worker2 # [ 0.616821] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1552worker1 # [ 0.587092] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1553worker1 # [ 0.588684] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1554client # [ 0.638712] Initialise system trusted keyrings1555worker1 # [ 0.590057] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1556worker1 # [ 0.591410] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1557worker1 # [ 0.592731] TCP: Hash tables configured (established 16384 bind 16384)1558client # [ 0.641835] workingset: timestamp_bits=40 max_order=19 bucket_order=01559worker1 # [ 0.594024] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1560worker1 # [ 0.595380] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1561worker1 # [ 0.596622] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1562worker1 # [ 0.597883] NET: Registered PF_UNIX/PF_LOCAL protocol family1563worker1 # [ 0.598859] NET: Registered PF_XDP protocol family1564worker1 # [ 0.599719] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1565worker1 # [ 0.600782] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1566lb # [ 0.643621] Initialise system trusted keyrings1567worker1 # [ 0.601805] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1568worker1 # [ 0.602891] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1569worker1 # [ 0.604051] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1570lb # [ 0.646743] workingset: timestamp_bits=40 max_order=19 bucket_order=01571worker1 # [ 0.605195] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1572worker1 # [ 0.606986] ACPI: \_SB_.GSIA: Enabled at IRQ 161573worker1 # [ 0.609080] ACPI: \_SB_.GSIB: Enabled at IRQ 171574worker1 # [ 0.611121] ACPI: \_SB_.GSIC: Enabled at IRQ 181575worker1 # [ 0.613527] ACPI: \_SB_.GSID: Enabled at IRQ 191576worker2 # [ 0.655624] Initialise system trusted keyrings1577worker1 # [ 0.615297] PCI: CLS 0 bytes, default 641578client # [ 0.663843] Key type asymmetric registered1579client # [ 0.664567] Asymmetric key parser 'x509' registered1580worker1 # [ 0.616277] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1581client # [ 0.665451] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582worker2 # [ 0.658795] workingset: timestamp_bits=40 max_order=19 bucket_order=01583worker1 # [ 0.618045] Trying to unpack rootfs image as initramfs...1584client # [ 0.669916] io scheduler mq-deadline registered1585client # [ 0.670718] io scheduler kyber registered1586client # [ 0.673468] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1587lb # [ 0.667894] Key type asymmetric registered1588lb # [ 0.668576] Asymmetric key parser 'x509' registered1589client # [ 0.674748] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1590lb # [ 0.669427] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1591lb # [ 0.672849] io scheduler mq-deadline registered1592lb # [ 0.673578] io scheduler kyber registered1593client # [ 0.680210] Linux agpgart interface v0.1031594client # [ 0.681003] ACPI: bus type drm_connector registered1595lb # [ 0.677730] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1596client # [ 0.684227] usbcore: registered new interface driver usbserial_generic1597client # [ 0.685311] usbserial: USB Serial support registered for generic1598lb # [ 0.678970] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1599worker2 # [ 0.679965] Key type asymmetric registered1600worker2 # [ 0.680664] Asymmetric key parser 'x509' registered1601worker2 # [ 0.681597] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1602client # [ 0.688802] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1603client # [ 0.688802] Please enable it if your BIOS has the CPPC option.1604lb # [ 0.684574] Linux agpgart interface v0.1031605client # [ 0.691051] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1606lb # [ 0.685351] ACPI: bus type drm_connector registered1607worker2 # [ 0.686879] io scheduler mq-deadline registered1608client # [ 0.693912] drop_monitor: Initializing network drop monitor service1609worker2 # [ 0.687638] io scheduler kyber registered1610client # [ 0.695073] NET: Registered PF_INET6 protocol family1611lb # [ 0.690772] usbcore: registered new interface driver usbserial_generic1612worker2 # [ 0.690352] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1613lb # [ 0.691798] usbserial: USB Serial support registered for generic1614client # [ 0.698255] Segment Routing with IPv61615worker2 # [ 0.691672] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1616lb # [ 0.692746] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1617lb # [ 0.692746] Please enable it if your BIOS has the CPPC option.1618client # [ 0.700820] In-situ OAM (IOAM) with IPv61619lb # [ 0.694939] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1620client # [ 0.701878] IPI shorthand broadcast: enabled1621worker2 # [ 0.697370] Linux agpgart interface v0.1031622lb # [ 0.698848] drop_monitor: Initializing network drop monitor service1623worker2 # [ 0.698195] ACPI: bus type drm_connector registered1624lb # [ 0.700027] NET: Registered PF_INET6 protocol family1625worker2 # [ 0.701167] usbcore: registered new interface driver usbserial_generic1626lb # [ 0.703218] Segment Routing with IPv61627worker2 # [ 0.702304] usbserial: USB Serial support registered for generic1628client # [ 0.710156] sched_clock: Marking stable (576014696, 133768901)->(783269433, -73485836)1629worker1 # [ 0.662619] Initialise system trusted keyrings1630lb # [ 0.705720] In-situ OAM (IOAM) with IPv61631lb # [ 0.706760] IPI shorthand broadcast: enabled1632worker2 # [ 0.705762] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1633worker1 # [ 0.665544] workingset: timestamp_bits=40 max_order=19 bucket_order=01634worker2 # [ 0.705762] Please enable it if your BIOS has the CPPC option.1635client # [ 0.715025] registered taskstats version 11636worker2 # [ 0.708089] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1637client # [ 0.715990] Loading compiled-in X.509 certificates1638worker2 # [ 0.711869] drop_monitor: Initializing network drop monitor service1639worker2 # [ 0.713107] NET: Registered PF_INET6 protocol family1640lb # [ 0.714893] sched_clock: Marking stable (581013870, 133673941)->(788462587, -73774776)1641worker2 # [ 0.716202] Segment Routing with IPv61642worker2 # [ 0.718774] In-situ OAM (IOAM) with IPv61643lb # [ 0.719787] registered taskstats version 11644worker2 # [ 0.719852] IPI shorthand broadcast: enabled1645lb # [ 0.720739] Loading compiled-in X.509 certificates1646client # [ 0.733802] Demotion targets for Node 0: null1647client # [ 0.735836] Key type .fscrypt registered1648worker2 # [ 0.728167] sched_clock: Marking stable (592013992, 135724361)->(806458927, -78720574)1649client # [ 0.736498] Key type fscrypt-provisioning registered1650worker1 # [ 0.688668] Key type asymmetric registered1651client # [ 0.737477] ima: No TPM chip found, activating TPM-bypass!1652worker1 # [ 0.689399] Asymmetric key parser 'x509' registered1653worker1 # [ 0.690271] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1654worker2 # [ 0.733929] registered taskstats version 11655client # [ 0.741800] ima: Allocated hash algorithm: sha11656worker2 # [ 0.734938] Loading compiled-in X.509 certificates1657worker1 # [ 0.693672] io scheduler mq-deadline registered1658client # [ 0.742579] ima: No architecture policies found1659worker1 # [ 0.694443] io scheduler kyber registered1660client # [ 0.743544] PM: Magic number: 10:253:1341661client # [ 0.744322] pci 0000:00:03.0: hash matches1662lb # [ 0.738710] Demotion targets for Node 0: null1663lb # [ 0.739584] Key type .fscrypt registered1664lb # [ 0.740283] Key type fscrypt-provisioning registered1665worker1 # [ 0.698052] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1666client # [ 0.746812] acpi device:09: hash matches1667lb # [ 0.741223] ima: No TPM chip found, activating TPM-bypass!1668worker1 # [ 0.699315] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1669client # [ 0.748345] RAS: Correctable Errors collector initialized.1670lb # [ 0.745708] ima: Allocated hash algorithm: sha11671lb # [ 0.746503] ima: No architecture policies found1672worker1 # [ 0.704646] Linux agpgart interface v0.1031673lb # [ 0.747452] PM: Magic number: 10:253:1341674worker1 # [ 0.705437] ACPI: bus type drm_connector registered1675lb # [ 0.748204] pci 0000:00:03.0: hash matches1676lb # [ 0.750705] acpi device:09: hash matches1677client # [ 0.757433] clk: Disabling unused clocks1678worker1 # [ 0.708837] usbcore: registered new interface driver usbserial_generic1679worker2 # [ 0.751404] Demotion targets for Node 0: null1680worker1 # [ 0.709883] usbserial: USB Serial support registered for generic1681lb # [ 0.752269] RAS: Correctable Errors collector initialized.1682worker1 # [ 0.712513] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1683worker2 # [ 0.754796] Key type .fscrypt registered1684client # [ 0.761797] PM: genpd: Disabling unused power domains1685worker1 # [ 0.712513] Please enable it if your BIOS has the CPPC option.1686worker2 # [ 0.755479] Key type fscrypt-provisioning registered1687worker2 # [ 0.756472] ima: No TPM chip found, activating TPM-bypass!1688worker1 # [ 0.714771] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1689worker2 # [ 0.758753] ima: Allocated hash algorithm: sha11690worker2 # [ 0.759596] ima: No architecture policies found1691worker1 # [ 0.718653] drop_monitor: Initializing network drop monitor service1692lb # [ 0.761378] clk: Disabling unused clocks1693worker1 # [ 0.719849] NET: Registered PF_INET6 protocol family1694worker2 # [ 0.763761] PM: Magic number: 10:253:1341695worker1 # [ 0.722990] Segment Routing with IPv61696worker2 # [ 0.764556] pci 0000:00:03.0: hash matches1697lb # [ 0.765707] PM: genpd: Disabling unused power domains1698worker2 # [ 0.765324] acpi device:09: hash matches1699worker1 # [ 0.725525] In-situ OAM (IOAM) with IPv61700worker1 # [ 0.726619] IPI shorthand broadcast: enabled1701worker2 # [ 0.769659] RAS: Correctable Errors collector initialized.1702worker2 # [ 0.776280] clk: Disabling unused clocks1703worker1 # [ 0.734936] sched_clock: Marking stable (599013828, 135472208)->(818680686, -84194650)1704worker2 # [ 0.780759] PM: genpd: Disabling unused power domains1705worker1 # [ 0.740756] registered taskstats version 11706worker1 # [ 0.741775] Loading compiled-in X.509 certificates1707worker1 # [ 0.760507] Demotion targets for Node 0: null1708worker1 # [ 0.761448] Key type .fscrypt registered1709worker1 # [ 0.762190] Key type fscrypt-provisioning registered1710worker1 # [ 0.765611] ima: No TPM chip found, activating TPM-bypass!1711worker1 # [ 0.766588] ima: Allocated hash algorithm: sha11712worker1 # [ 0.767399] ima: No architecture policies found1713worker1 # [ 0.769674] PM: Magic number: 10:253:1341714worker1 # [ 0.770461] pci 0000:00:03.0: hash matches1715worker1 # [ 0.771233] acpi device:09: hash matches1716worker1 # [ 0.776395] RAS: Correctable Errors collector initialized.1717worker1 # [ 0.785080] clk: Disabling unused clocks1718worker1 # [ 0.787513] PM: genpd: Disabling unused power domains1719client # [ 0.922747] Freeing initrd memory: 29096K1720client # [ 0.925895] Freeing unused decrypted memory: 2028K1721client # [ 0.928508] Freeing unused kernel image (initmem) memory: 3644K1722client # [ 0.929613] Write protecting the kernel read-only data: 32768k1723client # [ 0.931498] Freeing unused kernel image (text/rodata gap) memory: 1208K1724client # [ 0.933038] Freeing unused kernel image (rodata/data gap) memory: 752K1725lb # [ 0.943864] Freeing initrd memory: 29092K1726lb # [ 0.947045] Freeing unused decrypted memory: 2028K1727lb # [ 0.949749] Freeing unused kernel image (initmem) memory: 3644K1728worker2 # [ 0.950130] Freeing initrd memory: 29092K1729lb # [ 0.950862] Write protecting the kernel read-only data: 32768k1730lb # [ 0.952719] Freeing unused kernel image (text/rodata gap) memory: 1208K1731worker2 # [ 0.953347] Freeing unused decrypted memory: 2028K1732lb # [ 0.954334] Freeing unused kernel image (rodata/data gap) memory: 752K1733worker2 # [ 0.956127] Freeing unused kernel image (initmem) memory: 3644K1734worker2 # [ 0.957254] Write protecting the kernel read-only data: 32768k1735worker2 # [ 0.959221] Freeing unused kernel image (text/rodata gap) memory: 1208K1736worker2 # [ 0.960839] Freeing unused kernel image (rodata/data gap) memory: 752K1737client # [ 0.984154] x86/mm: Checked W+X mappings: passed, no W+X pages found.1738client # [ 0.985260] Run /init as init process1739client # [ 0.995279] systemd[1]: Inserted module 'autofs4'1740worker1 # [ 0.950094] Freeing initrd memory: 29092K1741worker1 # [ 0.953306] Freeing unused decrypted memory: 2028K1742worker1 # [ 0.956063] Freeing unused kernel image (initmem) memory: 3644K1743worker1 # [ 0.957199] Write protecting the kernel read-only data: 32768k1744worker1 # [ 0.959171] Freeing unused kernel image (text/rodata gap) memory: 1208K1745worker1 # [ 0.960772] Freeing unused kernel image (rodata/data gap) memory: 752K1746client # [ 1.011542] fuse: init (API version 7.45)1747lb # [ 1.005515] x86/mm: Checked W+X mappings: passed, no W+X pages found.1748lb # [ 1.006646] Run /init as init process1749client # [ 1.017345] ACPI: \_SB_.GSIG: Enabled at IRQ 221750worker2 # [ 1.012244] x86/mm: Checked W+X mappings: passed, no W+X pages found.1751client # [ 1.019732] ACPI: \_SB_.GSIH: Enabled at IRQ 231752worker2 # [ 1.013339] Run /init as init process1753client # [ 1.022731] ACPI: \_SB_.GSIE: Enabled at IRQ 201754lb # [ 1.017142] systemd[1]: Inserted module 'autofs4'1755client # [ 1.025043] ACPI: \_SB_.GSIF: Enabled at IRQ 211756worker2 # [ 1.023908] systemd[1]: Inserted module 'autofs4'1757lb # [ 1.033421] fuse: init (API version 7.45)1758lb # [ 1.039316] ACPI: \_SB_.GSIG: Enabled at IRQ 221759worker2 # [ 1.040618] fuse: init (API version 7.45)1760lb # [ 1.041835] ACPI: \_SB_.GSIH: Enabled at IRQ 231761client # [ 1.049446] systemd[1]: Successfully made /usr/ read-only.1762lb # [ 1.044793] ACPI: \_SB_.GSIE: Enabled at IRQ 201763lb # [ 1.047009] ACPI: \_SB_.GSIF: Enabled at IRQ 211764worker2 # [ 1.046538] ACPI: \_SB_.GSIG: Enabled at IRQ 221765worker2 # [ 1.049011] ACPI: \_SB_.GSIH: Enabled at IRQ 231766worker2 # [ 1.052116] ACPI: \_SB_.GSIE: Enabled at IRQ 201767worker1 # [ 1.011910] x86/mm: Checked W+X mappings: passed, no W+X pages found.1768worker1 # [ 1.013072] Run /init as init process1769worker2 # [ 1.054420] ACPI: \_SB_.GSIF: Enabled at IRQ 211770worker1 # [ 1.023656] systemd[1]: Inserted module 'autofs4'1771lb # [ 1.073963] systemd[1]: Successfully made /usr/ read-only.1772worker2 # [ 1.079467] systemd[1]: Successfully made /usr/ read-only.1773worker1 # [ 1.040032] fuse: init (API version 7.45)1774worker1 # [ 1.045922] ACPI: \_SB_.GSIG: Enabled at IRQ 221775worker1 # [ 1.048348] ACPI: \_SB_.GSIH: Enabled at IRQ 231776worker1 # [ 1.051552] ACPI: \_SB_.GSIE: Enabled at IRQ 201777worker1 # [ 1.054121] ACPI: \_SB_.GSIF: Enabled at IRQ 211778worker1 # [ 1.079184] systemd[1]: Successfully made /usr/ read-only.1779client # [ 1.385904] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1780client # [ 1.397848] systemd[1]: Detected virtualization kvm.1781client # [ 1.399943] systemd[1]: Detected architecture x86-64.1782client # [ 1.402021] systemd[1]: Running in initrd.1783client # [ 1.404444] systemd[1]: Initializing machine ID from random generator.1784client # [ 1.407306] systemd[1]: Hostname set to <client>.1785lb # [ 1.409631] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1786lb # [ 1.414864] systemd[1]: Detected virtualization kvm.1787lb # [ 1.415790] systemd[1]: Detected architecture x86-64.1788lb # [ 1.416661] systemd[1]: Running in initrd.1789lb # [ 1.417830] systemd[1]: Initializing machine ID from random generator.1790lb # [ 1.419038] systemd[1]: Hostname set to <lb>.1791worker2 # [ 1.415158] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1792worker2 # [ 1.420192] systemd[1]: Detected virtualization kvm.1793worker2 # [ 1.421107] systemd[1]: Detected architecture x86-64.1794worker2 # [ 1.422012] systemd[1]: Running in initrd.1795worker2 # [ 1.423104] systemd[1]: Initializing machine ID from random generator.1796worker2 # [ 1.424338] systemd[1]: Hostname set to <worker2>.1797worker1 # [ 1.414739] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1798worker1 # [ 1.419892] systemd[1]: Detected virtualization kvm.1799worker1 # [ 1.420787] systemd[1]: Detected architecture x86-64.1800worker1 # [ 1.421669] systemd[1]: Running in initrd.1801worker1 # [ 1.422796] systemd[1]: Initializing machine ID from random generator.1802worker1 # [ 1.424036] systemd[1]: Hostname set to <worker1>.1803lb # [ 1.607336] systemd[1]: bpf-restrict-fs: LSM BPF program attached1804client # [ 1.625413] systemd[1]: bpf-restrict-fs: LSM BPF program attached1805worker2 # [ 1.632254] systemd[1]: bpf-restrict-fs: LSM BPF program attached1806lb # [ 1.647823] systemd[1]: Queued start job for default target Initrd Default Target.1807worker1 # [ 1.609665] systemd[1]: bpf-restrict-fs: LSM BPF program attached1808lb # [ 1.652093] systemd[1]: Created slice Slice /system/modprobe.1809lb # [ 1.653322] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1810lb # [ 1.654786] systemd[1]: Expecting device /dev/disk/by-label/nixos...1811lb # [ 1.655925] systemd[1]: Reached target Path Units.1812lb # [ 1.656809] systemd[1]: Reached target Slice Units.1813lb # [ 1.657722] systemd[1]: Reached target Swaps.1814lb # [ 1.658541] systemd[1]: Reached target Timer Units.1815lb # [ 1.659561] systemd[1]: Listening on D-Bus System Message Bus Socket.1816lb # [ 1.660807] systemd[1]: Listening on Journal Socket (/dev/log).1817lb # [ 1.661996] systemd[1]: Listening on Journal Sockets.1818lb # [ 1.663041] systemd[1]: Listening on udev Control Socket.1819lb # [ 1.664159] systemd[1]: Listening on udev Kernel Socket.1820lb # [ 1.665161] systemd[1]: Reached target Socket Units.1821lb # [ 1.667078] systemd[1]: Starting Create List of Static Device Nodes...1822lb # [ 1.670761] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1823client # [ 1.679140] systemd[1]: Queued start job for default target Initrd Default Target.1824client # [ 1.683067] systemd[1]: Created slice Slice /system/modprobe.1825client # [ 1.684237] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1826client # [ 1.685677] systemd[1]: Expecting device /dev/disk/by-label/nixos...1827lb # [ 1.679765] systemd[1]: Starting Load Kernel Module configfs...1828client # [ 1.686818] systemd[1]: Reached target Path Units.1829client # [ 1.687695] systemd[1]: Reached target Slice Units.1830client # [ 1.688605] systemd[1]: Reached target Swaps.1831client # [ 1.689447] systemd[1]: Reached target Timer Units.1832client # [ 1.690498] systemd[1]: Listening on D-Bus System Message Bus Socket.1833client # [ 1.691730] systemd[1]: Listening on Journal Socket (/dev/log).1834client # [ 1.692933] systemd[1]: Listening on Journal Sockets.1835worker2 # [ 1.685888] systemd[1]: Queued start job for default target Initrd Default Target.1836client # [ 1.693943] systemd[1]: Listening on udev Control Socket.1837client # [ 1.694980] systemd[1]: Listening on udev Kernel Socket.1838client # [ 1.695980] systemd[1]: Reached target Socket Units.1839worker2 # [ 1.689058] systemd[1]: Created slice Slice /system/modprobe.1840worker1 # [ 1.648027] systemd[1]: Queued start job for default target Initrd Default Target.1841worker2 # [ 1.690251] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1842client # [ 1.697798] systemd[1]: Starting Create List of Static Device Nodes...1843worker2 # [ 1.691668] systemd[1]: Expecting device /dev/disk/by-label/nixos...1844worker2 # [ 1.692832] systemd[1]: Reached target Path Units.1845worker2 # [ 1.693693] systemd[1]: Reached target Slice Units.1846worker1 # [ 1.652821] systemd[1]: Created slice Slice /system/modprobe.1847worker2 # [ 1.694628] systemd[1]: Reached target Swaps.1848client # [ 1.701630] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1849worker2 # [ 1.695502] systemd[1]: Reached target Timer Units.1850worker1 # [ 1.653961] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1851worker2 # [ 1.696502] systemd[1]: Listening on D-Bus System Message Bus Socket.1852worker1 # [ 1.655437] systemd[1]: Expecting device /dev/disk/by-label/nixos...1853worker1 # [ 1.656548] systemd[1]: Reached target Path Units.1854worker2 # [ 1.697760] systemd[1]: Listening on Journal Socket (/dev/log).1855worker1 # [ 1.657405] systemd[1]: Reached target Slice Units.1856worker2 # [ 1.698940] systemd[1]: Listening on Journal Sockets.1857worker1 # [ 1.658316] systemd[1]: Reached target Swaps.1858worker2 # [ 1.699963] systemd[1]: Listening on udev Control Socket.1859worker1 # [ 1.659155] systemd[1]: Reached target Timer Units.1860worker2 # [ 1.700989] systemd[1]: Listening on udev Kernel Socket.1861client # [ 1.708340] systemd[1]: Starting Load Kernel Module configfs...1862worker1 # [ 1.660149] systemd[1]: Listening on D-Bus System Message Bus Socket.1863worker2 # [ 1.701946] systemd[1]: Reached target Socket Units.1864worker1 # [ 1.661359] systemd[1]: Listening on Journal Socket (/dev/log).1865worker1 # [ 1.662565] systemd[1]: Listening on Journal Sockets.1866worker2 # [ 1.703771] systemd[1]: Starting Create List of Static Device Nodes...1867worker1 # [ 1.663577] systemd[1]: Listening on udev Control Socket.1868lb # [ 1.694764] systemd[1]: Starting Journal Service...1869worker1 # [ 1.664602] systemd[1]: Listening on udev Kernel Socket.1870worker1 # [ 1.665617] systemd[1]: Reached target Socket Units.1871worker2 # [ 1.707565] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1872worker1 # [ 1.667398] systemd[1]: Starting Create List of Static Device Nodes...1873worker1 # [ 1.671315] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1874worker2 # [ 1.714811] systemd[1]: Starting Load Kernel Module configfs...1875lb # [ 1.718642] netfs: FS-Cache loaded1876worker1 # [ 1.678697] systemd[1]: Starting Load Kernel Module configfs...1877client # [ 1.723163] systemd[1]: Starting Journal Service...1878lb # [ 1.729962] 9pnet: Installing 9P2000 support1879lb # [ 1.735787] systemd[1]: Starting Load Kernel Modules...1880worker2 # [ 1.728924] systemd[1]: Starting Journal Service...1881lb # [ 1.741818] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1882client # [ 1.751165] netfs: FS-Cache loaded1883worker1 # [ 1.691063] systemd[1]: Starting Journal Service...1884lb # [ 1.752763] systemd[1]: Starting Coldplug All udev Devices...1885client # [ 1.759151] 9pnet: Installing 9P2000 support1886worker2 # [ 1.756978] netfs: FS-Cache loaded1887client # [ 1.765559] systemd[1]: Starting Load Kernel Modules...1888lb # [ 1.761528] systemd-journald[67]: Collecting audit messages is disabled.1889worker1 # [ 1.720370] netfs: FS-Cache loaded1890client # [ 1.770871] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1891worker2 # [ 1.765686] 9pnet: Installing 9P2000 support1892worker2 # [ 1.767390] systemd[1]: Starting Load Kernel Modules...1893lb # [ 1.769823] systemd[1]: Finished Create List of Static Device Nodes.1894worker1 # [ 1.729581] 9pnet: Installing 9P2000 support1895worker1 # [ 1.731260] systemd[1]: Starting Load Kernel Modules...1896lb # [ 1.774390] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897client # [ 1.781013] systemd[1]: Starting Coldplug All udev Devices...1898worker2 # [ 1.773825] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1899client # [ 1.784663] systemd-journald[67]: Collecting audit messages is disabled.1900worker1 # [ 1.738587] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1901lb # [ 1.782761] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1902worker2 # [ 1.781927] systemd[1]: Starting Coldplug All udev Devices...1903client # [ 1.793918] systemd[1]: Finished Create List of Static Device Nodes.1904worker1 # [ 1.747710] systemd[1]: Starting Coldplug All udev Devices...1905lb # [ 1.790114] systemd[1]: modprobe@configfs.service: Deactivated successfully.1906client # [ 1.800337] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1907lb # [ 1.796096] systemd[1]: Finished Load Kernel Module configfs.1908worker2 # [ 1.795562] systemd-journald[67]: Collecting audit messages is disabled.1909worker2 # [ 1.798045] systemd[1]: Finished Create List of Static Device Nodes.1910worker1 # [ 1.758661] systemd-journald[67]: Collecting audit messages is disabled.1911lb # [ 1.800164] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1912client # [ 1.808182] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1913worker2 # [ 1.802296] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1914worker1 # [ 1.761817] systemd[1]: Finished Create List of Static Device Nodes.1915lb # [ 1.804262] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1916client # [ 1.813165] systemd[1]: modprobe@configfs.service: Deactivated successfully.1917worker1 # [ 1.767101] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1918worker2 # [ 1.810117] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1919lb # [ 1.812714] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1920client # [ 1.820157] systemd[1]: Finished Load Kernel Module configfs.1921lb # [ 1.814591] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1922worker1 # [ 1.774875] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1923worker2 # [ 1.817140] systemd[1]: modprobe@configfs.service: Deactivated successfully.1924client # [ 1.825753] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1925worker2 # [ 1.822089] systemd[1]: Finished Load Kernel Module configfs.1926client # [ 1.828279] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1927worker1 # [ 1.780882] systemd[1]: modprobe@configfs.service: Deactivated successfully.1928worker2 # [ 1.826532] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1929worker1 # [ 1.787871] systemd[1]: Finished Load Kernel Module configfs.1930worker2 # [ 1.830169] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1931client # [ 1.837867] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1932client # [ 1.839265] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1933worker1 # [ 1.793912] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1934worker2 # [ 1.838999] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1935worker1 # [ 1.798916] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1936worker2 # [ 1.841848] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1937lb # [ 1.844266] systemd[1]: Finished Load Kernel Modules.1938worker1 # [ 1.806561] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1939worker1 # [ 1.807912] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1940lb # [ 1.852932] systemd[1]: Starting Apply Kernel Variables...1941client # [ 1.869332] systemd[1]: Finished Load Kernel Modules.1942lb # [ 1.865873] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1943worker2 # [ 1.867218] systemd[1]: Finished Load Kernel Modules.1944client # [ 1.876975] systemd[1]: Starting Apply Kernel Variables...1945lb # [ 1.874906] systemd[1]: Starting Create Static Device Nodes in /dev...1946worker2 # [ 1.875254] systemd[1]: Starting Apply Kernel Variables...1947client # [ 1.885934] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1948worker1 # [ 1.838041] systemd[1]: Finished Load Kernel Modules.1949worker1 # [ 1.844568] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1950lb # [ 1.888032] systemd[1]: Started Journal Service.1951worker2 # [ 1.887841] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1952lb # [ 1.751886] systemd-modules-load[69]: Inserted module 'dm_mod'1953client # [ 1.895468] systemd[1]: Starting Create Static Device Nodes in /dev...1954lb # [ 1.756939] systemd-modules-load[69]: Inserted module 'virtio_balloon'1955lb # [ 1.759048] systemd-modules-load[69]: Inserted module 'virtio_gpu'1956worker1 # [ 1.851673] systemd[1]: Starting Apply Kernel Variables...1957worker2 # [ 1.897935] systemd[1]: Starting Create Static Device Nodes in /dev...1958lb # [ 1.766390] systemd[1]: Finished Apply Kernel Variables.1959client # [ 1.773278] systemd-modules-load[69]: Inserted module 'dm_mod'1960client # [ 1.775255] systemd-modules-load[69]: Inserted module 'virtio_balloon'1961client # [ 1.911242] systemd[1]: Started Journal Service.1962client # [ 1.778461] systemd-modules-load[69]: Inserted module 'virtio_gpu'1963worker1 # [ 1.864562] systemd[1]: Starting Create Static Device Nodes in /dev...1964client # [ 1.784785] systemd[1]: Finished Apply Kernel Variables.1965worker2 # [ 1.911349] systemd[1]: Finished Apply Kernel Variables.1966lb # [ 1.782119] systemd[1]: Finished Create Static Device Nodes in /dev.1967lb # [ 1.783127] systemd[1]: Reached target Preparation for Local File Systems.1968lb # [ 1.784058] systemd[1]: Reached target Local File Systems.1969worker2 # [ 1.782215] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.919005] systemd[1]: Started Journal Service.1970lb # [ 1.787135] systemd[1]: Starting Create System Files and Directories...1971worker1 # [ 1.741372] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.879620] systemd[1]: Started Journal Service.1972worker2 # 1973worker1 # 1974worker2 # [ 1.786311] systemd-modules-load[68]: Inserted module 'virtio_balloon'1975worker1 # [ 1.745311] systemd-modules-load[68]: Inserted module 'virtio_balloon'1976worker2 # [ 1.787348] systemd-modules-load[68]: Inserted module 'virtio_gpu'1977worker1 # [ 1.748287] systemd-modules-load[68]: Inserted module 'virtio_gpu'1978lb # [ 1.792273] systemd[1]: Starting Rule-based Manager for Device Events and Files...1979client # [ 1.799337] systemd[1]: Finished Create Static Device Nodes in /dev.1980client # [ 1.801131] systemd[1]: Reached target Preparation for Local File Systems.1981client # [ 1.803091] systemd[1]: Reached target Local File Systems.1982client # [ 1.805848] systemd[1]: Starting Create System Files and Directories...1983worker2 # [ 1.800666] systemd[1]: Finished Create Static Device Nodes in /dev.1984worker2 # [ 1.803690] systemd[1]: Reached target Preparation for Local File Systems.1985client # [ 1.813076] systemd[1]: Starting Rule-based Manager for Device Events and Files...1986worker2 # [ 1.804704] systemd[1]: Reached target Local File Systems.1987worker1 # [ 1.765196] systemd[1]: Finished Apply Kernel Variables.1988worker2 # [ 1.807080] systemd[1]: Starting Create System Files and Directories...1989worker1 # [ 1.769105] systemd[1]: Finished Create Static Device Nodes in /dev.1990worker2 # [ 1.810591] systemd[1]: Starting Rule-based Manager for Device Events and Files...1991worker1 # [ 1.770114] systemd[1]: Reached target Preparation for Local File Systems.1992worker1 # [ 1.771088] systemd[1]: Reached target Local File Systems.1993lb # [ 1.817692] systemd[1]: Finished Create System Files and Directories.1994worker1 # [ 1.773906] systemd[1]: Starting Create System Files and Directories...1995worker1 # [ 1.779138] systemd[1]: Starting Rule-based Manager for Device Events and Files...1996client # [ 1.834235] systemd[1]: Finished Create System Files and Directories.1997worker2 # [ 1.838176] systemd[1]: Finished Create System Files and Directories.1998lb # [ 1.844953] systemd-udevd[83]: Using default interface naming scheme 'v261'.1999worker1 # [ 1.804168] systemd[1]: Finished Create System Files and Directories.2000client # [ 1.857339] systemd-udevd[83]: Using default interface naming scheme 'v261'.
loading 2,518 hidden lines…
4519worker2 # [ 35.628951] nix-grpc-daemon[786]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4520worker1 # [ 35.788060] nix-grpc-daemon[784]: ts=2026-09-21T15:07:50Z level=info event=scheduler_connected addr=lb:500514521worker2 # [ 35.829900] nix-grpc-daemon[786]: ts=2026-09-21T15:07:50Z level=info event=scheduler_connected addr=lb:500514522worker1 # [ 35.791935] nix-grpc-daemon[784]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4523worker2 # [ 35.834091] nix-grpc-daemon[786]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4524lb: must succeed: curl -sf localhost:9901/clusters4525worker2 # [ 36.034291] nix-grpc-daemon[786]: ts=2026-09-21T15:07:50Z level=info event=scheduler_connected addr=lb:500514526worker2 # [ 36.036059] nix-grpc-daemon[786]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4527worker1 # [ 35.993137] nix-grpc-daemon[784]: ts=2026-09-21T15:07:50Z level=info event=scheduler_connected addr=lb:500514528worker1 # [ 35.997479] nix-grpc-daemon[784]: ts=2026-09-21T15:07:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4529lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4530worker2 # [ 36.238291] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514531worker1 # [ 36.198178] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514532worker2 # [ 36.242113] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4533worker1 # [ 36.201994] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4534worker2 # [ 36.443579] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514535worker1 # [ 36.403196] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514536worker2 # [ 36.447654] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4537worker1 # [ 36.406964] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4538worker1 # [ 36.605946] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514539worker2 # [ 36.648832] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514540worker1 # [ 36.609866] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4541worker2 # [ 36.652653] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4542worker1 # [ 36.810615] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514543worker2 # [ 36.853868] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514544worker1 # [ 36.814554] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4545worker2 # [ 36.857765] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4546worker1 # [ 37.015505] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514547worker2 # [ 37.058973] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_connected addr=lb:500514548worker1 # [ 37.019519] nix-grpc-daemon[784]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4549lb: must succeed: curl -sf localhost:9901/clusters4550worker2 # [ 37.062892] nix-grpc-daemon[786]: ts=2026-09-21T15:07:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4551lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4552worker1 # [ 37.220465] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514553worker2 # [ 37.265808] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514554worker1 # [ 37.224527] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4555worker2 # [ 37.269121] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4556worker2 # [ 37.466772] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514557worker1 # [ 37.425618] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514558worker1 # [ 37.429702] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4559worker2 # [ 37.470814] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4560worker2 # [ 37.672133] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514561worker1 # [ 37.631389] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514562worker2 # [ 37.676120] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4563worker1 # [ 37.635166] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4564worker2 # [ 37.877242] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514565worker1 # [ 37.836097] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514566worker2 # [ 37.881168] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4567worker1 # [ 37.840096] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4568worker1 # [ 38.040943] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514569worker2 # [ 38.082432] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_connected addr=lb:500514570worker1 # [ 38.044917] nix-grpc-daemon[784]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4571worker2 # [ 38.086368] nix-grpc-daemon[786]: ts=2026-09-21T15:07:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4572lb: must succeed: curl -sf localhost:9901/clusters4573lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4574worker1 # [ 38.245787] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514575worker2 # [ 38.288439] nix-grpc-daemon[786]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514576worker1 # [ 38.249764] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4577worker2 # [ 38.292504] nix-grpc-daemon[786]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4578worker1 # [ 38.450636] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514579worker2 # [ 38.493890] nix-grpc-daemon[786]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514580worker1 # [ 38.454653] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4581worker2 # [ 38.497831] nix-grpc-daemon[786]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4582worker1 # [ 38.656560] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514583worker2 # [ 38.698843] nix-grpc-daemon[786]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514584worker1 # [ 38.660419] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4585worker2 # [ 38.702726] nix-grpc-daemon[786]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4586worker1 # [ 38.861371] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514587worker2 # [ 38.903956] nix-grpc-daemon[786]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514588worker1 # [ 38.865234] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4589worker2 # [ 38.908072] nix-grpc-daemon[786]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4590worker1 # [ 39.066107] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_connected addr=lb:500514591worker2 # [ 39.109184] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514592worker1 # [ 39.070299] nix-grpc-daemon[784]: ts=2026-09-21T15:07:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4593worker2 # [ 39.113783] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4594lb: must succeed: curl -sf localhost:9901/clusters4595lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4596worker1 # [ 39.270389] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514597worker1 # [ 39.272241] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4598worker2 # [ 39.316198] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514599worker2 # [ 39.319365] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4600worker1 # [ 39.473327] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514601worker2 # [ 39.517116] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514602worker1 # [ 39.477230] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4603worker2 # [ 39.520918] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4604worker1 # [ 39.508743] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=scheduler_take_over4605lb # [ 39.552972] niks3-server[1108]: 2026/09/21 15:07:53 INFO lead: acquired remote=192.168.1.3:536584606worker1 # [ 39.679451] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514607worker2 # [ 39.722343] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514608worker1 # [ 39.682647] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4609worker2 # [ 39.726180] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4610worker1 # [ 39.880212] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514611worker2 # [ 39.927096] nix-grpc-daemon[786]: ts=2026-09-21T15:07:54Z level=info event=scheduler_connected addr=lb:500514612worker1 # [ 39.899828] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:401904613worker1 # [ 39.904705] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04614worker1 # [ 39.927494] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=oidc_jwks provider=mock keys=14615worker1 # [ 39.931807] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:401904616worker1 # [ 39.935624] nix-grpc-daemon[784]: ts=2026-09-21T15:07:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04617lb: must succeed: curl -sf localhost:9901/clusters4618lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4619worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4620worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4621lb: must succeed: curl -sf localhost:9901/clusters4622lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4623lb: must succeed: curl -sf localhost:9901/clusters4624lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4625lb: must succeed: curl -sf localhost:9901/clusters4626lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4627lb: must succeed: curl -sf localhost:9901/clusters4628lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4629worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4630worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)4631worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4632worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4633lb: must succeed: curl -sf localhost:9901/clusters4634lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4635lb: must succeed: curl -sf localhost:9901/clusters4636lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4637worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4638worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4639worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4640worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4641worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4642worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4643worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4644worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4645(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.66 seconds)4646subtest: balancer auth: client cert, bearer token, nothing, unknown CN4647worker1 # [ 41.539237] nix-grpc-daemon[784]: ts=2026-09-21T15:07:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401964648worker1 # [ 41.553323] systemd[1]: Started Nix Daemon.4649worker1 # [ 41.818062] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4650worker1 # [ 41.845901] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5)4651worker1 # [ 41.849359] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 2/5)4652worker1 # [ 42.033151] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 394 ms (attempt 3/5)4653worker1 # [ 42.432955] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 4/5)4654worker1 # [ 42.576723] nix-grpc-daemon[784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4655worker1 # [ 42.580201] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4656worker2 # [ 42.668478] nix-grpc-daemon[786]: ts=2026-09-21T15:07:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467024657worker2 # [ 42.680856] systemd[1]: Started Nix Daemon.4658worker2 # [ 42.934688] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4659worker2 # [ 42.959873] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)4660worker2 # [ 43.041849] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 2/5)4661worker2 # [ 43.089349] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 336 ms (attempt 3/5)4662worker2 # [ 43.425974] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 110 ms (attempt 4/5)4663worker2 # [ 43.540187] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4664worker2 # [ 43.544927] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4665worker2 # [ 43.565087] nix-grpc-daemon[786]: ts=2026-09-21T15:07:58Z level=debug event=path_info_miss path=6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test error="error: path '/nix/store/6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test' is required, but there is no substituter that can build it"4666worker2 # [ 43.580903] nix-grpc-daemon[786]: ts=2026-09-21T15:07:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467024667worker2 # [ 43.590689] nix-grpc-daemon[786]: ts=2026-09-21T15:07:58Z level=debug event=path_info_miss path=6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test error="error: path '/nix/store/6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test' is required, but there is no substituter that can build it"4668client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4669client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4670client: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4671client: (finished: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4672worker1 # [ 43.921414] nix-grpc-daemon[784]: ts=2026-09-21T15:07:58Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:401964673worker2 # [ 43.995551] nix-grpc-daemon[786]: ts=2026-09-21T15:07:58Z level=info event=oidc_jwks provider=mock keys=14674worker2 # [ 43.999432] nix-grpc-daemon[786]: ts=2026-09-21T15:07:58Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:467024675worker2 # [ 44.005337] nix-grpc-daemon[786]: ts=2026-09-21T15:07:58Z level=debug event=path_info_miss path=6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test error="error: path '/nix/store/6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test' is required, but there is no substituter that can build it"4676worker2 # [ 44.014493] nix-grpc-daemon[786]: ts=2026-09-21T15:07:58Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:467024677worker2 # [ 44.023484] nix-grpc-daemon[786]: ts=2026-09-21T15:07:58Z level=debug event=path_info_miss path=6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test error="error: path '/nix/store/6ya3np4gk5g3zb085kx6dfrg466qaw4r-nixos-system-client-test' is required, but there is no substituter that can build it"4678client: must succeed: rm -r /var/lib/nix-grpc-store4679client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4680worker2 # [ 44.295807] nix-grpc-daemon[786]: ts=2026-09-21T15:07:59Z level=info event=denied method=QueryMissing cn=- role=none4681worker2 # [ 44.537968] nix-grpc-daemon[786]: ts=2026-09-21T15:07:59Z level=info event=denied method=QueryMissing cn=stranger role=none4682(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.31 seconds)4683subtest: DAG build is scheduled across workers and lands in the cache4684client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24685worker1 # [ 44.726595] nix-grpc-daemon[784]: ts=2026-09-21T15:07:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401964686client # don't know how to build these paths:4687client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4688worker1 # [ 44.740402] nix-grpc-daemon[784]: ts=2026-09-21T15:07:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401964689worker1 # [ 44.773326] nix-grpc-daemon[784]: ts=2026-09-21T15:07:59Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04690worker1 # [ 44.778361] nix-grpc-daemon[784]: ts=2026-09-21T15:07:59Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14691worker1 # [ 44.831355] nix-grpc-daemon[784]: ts=2026-09-21T15:07:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401964692worker1 # [ 44.860770] nix-grpc-daemon[784]: ts=2026-09-21T15:07:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401964693client # copying 1 paths to 'lb:50051'...4694worker1 # [ 44.874979] nix-grpc-daemon[784]: ts=2026-09-21T15:07:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401964695client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4696worker1 # [ 44.901859] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4697worker1 # [ 44.927833] nix-grpc-daemon[784]: ts=2026-09-21T15:07:59Z level=info event=niks3_push_started pid=8834698worker1 # [ 45.267301] nix-grpc-daemon[883]: time=2026-09-21T15:08:00.105Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4699worker1 # [ 45.507644] nix-daemon[868]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4700worker1 # [ 45.522951] nix-daemon[868]: reaped child process 899, status = succeeded4701worker1 # [ 45.734667] nix-daemon[868]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4702worker1 # [ 45.746433] nix-daemon[868]: reaped child process 904, status = succeeded4703lb # [ 45.813143] niks3-server[1108]: 2026/09/21 15:08:00 INFO Received uploads request method=POST path=/api/pending_closures4704worker1 # [ 45.793229] nix-grpc-daemon[883]: time=2026-09-21T15:08:00.635Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4705worker1 # [ 45.795597] nix-grpc-daemon[883]: time=2026-09-21T15:08:00.638Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4706lb # [ 45.883933] niks3-server[1108]: 2026/09/21 15:08:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4707lb # [ 45.895266] niks3-server[1108]: 2026/09/21 15:08:00 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4708lb # [ 45.919357] niks3-server[1108]: 2026/09/21 15:08:00 INFO Signed narinfos id=1 count=14709worker1 # [ 45.877344] nix-grpc-daemon[883]: time=2026-09-21T15:08:00.719Z level=INFO msg="Uploading 1 narinfos"4710lb # [ 45.926281] niks3-server[1108]: 2026/09/21 15:08:00 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4711lb # [ 45.934509] niks3-server[1108]: 2026/09/21 15:08:00 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4712lb # [ 45.945387] niks3-server[1108]: 2026/09/21 15:08:00 INFO Completed upload id=14713worker1 # [ 45.901795] nix-grpc-daemon[883]: time=2026-09-21T15:08:00.744Z level=INFO msg="Upload complete. (609ms)"4714lb # [ 45.954685] niks3-server[1108]: 2026/09/21 15:08:00 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4715worker1 # [ 45.906400] nix-grpc-daemon[784]: ts=2026-09-21T15:08:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=1 paths=1 nar_bytes_in=5044716worker1 # [ 45.913747] nix-grpc-daemon[784]: ts=2026-09-21T15:08:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401964717worker1 # [ 45.924872] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4718worker1 # [ 45.937073] nix-daemon[868]: reaped child process 881, status = succeeded4719client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4720client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4721worker1 # [ 45.944262] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4722client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4723client # farm-a> LOG-farm-a-t14724worker1 # [ 46.958376] nix-daemon[868]: reaped child process 908, status = succeeded4725worker1 # [ 47.131042] nix-daemon[868]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4726worker1 # [ 47.142917] nix-daemon[868]: reaped child process 927, status = succeeded4727lb # [ 47.205799] niks3-server[1108]: 2026/09/21 15:08:01 INFO Received uploads request method=POST path=/api/pending_closures4728worker1 # [ 47.174833] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.017Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4729worker1 # [ 47.176505] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.017Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4730lb # [ 47.253828] niks3-server[1108]: 2026/09/21 15:08:01 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4731lb # [ 47.270645] niks3-server[1108]: 2026/09/21 15:08:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4732lb # [ 47.276457] niks3-server[1108]: 2026/09/21 15:08:01 INFO Signed narinfos id=2 count=14733worker1 # [ 47.235876] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.077Z level=INFO msg="Uploading 1 narinfos"4734lb # [ 47.280921] niks3-server[1108]: 2026/09/21 15:08:01 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4735lb # [ 47.285519] niks3-server[1108]: 2026/09/21 15:08:01 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4736lb # [ 47.295686] niks3-server[1108]: 2026/09/21 15:08:01 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4737lb # [ 47.298539] niks3-server[1108]: 2026/09/21 15:08:01 INFO Completed upload id=24738worker1 # [ 47.254855] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.097Z level=INFO msg="Upload complete. (301ms)"4739lb # [ 47.300269] niks3-server[1108]: 2026/09/21 15:08:01 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4740worker1 # [ 47.263427] nix-grpc-daemon[784]: ts=2026-09-21T15:08:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14741worker1 # [ 47.280444] nix-daemon[868]: reaped child process 906, status = succeeded4742worker1 # [ 47.284248] nix-grpc-daemon[784]: ts=2026-09-21T15:08:02Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04743worker1 # [ 47.286302] nix-grpc-daemon[784]: ts=2026-09-21T15:08:02Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14744worker1 # [ 47.335863] nix-grpc-daemon[784]: ts=2026-09-21T15:08:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401964745worker1 # [ 47.348076] nix-grpc-daemon[784]: ts=2026-09-21T15:08:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401964746worker1 # [ 47.361466] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.203Z level=INFO msg="All 1 paths already cached"4747client # copying 1 paths to 'lb:50051'...4748worker1 # [ 47.369168] nix-grpc-daemon[784]: ts=2026-09-21T15:08:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401964749client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4750worker1 # [ 47.387743] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4751worker1 # [ 47.603489] nix-daemon[868]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4752worker1 # [ 47.616457] nix-daemon[868]: reaped child process 936, status = succeeded4753worker1 # [ 47.811939] nix-daemon[868]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4754worker1 # [ 47.823789] nix-daemon[868]: reaped child process 941, status = succeeded4755lb # [ 47.887728] niks3-server[1108]: 2026/09/21 15:08:02 INFO Received uploads request method=POST path=/api/pending_closures4756worker1 # [ 47.853178] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.695Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4757worker1 # [ 47.855272] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.697Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4758lb # [ 47.918566] niks3-server[1108]: 2026/09/21 15:08:02 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4759lb # [ 47.928331] niks3-server[1108]: 2026/09/21 15:08:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4760worker1 # [ 47.891410] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.733Z level=INFO msg="Uploading 1 narinfos"4761lb # [ 47.935199] niks3-server[1108]: 2026/09/21 15:08:02 INFO Signed narinfos id=3 count=14762lb # [ 47.939551] niks3-server[1108]: 2026/09/21 15:08:02 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4763lb # [ 47.951743] niks3-server[1108]: 2026/09/21 15:08:02 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4764lb # [ 47.958057] niks3-server[1108]: 2026/09/21 15:08:02 INFO Completed upload id=34765worker1 # [ 47.914045] nix-grpc-daemon[883]: time=2026-09-21T15:08:02.756Z level=INFO msg="Upload complete. (511ms)"4766lb # [ 47.961190] niks3-server[1108]: 2026/09/21 15:08:02 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4767worker1 # [ 47.919141] nix-grpc-daemon[784]: ts=2026-09-21T15:08:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_in=6244768worker1 # [ 47.923830] nix-grpc-daemon[784]: ts=2026-09-21T15:08:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401964769worker1 # [ 47.931482] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4770worker1 # [ 47.938116] nix-daemon[868]: reaped child process 931, status = succeeded4771worker1 # [ 47.942959] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4772client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4773client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4774lb # [ 48.461695] envoy[979]: conn peer=[2001:db8:1::1]:33478 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=406 ms=20614775lb # [ 48.465391] envoy[979]: conn peer=[2001:db8:1::1]:44904 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=388 ms=734776lb # [ 48.468792] envoy[979]: conn peer=[2001:db8:1::1]:44912 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134777lb # [ 48.472188] envoy[979]: conn peer=[2001:db8:1::1]:44916 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=94778worker1 # [ 48.785812] nix-grpc-daemon[784]: E0000 00:00:1790003283.626857 823 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4779worker1 # [ 48.816146] nix-grpc-daemon[784]: E0000 00:00:1790003283.658148 823 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4780worker1 # [ 48.821293] nix-grpc-daemon[784]: E0000 00:00:1790003283.658381 823 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4781worker1 # [ 48.845796] nix-grpc-daemon[784]: E0000 00:00:1790003283.687951 823 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4782client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4783client # farm-b> LOG-farm-b-t14784worker1 # [ 49.155490] nix-daemon[868]: reaped child process 945, status = succeeded4785worker1 # [ 49.327755] nix-daemon[868]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4786worker1 # [ 49.338397] nix-daemon[868]: reaped child process 964, status = succeeded4787lb # [ 49.401637] niks3-server[1108]: 2026/09/21 15:08:03 INFO Received uploads request method=POST path=/api/pending_closures4788worker1 # [ 49.366648] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.209Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4789worker1 # [ 49.368243] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.210Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4790lb # [ 49.443985] niks3-server[1108]: 2026/09/21 15:08:03 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4791lb # [ 49.449589] niks3-server[1108]: 2026/09/21 15:08:03 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4792lb # [ 49.457942] niks3-server[1108]: 2026/09/21 15:08:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4793worker1 # [ 49.419251] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.261Z level=INFO msg="Uploading 1 narinfos"4794lb # [ 49.463288] niks3-server[1108]: 2026/09/21 15:08:03 INFO Signed narinfos id=4 count=14795lb # [ 49.468512] niks3-server[1108]: 2026/09/21 15:08:03 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4796lb # [ 49.478065] niks3-server[1108]: 2026/09/21 15:08:03 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4797lb # [ 49.482770] niks3-server[1108]: 2026/09/21 15:08:03 INFO Completed upload id=44798lb # [ 49.485737] niks3-server[1108]: 2026/09/21 15:08:03 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4799worker1 # [ 49.440547] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.283Z level=INFO msg="Upload complete. (291ms)"4800worker1 # [ 49.446728] nix-grpc-daemon[784]: ts=2026-09-21T15:08:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14801worker1 # [ 49.458486] nix-grpc-daemon[784]: ts=2026-09-21T15:08:04Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04802worker1 # [ 49.463192] nix-grpc-daemon[784]: ts=2026-09-21T15:08:04Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14803worker1 # [ 49.471396] nix-daemon[868]: reaped child process 943, status = succeeded4804worker1 # [ 49.502992] nix-grpc-daemon[784]: ts=2026-09-21T15:08:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401964805worker1 # [ 49.521633] nix-grpc-daemon[784]: ts=2026-09-21T15:08:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401964806worker1 # [ 49.534625] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.377Z level=INFO msg="All 2 paths already cached"4807client # copying 1 paths to 'lb:50051'...4808worker1 # [ 49.542685] nix-grpc-daemon[784]: ts=2026-09-21T15:08:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401964809client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4810worker1 # [ 49.560726] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4811worker1 # [ 49.759303] nix-daemon[868]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4812worker1 # [ 49.770616] nix-daemon[868]: reaped child process 973, status = succeeded4813worker1 # [ 49.967300] nix-daemon[868]: accepted connection from pid 975, user nix-grpc-daemon (trusted)4814worker1 # [ 49.980124] nix-daemon[868]: reaped child process 978, status = succeeded4815lb # [ 50.044071] niks3-server[1108]: 2026/09/21 15:08:04 INFO Received uploads request method=POST path=/api/pending_closures4816worker1 # [ 50.010579] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.853Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4817worker1 # [ 50.012437] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.854Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4818lb # [ 50.072458] niks3-server[1108]: 2026/09/21 15:08:04 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4819lb # [ 50.085654] niks3-server[1108]: 2026/09/21 15:08:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4820lb # [ 50.092766] niks3-server[1108]: 2026/09/21 15:08:04 INFO Signed narinfos id=5 count=14821worker1 # [ 50.052104] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.894Z level=INFO msg="Uploading 1 narinfos"4822lb # [ 50.095995] niks3-server[1108]: 2026/09/21 15:08:04 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4823lb # [ 50.107534] niks3-server[1108]: 2026/09/21 15:08:04 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4824lb # [ 50.117983] niks3-server[1108]: 2026/09/21 15:08:04 INFO Completed upload id=54825worker1 # [ 50.074208] nix-grpc-daemon[883]: time=2026-09-21T15:08:04.916Z level=INFO msg="Upload complete. (499ms)"4826lb # [ 50.120220] niks3-server[1108]: 2026/09/21 15:08:04 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4827worker1 # [ 50.077757] nix-grpc-daemon[784]: ts=2026-09-21T15:08:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_in=7524828worker1 # [ 50.084336] nix-grpc-daemon[784]: ts=2026-09-21T15:08:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401964829worker1 # [ 50.092484] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4830worker1 # [ 50.100337] nix-daemon[868]: reaped child process 968, status = succeeded4831worker1 # [ 50.105418] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4832client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4833client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4834client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4835client # farm-top> LOG-farm-top-t14836worker1 # [ 51.329758] nix-daemon[868]: reaped child process 982, status = succeeded4837worker1 # [ 51.523294] nix-daemon[868]: accepted connection from pid 998, user nix-grpc-daemon (trusted)4838worker1 # [ 51.535366] nix-daemon[868]: reaped child process 1001, status = succeeded4839lb # [ 51.601289] niks3-server[1108]: 2026/09/21 15:08:06 INFO Received uploads request method=POST path=/api/pending_closures4840worker1 # [ 51.566357] nix-grpc-daemon[883]: time=2026-09-21T15:08:06.408Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4841worker1 # [ 51.567912] nix-grpc-daemon[883]: time=2026-09-21T15:08:06.410Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4842lb # [ 51.640809] niks3-server[1108]: 2026/09/21 15:08:06 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4843lb # [ 51.652421] niks3-server[1108]: 2026/09/21 15:08:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4844lb # [ 51.660523] niks3-server[1108]: 2026/09/21 15:08:06 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4845lb # [ 51.665446] niks3-server[1108]: 2026/09/21 15:08:06 INFO Signed narinfos id=6 count=14846worker1 # [ 51.621958] nix-grpc-daemon[883]: time=2026-09-21T15:08:06.464Z level=INFO msg="Uploading 1 narinfos"4847lb # [ 51.671097] niks3-server[1108]: 2026/09/21 15:08:06 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4848lb # [ 51.679508] niks3-server[1108]: 2026/09/21 15:08:06 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4849lb # [ 51.688451] niks3-server[1108]: 2026/09/21 15:08:06 INFO Completed upload id=64850lb # [ 51.689799] niks3-server[1108]: 2026/09/21 15:08:06 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4851worker1 # [ 51.644838] nix-grpc-daemon[883]: time=2026-09-21T15:08:06.487Z level=INFO msg="Upload complete. (330ms)"4852worker1 # [ 51.653419] nix-grpc-daemon[784]: ts=2026-09-21T15:08:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14853worker1 # [ 51.666267] nix-daemon[868]: reaped child process 980, status = succeeded4854client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.17 seconds)4855client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4856client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4857client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4858client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4859client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4860client # copying 3 paths...4861client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4862client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4863client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4864client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4865(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.74 seconds)4866subtest: repeat is answered Cached by the scheduler without building4867worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4868worker1 # finding garbage collector roots...4869worker1 # deleting specified paths...4870worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4871worker1 # deleting unused links...4872worker1 # note: hard linking is currently saving 0.0 KiB4873worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.30 seconds)4874worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4875worker2 # finding garbage collector roots...4876worker2 # deleting specified paths...4877worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4878worker2 # deleting unused links...4879worker2 # note: hard linking is currently saving 0.0 KiB4880worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.31 seconds)4881client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24882worker1 # [ 53.082173] nix-grpc-daemon[784]: ts=2026-09-21T15:08:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401964883client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4884client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4885worker2 # [ 53.144583] nix-grpc-daemon[786]: ts=2026-09-21T15:08:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467024886worker1 # [ 53.130651] nix-grpc-daemon[784]: ts=2026-09-21T15:08:07Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14887client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.29 seconds)4888client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4889client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4890lb: must succeed: curl -sf localhost:9901/clusters4891lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4892lb: must succeed: curl -sf localhost:9901/clusters4893lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4894worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4895worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4896(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.42 seconds)4897subtest: inputs already in the cache are not sent to the scheduler4898lb: must succeed: curl -sf localhost:9901/clusters4899lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4900lb: must succeed: curl -sf localhost:9901/clusters4901lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4902worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4903worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4904client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24905worker2 # [ 54.148559] nix-grpc-daemon[786]: ts=2026-09-21T15:08:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467024906client # don't know how to build these paths:4907client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4908worker2 # [ 54.158697] nix-grpc-daemon[786]: ts=2026-09-21T15:08:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467024909worker1 # [ 54.134251] nix-grpc-daemon[784]: ts=2026-09-21T15:08:08Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04910worker1 # [ 54.138353] nix-grpc-daemon[784]: ts=2026-09-21T15:08:08Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14911worker1 # [ 54.177152] nix-grpc-daemon[784]: ts=2026-09-21T15:08:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401964912worker1 # [ 54.189895] nix-grpc-daemon[784]: ts=2026-09-21T15:08:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401964913worker1 # [ 54.202612] nix-grpc-daemon[883]: time=2026-09-21T15:08:09.044Z level=INFO msg="All 2 paths already cached"4914client # copying 1 paths to 'lb:50051'...4915client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4916worker1 # [ 54.210568] nix-grpc-daemon[784]: ts=2026-09-21T15:08:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401964917worker1 # [ 54.221922] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4918worker1 # [ 54.438598] nix-daemon[868]: accepted connection from pid 1030, user nix-grpc-daemon (trusted)4919worker1 # [ 54.451549] nix-daemon[868]: reaped child process 1033, status = succeeded4920worker1 # [ 54.655631] nix-daemon[868]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4921worker1 # [ 54.667666] nix-daemon[868]: reaped child process 1038, status = succeeded4922lb # [ 54.730438] niks3-server[1108]: 2026/09/21 15:08:09 INFO Received uploads request method=POST path=/api/pending_closures4923worker1 # [ 54.694687] nix-grpc-daemon[883]: time=2026-09-21T15:08:09.536Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4924worker1 # [ 54.696075] nix-grpc-daemon[883]: time=2026-09-21T15:08:09.537Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4925lb # [ 54.755379] niks3-server[1108]: 2026/09/21 15:08:09 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4926lb # [ 54.767917] niks3-server[1108]: 2026/09/21 15:08:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4927lb # [ 54.773117] niks3-server[1108]: 2026/09/21 15:08:09 INFO Signed narinfos id=7 count=14928worker1 # [ 54.732245] nix-grpc-daemon[883]: time=2026-09-21T15:08:09.574Z level=INFO msg="Uploading 1 narinfos"4929lb # [ 54.779518] niks3-server[1108]: 2026/09/21 15:08:09 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4930lb # [ 54.789962] niks3-server[1108]: 2026/09/21 15:08:09 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4931lb # [ 54.794907] niks3-server[1108]: 2026/09/21 15:08:09 INFO Completed upload id=74932worker1 # [ 54.753333] nix-grpc-daemon[883]: time=2026-09-21T15:08:09.595Z level=INFO msg="Upload complete. (519ms)"4933lb # [ 54.799283] niks3-server[1108]: 2026/09/21 15:08:09 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4934worker1 # [ 54.756794] nix-grpc-daemon[784]: ts=2026-09-21T15:08:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_in=7524935worker1 # [ 54.761723] nix-grpc-daemon[784]: ts=2026-09-21T15:08:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401964936worker1 # [ 54.770685] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4937worker1 # [ 54.776536] nix-daemon[868]: reaped child process 1028, status = succeeded4938client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4939worker1 # [ 54.780983] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4940client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4941client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4942client # farm-top> LOG-farm-top-t1b4943worker1 # [ 56.206608] nix-daemon[868]: reaped child process 1042, status = succeeded4944worker1 # [ 56.399585] nix-daemon[868]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)4945worker1 # [ 56.411491] nix-daemon[868]: reaped child process 1061, status = succeeded4946lb # [ 56.477806] niks3-server[1108]: 2026/09/21 15:08:10 INFO Received uploads request method=POST path=/api/pending_closures4947worker1 # [ 56.441295] nix-grpc-daemon[883]: time=2026-09-21T15:08:11.282Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4948worker1 # [ 56.442671] nix-grpc-daemon[883]: time=2026-09-21T15:08:11.282Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4949lb # [ 56.509533] niks3-server[1108]: 2026/09/21 15:08:10 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4950lb # [ 56.521554] niks3-server[1108]: 2026/09/21 15:08:10 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4951lb # [ 56.533737] niks3-server[1108]: 2026/09/21 15:08:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4952lb # [ 56.540481] niks3-server[1108]: 2026/09/21 15:08:10 INFO Signed narinfos id=8 count=14953worker1 # [ 56.496115] nix-grpc-daemon[883]: time=2026-09-21T15:08:11.338Z level=INFO msg="Uploading 1 narinfos"4954lb # [ 56.544388] niks3-server[1108]: 2026/09/21 15:08:10 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4955lb # [ 56.554231] niks3-server[1108]: 2026/09/21 15:08:10 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4956lb # [ 56.558362] niks3-server[1108]: 2026/09/21 15:08:10 INFO Completed upload id=84957worker1 # [ 56.516123] nix-grpc-daemon[883]: time=2026-09-21T15:08:11.358Z level=INFO msg="Upload complete. (327ms)"4958lb # [ 56.561314] niks3-server[1108]: 2026/09/21 15:08:10 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4959worker1 # [ 56.524395] nix-grpc-daemon[784]: ts=2026-09-21T15:08:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14960worker1 # [ 56.536196] nix-daemon[868]: reaped child process 1040, status = succeeded4961client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.70 seconds)4962client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4963client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds)4964lb: must succeed: curl -sf localhost:9901/clusters4965lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4966lb: must succeed: curl -sf localhost:9901/clusters4967lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4968worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4969worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4970(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.28 seconds)4971subtest: two clients wanting the same drv build it once4972client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4973client # [ 57.056426] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4974client # Running as unit: dedup1.service; invocation ID: 5176bde163ff4f778afad2465e5964564975client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)4976client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4977client # [ 57.261680] nix[895]: don't know how to build these paths:4978client # [ 57.262976] nix[895]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4979worker2 # [ 57.252566] nix-grpc-daemon[786]: ts=2026-09-21T15:08:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467024980worker1 # [ 57.221236] nix-grpc-daemon[784]: ts=2026-09-21T15:08:12Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04981worker1 # [ 57.225953] nix-grpc-daemon[784]: ts=2026-09-21T15:08:12Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14982worker2 # [ 57.269642] nix-grpc-daemon[786]: ts=2026-09-21T15:08:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467024983worker2 # [ 57.284262] nix-grpc-daemon[786]: ts=2026-09-21T15:08:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467024984client # [ 57.303078] nix[895]: copying 1 paths to 'lb:50051'...4985client # [ 57.303902] nix[895]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4986worker2 # [ 57.295428] nix-grpc-daemon[786]: ts=2026-09-21T15:08:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467024987worker2 # [ 57.308220] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4988worker2 # [ 57.330461] nix-grpc-daemon[786]: ts=2026-09-21T15:08:12Z level=info event=niks3_push_started pid=8824989worker2 # [ 57.662068] nix-grpc-daemon[882]: time=2026-09-21T15:08:12.554Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""4990worker2 # [ 57.900711] nix-daemon[857]: accepted connection from pid 895, user nix-grpc-daemon (trusted)4991worker2 # [ 57.914950] nix-daemon[857]: reaped child process 898, status = succeeded4992worker2 # [ 58.123309] nix-daemon[857]: accepted connection from pid 900, user nix-grpc-daemon (trusted)4993client # [ 58.130720] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4994client # Running as unit: dedup2.service; invocation ID: 31ec4863056f4a64b91133172c079ed84995worker2 # [ 58.135218] nix-daemon[857]: reaped child process 903, status = succeeded4996client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4997lb # [ 58.159693] niks3-server[1108]: 2026/09/21 15:08:12 INFO Received uploads request method=POST path=/api/pending_closures4998worker2 # [ 58.166165] nix-grpc-daemon[882]: time=2026-09-21T15:08:13.062Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4999worker2 # [ 58.168548] nix-grpc-daemon[882]: time=2026-09-21T15:08:13.065Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5000lb # [ 58.197908] niks3-server[1108]: 2026/09/21 15:08:12 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5001lb # [ 58.209096] niks3-server[1108]: 2026/09/21 15:08:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign5002worker2 # [ 58.214113] nix-grpc-daemon[882]: time=2026-09-21T15:08:13.110Z level=INFO msg="Uploading 1 narinfos"5003lb # [ 58.216851] niks3-server[1108]: 2026/09/21 15:08:12 INFO Signed narinfos id=9 count=15004lb # [ 58.226337] niks3-server[1108]: 2026/09/21 15:08:12 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5005lb # [ 58.236602] niks3-server[1108]: 2026/09/21 15:08:12 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete5006lb # [ 58.242618] niks3-server[1108]: 2026/09/21 15:08:12 INFO Completed upload id=95007worker2 # [ 58.240165] nix-grpc-daemon[882]: time=2026-09-21T15:08:13.136Z level=INFO msg="Upload complete. (553ms)"5008lb # [ 58.244825] niks3-server[1108]: 2026/09/21 15:08:12 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5009worker2 # [ 58.242511] nix-grpc-daemon[786]: ts=2026-09-21T15:08:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=0 paths=1 nar_bytes_in=4325010worker2 # [ 58.247062] nix-grpc-daemon[786]: ts=2026-09-21T15:08:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467025011worker2 # [ 58.255560] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5012worker2 # [ 58.257974] nix-daemon[857]: reaped child process 880, status = succeeded5013client # [ 58.272394] nix[895]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5014worker2 # [ 58.265778] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5015client # [ 58.384115] nix[911]: don't know how to build these paths:5016client # [ 58.385199] nix[911]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5017worker1 # [ 58.332824] nix-grpc-daemon[784]: ts=2026-09-21T15:08:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401965018worker1 # [ 58.345822] nix-grpc-daemon[784]: ts=2026-09-21T15:08:13Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05019worker2 # [ 58.393093] nix-grpc-daemon[786]: ts=2026-09-21T15:08:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467025020client # [ 58.406537] nix[911]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5021lb # [ 58.462920] envoy[979]: conn peer=[2001:db8:1::1]:44918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9567 tx=4092 ms=69545022lb # [ 58.468951] envoy[979]: conn peer=[2001:db8:1::1]:57844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=635023lb # [ 58.472728] envoy[979]: conn peer=[2001:db8:1::1]:57854 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1783 ms=24275024client: must succeed: sleep 25025client: (finished: must succeed: sleep 2, in 2.05 seconds)5026client: waiting for success: ! systemctl is-active dedup1 dedup25027worker2 # [ 62.883551] nix-daemon[857]: reaped child process 911, status = succeeded5028worker2 # [ 63.087425] nix-daemon[857]: accepted connection from pid 941, user nix-grpc-daemon (trusted)5029worker2 # [ 63.100201] nix-daemon[857]: reaped child process 944, status = succeeded5030lb # [ 63.124386] niks3-server[1108]: 2026/09/21 15:08:17 INFO Received uploads request method=POST path=/api/pending_closures5031worker2 # [ 63.131953] nix-grpc-daemon[882]: time=2026-09-21T15:08:18.028Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5032worker2 # [ 63.133724] nix-grpc-daemon[882]: time=2026-09-21T15:08:18.030Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5033lb # [ 63.169634] niks3-server[1108]: 2026/09/21 15:08:17 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5034lb # [ 63.180725] niks3-server[1108]: 2026/09/21 15:08:17 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5035lb # [ 63.189290] niks3-server[1108]: 2026/09/21 15:08:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign5036lb # [ 63.194665] niks3-server[1108]: 2026/09/21 15:08:17 INFO Signed narinfos id=10 count=15037worker2 # [ 63.194884] nix-grpc-daemon[882]: time=2026-09-21T15:08:18.090Z level=INFO msg="Uploading 1 narinfos"5038lb # [ 63.197587] niks3-server[1108]: 2026/09/21 15:08:17 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5039lb # [ 63.207847] niks3-server[1108]: 2026/09/21 15:08:17 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete5040lb # [ 63.214111] niks3-server[1108]: 2026/09/21 15:08:17 INFO Completed upload id=105041worker2 # [ 63.211476] nix-grpc-daemon[882]: time=2026-09-21T15:08:18.108Z level=INFO msg="Upload complete. (332ms)"5042lb # [ 63.216587] niks3-server[1108]: 2026/09/21 15:08:17 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5043worker2 # [ 63.217821] nix-grpc-daemon[786]: ts=2026-09-21T15:08:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=15044worker2 # [ 63.223226] nix-grpc-daemon[786]: ts=2026-09-21T15:08:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=15045worker2 # [ 63.233769] nix-daemon[857]: reaped child process 909, status = succeeded5046client # [ 63.267451] systemd[1]: dedup1.service: Deactivated successfully.5047client # [ 63.273474] systemd[1]: dedup1.service: Consumed 119ms CPU time over 6.215s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.5K outgoing IP traffic.5048client # [ 63.311189] systemd[1]: dedup2.service: Deactivated successfully.5049client # [ 63.312599] systemd[1]: dedup2.service: Consumed 141ms CPU time over 5.181s wall clock time, 35.3M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic.5050client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)5051client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5052client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)5053worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5054worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)5055worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5056worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)5057worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5058worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5059worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5060worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5061(finished: subtest: two clients wanting the same drv build it once, in 7.24 seconds)5062subtest: build hook: nix-daemon with builders = grpc://lb5063client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log5064client # [ 64.444069] systemd[1]: Started Nix Daemon.5065client # [ 64.622051] nix-daemon[953]: accepted connection from pid 950, user root (trusted)5066worker1 # [ 64.765664] nix-grpc-daemon[784]: ts=2026-09-21T15:08:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401965067worker2 # [ 64.811199] nix-grpc-daemon[786]: ts=2026-09-21T15:08:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467025068worker1 # [ 64.785231] nix-grpc-daemon[784]: ts=2026-09-21T15:08:19Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05069worker1 # [ 64.789207] nix-grpc-daemon[784]: ts=2026-09-21T15:08:19Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15070worker1 # [ 64.837702] nix-grpc-daemon[784]: ts=2026-09-21T15:08:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401965071worker1 # [ 64.850415] nix-grpc-daemon[784]: ts=2026-09-21T15:08:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401965072worker1 # [ 64.861824] nix-grpc-daemon[784]: ts=2026-09-21T15:08:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401965073worker1 # [ 64.870869] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5074worker1 # [ 65.097125] nix-daemon[868]: accepted connection from pid 1101, user nix-grpc-daemon (trusted)5075worker1 # [ 65.110090] nix-daemon[868]: reaped child process 1104, status = succeeded5076worker1 # [ 65.317697] nix-daemon[868]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5077worker1 # [ 65.329968] nix-daemon[868]: reaped child process 1109, status = succeeded5078lb # [ 65.394257] niks3-server[1108]: 2026/09/21 15:08:19 INFO Received uploads request method=POST path=/api/pending_closures5079worker1 # [ 65.358252] nix-grpc-daemon[883]: time=2026-09-21T15:08:20.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5080worker1 # [ 65.359918] nix-grpc-daemon[883]: time=2026-09-21T15:08:20.202Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5081lb # [ 65.425160] niks3-server[1108]: 2026/09/21 15:08:19 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5082lb # [ 65.437551] niks3-server[1108]: 2026/09/21 15:08:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5083worker1 # [ 65.398357] nix-grpc-daemon[883]: time=2026-09-21T15:08:20.240Z level=INFO msg="Uploading 1 narinfos"5084lb # [ 65.442373] niks3-server[1108]: 2026/09/21 15:08:19 INFO Signed narinfos id=11 count=15085lb # [ 65.448585] niks3-server[1108]: 2026/09/21 15:08:19 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5086lb # [ 65.459662] niks3-server[1108]: 2026/09/21 15:08:19 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5087lb # [ 65.466947] niks3-server[1108]: 2026/09/21 15:08:19 INFO Completed upload id=115088worker1 # [ 65.423160] nix-grpc-daemon[883]: time=2026-09-21T15:08:20.265Z level=INFO msg="Upload complete. (538ms)"5089lb # [ 65.469293] niks3-server[1108]: 2026/09/21 15:08:19 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5090worker1 # [ 65.427126] nix-grpc-daemon[784]: ts=2026-09-21T15:08:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_in=5125091worker1 # [ 65.432558] nix-grpc-daemon[784]: ts=2026-09-21T15:08:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401965092worker1 # [ 65.442202] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5093worker1 # [ 65.448968] nix-daemon[868]: reaped child process 1099, status = succeeded5094worker1 # [ 65.454176] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5095worker1 # [ 67.079662] nix-daemon[868]: reaped child process 1113, status = succeeded5096worker1 # [ 67.283900] nix-daemon[868]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)5097worker1 # [ 67.294893] nix-daemon[868]: reaped child process 1132, status = succeeded5098lb # [ 67.360533] niks3-server[1108]: 2026/09/21 15:08:21 INFO Received uploads request method=POST path=/api/pending_closures5099worker1 # [ 67.324823] nix-grpc-daemon[883]: time=2026-09-21T15:08:22.167Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5100worker1 # [ 67.326298] nix-grpc-daemon[883]: time=2026-09-21T15:08:22.168Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5101lb # [ 67.399240] niks3-server[1108]: 2026/09/21 15:08:21 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5102lb # [ 67.409280] niks3-server[1108]: 2026/09/21 15:08:21 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5103lb # [ 67.415436] niks3-server[1108]: 2026/09/21 15:08:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5104worker1 # [ 67.376193] nix-grpc-daemon[883]: time=2026-09-21T15:08:22.218Z level=INFO msg="Uploading 1 narinfos"5105lb # [ 67.420330] niks3-server[1108]: 2026/09/21 15:08:21 INFO Signed narinfos id=12 count=15106lb # [ 67.431178] niks3-server[1108]: 2026/09/21 15:08:21 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5107lb # [ 67.463118] niks3-server[1108]: 2026/09/21 15:08:21 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5108lb # [ 67.468800] niks3-server[1108]: 2026/09/21 15:08:21 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5109lb # [ 67.473365] niks3-server[1108]: 2026/09/21 15:08:21 INFO Completed upload id=125110worker1 # [ 67.429598] nix-grpc-daemon[883]: time=2026-09-21T15:08:22.272Z level=INFO msg="Upload complete. (365ms)"5111worker1 # [ 67.436741] nix-grpc-daemon[784]: ts=2026-09-21T15:08:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=2 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=15112worker2 # [ 67.481465] nix-grpc-daemon[786]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467025113worker1 # [ 67.450401] nix-daemon[868]: reaped child process 1111, status = succeeded5114worker2 # [ 67.501753] nix-grpc-daemon[786]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5115worker2 # [ 67.531558] nix-grpc-daemon[786]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467025116worker2 # [ 67.537518] nix-grpc-daemon[786]: ts=2026-09-21T15:08:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=0 paths=1 nar_bytes_out=1285117worker1 # [ 67.736804] nix-grpc-daemon[784]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401965118worker1 # [ 67.741555] nix-grpc-daemon[784]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401965119worker1 # [ 67.750304] nix-grpc-daemon[883]: time=2026-09-21T15:08:22.592Z level=INFO msg="All 1 paths already cached"5120worker1 # [ 67.764523] nix-grpc-daemon[784]: ts=2026-09-21T15:08:22Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05121worker1 # [ 67.770487] nix-grpc-daemon[784]: ts=2026-09-21T15:08:22Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15122worker1 # [ 67.775448] nix-grpc-daemon[784]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401965123worker1 # [ 67.787067] nix-grpc-daemon[784]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401965124worker1 # [ 67.799065] nix-grpc-daemon[883]: time=2026-09-21T15:08:22.641Z level=INFO msg="All 1 paths already cached"5125worker1 # [ 67.805821] nix-grpc-daemon[784]: ts=2026-09-21T15:08:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401965126worker1 # [ 67.815155] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5127worker1 # [ 68.039294] nix-daemon[868]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)5128worker1 # [ 68.052289] nix-daemon[868]: reaped child process 1141, status = succeeded5129worker1 # [ 68.262640] nix-daemon[868]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)5130worker1 # [ 68.274622] nix-daemon[868]: reaped child process 1146, status = succeeded5131lb # [ 68.338642] niks3-server[1108]: 2026/09/21 15:08:22 INFO Received uploads request method=POST path=/api/pending_closures5132worker1 # [ 68.302762] nix-grpc-daemon[883]: time=2026-09-21T15:08:23.145Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5133worker1 # [ 68.304157] nix-grpc-daemon[883]: time=2026-09-21T15:08:23.145Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5134lb # [ 68.364186] niks3-server[1108]: 2026/09/21 15:08:22 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5135lb # [ 68.375872] niks3-server[1108]: 2026/09/21 15:08:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5136worker1 # [ 68.336910] nix-grpc-daemon[883]: time=2026-09-21T15:08:23.179Z level=INFO msg="Uploading 1 narinfos"5137lb # [ 68.380900] niks3-server[1108]: 2026/09/21 15:08:22 INFO Signed narinfos id=13 count=15138lb # [ 68.387059] niks3-server[1108]: 2026/09/21 15:08:22 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5139lb # [ 68.399292] niks3-server[1108]: 2026/09/21 15:08:22 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5140lb # [ 68.404102] niks3-server[1108]: 2026/09/21 15:08:22 INFO Completed upload id=135141worker1 # [ 68.360168] nix-grpc-daemon[883]: time=2026-09-21T15:08:23.202Z level=INFO msg="Upload complete. (533ms)"5142lb # [ 68.406357] niks3-server[1108]: 2026/09/21 15:08:22 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5143worker1 # [ 68.363310] nix-grpc-daemon[784]: ts=2026-09-21T15:08:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_in=6245144worker1 # [ 68.368797] nix-grpc-daemon[784]: ts=2026-09-21T15:08:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401965145worker1 # [ 68.378075] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5146worker1 # [ 68.383232] nix-daemon[868]: reaped child process 1136, status = succeeded5147worker1 # [ 68.387177] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5148lb # [ 68.462419] envoy[979]: conn peer=[2001:db8:1::1]:57860 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3011 tx=1451 ms=59785149lb # [ 68.466104] envoy[979]: conn peer=[2001:db8:1::1]:57862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=49165150lb # [ 68.469762] envoy[979]: conn peer=[2001:db8:1::1]:43962 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3421 tx=1852 ms=27835151lb # [ 68.473404] envoy[979]: conn peer=[2001:db8:1::1]:43976 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=333 ms=565152worker1 # [ 69.410438] nix-daemon[868]: reaped child process 1150, status = succeeded5153worker1 # [ 69.619675] nix-daemon[868]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)5154worker1 # [ 69.630302] nix-daemon[868]: reaped child process 1169, status = succeeded5155lb # [ 69.695557] niks3-server[1108]: 2026/09/21 15:08:24 INFO Received uploads request method=POST path=/api/pending_closures5156worker1 # [ 69.660256] nix-grpc-daemon[883]: time=2026-09-21T15:08:24.502Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5157worker1 # [ 69.662866] nix-grpc-daemon[883]: time=2026-09-21T15:08:24.504Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5158lb # [ 69.727671] niks3-server[1108]: 2026/09/21 15:08:24 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5159lb # [ 69.738518] niks3-server[1108]: 2026/09/21 15:08:24 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5160lb # [ 69.750087] niks3-server[1108]: 2026/09/21 15:08:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5161worker1 # [ 69.710994] nix-grpc-daemon[883]: time=2026-09-21T15:08:24.553Z level=INFO msg="Uploading 1 narinfos"5162lb # [ 69.754901] niks3-server[1108]: 2026/09/21 15:08:24 INFO Signed narinfos id=14 count=15163lb # [ 69.760314] niks3-server[1108]: 2026/09/21 15:08:24 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5164lb # [ 69.770987] niks3-server[1108]: 2026/09/21 15:08:24 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5165lb # [ 69.775134] niks3-server[1108]: 2026/09/21 15:08:24 INFO Completed upload id=145166lb # [ 69.776977] niks3-server[1108]: 2026/09/21 15:08:24 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5167worker1 # [ 69.732601] nix-grpc-daemon[883]: time=2026-09-21T15:08:24.575Z level=INFO msg="Upload complete. (337ms)"5168worker1 # [ 69.738506] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=15169worker1 # [ 69.752886] nix-daemon[868]: reaped child process 1148, status = succeeded5170worker2 # [ 69.832806] nix-grpc-daemon[786]: ts=2026-09-21T15:08:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467025171worker2 # [ 69.844772] nix-grpc-daemon[786]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5172worker1 # [ 69.831149] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:401965173worker1 # [ 69.837262] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_out=1765174worker1 # [ 70.068873] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401965175worker1 # [ 70.075156] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401965176worker1 # [ 70.080278] nix-grpc-daemon[883]: time=2026-09-21T15:08:24.921Z level=INFO msg="All 2 paths already cached"5177worker1 # [ 70.090226] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05178worker1 # [ 70.094161] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15179worker1 # [ 70.101802] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401965180worker1 # [ 70.113671] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401965181worker1 # [ 70.126000] nix-grpc-daemon[883]: time=2026-09-21T15:08:24.968Z level=INFO msg="All 2 paths already cached"5182worker1 # [ 70.132602] nix-grpc-daemon[784]: ts=2026-09-21T15:08:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401965183worker1 # [ 70.142158] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5184worker1 # [ 70.362625] nix-daemon[868]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)5185worker1 # [ 70.375097] nix-daemon[868]: reaped child process 1178, status = succeeded5186worker1 # [ 70.593769] nix-daemon[868]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)5187worker1 # [ 70.605538] nix-daemon[868]: reaped child process 1183, status = succeeded5188lb # [ 70.671317] niks3-server[1108]: 2026/09/21 15:08:25 INFO Received uploads request method=POST path=/api/pending_closures5189worker1 # [ 70.635749] nix-grpc-daemon[883]: time=2026-09-21T15:08:25.478Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5190worker1 # [ 70.637259] nix-grpc-daemon[883]: time=2026-09-21T15:08:25.479Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5191lb # [ 70.702601] niks3-server[1108]: 2026/09/21 15:08:25 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5192lb # [ 70.715174] niks3-server[1108]: 2026/09/21 15:08:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5193worker1 # [ 70.677069] nix-grpc-daemon[883]: time=2026-09-21T15:08:25.519Z level=INFO msg="Uploading 1 narinfos"5194lb # [ 70.720267] niks3-server[1108]: 2026/09/21 15:08:25 INFO Signed narinfos id=15 count=15195lb # [ 70.727766] niks3-server[1108]: 2026/09/21 15:08:25 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5196lb # [ 70.735121] niks3-server[1108]: 2026/09/21 15:08:25 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5197lb # [ 70.740932] niks3-server[1108]: 2026/09/21 15:08:25 INFO Completed upload id=155198worker1 # [ 70.699549] nix-grpc-daemon[883]: time=2026-09-21T15:08:25.542Z level=INFO msg="Upload complete. (545ms)"5199lb # [ 70.745879] niks3-server[1108]: 2026/09/21 15:08:25 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5200worker1 # [ 70.703433] nix-grpc-daemon[784]: ts=2026-09-21T15:08:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_in=7525201worker1 # [ 70.709104] nix-grpc-daemon[784]: ts=2026-09-21T15:08:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401965202worker1 # [ 70.718480] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5203worker1 # [ 70.723502] nix-daemon[868]: reaped child process 1173, status = succeeded5204worker1 # [ 70.727994] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5205worker1 # [ 71.339869] nix-daemon[868]: reaped child process 1187, status = succeeded5206worker1 # [ 71.528414] nix-daemon[868]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5207worker1 # [ 71.540820] nix-daemon[868]: reaped child process 1206, status = succeeded5208lb # [ 71.604368] niks3-server[1108]: 2026/09/21 15:08:26 INFO Received uploads request method=POST path=/api/pending_closures5209worker1 # [ 71.568911] nix-grpc-daemon[883]: time=2026-09-21T15:08:26.411Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5210worker1 # [ 71.570377] nix-grpc-daemon[883]: time=2026-09-21T15:08:26.412Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5211lb # [ 71.651704] niks3-server[1108]: 2026/09/21 15:08:26 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5212lb # [ 71.660406] niks3-server[1108]: 2026/09/21 15:08:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5213lb # [ 71.664477] niks3-server[1108]: 2026/09/21 15:08:26 INFO Signed narinfos id=16 count=15214worker1 # [ 71.624327] nix-grpc-daemon[883]: time=2026-09-21T15:08:26.466Z level=INFO msg="Uploading 1 narinfos"5215lb # [ 71.671591] niks3-server[1108]: 2026/09/21 15:08:26 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5216lb # [ 71.677785] niks3-server[1108]: 2026/09/21 15:08:26 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5217lb # [ 71.687151] niks3-server[1108]: 2026/09/21 15:08:26 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5218lb # [ 71.690413] niks3-server[1108]: 2026/09/21 15:08:26 INFO Completed upload id=165219lb # [ 71.692560] niks3-server[1108]: 2026/09/21 15:08:26 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5220worker1 # [ 71.648323] nix-grpc-daemon[883]: time=2026-09-21T15:08:26.490Z level=INFO msg="Upload complete. (314ms)"5221worker1 # [ 71.655599] nix-grpc-daemon[784]: ts=2026-09-21T15:08:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15222worker2 # [ 71.701302] nix-grpc-daemon[786]: ts=2026-09-21T15:08:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467025223worker1 # [ 71.669176] nix-daemon[868]: reaped child process 1185, status = succeeded5224worker2 # [ 71.709699] nix-grpc-daemon[786]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5225worker1 # [ 71.693179] nix-grpc-daemon[784]: ts=2026-09-21T15:08:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:401965226worker1 # [ 71.697570] nix-grpc-daemon[784]: ts=2026-09-21T15:08:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_out=2325227client # [ 71.859417] nix-daemon[953]: reaped child process 955, status = succeeded5228client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.63 seconds)5229client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5230client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5231client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5232client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)5233client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5234client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5235(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.72 seconds)5236subtest: an input only one worker has still reaches the builder5237client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5238client # this derivation will be built:5239client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5240client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5241client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.39 seconds)5242client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5243client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds)5244worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5245worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.27 seconds)5246worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5247worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5248client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25249client # [ 73.039448] nix-daemon[953]: accepted connection from pid 1034, user root (trusted)5250client # this derivation will be built:5251client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5252worker1 # [ 73.189841] nix-grpc-daemon[784]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401965253client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5254worker2 # [ 73.234347] nix-grpc-daemon[786]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467025255client # copying 1 paths...5256client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5257worker2 # [ 73.246692] nix-grpc-daemon[786]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467025258worker2 # [ 73.259494] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5259worker2 # [ 73.486371] nix-daemon[857]: accepted connection from pid 963, user nix-grpc-daemon (trusted)5260worker2 # [ 73.499883] nix-daemon[857]: reaped child process 966, status = succeeded5261lb # [ 73.523819] niks3-server[1108]: 2026/09/21 15:08:27 INFO Received uploads request method=POST path=/api/pending_closures5262worker2 # [ 73.529393] nix-grpc-daemon[882]: time=2026-09-21T15:08:28.425Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5263worker2 # [ 73.531149] nix-grpc-daemon[882]: time=2026-09-21T15:08:28.427Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5264lb # [ 73.552952] niks3-server[1108]: 2026/09/21 15:08:27 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5265lb # [ 73.564688] niks3-server[1108]: 2026/09/21 15:08:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5266worker2 # [ 73.566231] nix-grpc-daemon[882]: time=2026-09-21T15:08:28.462Z level=INFO msg="Uploading 1 narinfos"5267lb # [ 73.568900] niks3-server[1108]: 2026/09/21 15:08:27 INFO Signed narinfos id=17 count=15268lb # [ 73.575506] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5269lb # [ 73.584948] niks3-server[1108]: 2026/09/21 15:08:28 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5270lb # [ 73.591759] niks3-server[1108]: 2026/09/21 15:08:28 INFO Completed upload id=175271worker2 # [ 73.589100] nix-grpc-daemon[882]: time=2026-09-21T15:08:28.485Z level=INFO msg="Upload complete. (318ms)"5272lb # [ 73.594490] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5273worker2 # [ 73.592589] nix-grpc-daemon[786]: ts=2026-09-21T15:08:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=0 paths=1 nar_bytes_in=1205274worker2 # [ 73.603395] nix-daemon[857]: reaped child process 961, status = succeeded5275worker1 # [ 73.568243] nix-grpc-daemon[784]: ts=2026-09-21T15:08:28Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05276worker1 # [ 73.572481] nix-grpc-daemon[784]: ts=2026-09-21T15:08:28Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15277worker2 # [ 73.621772] nix-grpc-daemon[786]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467025278worker2 # [ 73.632679] nix-grpc-daemon[786]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467025279client # copying 2 paths to 'lb:50051'...5280client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5281worker2 # [ 73.646505] nix-grpc-daemon[786]: ts=2026-09-21T15:08:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467025282client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5283worker2 # [ 73.657282] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5284worker2 # [ 73.861151] nix-daemon[857]: accepted connection from pid 970, user nix-grpc-daemon (trusted)5285worker2 # [ 73.873802] nix-daemon[857]: reaped child process 973, status = succeeded5286worker2 # [ 74.085863] nix-daemon[857]: accepted connection from pid 975, user nix-grpc-daemon (trusted)5287worker2 # [ 74.097479] nix-daemon[857]: reaped child process 978, status = succeeded5288lb # [ 74.122202] niks3-server[1108]: 2026/09/21 15:08:28 INFO Received uploads request method=POST path=/api/pending_closures5289lb # [ 74.131486] niks3-server[1108]: 2026/09/21 15:08:28 INFO Received uploads request method=POST path=/api/pending_closures5290worker2 # [ 74.136618] nix-grpc-daemon[882]: time=2026-09-21T15:08:29.033Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5291worker2 # [ 74.138879] nix-grpc-daemon[882]: time=2026-09-21T15:08:29.034Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5292worker2 # [ 74.140522] nix-grpc-daemon[882]: time=2026-09-21T15:08:29.035Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5293lb # [ 74.165981] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5294lb # [ 74.171295] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5295lb # [ 74.188085] niks3-server[1108]: 2026/09/21 15:08:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5296lb # [ 74.192480] niks3-server[1108]: 2026/09/21 15:08:28 INFO Signed narinfos id=18 count=15297lb # [ 74.197468] niks3-server[1108]: 2026/09/21 15:08:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5298lb # [ 74.199883] niks3-server[1108]: 2026/09/21 15:08:28 INFO Signed narinfos id=19 count=25299lb # [ 74.201481] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5300worker2 # [ 74.199295] nix-grpc-daemon[882]: time=2026-09-21T15:08:29.095Z level=INFO msg="Uploading 3 narinfos"5301lb # [ 74.203650] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5302lb # [ 74.217803] niks3-server[1108]: 2026/09/21 15:08:28 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5303lb # [ 74.220438] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5304lb # [ 74.223784] niks3-server[1108]: 2026/09/21 15:08:28 INFO Completed upload id=185305lb # [ 74.225130] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5306lb # [ 74.226869] niks3-server[1108]: 2026/09/21 15:08:28 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5307lb # [ 74.229225] niks3-server[1108]: 2026/09/21 15:08:28 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5308lb # [ 74.231507] niks3-server[1108]: 2026/09/21 15:08:28 INFO Completed upload id=195309worker2 # [ 74.228976] nix-grpc-daemon[882]: time=2026-09-21T15:08:29.125Z level=INFO msg="Upload complete. (558ms)"5310worker2 # [ 74.235465] nix-grpc-daemon[786]: ts=2026-09-21T15:08:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=0 paths=2 nar_bytes_in=9765311worker2 # [ 74.241629] nix-grpc-daemon[786]: ts=2026-09-21T15:08:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467025312worker2 # [ 74.250709] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5313worker2 # [ 74.258585] nix-daemon[857]: reaped child process 968, status = succeeded5314client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5315worker2 # [ 74.264752] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5316client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5317client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5318worker2 # [ 75.077182] nix-daemon[857]: reaped child process 982, status = succeeded5319worker2 # [ 75.260311] nix-daemon[857]: accepted connection from pid 998, user nix-grpc-daemon (trusted)5320worker2 # [ 75.270824] nix-daemon[857]: reaped child process 1001, status = succeeded5321lb # [ 75.294164] niks3-server[1108]: 2026/09/21 15:08:29 INFO Received uploads request method=POST path=/api/pending_closures5322worker2 # [ 75.300749] nix-grpc-daemon[882]: time=2026-09-21T15:08:30.197Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5323lb # [ 75.322899] niks3-server[1108]: 2026/09/21 15:08:29 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5324lb # [ 75.330893] niks3-server[1108]: 2026/09/21 15:08:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5325worker2 # [ 75.332844] nix-grpc-daemon[882]: time=2026-09-21T15:08:30.229Z level=INFO msg="Uploading 1 narinfos"5326lb # [ 75.335694] niks3-server[1108]: 2026/09/21 15:08:29 INFO Signed narinfos id=20 count=15327lb # [ 75.345755] niks3-server[1108]: 2026/09/21 15:08:29 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5328lb # [ 75.353434] niks3-server[1108]: 2026/09/21 15:08:29 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5329lb # [ 75.359280] niks3-server[1108]: 2026/09/21 15:08:29 INFO Completed upload id=205330lb # [ 75.360441] niks3-server[1108]: 2026/09/21 15:08:29 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5331worker2 # [ 75.356814] nix-grpc-daemon[882]: time=2026-09-21T15:08:30.253Z level=INFO msg="Upload complete. (286ms)"5332client # copying 1 paths...5333worker2 # [ 75.361610] nix-grpc-daemon[786]: ts=2026-09-21T15:08:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15334worker2 # [ 75.368322] nix-grpc-daemon[786]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467025335client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5336worker2 # [ 75.376398] nix-daemon[857]: reaped child process 980, status = succeeded5337worker2 # [ 75.382290] nix-grpc-daemon[786]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467025338worker2 # [ 75.384261] nix-grpc-daemon[786]: ts=2026-09-21T15:08:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=0 paths=1 nar_bytes_out=1205339client # [ 75.499113] nix-daemon[953]: reaped child process 1037, status = succeeded5340client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.68 seconds)5341client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25342client # [ 75.679081] nix-daemon[953]: accepted connection from pid 1056, user root (trusted)5343client # this derivation will be built:5344client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5345worker1 # [ 75.822320] nix-grpc-daemon[784]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401965346client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5347worker2 # [ 75.870786] nix-grpc-daemon[786]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467025348worker2 # [ 75.880944] nix-grpc-daemon[882]: time=2026-09-21T15:08:30.777Z level=INFO msg="All 1 paths already cached"5349worker1 # [ 75.854413] nix-grpc-daemon[784]: ts=2026-09-21T15:08:30Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05350worker1 # [ 75.860165] nix-grpc-daemon[784]: ts=2026-09-21T15:08:30Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15351worker1 # [ 75.865768] nix-grpc-daemon[784]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401965352worker1 # [ 75.877693] nix-grpc-daemon[784]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401965353client # copying 1 paths to 'lb:50051'...5354client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5355worker1 # [ 75.895529] nix-grpc-daemon[784]: ts=2026-09-21T15:08:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401965356worker1 # [ 75.906868] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5357worker1 # [ 75.922757] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 1/5)5358worker1 # [ 76.013582] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 2/5)5359worker1 # [ 76.200973] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 249 ms (attempt 3/5)5360worker1 # [ 76.450341] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 365 ms (attempt 4/5)5361worker1 # [ 76.816091] nix-grpc-daemon[784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5362worker1 # [ 76.818978] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5363worker1 # [ 76.825679] nix-grpc-daemon[784]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5364worker1 # [ 77.058636] nix-daemon[868]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5365worker1 # [ 77.070394] nix-daemon[868]: reaped child process 1227, status = succeeded5366worker1 # [ 77.261085] nix-daemon[868]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5367worker1 # [ 77.272793] nix-daemon[868]: reaped child process 1232, status = succeeded5368lb # [ 77.335545] niks3-server[1108]: 2026/09/21 15:08:31 INFO Received uploads request method=POST path=/api/pending_closures5369worker1 # [ 77.301510] nix-grpc-daemon[883]: time=2026-09-21T15:08:32.143Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5370worker1 # [ 77.302985] nix-grpc-daemon[883]: time=2026-09-21T15:08:32.143Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5371lb # [ 77.363854] niks3-server[1108]: 2026/09/21 15:08:31 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5372lb # [ 77.375777] niks3-server[1108]: 2026/09/21 15:08:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5373worker1 # [ 77.338929] nix-grpc-daemon[883]: time=2026-09-21T15:08:32.181Z level=INFO msg="Uploading 1 narinfos"5374lb # [ 77.382665] niks3-server[1108]: 2026/09/21 15:08:31 INFO Signed narinfos id=21 count=15375lb # [ 77.387446] niks3-server[1108]: 2026/09/21 15:08:31 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5376lb # [ 77.397389] niks3-server[1108]: 2026/09/21 15:08:31 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5377lb # [ 77.399863] niks3-server[1108]: 2026/09/21 15:08:31 INFO Completed upload id=215378worker1 # [ 77.357265] nix-grpc-daemon[883]: time=2026-09-21T15:08:32.199Z level=INFO msg="Upload complete. (507ms)"5379lb # [ 77.401900] niks3-server[1108]: 2026/09/21 15:08:31 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5380worker1 # [ 77.360218] nix-grpc-daemon[784]: ts=2026-09-21T15:08:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=1 paths=1 nar_bytes_in=5605381worker1 # [ 77.364834] nix-grpc-daemon[784]: ts=2026-09-21T15:08:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401965382worker1 # [ 77.375334] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5383worker1 # [ 77.383303] nix-daemon[868]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5384client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5385worker1 # [ 77.385385] nix-daemon[868]: reaped child process 1217, status = succeeded5386client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5387client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5388worker1 # [ 78.408891] nix-daemon[868]: reaped child process 1236, status = succeeded5389lb # [ 78.462852] envoy[979]: conn peer=[2001:db8:1::1]:43992 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=2067 ms=21125390lb # [ 78.466670] envoy[979]: conn peer=[2001:db8:1::1]:44008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=195391lb # [ 78.470137] envoy[979]: conn peer=[2001:db8:1::1]:44024 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4314 tx=2276 ms=16825392lb # [ 78.473599] envoy[979]: conn peer=[2001:db8:1::1]:44030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=575393lb # [ 78.476936] envoy[979]: conn peer=[2001:db8:1::1]:35226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=525394lb # [ 78.480331] envoy[979]: conn peer=[2001:db8:1::1]:35216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4244 tx=1848 ms=22075395worker1 # [ 78.602315] nix-daemon[868]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5396worker1 # [ 78.613089] nix-daemon[868]: reaped child process 1255, status = succeeded5397lb # [ 78.677233] niks3-server[1108]: 2026/09/21 15:08:33 INFO Received uploads request method=POST path=/api/pending_closures5398worker1 # [ 78.641449] nix-grpc-daemon[883]: time=2026-09-21T15:08:33.483Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5399lb # [ 78.706684] niks3-server[1108]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5400lb # [ 78.715699] niks3-server[1108]: 2026/09/21 15:08:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5401lb # [ 78.721075] niks3-server[1108]: 2026/09/21 15:08:33 INFO Signed narinfos id=22 count=15402worker1 # [ 78.680320] nix-grpc-daemon[883]: time=2026-09-21T15:08:33.522Z level=INFO msg="Uploading 1 narinfos"5403lb # [ 78.727369] niks3-server[1108]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5404lb # [ 78.738288] niks3-server[1108]: 2026/09/21 15:08:33 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5405lb # [ 78.744681] niks3-server[1108]: 2026/09/21 15:08:33 INFO Completed upload id=225406worker1 # [ 78.703167] nix-grpc-daemon[883]: time=2026-09-21T15:08:33.545Z level=INFO msg="Upload complete. (311ms)"5407lb # [ 78.747559] niks3-server[1108]: 2026/09/21 15:08:33 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5408worker1 # [ 78.708800] nix-grpc-daemon[784]: ts=2026-09-21T15:08:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15409client # copying 1 paths...5410worker1 # [ 78.720944] nix-daemon[868]: reaped child process 1234, status = succeeded5411worker2 # [ 78.759970] nix-grpc-daemon[786]: ts=2026-09-21T15:08:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467025412worker2 # [ 78.770571] nix-grpc-daemon[786]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5413client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5414worker1 # [ 78.759366] nix-grpc-daemon[784]: ts=2026-09-21T15:08:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:401965415worker1 # [ 78.764622] nix-grpc-daemon[784]: ts=2026-09-21T15:08:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_out=1205416client # [ 78.878584] nix-daemon[953]: reaped child process 1059, status = succeeded5417client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.38 seconds)5418client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25419client # [ 79.051432] nix-daemon[953]: accepted connection from pid 1078, user root (trusted)5420client # this derivation will be built:5421client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5422worker1 # [ 79.172923] nix-grpc-daemon[784]: ts=2026-09-21T15:08:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:401965423client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5424worker2 # [ 79.217629] nix-grpc-daemon[786]: ts=2026-09-21T15:08:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467025425worker2 # [ 79.227706] nix-grpc-daemon[882]: time=2026-09-21T15:08:34.123Z level=INFO msg="All 1 paths already cached"5426worker1 # [ 79.201816] nix-grpc-daemon[784]: ts=2026-09-21T15:08:34Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05427worker2 # [ 79.245579] nix-grpc-daemon[786]: ts=2026-09-21T15:08:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467025428worker1 # [ 79.206468] nix-grpc-daemon[784]: ts=2026-09-21T15:08:34Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15429worker2 # [ 79.257406] nix-grpc-daemon[786]: ts=2026-09-21T15:08:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467025430client # copying 1 paths to 'lb:50051'...5431worker2 # [ 79.272218] nix-grpc-daemon[882]: time=2026-09-21T15:08:34.168Z level=INFO msg="All 1 paths already cached"5432client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5433worker2 # [ 79.277212] nix-grpc-daemon[786]: ts=2026-09-21T15:08:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467025434worker2 # [ 79.288500] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5435worker2 # [ 79.494709] nix-daemon[857]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)5436worker2 # [ 79.506867] nix-daemon[857]: reaped child process 1009, status = succeeded5437worker2 # [ 79.682860] nix-daemon[857]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)5438worker2 # [ 79.694511] nix-daemon[857]: reaped child process 1014, status = succeeded5439lb # [ 79.716364] niks3-server[1108]: 2026/09/21 15:08:34 INFO Received uploads request method=POST path=/api/pending_closures5440worker2 # [ 79.720872] nix-grpc-daemon[882]: time=2026-09-21T15:08:34.617Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5441worker2 # [ 79.722370] nix-grpc-daemon[882]: time=2026-09-21T15:08:34.619Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5442lb # [ 79.744775] niks3-server[1108]: 2026/09/21 15:08:34 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5443lb # [ 79.755126] niks3-server[1108]: 2026/09/21 15:08:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5444lb # [ 79.759672] niks3-server[1108]: 2026/09/21 15:08:34 INFO Signed narinfos id=23 count=15445worker2 # [ 79.759768] nix-grpc-daemon[882]: time=2026-09-21T15:08:34.656Z level=INFO msg="Uploading 1 narinfos"5446lb # [ 79.765827] niks3-server[1108]: 2026/09/21 15:08:34 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5447lb # [ 79.777657] niks3-server[1108]: 2026/09/21 15:08:34 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5448worker2 # [ 79.781765] nix-grpc-daemon[882]: time=2026-09-21T15:08:34.678Z level=INFO msg="Upload complete. (481ms)"5449lb # [ 79.784291] niks3-server[1108]: 2026/09/21 15:08:34 INFO Completed upload id=235450lb # [ 79.786709] niks3-server[1108]: 2026/09/21 15:08:34 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5451worker2 # [ 79.784770] nix-grpc-daemon[786]: ts=2026-09-21T15:08:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=0 paths=1 nar_bytes_in=5605452worker2 # [ 79.790867] nix-grpc-daemon[786]: ts=2026-09-21T15:08:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467025453worker2 # [ 79.800110] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5454worker2 # [ 79.805633] nix-daemon[857]: reaped child process 1004, status = succeeded5455worker2 # [ 79.809202] nix-daemon[857]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5456client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5457client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5458client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5459worker2 # [ 81.237410] nix-daemon[857]: reaped child process 1018, status = succeeded5460worker2 # [ 81.410792] nix-daemon[857]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5461worker2 # [ 81.422582] nix-daemon[857]: reaped child process 1037, status = succeeded5462lb # [ 81.444908] niks3-server[1108]: 2026/09/21 15:08:35 INFO Received uploads request method=POST path=/api/pending_closures5463worker2 # [ 81.449964] nix-grpc-daemon[882]: time=2026-09-21T15:08:36.346Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5464lb # [ 81.473129] niks3-server[1108]: 2026/09/21 15:08:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5465lb # [ 81.478177] niks3-server[1108]: 2026/09/21 15:08:35 INFO Signed narinfos id=24 count=15466worker2 # [ 81.478422] nix-grpc-daemon[882]: time=2026-09-21T15:08:36.375Z level=INFO msg="Uploading 1 narinfos"5467lb # [ 81.487723] niks3-server[1108]: 2026/09/21 15:08:35 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5468lb # [ 81.497192] niks3-server[1108]: 2026/09/21 15:08:35 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5469lb # [ 81.504196] niks3-server[1108]: 2026/09/21 15:08:35 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5470lb # [ 81.510518] niks3-server[1108]: 2026/09/21 15:08:35 INFO Completed upload id=245471worker2 # [ 81.507906] nix-grpc-daemon[882]: time=2026-09-21T15:08:36.404Z level=INFO msg="Upload complete. (290ms)"5472lb # [ 81.512711] niks3-server[1108]: 2026/09/21 15:08:35 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5473worker2 # [ 81.513506] nix-grpc-daemon[786]: ts=2026-09-21T15:08:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46702 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15474client # copying 1 paths...5475worker1 # [ 81.479272] nix-grpc-daemon[784]: ts=2026-09-21T15:08:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:401965476worker2 # [ 81.524314] nix-daemon[857]: reaped child process 1016, status = succeeded5477worker1 # [ 81.492928] nix-grpc-daemon[784]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5478client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5479worker1 # [ 81.523590] nix-grpc-daemon[784]: ts=2026-09-21T15:08:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:401965480worker1 # [ 81.528770] nix-grpc-daemon[784]: ts=2026-09-21T15:08:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40196 duration_s=0 paths=1 nar_bytes_out=1205481client # [ 81.683959] nix-daemon[953]: reaped child process 1081, status = succeeded5482client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.81 seconds)5483(finished: subtest: an input only one worker has still reaches the builder, in 9.74 seconds)5484subtest: upload whose reference the worker lacks is completed from the cache5485client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5486client # this derivation will be built:5487client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5488client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5489client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.40 seconds)5490client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5491client # this derivation will be built:5492client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5493client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5494client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5495client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5496client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5497worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5498worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.27 seconds)5499worker2 # [ 82.982440] systemd[1]: Stopping Nix worker-protocol over gRPC...5500worker2 # [ 83.939415] nix-grpc-daemon[786]: ts=2026-09-21T15:08:38Z level=info event=signal_exit5501worker2 # [ 85.212713] nix-grpc-daemon[786]: ts=2026-09-21T15:08:40Z level=info event=scheduler_restarting5502worker1 # [ 85.181788] nix-grpc-daemon[784]: ts=2026-09-21T15:08:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401905503worker1 # [ 85.185804] nix-grpc-daemon[784]: ts=2026-09-21T15:08:40Z level=info event=worker_gone addr=192.168.1.4:500515504worker2 # [ 85.231579] nix-grpc-daemon[786]: ts=2026-09-21T15:08:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5505worker2 # [ 85.252589] nix-daemon[857]: reaped child process 859, status = succeeded5506worker2 # [ 86.073557] nix-grpc-daemon[786]: E0000 00:00:1790003320.970105 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5507worker2 # [ 86.078376] nix-grpc-daemon[786]: E0000 00:00:1790003320.970240 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5508worker2 # [ 86.291896] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5509worker2 # [ 86.295741] systemd[1]: Stopped Nix worker-protocol over gRPC.5510worker2 # [ 86.298074] systemd[1]: nix-grpc-daemon.service: Consumed 1.580s CPU time over 1min 7.702s wall clock time, 50.4M memory peak, 60.5K incoming IP traffic, 180.7K outgoing IP traffic.5511worker2 # [ 86.308810] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5512worker2 # [ 86.313415] systemd[1]: Closed Nix worker-protocol over gRPC.5513worker2 # [ 86.315862] systemd[1]: nix-grpc-daemon.socket: Consumed 289us CPU time over 1min 16.698s wall clock time, 256K memory peak, 65.5K incoming IP traffic, 58K outgoing IP traffic.5514lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5515lb: must succeed: curl -sf localhost:9901/clusters5516lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5517lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.09 seconds)5518client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25519worker1 # [ 86.614637] nix-grpc-daemon[784]: ts=2026-09-21T15:08:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401965520client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5521worker1 # [ 86.619937] nix-grpc-daemon[784]: ts=2026-09-21T15:08:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:401965522client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.28 seconds)5523worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5524worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5525worker2 # [ 86.777029] systemd[1]: Listening on Nix worker-protocol over gRPC.5526worker1 # [ 86.803641] systemd[1]: Stopping Nix worker-protocol over gRPC...5527worker2 # [ 87.274748] systemd[1]: Starting scheduler-token.service...5528worker2 # [ 87.371989] systemd[1]: scheduler-token.service: Deactivated successfully.5529worker2 # [ 87.375134] systemd[1]: Finished scheduler-token.service.5530worker2 # [ 87.377220] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 100ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5531worker2 # [ 87.382634] systemd[1]: Starting Nix worker-protocol over gRPC...5532worker2 # [ 87.550868] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:42Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5533worker2 # [ 87.552693] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5534worker2 # [ 87.556073] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15535worker2 # [ 87.564066] nix-grpc-daemon[1061]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5536worker2 # [ 87.565507] nix-grpc-daemon[1061]: E0000 00:00:1790003322.459826 1061 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5537worker2 # [ 87.571175] nix-grpc-daemon[1061]: E0000 00:00:1790003322.465202 1061 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5538worker2 # [ 87.573245] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5539worker2 # [ 87.574732] systemd[1]: Started Nix worker-protocol over gRPC.5540worker2 # [ 87.584106] nix-grpc-daemon[1061]: E0000 00:00:1790003322.480201 1102 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5541worker2 # [ 87.597056] nix-grpc-daemon[1061]: E0000 00:00:1790003322.493796 1102 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5542worker2 # [ 87.600197] nix-grpc-daemon[1061]: E0000 00:00:1790003322.495972 1102 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5543worker2 # [ 87.611287] nix-grpc-daemon[1061]: E0000 00:00:1790003322.508028 1102 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5544worker2 # [ 87.619996] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:42Z level=info event=scheduler_connected addr=lb:500515545worker1 # [ 87.580953] nix-grpc-daemon[784]: ts=2026-09-21T15:08:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:401905546worker1 # [ 87.585342] nix-grpc-daemon[784]: ts=2026-09-21T15:08:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05547worker1 # [ 87.783714] nix-grpc-daemon[784]: ts=2026-09-21T15:08:42Z level=info event=signal_exit5548lb # [ 88.096141] niks3-server[1108]: 2026/09/21 15:08:42 INFO lead: released remote=192.168.1.3:536585549lb # [ 88.464304] envoy[979]: conn peer=[2001:db8:1::1]:35232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2010 ms=29555550lb # [ 88.467915] envoy[979]: conn peer=[2001:db8:1::1]:35236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=205551lb # [ 88.471332] envoy[979]: conn peer=[2001:db8:1::1]:35266 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=565552lb # [ 88.474730] envoy[979]: conn peer=[2001:db8:1::1]:35250 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=1991 ms=24085553lb # [ 88.478234] envoy[979]: conn peer=[2001:db8:1::4]:35768 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=85913 tx=3792 ms=662295554lb # [ 88.481842] envoy[979]: conn peer=[2001:db8:1::1]:32930 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=155555worker1 # [ 89.050924] nix-grpc-daemon[784]: ts=2026-09-21T15:08:43Z level=info event=scheduler_restarting5556worker2 # [ 89.093690] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:43Z level=info event=scheduler_restarting addr=lb:500515557worker1 # [ 89.059834] nix-grpc-daemon[784]: ts=2026-09-21T15:08:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401905558worker1 # [ 89.063414] nix-grpc-daemon[784]: ts=2026-09-21T15:08:43Z level=info event=worker_gone addr=192.168.1.4:500515559worker1 # [ 89.066823] nix-grpc-daemon[784]: ts=2026-09-21T15:08:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401905560worker1 # [ 89.070293] nix-grpc-daemon[784]: ts=2026-09-21T15:08:43Z level=info event=worker_gone addr=192.168.1.3:500515561worker1 # [ 89.073604] nix-grpc-daemon[784]: ts=2026-09-21T15:08:43Z level=info event=scheduler_restarting addr=lb:500515562worker1 # [ 89.093304] nix-daemon[868]: reaped child process 870, status = succeeded5563worker2 # [ 89.294089] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:44Z level=info event=scheduler_connected addr=lb:500515564worker2 # [ 89.298237] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5565worker1 # [ 89.274459] nix-grpc-daemon[784]: ts=2026-09-21T15:08:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5566worker2 # [ 89.499317] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:44Z level=info event=scheduler_connected addr=lb:500515567worker2 # [ 89.503236] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5568worker2 # [ 89.704292] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:44Z level=info event=scheduler_connected addr=lb:500515569worker2 # [ 89.708245] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5570worker2 # [ 89.909274] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:44Z level=info event=scheduler_connected addr=lb:500515571worker2 # [ 89.913155] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5572worker1 # [ 89.961826] nix-grpc-daemon[784]: E0000 00:00:1790003324.804143 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5573worker1 # [ 89.966710] nix-grpc-daemon[784]: E0000 00:00:1790003324.804343 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5574worker2 # [ 90.115380] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_connected addr=lb:500515575worker2 # [ 90.119197] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5576worker1 # [ 90.175585] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5577worker1 # [ 90.178642] systemd[1]: Stopped Nix worker-protocol over gRPC.5578worker1 # [ 90.183312] systemd[1]: nix-grpc-daemon.service: Consumed 2.873s CPU time over 1min 11.768s wall clock time, 64.4M memory peak, 95.8K incoming IP traffic, 221.9K outgoing IP traffic.5579worker1 # [ 90.194678] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5580worker1 # [ 90.197750] systemd[1]: Closed Nix worker-protocol over gRPC.5581worker1 # [ 90.200479] systemd[1]: nix-grpc-daemon.socket: Consumed 290us CPU time over 1min 20.588s wall clock time, 256K memory peak, 101.4K incoming IP traffic, 83K outgoing IP traffic.5582lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5583lb: must succeed: curl -sf localhost:9901/clusters5584worker2 # [ 90.320639] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_connected addr=lb:500515585worker2 # [ 90.324885] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5587lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.09 seconds)5588client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25589worker2 # [ 90.525939] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_connected addr=lb:500515590worker2 # [ 90.529878] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5591worker2 # [ 90.591746] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601505592worker2 # [ 90.602980] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5593worker2 # [ 90.622542] nix-grpc-daemon[1061]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 1/5)5594worker2 # [ 90.727460] nix-grpc-daemon[1061]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 2/5)5595worker2 # [ 90.734155] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_connected addr=lb:500515596worker2 # [ 90.737746] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5597worker2 # [ 90.873519] nix-grpc-daemon[1061]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 3/5)5598worker2 # [ 90.938968] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_connected addr=lb:500515599worker2 # [ 90.942919] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5600worker2 # [ 90.963620] nix-grpc-daemon[1061]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 236 ms (attempt 4/5)5601worker2 # [ 91.144966] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500515602worker2 # [ 91.148884] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5603worker2 # [ 91.205582] nix-grpc-daemon[1061]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5604worker2 # [ 91.209408] nix-grpc-daemon[1061]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5605client # copying 2 paths...5606client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5607worker2 # [ 91.232121] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:601505608client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5609worker2 # [ 91.243171] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5610worker2 # [ 91.265918] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=niks3_push_started pid=11215611worker2 # [ 91.352197] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5612worker2 # [ 91.480419] nix-grpc-daemon[1121]: time=2026-09-21T15:08:46.376Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5613worker2 # [ 91.659571] nix-daemon[857]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)5614worker2 # [ 91.671688] nix-daemon[857]: reaped child process 1137, status = succeeded5615lb # [ 91.693523] niks3-server[1108]: 2026/09/21 15:08:46 INFO Received uploads request method=POST path=/api/pending_closures5616lb # [ 91.701986] niks3-server[1108]: 2026/09/21 15:08:46 INFO Received uploads request method=POST path=/api/pending_closures5617worker2 # [ 91.706886] nix-grpc-daemon[1121]: time=2026-09-21T15:08:46.603Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5618worker2 # [ 91.708496] nix-grpc-daemon[1121]: time=2026-09-21T15:08:46.605Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5619worker2 # [ 91.710361] nix-grpc-daemon[1121]: time=2026-09-21T15:08:46.607Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5620lb # [ 91.748810] niks3-server[1108]: 2026/09/21 15:08:46 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5621worker2 # [ 91.754477] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500515622worker2 # [ 91.756179] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5623lb # [ 91.763136] niks3-server[1108]: 2026/09/21 15:08:46 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5624lb # [ 91.768329] niks3-server[1108]: 2026/09/21 15:08:46 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5625lb # [ 91.773804] niks3-server[1108]: 2026/09/21 15:08:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5626lb # [ 91.775756] niks3-server[1108]: 2026/09/21 15:08:46 INFO Signed narinfos id=25 count=15627lb # [ 91.777656] niks3-server[1108]: 2026/09/21 15:08:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5628lb # [ 91.780167] niks3-server[1108]: 2026/09/21 15:08:46 INFO Signed narinfos id=26 count=25629worker2 # [ 91.778764] nix-grpc-daemon[1121]: time=2026-09-21T15:08:46.674Z level=INFO msg="Uploading 3 narinfos"5630lb # [ 91.782603] niks3-server[1108]: 2026/09/21 15:08:46 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5631lb # [ 91.792724] niks3-server[1108]: 2026/09/21 15:08:46 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5632lb # [ 91.796350] niks3-server[1108]: 2026/09/21 15:08:46 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5633lb # [ 91.799563] niks3-server[1108]: 2026/09/21 15:08:46 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5634lb # [ 91.802643] niks3-server[1108]: 2026/09/21 15:08:46 INFO Completed upload id=255635lb # [ 91.804365] niks3-server[1108]: 2026/09/21 15:08:46 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5636lb # [ 91.806980] niks3-server[1108]: 2026/09/21 15:08:46 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5637lb # [ 91.809352] niks3-server[1108]: 2026/09/21 15:08:46 INFO Completed upload id=265638worker2 # [ 91.807931] nix-grpc-daemon[1121]: time=2026-09-21T15:08:46.704Z level=INFO msg="Upload complete. (324ms)"5639worker2 # [ 91.812703] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=0 paths=2 nar_bytes_in=3125640worker2 # [ 91.824592] nix-daemon[857]: reaped child process 1119, status = succeeded5641client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.52 seconds)5642worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5643worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5644worker1 # [ 91.887262] systemd[1]: Listening on Nix worker-protocol over gRPC.5645worker1 # [ 91.891364] systemd[1]: Starting Nix worker-protocol over gRPC...5646worker2 # [ 91.957773] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500515647worker2 # [ 91.961783] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5648worker1 # [ 92.079730] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5649worker1 # [ 92.082131] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5650worker1 # [ 92.085907] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15651worker1 # [ 92.094068] nix-grpc-daemon[1281]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5652worker1 # [ 92.095450] nix-grpc-daemon[1281]: E0000 00:00:1790003326.936034 1281 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5653worker1 # [ 92.101232] nix-grpc-daemon[1281]: E0000 00:00:1790003326.941267 1281 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5654worker1 # [ 92.105170] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5655worker1 # [ 92.108122] systemd[1]: Started Nix worker-protocol over gRPC.5656lb # [ 92.153460] niks3-server[1108]: 2026/09/21 15:08:46 INFO lead: acquired remote=192.168.1.3:554705657worker1 # [ 92.113047] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:46Z level=info event=scheduler_take_over5658lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5659lb: must succeed: curl -sf localhost:9901/clusters5660worker1 # [ 92.116122] nix-grpc-daemon[1281]: E0000 00:00:1790003326.957416 1323 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5661worker2 # [ 92.163447] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:47Z level=info event=scheduler_connected addr=lb:500515662worker1 # [ 92.126951] nix-grpc-daemon[1281]: E0000 00:00:1790003326.969521 1323 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5663worker2 # [ 92.167286] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5664worker1 # [ 92.132057] nix-grpc-daemon[1281]: E0000 00:00:1790003326.971811 1323 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5665worker1 # [ 92.141100] nix-grpc-daemon[1281]: E0000 00:00:1790003326.983649 1323 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5666worker1 # [ 92.154206] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:46Z level=info event=scheduler_connected addr=lb:500515667worker1 # [ 92.156133] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5668lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5669lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5670(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.54 seconds)5671subtest: a niks3 restart does not cost the leader its role5672lb: must succeed: curl -sf localhost:9901/clusters5673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5674lb: must succeed: curl -sf localhost:9901/clusters5675lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5676worker2 # [ 92.368480] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:47Z level=info event=scheduler_connected addr=lb:500515677worker1 # [ 92.337395] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:47Z level=info event=oidc_jwks provider=mock keys=15678worker1 # [ 92.340494] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:386605679worker1 # [ 92.344473] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05680lb # [ 92.398982] niks3-server[1108]: 2026/09/21 15:08:46 INFO Shutdown signal received, draining in-flight requests timeout=10s5681lb # [ 92.401732] systemd[1]: Stopping niks3 server...5682worker1 # [ 92.356567] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:47Z level=info event=scheduler_connected addr=lb:500515683worker2 # [ 92.402155] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:47Z level=info event=scheduler_lock_lost status=2005684worker1 # [ 92.360584] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:386605685lb # [ 92.409214] niks3-server[1108]: 2026/09/21 15:08:46 INFO lead: released remote=192.168.1.3:554705686worker1 # [ 92.365103] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05687worker1 # [ 92.370764] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:47Z level=info event=scheduler_lock_lost status=2005688lb # [ 92.416882] systemd[1]: niks3.service: Deactivated successfully.5689lb # [ 92.418472] systemd[1]: Stopped niks3 server.5690lb # [ 92.419357] systemd[1]: niks3.service: Consumed 616ms CPU time over 1min 3.383s wall clock time, 20.5M memory peak, 71.5K incoming IP traffic, 101.5K outgoing IP traffic.5691lb # [ 92.427931] systemd[1]: Starting niks3 server...5692lb # [ 92.501366] niks3-server[1238]: 2026/09/21 15:08:46 goose: no migrations to run. current version: 202609200000005693lb # [ 92.507796] niks3-server[1238]: 2026/09/21 15:08:46 OK 1_commit_pending_closure.sql (4.87ms)5694lb # [ 92.511151] niks3-server[1238]: 2026/09/21 15:08:46 OK 2_object_stats_trigger.sql (1.83ms)5695lb # [ 92.512281] niks3-server[1238]: 2026/09/21 15:08:46 goose: up to current file version: 25696lb # [ 92.514752] niks3-server[1238]: 2026/09/21 15:08:46 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key5697lb # [ 92.526830] niks3-server[1238]: 2026/09/21 15:08:46 INFO Read proxy enabled — serving cache objects from S35698lb # [ 92.528139] niks3-server[1238]: 2026/09/21 15:08:46 INFO Using socket-activated listener address=0.0.0.0:57515699lb # [ 92.531259] systemd[1]: Started niks3 server.5700lb # [ 92.531923] niks3-server[1238]: 2026/09/21 15:08:46 INFO systemd watchdog enabled interval=15s5701lb # [ 92.533232] niks3-server[1238]: 2026/09/21 15:08:46 INFO Starting HTTP server address=0.0.0.0:57515702lb: waiting for unit niks3.service5703lb: (finished: waiting for unit niks3.service, in 0.05 seconds)5704lb: must succeed: curl -sf localhost:9901/clusters5705lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5706worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5707worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5708lb: must succeed: curl -sf localhost:9901/clusters5709lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5710lb: must succeed: curl -sf localhost:9901/clusters5711lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5712worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5713worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5714client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart >&25715worker1 # [ 93.107766] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:554605716worker1 # [ 93.119371] nix-daemon[868]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5717client # don't know how to build these paths:5718client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5719worker2 # [ 93.177242] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601505720worker1 # [ 93.162510] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:48Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05721worker1 # [ 93.168072] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:48Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15722worker2 # [ 93.253391] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505723worker2 # [ 93.266068] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601505724client # copying 1 paths to 'lb:50051'...5725client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5726worker2 # [ 93.278630] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:601505727worker2 # [ 93.290294] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5728lb # [ 93.414199] niks3-server[1238]: 2026/09/21 15:08:47 INFO lead: acquired remote=192.168.1.3:596825729worker2 # [ 93.500710] nix-daemon[857]: accepted connection from pid 1144, user nix-grpc-daemon (trusted)5730worker2 # [ 93.512827] nix-daemon[857]: reaped child process 1149, status = succeeded5731worker2 # [ 93.705297] nix-daemon[857]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)5732worker2 # [ 93.716910] nix-daemon[857]: reaped child process 1154, status = succeeded5733lb # [ 93.740543] niks3-server[1238]: 2026/09/21 15:08:48 INFO Received uploads request method=POST path=/api/pending_closures5734worker2 # [ 93.762423] nix-grpc-daemon[1121]: time=2026-09-21T15:08:48.658Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5735worker2 # [ 93.763909] nix-grpc-daemon[1121]: time=2026-09-21T15:08:48.658Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5736lb # [ 93.792773] niks3-server[1238]: 2026/09/21 15:08:48 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5737lb # [ 93.797887] niks3-server[1238]: 2026/09/21 15:08:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5738lb # [ 93.814536] niks3-server[1238]: 2026/09/21 15:08:48 INFO Signed narinfos id=27 count=15739worker2 # [ 93.814431] nix-grpc-daemon[1121]: time=2026-09-21T15:08:48.711Z level=INFO msg="Uploading 1 narinfos"5740lb # [ 93.826932] niks3-server[1238]: 2026/09/21 15:08:48 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5741lb # [ 93.832089] niks3-server[1238]: 2026/09/21 15:08:48 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5742lb # [ 93.836733] niks3-server[1238]: 2026/09/21 15:08:48 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5743lb # [ 93.844193] niks3-server[1238]: 2026/09/21 15:08:48 INFO Completed upload id=275744worker2 # [ 93.842826] nix-grpc-daemon[1121]: time=2026-09-21T15:08:48.739Z level=INFO msg="Upload complete. (540ms)"5745worker2 # [ 93.847373] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=0 paths=1 nar_bytes_in=5365746worker2 # [ 93.853359] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505747worker2 # [ 93.866182] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5748worker2 # [ 93.876216] nix-daemon[857]: reaped child process 1142, status = succeeded5749client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5750client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5751worker2 # [ 93.882841] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5752client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5753client # farm-a> LOG-farm-a-after-niks3-restart5754worker2 # [ 94.709444] nix-daemon[857]: reaped child process 1158, status = succeeded5755worker2 # [ 94.892844] nix-daemon[857]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5756worker2 # [ 94.904766] nix-daemon[857]: reaped child process 1177, status = succeeded5757lb # [ 94.926401] niks3-server[1238]: 2026/09/21 15:08:49 INFO Received uploads request method=POST path=/api/pending_closures5758worker2 # [ 94.934396] nix-grpc-daemon[1121]: time=2026-09-21T15:08:49.830Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5759worker2 # [ 94.937062] nix-grpc-daemon[1121]: time=2026-09-21T15:08:49.832Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5760lb # [ 94.960350] niks3-server[1238]: 2026/09/21 15:08:49 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5761lb # [ 94.977245] niks3-server[1238]: 2026/09/21 15:08:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5762lb # [ 94.981727] niks3-server[1238]: 2026/09/21 15:08:49 INFO Signed narinfos id=28 count=15763worker2 # [ 94.982969] nix-grpc-daemon[1121]: time=2026-09-21T15:08:49.878Z level=INFO msg="Uploading 1 narinfos"5764lb # [ 94.988380] niks3-server[1238]: 2026/09/21 15:08:49 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5765lb # [ 94.998515] niks3-server[1238]: 2026/09/21 15:08:49 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5766lb # [ 95.002291] niks3-server[1238]: 2026/09/21 15:08:49 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5767lb # [ 95.013068] niks3-server[1238]: 2026/09/21 15:08:49 INFO Completed upload id=285768worker2 # [ 95.010578] nix-grpc-daemon[1121]: time=2026-09-21T15:08:49.907Z level=INFO msg="Upload complete. (318ms)"5769lb # [ 95.019095] niks3-server[1238]: 2026/09/21 15:08:49 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5770worker2 # [ 95.016705] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15771worker1 # [ 94.985598] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:49Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05772worker2 # [ 95.031231] nix-daemon[857]: reaped child process 1156, status = succeeded5773worker1 # [ 94.989795] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:49Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15774worker2 # [ 95.070420] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505775worker2 # [ 95.083520] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601505776worker2 # [ 95.097225] nix-grpc-daemon[1121]: time=2026-09-21T15:08:49.993Z level=INFO msg="All 1 paths already cached"5777client # copying 1 paths to 'lb:50051'...5778client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5779worker2 # [ 95.105366] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:601505780worker2 # [ 95.113957] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5781worker2 # [ 95.306747] nix-daemon[857]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)5782worker2 # [ 95.318860] nix-daemon[857]: reaped child process 1184, status = succeeded5783worker2 # [ 95.497466] nix-daemon[857]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5784worker2 # [ 95.509284] nix-daemon[857]: reaped child process 1189, status = succeeded5785lb # [ 95.530076] niks3-server[1238]: 2026/09/21 15:08:49 INFO Received uploads request method=POST path=/api/pending_closures5786worker2 # [ 95.539964] nix-grpc-daemon[1121]: time=2026-09-21T15:08:50.436Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5787worker2 # [ 95.541465] nix-grpc-daemon[1121]: time=2026-09-21T15:08:50.436Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5788lb # [ 95.560964] niks3-server[1238]: 2026/09/21 15:08:49 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5789lb # [ 95.571877] niks3-server[1238]: 2026/09/21 15:08:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5790worker2 # [ 95.576389] nix-grpc-daemon[1121]: time=2026-09-21T15:08:50.472Z level=INFO msg="Uploading 1 narinfos"5791lb # [ 95.579115] niks3-server[1238]: 2026/09/21 15:08:50 INFO Signed narinfos id=29 count=15792lb # [ 95.584808] niks3-server[1238]: 2026/09/21 15:08:50 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5793lb # [ 95.594429] niks3-server[1238]: 2026/09/21 15:08:50 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5794lb # [ 95.601727] niks3-server[1238]: 2026/09/21 15:08:50 INFO Completed upload id=295795worker2 # [ 95.599153] nix-grpc-daemon[1121]: time=2026-09-21T15:08:50.495Z level=INFO msg="Upload complete. (471ms)"5796lb # [ 95.606080] niks3-server[1238]: 2026/09/21 15:08:50 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5797worker2 # [ 95.603086] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=0 paths=1 nar_bytes_in=6565798worker2 # [ 95.610220] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505799worker2 # [ 95.618818] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5800worker2 # [ 95.623567] nix-daemon[857]: reaped child process 1179, status = succeeded5801client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5802worker2 # [ 95.629513] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5803client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5804client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5805client # farm-b> LOG-farm-b-after-niks3-restart5806worker2 # [ 96.854321] nix-daemon[857]: reaped child process 1193, status = succeeded5807worker2 # [ 97.041377] nix-daemon[857]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5808worker2 # [ 97.053263] nix-daemon[857]: reaped child process 1212, status = succeeded5809lb # [ 97.075613] niks3-server[1238]: 2026/09/21 15:08:51 INFO Received uploads request method=POST path=/api/pending_closures5810worker2 # [ 97.081703] nix-grpc-daemon[1121]: time=2026-09-21T15:08:51.978Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5811worker2 # [ 97.083275] nix-grpc-daemon[1121]: time=2026-09-21T15:08:51.980Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5812lb # [ 97.121709] niks3-server[1238]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5813lb # [ 97.127178] niks3-server[1238]: 2026/09/21 15:08:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5814lb # [ 97.133245] niks3-server[1238]: 2026/09/21 15:08:51 INFO Signed narinfos id=30 count=15815worker2 # [ 97.130983] nix-grpc-daemon[1121]: time=2026-09-21T15:08:52.027Z level=INFO msg="Uploading 1 narinfos"5816lb # [ 97.138164] niks3-server[1238]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5817lb # [ 97.143388] niks3-server[1238]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5818lb # [ 97.150530] niks3-server[1238]: 2026/09/21 15:08:51 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5819lb # [ 97.154048] niks3-server[1238]: 2026/09/21 15:08:51 INFO Completed upload id=305820lb # [ 97.155649] niks3-server[1238]: 2026/09/21 15:08:51 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5821worker2 # [ 97.152755] nix-grpc-daemon[1121]: time=2026-09-21T15:08:52.049Z level=INFO msg="Upload complete. (315ms)"5822worker2 # [ 97.159282] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15823worker1 # [ 97.123784] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:51Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05824worker1 # [ 97.128317] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:51Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15825worker2 # [ 97.172244] nix-daemon[857]: reaped child process 1191, status = succeeded5826worker2 # [ 97.213475] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505827worker2 # [ 97.226549] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601505828worker2 # [ 97.239166] nix-grpc-daemon[1121]: time=2026-09-21T15:08:52.135Z level=INFO msg="All 2 paths already cached"5829client # copying 1 paths to 'lb:50051'...5830client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5831worker2 # [ 97.247526] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:601505832worker2 # [ 97.256310] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5833worker2 # [ 97.456394] nix-daemon[857]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)5834worker2 # [ 97.467682] nix-daemon[857]: reaped child process 1219, status = succeeded5835worker2 # [ 97.651648] nix-daemon[857]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5836worker2 # [ 97.663747] nix-daemon[857]: reaped child process 1224, status = succeeded5837lb # [ 97.684501] niks3-server[1238]: 2026/09/21 15:08:52 INFO Received uploads request method=POST path=/api/pending_closures5838worker2 # [ 97.690737] nix-grpc-daemon[1121]: time=2026-09-21T15:08:52.587Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5839worker2 # [ 97.692696] nix-grpc-daemon[1121]: time=2026-09-21T15:08:52.589Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5840lb # [ 97.712681] niks3-server[1238]: 2026/09/21 15:08:52 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5841lb # [ 97.724713] niks3-server[1238]: 2026/09/21 15:08:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5842worker2 # [ 97.727585] nix-grpc-daemon[1121]: time=2026-09-21T15:08:52.624Z level=INFO msg="Uploading 1 narinfos"5843lb # [ 97.730150] niks3-server[1238]: 2026/09/21 15:08:52 INFO Signed narinfos id=31 count=15844lb # [ 97.735913] niks3-server[1238]: 2026/09/21 15:08:52 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5845lb # [ 97.746305] niks3-server[1238]: 2026/09/21 15:08:52 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5846lb # [ 97.751126] niks3-server[1238]: 2026/09/21 15:08:52 INFO Completed upload id=315847worker2 # [ 97.749583] nix-grpc-daemon[1121]: time=2026-09-21T15:08:52.646Z level=INFO msg="Upload complete. (479ms)"5848lb # [ 97.752860] niks3-server[1238]: 2026/09/21 15:08:52 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5849worker2 # [ 97.751754] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=0 paths=1 nar_bytes_in=7845850worker2 # [ 97.757606] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505851worker2 # [ 97.766079] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5852worker2 # [ 97.771166] nix-daemon[857]: reaped child process 1214, status = succeeded5853worker2 # [ 97.776091] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5854client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5855client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5856lb # [ 98.465254] envoy[979]: conn peer=192.168.1.3:49666 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=21237 tx=4116 ms=694055857lb # [ 98.469125] envoy[979]: conn peer=[2001:db8:1::1]:32934 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1002 tx=282 ms=12345858client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5859client # farm-top> LOG-farm-top-after-niks3-restart5860worker2 # [ 99.001916] nix-daemon[857]: reaped child process 1228, status = succeeded5861worker2 # [ 99.194648] nix-daemon[857]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5862worker2 # [ 99.206642] nix-daemon[857]: reaped child process 1247, status = succeeded5863lb # [ 99.228712] niks3-server[1238]: 2026/09/21 15:08:53 INFO Received uploads request method=POST path=/api/pending_closures5864worker2 # [ 99.235759] nix-grpc-daemon[1121]: time=2026-09-21T15:08:54.132Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5865worker2 # [ 99.237494] nix-grpc-daemon[1121]: time=2026-09-21T15:08:54.134Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5866lb # [ 99.268424] niks3-server[1238]: 2026/09/21 15:08:53 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5867lb # [ 99.278090] niks3-server[1238]: 2026/09/21 15:08:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5868lb # [ 99.287978] niks3-server[1238]: 2026/09/21 15:08:53 INFO Signed narinfos id=32 count=15869worker2 # [ 99.287962] nix-grpc-daemon[1121]: time=2026-09-21T15:08:54.184Z level=INFO msg="Uploading 1 narinfos"5870lb # [ 99.292497] niks3-server[1238]: 2026/09/21 15:08:53 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5871lb # [ 99.297501] niks3-server[1238]: 2026/09/21 15:08:53 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5872lb # [ 99.305419] niks3-server[1238]: 2026/09/21 15:08:53 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5873lb # [ 99.312334] niks3-server[1238]: 2026/09/21 15:08:53 INFO Completed upload id=325874worker2 # [ 99.309822] nix-grpc-daemon[1121]: time=2026-09-21T15:08:54.206Z level=INFO msg="Upload complete. (325ms)"5875lb # [ 99.314302] niks3-server[1238]: 2026/09/21 15:08:53 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5876worker2 # [ 99.315925] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15877worker2 # [ 99.329611] nix-daemon[857]: reaped child process 1226, status = succeeded5878client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart >&2, in 6.46 seconds)5879client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart outPath5880client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart outPath, in 0.22 seconds)5881(finished: subtest: a niks3 restart does not cost the leader its role, in 7.34 seconds)5882subtest: scheduler down: the other node takes the lock and keeps it5883lb: must succeed: curl -sf localhost:9901/clusters5884lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5885lb: must succeed: curl -sf localhost:9901/clusters5886lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5887lb: must succeed: curl -sf localhost:9901/clusters5888lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5889lb: must succeed: curl -sf localhost:9901/clusters5890lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5891worker1 # [ 99.858441] systemd[1]: Stopping Nix worker-protocol over gRPC...5892worker1 # [ 100.104666] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:54Z level=info event=signal_exit5893lb # [ 100.624826] niks3-server[1238]: 2026/09/21 15:08:55 INFO lead: released remote=192.168.1.3:596825894worker2 # [ 101.623640] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:56Z level=info event=scheduler_restarting addr=lb:500515895worker1 # [ 101.586684] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:56Z level=info event=scheduler_restarting5896worker1 # [ 101.590565] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386605897worker1 # [ 101.594226] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:56Z level=info event=scheduler_restarting addr=lb:500515898worker1 # [ 101.598267] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:56Z level=info event=worker_gone addr=192.168.1.4:500515899worker1 # [ 101.601313] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386605900worker1 # [ 101.604592] nix-grpc-daemon[1281]: ts=2026-09-21T15:08:56Z level=info event=worker_gone addr=192.168.1.3:500515901worker1 # [ 101.614391] nix-daemon[868]: reaped child process 1341, status = succeeded5902worker2 # [ 101.824679] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:56Z level=info event=scheduler_connected addr=lb:500515903worker2 # [ 101.828689] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5904worker2 # [ 102.029757] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:56Z level=info event=scheduler_connected addr=lb:500515905worker2 # [ 102.033654] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5906worker1 # [ 102.185557] nix-grpc-daemon[1281]: E0000 00:00:1790003337.027948 1326 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5907worker1 # [ 102.190450] nix-grpc-daemon[1281]: E0000 00:00:1790003337.028069 1326 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5908worker2 # [ 102.234750] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_connected addr=lb:500515909worker2 # [ 102.238671] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5910worker2 # [ 102.439821] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_connected addr=lb:500515911worker2 # [ 102.444380] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5912worker2 # [ 102.645939] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_connected addr=lb:500515913worker2 # [ 102.649645] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5914worker1 # [ 102.713644] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5915worker1 # [ 102.716620] systemd[1]: Stopped Nix worker-protocol over gRPC.5916worker1 # [ 102.719911] systemd[1]: nix-grpc-daemon.service: Consumed 211ms CPU time over 10.821s wall clock time, 13.7M memory peak, 10.9K incoming IP traffic, 14.6K outgoing IP traffic.5917worker1 # [ 102.731193] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5918worker1 # [ 102.734835] systemd[1]: Closed Nix worker-protocol over gRPC.5919worker1 # [ 102.737704] systemd[1]: nix-grpc-daemon.socket: Consumed 291us CPU time over 10.852s wall clock time, 256K memory peak, 23.6K incoming IP traffic, 18K outgoing IP traffic.5920worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5921worker2 # [ 102.850445] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_connected addr=lb:500515922worker2 # [ 102.854052] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5923worker2 # [ 103.055158] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_connected addr=lb:500515924worker2 # [ 103.059111] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5925worker2 # [ 103.260154] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=scheduler_connected addr=lb:500515926worker2 # [ 103.264204] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5927worker2 # [ 103.412058] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=scheduler_take_over5928lb # [ 103.414974] niks3-server[1238]: 2026/09/21 15:08:57 INFO lead: acquired remote=192.168.1.4:423865929worker2 # [ 103.465129] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=scheduler_connected addr=lb:500515930worker2 # [ 103.469095] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5931worker2 # [ 103.670723] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=scheduler_connected addr=lb:500515932worker2 # [ 103.688136] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=oidc_jwks provider=mock keys=15933worker2 # [ 103.691542] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:393645934worker2 # [ 103.696309] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:58Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05935worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.16 seconds)5936worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5937worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5938client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25939worker2 # [ 104.224769] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:601505940client # don't know how to build these paths:5941client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5942worker2 # [ 104.234229] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601505943worker2 # [ 104.257815] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05944worker2 # [ 104.262639] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15945worker2 # [ 104.307137] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505946worker2 # [ 104.319610] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601505947client # copying 1 paths to 'lb:50051'...5948client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5949worker2 # [ 104.332207] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:601505950worker2 # [ 104.343783] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5951worker2 # [ 104.562568] nix-daemon[857]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5952worker2 # [ 104.575577] nix-daemon[857]: reaped child process 1271, status = succeeded5953worker2 # [ 104.780899] nix-daemon[857]: accepted connection from pid 1273, user nix-grpc-daemon (trusted)5954worker2 # [ 104.792705] nix-daemon[857]: reaped child process 1276, status = succeeded5955lb # [ 104.816828] niks3-server[1238]: 2026/09/21 15:08:59 INFO Received uploads request method=POST path=/api/pending_closures5956worker2 # [ 104.822469] nix-grpc-daemon[1121]: time=2026-09-21T15:08:59.718Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5957worker2 # [ 104.824255] nix-grpc-daemon[1121]: time=2026-09-21T15:08:59.720Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5958lb # [ 104.842362] niks3-server[1238]: 2026/09/21 15:08:59 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5959lb # [ 104.854970] niks3-server[1238]: 2026/09/21 15:08:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5960worker2 # [ 104.859687] nix-grpc-daemon[1121]: time=2026-09-21T15:08:59.756Z level=INFO msg="Uploading 1 narinfos"5961lb # [ 104.862360] niks3-server[1238]: 2026/09/21 15:08:59 INFO Signed narinfos id=33 count=15962lb # [ 104.866854] niks3-server[1238]: 2026/09/21 15:08:59 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5963lb # [ 104.878660] niks3-server[1238]: 2026/09/21 15:08:59 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5964lb # [ 104.883770] niks3-server[1238]: 2026/09/21 15:08:59 INFO Completed upload id=335965worker2 # [ 104.881163] nix-grpc-daemon[1121]: time=2026-09-21T15:08:59.777Z level=INFO msg="Upload complete. (525ms)"5966lb # [ 104.888691] niks3-server[1238]: 2026/09/21 15:08:59 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5967worker2 # [ 104.885575] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=0 paths=1 nar_bytes_in=5205968worker2 # [ 104.891278] nix-grpc-daemon[1061]: ts=2026-09-21T15:08:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505969worker2 # [ 104.901250] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5970worker2 # [ 104.905125] nix-daemon[857]: reaped child process 1266, status = succeeded5971worker2 # [ 104.910513] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)5972client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5973client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5974client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5975client # farm-a> LOG-farm-a-on-standby5976worker2 # [ 105.935738] nix-daemon[857]: reaped child process 1280, status = succeeded5977worker2 # [ 106.141619] nix-daemon[857]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5978worker2 # [ 106.154087] nix-daemon[857]: reaped child process 1299, status = succeeded5979lb # [ 106.178483] niks3-server[1238]: 2026/09/21 15:09:00 INFO Received uploads request method=POST path=/api/pending_closures5980worker2 # [ 106.184574] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.081Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5981worker2 # [ 106.187797] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.082Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5982lb # [ 106.217883] niks3-server[1238]: 2026/09/21 15:09:00 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5983lb # [ 106.229295] niks3-server[1238]: 2026/09/21 15:09:00 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5984lb # [ 106.236835] niks3-server[1238]: 2026/09/21 15:09:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5985worker2 # [ 106.238551] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.135Z level=INFO msg="Uploading 1 narinfos"5986lb # [ 106.242440] niks3-server[1238]: 2026/09/21 15:09:00 INFO Signed narinfos id=34 count=15987lb # [ 106.246243] niks3-server[1238]: 2026/09/21 15:09:00 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5988lb # [ 106.254657] niks3-server[1238]: 2026/09/21 15:09:00 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5989lb # [ 106.260893] niks3-server[1238]: 2026/09/21 15:09:00 INFO Completed upload id=345990worker2 # [ 106.258241] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.154Z level=INFO msg="Upload complete. (340ms)"5991lb # [ 106.263773] niks3-server[1238]: 2026/09/21 15:09:00 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5992worker2 # [ 106.271202] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15993worker2 # [ 106.284210] nix-daemon[857]: reaped child process 1278, status = succeeded5994worker2 # [ 106.290881] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:01Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05995worker2 # [ 106.294632] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:01Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=15996worker2 # [ 106.335479] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601505997worker2 # [ 106.349134] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601505998worker2 # [ 106.363488] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.260Z level=INFO msg="All 1 paths already cached"5999client # copying 1 paths to 'lb:50051'...6000client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6001worker2 # [ 106.371513] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:601506002worker2 # [ 106.379867] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6003worker2 # [ 106.594933] nix-daemon[857]: accepted connection from pid 1305, user nix-grpc-daemon (trusted)6004worker2 # [ 106.607885] nix-daemon[857]: reaped child process 1308, status = succeeded6005worker2 # [ 106.802507] nix-daemon[857]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)6006worker2 # [ 106.814417] nix-daemon[857]: reaped child process 1313, status = succeeded6007lb # [ 106.835872] niks3-server[1238]: 2026/09/21 15:09:01 INFO Received uploads request method=POST path=/api/pending_closures6008worker2 # [ 106.841701] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.738Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6009worker2 # [ 106.843177] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.738Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6010lb # [ 106.860479] niks3-server[1238]: 2026/09/21 15:09:01 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6011lb # [ 106.873071] niks3-server[1238]: 2026/09/21 15:09:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign6012worker2 # [ 106.874908] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.771Z level=INFO msg="Uploading 1 narinfos"6013lb # [ 106.878844] niks3-server[1238]: 2026/09/21 15:09:01 INFO Signed narinfos id=35 count=16014lb # [ 106.884219] niks3-server[1238]: 2026/09/21 15:09:01 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6015lb # [ 106.895097] niks3-server[1238]: 2026/09/21 15:09:01 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete6016lb # [ 106.901207] niks3-server[1238]: 2026/09/21 15:09:01 INFO Completed upload id=356017worker2 # [ 106.898323] nix-grpc-daemon[1121]: time=2026-09-21T15:09:01.795Z level=INFO msg="Upload complete. (505ms)"6018lb # [ 106.903400] niks3-server[1238]: 2026/09/21 15:09:01 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6019worker2 # [ 106.901753] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=0 paths=1 nar_bytes_in=6406020worker2 # [ 106.907240] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601506021worker2 # [ 106.916849] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6022worker2 # [ 106.923179] nix-daemon[857]: reaped child process 1303, status = succeeded6023worker2 # [ 106.926993] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6024client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6025client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6026client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6027client # farm-b> LOG-farm-b-on-standby6028worker2 # [ 107.751955] nix-daemon[857]: reaped child process 1317, status = succeeded6029worker2 # [ 107.951869] nix-daemon[857]: accepted connection from pid 1333, user nix-grpc-daemon (trusted)6030worker2 # [ 107.962464] nix-daemon[857]: reaped child process 1336, status = succeeded6031lb # [ 107.987288] niks3-server[1238]: 2026/09/21 15:09:02 INFO Received uploads request method=POST path=/api/pending_closures6032worker2 # [ 107.992950] nix-grpc-daemon[1121]: time=2026-09-21T15:09:02.889Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6033worker2 # [ 107.994448] nix-grpc-daemon[1121]: time=2026-09-21T15:09:02.891Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6034lb # [ 108.025273] niks3-server[1238]: 2026/09/21 15:09:02 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6035lb # [ 108.032888] niks3-server[1238]: 2026/09/21 15:09:02 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6036lb # [ 108.039981] niks3-server[1238]: 2026/09/21 15:09:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6037lb # [ 108.044954] niks3-server[1238]: 2026/09/21 15:09:02 INFO Signed narinfos id=36 count=16038worker2 # [ 108.045391] nix-grpc-daemon[1121]: time=2026-09-21T15:09:02.941Z level=INFO msg="Uploading 1 narinfos"6039lb # [ 108.052785] niks3-server[1238]: 2026/09/21 15:09:02 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6040lb # [ 108.062231] niks3-server[1238]: 2026/09/21 15:09:02 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6041lb # [ 108.068712] niks3-server[1238]: 2026/09/21 15:09:02 INFO Completed upload id=366042worker2 # [ 108.066070] nix-grpc-daemon[1121]: time=2026-09-21T15:09:02.962Z level=INFO msg="Upload complete. (332ms)"6043lb # [ 108.074728] niks3-server[1238]: 2026/09/21 15:09:02 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6044worker2 # [ 108.073496] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16045worker2 # [ 108.085635] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:02Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06046worker2 # [ 108.088942] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:02Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16047worker2 # [ 108.097381] nix-daemon[857]: reaped child process 1315, status = succeeded6048worker2 # [ 108.128780] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601506049worker2 # [ 108.147263] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601506050worker2 # [ 108.163381] nix-grpc-daemon[1121]: time=2026-09-21T15:09:03.060Z level=INFO msg="All 2 paths already cached"6051client # copying 1 paths to 'lb:50051'...6052client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6053worker2 # [ 108.171690] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:601506054worker2 # [ 108.180349] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6055worker2 # [ 108.394399] nix-daemon[857]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)6056worker2 # [ 108.406704] nix-daemon[857]: reaped child process 1345, status = succeeded6057lb # [ 108.465704] envoy[979]: conn peer=[2001:db8:1::1]:47364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9765 tx=4084 ms=61856058lb # [ 108.470510] envoy[979]: conn peer=[2001:db8:1::3]:41532 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=751 tx=265 ms=96396059worker2 # [ 108.612542] nix-daemon[857]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)6060worker2 # [ 108.624403] nix-daemon[857]: reaped child process 1350, status = succeeded6061lb # [ 108.648229] niks3-server[1238]: 2026/09/21 15:09:03 INFO Received uploads request method=POST path=/api/pending_closures6062worker2 # [ 108.654631] nix-grpc-daemon[1121]: time=2026-09-21T15:09:03.551Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6063worker2 # [ 108.656172] nix-grpc-daemon[1121]: time=2026-09-21T15:09:03.552Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6064lb # [ 108.680880] niks3-server[1238]: 2026/09/21 15:09:03 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6065lb # [ 108.693542] niks3-server[1238]: 2026/09/21 15:09:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6066lb # [ 108.699045] niks3-server[1238]: 2026/09/21 15:09:03 INFO Signed narinfos id=37 count=16067worker2 # [ 108.698567] nix-grpc-daemon[1121]: time=2026-09-21T15:09:03.595Z level=INFO msg="Uploading 1 narinfos"6068lb # [ 108.704349] niks3-server[1238]: 2026/09/21 15:09:03 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6069lb # [ 108.715184] niks3-server[1238]: 2026/09/21 15:09:03 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6070lb # [ 108.718155] niks3-server[1238]: 2026/09/21 15:09:03 INFO Completed upload id=376071worker2 # [ 108.715703] nix-grpc-daemon[1121]: time=2026-09-21T15:09:03.612Z level=INFO msg="Upload complete. (521ms)"6072lb # [ 108.720300] niks3-server[1238]: 2026/09/21 15:09:03 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6073worker2 # [ 108.719685] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=0 paths=1 nar_bytes_in=7686074worker2 # [ 108.725267] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601506075worker2 # [ 108.733835] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6076worker2 # [ 108.739306] nix-daemon[857]: reaped child process 1340, status = succeeded6077worker2 # [ 108.744849] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6078client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6079client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6080client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6081client # farm-top> LOG-farm-top-on-standby6082worker2 # [ 109.970214] nix-daemon[857]: reaped child process 1354, status = succeeded6083worker2 # [ 110.169597] nix-daemon[857]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6084worker2 # [ 110.181638] nix-daemon[857]: reaped child process 1373, status = succeeded6085lb # [ 110.204879] niks3-server[1238]: 2026/09/21 15:09:04 INFO Received uploads request method=POST path=/api/pending_closures6086worker2 # [ 110.212295] nix-grpc-daemon[1121]: time=2026-09-21T15:09:05.107Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6087worker2 # [ 110.213667] nix-grpc-daemon[1121]: time=2026-09-21T15:09:05.107Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6088lb # [ 110.240686] niks3-server[1238]: 2026/09/21 15:09:04 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6089lb # [ 110.253338] niks3-server[1238]: 2026/09/21 15:09:04 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6090lb # [ 110.264945] niks3-server[1238]: 2026/09/21 15:09:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6091worker2 # [ 110.267357] nix-grpc-daemon[1121]: time=2026-09-21T15:09:05.163Z level=INFO msg="Uploading 1 narinfos"6092lb # [ 110.271145] niks3-server[1238]: 2026/09/21 15:09:04 INFO Signed narinfos id=38 count=16093lb # [ 110.275649] niks3-server[1238]: 2026/09/21 15:09:04 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6094lb # [ 110.285920] niks3-server[1238]: 2026/09/21 15:09:04 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6095lb # [ 110.291608] niks3-server[1238]: 2026/09/21 15:09:04 INFO Completed upload id=386096worker2 # [ 110.289179] nix-grpc-daemon[1121]: time=2026-09-21T15:09:05.185Z level=INFO msg="Upload complete. (336ms)"6097lb # [ 110.294456] niks3-server[1238]: 2026/09/21 15:09:04 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6098worker2 # [ 110.297746] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16099worker2 # [ 110.310288] nix-daemon[857]: reaped child process 1352, status = succeeded6100client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.34 seconds)6101client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath6102client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.21 seconds)6103worker1 # [ 110.580788] systemd[1]: Listening on Nix worker-protocol over gRPC.6104worker1 # [ 110.588858] systemd[1]: Starting Nix worker-protocol over gRPC...6105worker1 # [ 110.794796] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6106worker1 # [ 110.796660] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6107worker1 # [ 110.800496] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16108worker1 # [ 110.809067] nix-grpc-daemon[1367]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6109worker1 # [ 110.810469] nix-grpc-daemon[1367]: E0000 00:00:1790003345.650743 1367 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6110worker1 # [ 110.817077] nix-grpc-daemon[1367]: E0000 00:00:1790003345.656007 1367 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6111worker1 # [ 110.819157] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6112worker1 # [ 110.820688] systemd[1]: Started Nix worker-protocol over gRPC.6113lb: must succeed: curl -sf localhost:9901/clusters6114worker1 # [ 110.831277] nix-grpc-daemon[1367]: E0000 00:00:1790003345.673764 1408 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6115worker1 # [ 110.845271] nix-grpc-daemon[1367]: E0000 00:00:1790003345.687838 1408 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6116worker1 # [ 110.849140] nix-grpc-daemon[1367]: E0000 00:00:1790003345.690135 1408 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6117worker1 # [ 110.859497] nix-grpc-daemon[1367]: E0000 00:00:1790003345.702073 1408 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6118worker1 # [ 110.869476] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:05Z level=info event=scheduler_connected addr=lb:500516119worker2 # [ 110.912988] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:393646120worker2 # [ 110.917186] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06121lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6122worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6123worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6124lb: must succeed: curl -sf localhost:9901/clusters6125lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6126lb: must succeed: curl -sf localhost:9901/clusters6127lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6128worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6129worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6130(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.66 seconds)6131subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6132client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs6133client # [ 111.300529] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.6134client # Running as unit: rs.service; invocation ID: 7269d5f90331479bbba88f23c8e509346135client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)6136client # [ 111.517597] nix[1205]: don't know how to build these paths:6137client # [ 111.518978] nix[1205]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6138worker2 # [ 111.508155] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:601506139worker2 # [ 111.519225] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:06Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06140worker2 # [ 111.523081] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:06Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16141worker2 # [ 111.572510] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601506142worker2 # [ 111.587064] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:601506143client # [ 111.607176] nix[1205]: copying 1 paths to 'lb:50051'...6144client # [ 111.609075] nix[1205]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6145worker2 # [ 111.599371] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:601506146worker2 # [ 111.609913] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6147worker2 # [ 111.845181] nix-daemon[857]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)6148worker2 # [ 111.857985] nix-daemon[857]: reaped child process 1389, status = succeeded6149worker2 # [ 112.077552] nix-daemon[857]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)6150worker2 # [ 112.089929] nix-daemon[857]: reaped child process 1394, status = succeeded6151lb # [ 112.115814] niks3-server[1238]: 2026/09/21 15:09:06 INFO Received uploads request method=POST path=/api/pending_closures6152worker2 # [ 112.121820] nix-grpc-daemon[1121]: time=2026-09-21T15:09:07.018Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6153worker2 # [ 112.123351] nix-grpc-daemon[1121]: time=2026-09-21T15:09:07.020Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6154lb # [ 112.145913] niks3-server[1238]: 2026/09/21 15:09:06 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6155lb # [ 112.158870] niks3-server[1238]: 2026/09/21 15:09:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6156worker2 # [ 112.160924] nix-grpc-daemon[1121]: time=2026-09-21T15:09:07.057Z level=INFO msg="Uploading 1 narinfos"6157lb # [ 112.164139] niks3-server[1238]: 2026/09/21 15:09:06 INFO Signed narinfos id=39 count=16158lb # [ 112.171182] niks3-server[1238]: 2026/09/21 15:09:06 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6159lb # [ 112.181331] niks3-server[1238]: 2026/09/21 15:09:06 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6160lb # [ 112.187962] niks3-server[1238]: 2026/09/21 15:09:06 INFO Completed upload id=396161worker2 # [ 112.185354] nix-grpc-daemon[1121]: time=2026-09-21T15:09:07.082Z level=INFO msg="Upload complete. (561ms)"6162lb # [ 112.190933] niks3-server[1238]: 2026/09/21 15:09:06 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6163worker2 # [ 112.189117] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=0 paths=1 nar_bytes_in=4166164worker2 # [ 112.194751] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:601506165worker2 # [ 112.203081] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6166worker2 # [ 112.210699] nix-daemon[857]: reaped child process 1384, status = succeeded6167worker2 # [ 112.216051] nix-daemon[857]: accepted connection from pid 1061, user nix-grpc-daemon (trusted)6168client # [ 112.224548] nix[1205]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6169lb: must succeed: curl -sf localhost:9901/clusters6170lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6171lb: must succeed: curl -sf localhost:9901/clusters6172lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6173lb: must succeed: curl -sf localhost:9901/clusters6174lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6175lb: must succeed: curl -sf localhost:9901/clusters6176lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6177worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6178worker1 # [ 114.287735] systemd-journald[289]: Received client request to rotate journal, rotating.6179worker1 # [ 114.308910] systemd-journald[289]: Vacuuming done, freed 0B of archived journals from /var/log/journal/ed08faf6ca1e446480de5e656cb7a071.6180worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds)6181worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6182worker2 # [ 114.269433] systemd[1]: Stopping Nix worker-protocol over gRPC...6183worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)6184client: must succeed: sleep 5.06185worker2 # [ 114.577849] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:09Z level=info event=draining6186lb # [ 118.465464] envoy[979]: conn peer=[2001:db8:1::1]:42668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9594 tx=4057 ms=60826187client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6188worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6189worker2 # [ 119.635423] nix-daemon[857]: reaped child process 1398, status = succeeded6190worker2 # [ 119.804662] nix-daemon[857]: accepted connection from pid 1439, user nix-grpc-daemon (trusted)6191worker2 # [ 119.816652] nix-daemon[857]: reaped child process 1442, status = succeeded6192lb # [ 119.836656] niks3-server[1238]: 2026/09/21 15:09:14 INFO Received uploads request method=POST path=/api/pending_closures6193worker2 # [ 119.844306] nix-grpc-daemon[1121]: time=2026-09-21T15:09:14.739Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6194worker2 # [ 119.847844] nix-grpc-daemon[1121]: time=2026-09-21T15:09:14.739Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6195lb # [ 119.879831] niks3-server[1238]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6196lb # [ 119.888892] niks3-server[1238]: 2026/09/21 15:09:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6197lb # [ 119.893633] niks3-server[1238]: 2026/09/21 15:09:14 INFO Signed narinfos id=40 count=16198worker2 # [ 119.894332] nix-grpc-daemon[1121]: time=2026-09-21T15:09:14.790Z level=INFO msg="Uploading 1 narinfos"6199lb # [ 119.904173] niks3-server[1238]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6200lb # [ 119.911928] niks3-server[1238]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6201lb # [ 119.919244] niks3-server[1238]: 2026/09/21 15:09:14 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6202lb # [ 119.923138] niks3-server[1238]: 2026/09/21 15:09:14 INFO Completed upload id=406203lb # [ 119.924950] niks3-server[1238]: 2026/09/21 15:09:14 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6204worker2 # [ 119.921768] nix-grpc-daemon[1121]: time=2026-09-21T15:09:14.818Z level=INFO msg="Upload complete. (293ms)"6205worker2 # [ 119.929765] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60150 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16206worker2 # [ 119.942626] nix-daemon[857]: reaped child process 1396, status = succeeded6207client # [ 119.974660] systemd[1]: rs.service: Deactivated successfully.6208client # [ 119.978174] systemd[1]: rs.service: Consumed 129ms CPU time over 8.677s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6209worker2 # [ 120.581391] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:15Z level=info event=signal_exit6210lb # [ 120.642259] niks3-server[1238]: 2026/09/21 15:09:15 INFO lead: released remote=192.168.1.4:423866211lb # [ 120.874707] niks3-server[1238]: 2026/09/21 15:09:15 INFO lead: acquired remote=192.168.1.3:540166212worker1 # [ 120.831310] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:15Z level=info event=scheduler_take_over6213worker1 # [ 121.599609] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:16Z level=info event=scheduler_restarting addr=lb:500516214worker2 # [ 121.645123] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:16Z level=info event=scheduler_restarting6215worker2 # [ 121.648544] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:16Z level=info event=scheduler_restarting addr=lb:500516216worker2 # [ 121.652119] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:393646217worker2 # [ 121.657202] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:16Z level=info event=worker_gone addr=192.168.1.4:500516218worker2 # [ 121.660579] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:393646219worker2 # [ 121.664129] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:16Z level=info event=worker_gone addr=192.168.1.3:500516220worker2 # [ 121.667360] nix-grpc-daemon[1061]: E0000 00:00:1790003356.552636 1106 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 12, started true, shutdown true6221worker2 # [ 121.672006] nix-grpc-daemon[1061]: E0000 00:00:1790003356.552832 1106 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6222worker2 # [ 121.691990] nix-daemon[857]: reaped child process 1113, status = succeeded6223worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.21 seconds)6224client: waiting for success: ! systemctl is-active rs6225worker1 # [ 121.800616] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:16Z level=info event=scheduler_connected addr=lb:500516226worker1 # [ 121.804804] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6227worker2 # [ 121.857699] nix-grpc-daemon[1061]: ts=2026-09-21T15:09:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6228client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)6229client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6230client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)6231client: must fail: journalctl -u rs | grep -q 'reconnecting'6232client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6233lb: must succeed: curl -sf localhost:9901/clusters6234lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6235worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6236worker1 # [ 122.006554] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:16Z level=info event=scheduler_connected addr=lb:500516237worker1 # [ 122.015074] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:369566238worker1 # [ 122.018712] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06239worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6240worker2 # [ 123.085780] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6241worker2 # [ 123.089610] systemd[1]: Stopped Nix worker-protocol over gRPC.6242worker2 # [ 123.091709] systemd[1]: nix-grpc-daemon.service: Consumed 2.207s CPU time over 35.701s wall clock time, 47.1M memory peak, 82.5K incoming IP traffic, 177.7K outgoing IP traffic.6243lb: must succeed: curl -sf localhost:9901/clusters6244worker2 # [ 123.107154] systemd[1]: Starting scheduler-token.service...6245lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6246worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6247worker2 # [ 123.196941] systemd[1]: scheduler-token.service: Deactivated successfully.6248worker2 # [ 123.198600] systemd[1]: Finished scheduler-token.service.6249worker2 # [ 123.199890] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 91ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6250worker2 # [ 123.205698] systemd[1]: Starting Nix worker-protocol over gRPC...6251worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6252worker2 # [ 123.410718] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6253worker2 # [ 123.413325] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6254worker2 # [ 123.417621] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16255worker2 # [ 123.425067] nix-grpc-daemon[1452]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6256worker2 # [ 123.426440] nix-grpc-daemon[1452]: E0000 00:00:1790003358.321541 1452 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6257worker2 # [ 123.431634] nix-grpc-daemon[1452]: E0000 00:00:1790003358.326798 1452 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6258worker2 # [ 123.434212] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6259worker2 # [ 123.436766] systemd[1]: Started Nix worker-protocol over gRPC.6260worker2 # [ 123.444036] nix-grpc-daemon[1452]: E0000 00:00:1790003358.340435 1493 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6261worker2 # [ 123.455755] nix-grpc-daemon[1452]: E0000 00:00:1790003358.352489 1493 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6262worker2 # [ 123.460053] nix-grpc-daemon[1452]: E0000 00:00:1790003358.354779 1493 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6263worker2 # [ 123.469916] nix-grpc-daemon[1452]: E0000 00:00:1790003358.366663 1493 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6264worker2 # [ 123.479420] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:18Z level=info event=scheduler_connected addr=lb:500516265worker1 # [ 123.446819] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:18Z level=info event=oidc_jwks provider=mock keys=16266worker1 # [ 123.450837] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:18Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:369566267worker1 # [ 123.454632] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:18Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06268lb: must succeed: curl -sf localhost:9901/clusters6269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6270worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6271worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6272(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.17 seconds)6273subtest: scheduler restart mid-build: build finishes, no double build6274client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid6275client # [ 124.473132] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.6276client # Running as unit: mid.service; invocation ID: e4b8fa4d95be497e9191a8c51379e4b16277client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)6278worker1 # [ 124.651641] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587646279worker1 # [ 124.663141] nix-daemon[868]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6280client # [ 124.726200] nix[1241]: don't know how to build these paths:6281client # [ 124.727112] nix[1241]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6282worker1 # [ 124.683977] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:19Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06283worker1 # [ 124.688899] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:19Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16284worker1 # [ 124.734271] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587646285worker1 # [ 124.747093] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)6286worker1 # [ 124.841124] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 151 ms (attempt 2/5)6287worker1 # [ 124.997952] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 213 ms (attempt 3/5)6288worker1 # [ 125.217085] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 4/5)6289worker1 # [ 125.259284] nix-grpc-daemon[1367]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6290worker1 # [ 125.263158] nix-grpc-daemon[1367]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6291worker1 # [ 125.281872] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587646292client # [ 125.341489] nix[1241]: copying 1 paths to 'lb:50051'...6293client # [ 125.345141] nix[1241]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6294worker1 # [ 125.295636] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587646295worker1 # [ 125.304375] nix-daemon[868]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6296worker1 # [ 125.327435] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:20Z level=info event=niks3_push_started pid=14926297worker1 # [ 125.573069] nix-grpc-daemon[1492]: time=2026-09-21T15:09:20.415Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6298worker1 # [ 125.757523] nix-daemon[868]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6299worker1 # [ 125.769290] nix-daemon[868]: reaped child process 1512, status = succeeded6300worker1 # [ 125.985720] nix-daemon[868]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6301worker1 # [ 125.997275] nix-daemon[868]: reaped child process 1517, status = succeeded6302lb # [ 126.063415] niks3-server[1238]: 2026/09/21 15:09:20 INFO Received uploads request method=POST path=/api/pending_closures6303worker1 # [ 126.029073] nix-grpc-daemon[1492]: time=2026-09-21T15:09:20.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6304worker1 # [ 126.030461] nix-grpc-daemon[1492]: time=2026-09-21T15:09:20.870Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6305lb # [ 126.097961] niks3-server[1238]: 2026/09/21 15:09:20 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6306lb # [ 126.105736] niks3-server[1238]: 2026/09/21 15:09:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6307worker1 # [ 126.070054] nix-grpc-daemon[1492]: time=2026-09-21T15:09:20.912Z level=INFO msg="Uploading 1 narinfos"6308lb # [ 126.113785] niks3-server[1238]: 2026/09/21 15:09:20 INFO Signed narinfos id=41 count=16309lb # [ 126.130918] niks3-server[1238]: 2026/09/21 15:09:20 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6310lb # [ 126.137952] niks3-server[1238]: 2026/09/21 15:09:20 INFO Completed upload id=416311worker1 # [ 126.094318] nix-grpc-daemon[1492]: time=2026-09-21T15:09:20.936Z level=INFO msg="Upload complete. (517ms)"6312lb # [ 126.142493] niks3-server[1238]: 2026/09/21 15:09:20 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6313worker1 # [ 126.099442] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58764 duration_s=0 paths=1 nar_bytes_in=4246314worker1 # [ 126.105111] nix-grpc-daemon[1367]: ts=2026-09-21T15:09:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587646315worker1 # [ 126.111982] nix-daemon[868]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6316worker1 # [ 126.116556] nix-daemon[868]: reaped child process 1490, status = succeeded6317worker1 # [ 126.121390] nix-daemon[868]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6318client # [ 126.171483] nix[1241]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6319lb: must succeed: curl -sf localhost:9901/clusters6320lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6321lb: must succeed: curl -sf localhost:9901/clusters6322lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6323worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6324worker1 # [ 128.317255] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1367 (nix-grpc-daemon) on client request.6325worker1 # [ 128.322318] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6326lb # [ 128.374827] niks3-server[1238]: 2026/09/21 15:09:22 INFO lead: released remote=192.168.1.3:540166327worker2 # [ 128.379885] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6328client # [ 128.389916] nix[1241]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6329worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.07 seconds)6330client: must succeed: sleep 5.06331client # [ 128.394559] nix[1241]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6332worker1 # [ 128.350615] nix-daemon[868]: reaped child process 1519, status = succeeded6333worker1 # [ 128.352227] nix-daemon[868]: reaped child process 1482, status = succeeded6334worker1 # [ 128.359359] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6335worker1 # [ 128.362363] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6336worker1 # [ 128.363916] systemd[1]: nix-grpc-daemon.service: Consumed 447ms CPU time over 17.771s wall clock time, 33.8M memory peak, 15.9K incoming IP traffic, 31.4K outgoing IP traffic.6337worker1 # [ 128.368805] nix-daemon[1521]: unexpected Nix daemon error: error: interrupted by the user6338worker1 # [ 128.372443] nix-daemon[868]: reaped child process 1521, status = failed with exit code 16339lb # [ 128.446193] niks3-server[1238]: 2026/09/21 15:09:22 INFO lead: acquired remote=192.168.1.4:458046340worker2 # [ 128.447104] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:23Z level=info event=scheduler_take_over6341lb # [ 128.465600] envoy[979]: conn peer=[2001:db8:1::1]:42682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=1429 ms=84336342lb # [ 128.469379] envoy[979]: conn peer=[2001:db8:1::4]:36812 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=23104 tx=1719 ms=342296343lb # [ 128.472988] envoy[979]: conn peer=[2001:db8:1::3]:53200 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=993 tx=363 ms=174736344worker1 # [ 128.477205] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6345worker1 # [ 128.481667] systemd[1]: Starting Nix worker-protocol over gRPC...6346worker2 # [ 128.580599] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:23Z level=info event=scheduler_connected addr=lb:500516347worker1 # [ 128.669306] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6348worker1 # [ 128.671408] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6349worker1 # [ 128.675371] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16350worker1 # [ 128.683067] nix-grpc-daemon[1558]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6351worker1 # [ 128.684377] nix-grpc-daemon[1558]: E0000 00:00:1790003363.525250 1558 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6352worker1 # [ 128.689065] nix-grpc-daemon[1558]: E0000 00:00:1790003363.530344 1558 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6353worker1 # [ 128.691158] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6354worker1 # [ 128.695882] systemd[1]: Started Nix worker-protocol over gRPC.6355worker2 # [ 128.741896] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6356worker1 # [ 128.704080] nix-grpc-daemon[1558]: E0000 00:00:1790003363.545888 1599 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6357worker1 # [ 128.715052] nix-grpc-daemon[1558]: E0000 00:00:1790003363.557646 1599 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6358worker1 # [ 128.719054] nix-grpc-daemon[1558]: E0000 00:00:1790003363.559920 1599 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6359worker1 # [ 128.729251] nix-grpc-daemon[1558]: E0000 00:00:1790003363.571820 1599 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6360worker1 # [ 128.738322] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=scheduler_connected addr=lb:500516361worker1 # [ 128.740749] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6362client # [ 128.898636] nix[1241]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting6363worker2 # [ 128.943463] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:23Z level=info event=scheduler_connected addr=lb:500516364worker2 # [ 128.947312] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6365worker1 # [ 128.942056] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=scheduler_connected addr=lb:500516366worker1 # [ 128.945949] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6367worker2 # [ 129.148113] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516368worker2 # [ 129.151970] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6369worker1 # [ 129.146768] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=scheduler_connected addr=lb:500516370worker1 # [ 129.150617] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6371worker2 # [ 129.353109] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516372worker2 # [ 129.356943] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6373worker1 # [ 129.351583] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516374worker1 # [ 129.355527] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6375worker2 # [ 129.558845] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516376worker2 # [ 129.562614] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6377worker1 # [ 129.556493] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516378worker1 # [ 129.560439] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6379worker2 # [ 129.763677] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516380worker2 # [ 129.767617] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6381worker1 # [ 129.762758] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516382worker1 # [ 129.765881] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6383client # [ 129.902270] nix[1241]: scheduler at lb:50051: no healthy upstream, reconnecting6384worker2 # [ 129.968780] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516385worker2 # [ 129.972744] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6386worker1 # [ 129.963752] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:24Z level=info event=scheduler_connected addr=lb:500516387worker1 # [ 129.967735] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6388worker2 # [ 130.173869] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:25Z level=info event=scheduler_connected addr=lb:500516389worker2 # [ 130.190143] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:25Z level=info event=oidc_jwks provider=mock keys=16390worker2 # [ 130.194059] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:450666391worker2 # [ 130.197883] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06392worker1 # [ 130.168616] nix-grpc-daemon[1558]: ts=2026-09-21T15:09:25Z level=info event=scheduler_connected addr=lb:500516393worker2 # [ 130.211361] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:450666394worker2 # [ 130.214946] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06395worker2 # [ 131.903865] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:26Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06396worker2 # [ 131.909466] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:26Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16397worker2 # [ 131.959764] nix-grpc-daemon[1452]: ts=2026-09-21T15:09:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450766398worker2 # [ 131.971324] nix-daemon[857]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)6399worker2 # [ 131.988942] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)6400worker2 # [ 132.077897] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 137 ms (attempt 2/5)6401worker2 # [ 132.220834] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 317 ms (attempt 3/5)6402worker2 # [ 132.543802] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 339 ms (attempt 4/5)6403worker2 # [ 132.889162] nix-grpc-daemon[1452]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6404worker2 # [ 132.893056] nix-grpc-daemon[1452]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6405worker2 # [ 132.909640] nix-grpc-daemon[1452]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6406worker2 # [ 132.931350] nix-daemon[857]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)6407client # [ 132.955073] nix[1241]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6408worker2 # [ 132.947297] nix-daemon[857]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)6409client: (finished: must succeed: sleep 5.0, in 5.06 seconds)6410client: waiting for success: ! systemctl is-active mid6411worker2 # [ 224.446886] nix-grpc-daemon[1452]: ts=2026-09-21T15:10:59Z level=info event=niks3_push_started pid=15606412worker2 # [ 224.467293] nix-daemon[857]: reaped child process 1535, status = succeeded6413worker2 # [ 224.695465] nix-grpc-daemon[1560]: time=2026-09-21T15:10:59.591Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6414worker2 # [ 224.907352] nix-daemon[857]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6415worker2 # [ 224.919301] nix-daemon[857]: reaped child process 1576, status = succeeded6416lb # [ 224.943596] niks3-server[1238]: 2026/09/21 15:10:59 INFO Received uploads request method=POST path=/api/pending_closures6417worker2 # [ 224.953699] nix-grpc-daemon[1560]: time=2026-09-21T15:10:59.850Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6418worker2 # [ 224.955275] nix-grpc-daemon[1560]: time=2026-09-21T15:10:59.852Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6419lb # [ 225.004145] niks3-server[1238]: 2026/09/21 15:10:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6420lb # [ 225.021605] niks3-server[1238]: 2026/09/21 15:10:59 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6421lb # [ 225.024091] niks3-server[1238]: 2026/09/21 15:10:59 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6422lb # [ 225.027366] niks3-server[1238]: 2026/09/21 15:10:59 INFO Signed narinfos id=42 count=16423worker2 # [ 225.024957] nix-grpc-daemon[1560]: time=2026-09-21T15:10:59.921Z level=INFO msg="Uploading 1 narinfos"6424lb # [ 225.028698] niks3-server[1238]: 2026/09/21 15:10:59 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6425lb # [ 225.038355] niks3-server[1238]: 2026/09/21 15:10:59 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6426lb # [ 225.044513] niks3-server[1238]: 2026/09/21 15:10:59 INFO Completed upload id=426427worker2 # [ 225.042138] nix-grpc-daemon[1560]: time=2026-09-21T15:10:59.938Z level=INFO msg="Upload complete. (342ms)"6428lb # [ 225.049903] niks3-server[1238]: 2026/09/21 15:10:59 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6429worker2 # [ 225.050145] nix-grpc-daemon[1452]: ts=2026-09-21T15:10:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45076 duration_s=93 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16430worker2 # [ 225.064246] nix-daemon[857]: reaped child process 1533, status = succeeded6431client # [ 225.107534] systemd[1]: mid.service: Deactivated successfully.6432client # [ 225.112189] systemd[1]: mid.service: Consumed 237ms CPU time over 1min 40.638s wall clock time, 41.4M memory peak, 4K written to disk, 9.1K incoming IP traffic, 11.1K outgoing IP traffic.6433client: (finished: waiting for success: ! systemctl is-active mid, in 91.63 seconds)6434client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6435client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6436(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 100.98 seconds)6437subtest: interrupting the client stops the build on the worker6438client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6439client # [ 225.431841] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6440client # Running as unit: intr.service; invocation ID: 767f877c7f674057b6f0e63a50c6a8796441client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6442worker1 # [ 225.584477] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:335386443worker1 # [ 225.595910] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6444client # [ 225.658350] nix[1707]: don't know how to build these paths:6445client # [ 225.659250] nix[1707]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6446worker2 # [ 225.660420] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:00Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06447worker2 # [ 225.664539] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:00Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16448worker1 # [ 225.663873] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386449worker1 # [ 225.676678] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)6450worker1 # [ 225.755576] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 2/5)6451worker1 # [ 225.900706] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 338 ms (attempt 3/5)6452worker1 # [ 226.244763] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 719 ms (attempt 4/5)6453worker1 # [ 226.965562] nix-grpc-daemon[1558]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6454worker1 # [ 226.968776] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6455worker1 # [ 226.985094] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335386456client # [ 227.045843] nix[1707]: copying 1 paths to 'lb:50051'...6457client # [ 227.047538] nix[1707]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6458worker1 # [ 226.996811] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335386459worker1 # [ 227.008491] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6460worker1 # [ 227.030429] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:01Z level=info event=niks3_push_started pid=16346461worker1 # [ 227.300345] nix-grpc-daemon[1634]: time=2026-09-21T15:11:02.142Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6462worker1 # [ 227.506773] nix-daemon[868]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6463worker1 # [ 227.517464] nix-daemon[868]: reaped child process 1650, status = succeeded6464worker1 # [ 227.735282] nix-daemon[868]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6465worker1 # [ 227.749187] nix-daemon[868]: reaped child process 1658, status = succeeded6466lb # [ 227.811777] niks3-server[1238]: 2026/09/21 15:11:02 INFO Received uploads request method=POST path=/api/pending_closures6467worker1 # [ 227.779556] nix-grpc-daemon[1634]: time=2026-09-21T15:11:02.622Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6468worker1 # [ 227.782701] nix-grpc-daemon[1634]: time=2026-09-21T15:11:02.623Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6469lb # [ 227.848106] niks3-server[1238]: 2026/09/21 15:11:02 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6470lb # [ 227.857563] niks3-server[1238]: 2026/09/21 15:11:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6471lb # [ 227.862372] niks3-server[1238]: 2026/09/21 15:11:02 INFO Signed narinfos id=43 count=16472worker1 # [ 227.821774] nix-grpc-daemon[1634]: time=2026-09-21T15:11:02.664Z level=INFO msg="Uploading 1 narinfos"6473lb # [ 227.876620] niks3-server[1238]: 2026/09/21 15:11:02 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6474lb # [ 227.881453] niks3-server[1238]: 2026/09/21 15:11:02 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6475lb # [ 227.891043] niks3-server[1238]: 2026/09/21 15:11:02 INFO Completed upload id=436476worker1 # [ 227.847205] nix-grpc-daemon[1634]: time=2026-09-21T15:11:02.689Z level=INFO msg="Upload complete. (543ms)"6477lb # [ 227.893107] niks3-server[1238]: 2026/09/21 15:11:02 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6478worker1 # [ 227.849764] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=4246479worker1 # [ 227.854609] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386480worker1 # [ 227.862162] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6481worker1 # [ 227.864502] nix-daemon[868]: reaped child process 1632, status = succeeded6482worker1 # [ 227.869952] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6483client # [ 227.919352] nix[1707]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6484lb # [ 228.470662] envoy[979]: conn peer=[2001:db8:1::1]:37160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4291 tx=2001 ms=1003856485client: must succeed: systemctl kill -s INT intr6486client # [ 229.087196] systemd[1]: intr.service: Sent signal SIGINT to main process 1707 (nix) on client request.6487worker1 # [ 229.044243] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:03Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6488client # [ 229.093798] nix[1707]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6489client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6490client # [ 229.100959] nix[1707]: error: interrupted by the user6491client # [ 229.130139] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6492client # [ 229.131569] systemd[1]: intr.service: Failed with result 'exit-code'.6493client # [ 229.133549] systemd[1]: intr.service: Consumed 112ms CPU time over 3.701s wall clock time, 43.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.6494worker1 # [ 229.087395] nix-daemon[1663]: unexpected Nix daemon error: error: interrupted by the user6495worker1 # [ 229.093572] nix-daemon[868]: reaped child process 1661, status = succeeded6496worker1 # [ 229.096628] nix-daemon[868]: reaped child process 1663, status = failed with exit code 16497(finished: subtest: interrupting the client stops the build on the worker, in 3.84 seconds)6498subtest: a deploy mid-build drains: the build finishes, the new generation takes over6499client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6500client # [ 229.285080] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6501client # Running as unit: sw.service; invocation ID: 369fe6de3bca47c29f2d01ca4bf915836502client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.06 seconds)6503client # [ 229.485694] nix[1728]: don't know how to build these paths:6504client # [ 229.486565] nix[1728]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6505worker1 # [ 229.434299] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:335386506worker2 # [ 229.486884] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:04Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06507worker2 # [ 229.491392] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:04Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16508worker2 # [ 229.529185] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450766509worker2 # [ 229.545779] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:450766510client # [ 229.565138] nix[1728]: copying 1 paths to 'lb:50051'...6511client # [ 229.566559] nix[1728]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6512worker2 # [ 229.556817] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:450766513worker2 # [ 229.565958] nix-daemon[857]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)6514worker2 # [ 229.789748] nix-daemon[857]: accepted connection from pid 1615, user nix-grpc-daemon (trusted)6515worker2 # [ 229.801707] nix-daemon[857]: reaped child process 1618, status = succeeded6516worker2 # [ 230.017314] nix-daemon[857]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6517worker2 # [ 230.029249] nix-daemon[857]: reaped child process 1623, status = succeeded6518lb # [ 230.055153] niks3-server[1238]: 2026/09/21 15:11:04 INFO Received uploads request method=POST path=/api/pending_closures6519worker2 # [ 230.059794] nix-grpc-daemon[1560]: time=2026-09-21T15:11:04.956Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6520worker2 # [ 230.061878] nix-grpc-daemon[1560]: time=2026-09-21T15:11:04.958Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6521lb # [ 230.084892] niks3-server[1238]: 2026/09/21 15:11:04 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6522lb # [ 230.095063] niks3-server[1238]: 2026/09/21 15:11:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6523worker2 # [ 230.100322] nix-grpc-daemon[1560]: time=2026-09-21T15:11:04.996Z level=INFO msg="Uploading 1 narinfos"6524lb # [ 230.102836] niks3-server[1238]: 2026/09/21 15:11:04 INFO Signed narinfos id=44 count=16525lb # [ 230.107109] niks3-server[1238]: 2026/09/21 15:11:04 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6526lb # [ 230.119144] niks3-server[1238]: 2026/09/21 15:11:04 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6527lb # [ 230.124048] niks3-server[1238]: 2026/09/21 15:11:04 INFO Completed upload id=446528worker2 # [ 230.124388] nix-grpc-daemon[1560]: time=2026-09-21T15:11:05.021Z level=INFO msg="Upload complete. (544ms)"6529lb # [ 230.127495] niks3-server[1238]: 2026/09/21 15:11:04 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6530worker2 # [ 230.128856] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45076 duration_s=0 paths=1 nar_bytes_in=4166531worker2 # [ 230.135604] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:450766532worker2 # [ 230.147755] nix-daemon[857]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)6533worker2 # [ 230.153617] nix-daemon[857]: reaped child process 1613, status = succeeded6534worker2 # [ 230.162548] nix-daemon[857]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)6535client # [ 230.170257] nix[1728]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6536worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26537worker2 # Checking switch inhibitors... done6538worker2 # [ 233.989354] nixos[1663]: switching to system configuration /nix/store/fpss1a9rmrhp3y01197y7pg9lknik4aw-nixos-system-worker2-test6539worker2 # activating the configuration...6540worker2 # [ 233.994222] systemd[1]: Stopped target Remote File Systems.6541worker2 # [ 233.995086] systemd[1]: Stopped target Local File Systems.6542worker2 # setting up /etc...6543worker2 # [ 234.748857] systemd[1]: Reload requested from client PID 1663 ('.switch-to-conf') (unit backdoor.service)...6544worker2 # [ 234.750544] systemd[1]: Reloading...6545worker2 # [ 235.650336] systemd[1]: Reloading finished in 896 ms.6546worker2 # restarting sysinit-reactivation.target6547worker2 # [ 235.708853] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6548worker2 # [ 235.863207] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6549worker2 # [ 235.866366] systemd[1]: Reached target Reactivate sysinit units.6550worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6551worker2 # [ 235.871766] dbus-broker-launch[533]: Caught SIGHUP, trigger reload.6552worker2 # [ 235.872983] systemd[1]: Reloading D-Bus System Message Bus...6553worker2 # [ 235.876917] systemd[1]: Reloading Nix worker-protocol over gRPC...6554worker2 # [ 235.905251] dbus-broker-launch[533]: Looking up NSS user entry for 'systemd-timesync'...6555worker2 # [ 235.913681] dbus-broker-launch[533]: NSS returned no entry for 'systemd-timesync'6556worker2 # [ 235.914791] dbus-broker-launch[533]: Invalid user-name in /nix/store/s93qka0z548yjbxqnmj4c4b32apww6jv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6557worker2 # [ 235.929385] systemd[1]: Reloaded Nix worker-protocol over gRPC.6558worker2 # [ 235.948065] systemd[1]: Reloaded D-Bus System Message Bus.6559worker2 # [ 235.952524] systemd[1]: Reached target Local File Systems.6560worker2 # [ 235.958403] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6561worker2 # [ 235.962312] systemd[1]: Starting Create SUID/SGID Wrappers...6562worker2 # [ 235.963449] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6563worker2 # [ 235.964687] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676564worker2 # [ 235.966701] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6565worker2 # [ 235.969408] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6566worker2 # [ 235.971607] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6567worker2 # [ 235.978038] systemd[1]: Reached target Remote File Systems.6568worker2 # [ 235.982787] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6569worker2 # [ 235.984054] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6570worker2 # [ 236.466878] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:11Z level=info event=draining6571worker2 # [ 236.512936] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6572worker2 # [ 236.514407] systemd[1]: Finished Create SUID/SGID Wrappers.6573worker2 # [ 236.520707] systemd[1]: Starting linger-users.service...6574worker2 # [ 236.571426] systemd[1]: linger-users.service: Deactivated successfully.6575worker2 # [ 236.572909] systemd[1]: Finished linger-users.service.6576worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6577worker2 # [ 236.829601] nixos[1663]: finished switching to system configuration /nix/store/fpss1a9rmrhp3y01197y7pg9lknik4aw-nixos-system-worker2-test6578worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.85 seconds)6579worker2: must succeed: pgrep -f 'read -t [9]0 x'6580worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6581client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26582client # don't know how to build these paths:6583client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6584worker1 # [ 237.041235] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:335386585worker1 # [ 237.046499] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335386586worker2 # [ 237.110240] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:12Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06587worker2 # [ 237.112103] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:12Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16588worker1 # [ 237.121413] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386589worker1 # [ 237.134384] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335386590client # copying 1 paths to 'lb:50051'...6591worker1 # [ 237.147171] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335386592client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6593worker1 # [ 237.168315] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6594worker1 # [ 237.386161] nix-daemon[868]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6595worker1 # [ 237.398172] nix-daemon[868]: reaped child process 1713, status = succeeded6596worker1 # [ 237.605586] nix-daemon[868]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6597worker1 # [ 237.617919] nix-daemon[868]: reaped child process 1718, status = succeeded6598lb # [ 237.682981] niks3-server[1238]: 2026/09/21 15:11:12 INFO Received uploads request method=POST path=/api/pending_closures6599worker1 # [ 237.646598] nix-grpc-daemon[1634]: time=2026-09-21T15:11:12.488Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6600worker1 # [ 237.648103] nix-grpc-daemon[1634]: time=2026-09-21T15:11:12.488Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6601lb # [ 237.708078] niks3-server[1238]: 2026/09/21 15:11:12 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6602lb # [ 237.722074] niks3-server[1238]: 2026/09/21 15:11:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6603worker1 # [ 237.686300] nix-grpc-daemon[1634]: time=2026-09-21T15:11:12.528Z level=INFO msg="Uploading 1 narinfos"6604lb # [ 237.730123] niks3-server[1238]: 2026/09/21 15:11:12 INFO Signed narinfos id=45 count=16605lb # [ 237.738750] niks3-server[1238]: 2026/09/21 15:11:12 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6606lb # [ 237.748747] niks3-server[1238]: 2026/09/21 15:11:12 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6607lb # [ 237.755943] niks3-server[1238]: 2026/09/21 15:11:12 INFO Completed upload id=456608worker1 # [ 237.712129] nix-grpc-daemon[1634]: time=2026-09-21T15:11:12.554Z level=INFO msg="Upload complete. (531ms)"6609lb # [ 237.758901] niks3-server[1238]: 2026/09/21 15:11:12 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6610worker1 # [ 237.716653] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=5286611worker1 # [ 237.722414] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386612worker1 # [ 237.731662] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6613worker1 # [ 237.741158] nix-daemon[868]: reaped child process 1708, status = succeeded6614client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6615client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6616worker1 # [ 237.746281] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6617client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6618client # farm-a> LOG-farm-a-during-drain6619lb # [ 238.471655] envoy[979]: conn peer=[2001:db8:1::1]:42898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2967 tx=852 ms=34866620worker1 # [ 238.570856] nix-daemon[868]: reaped child process 1722, status = succeeded6621worker1 # [ 238.771340] nix-daemon[868]: accepted connection from pid 1738, user nix-grpc-daemon (trusted)6622worker1 # [ 238.783251] nix-daemon[868]: reaped child process 1741, status = succeeded6623lb # [ 238.848497] niks3-server[1238]: 2026/09/21 15:11:13 INFO Received uploads request method=POST path=/api/pending_closures6624worker1 # [ 238.813646] nix-grpc-daemon[1634]: time=2026-09-21T15:11:13.656Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6625worker1 # [ 238.815319] nix-grpc-daemon[1634]: time=2026-09-21T15:11:13.657Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6626lb # [ 238.880845] niks3-server[1238]: 2026/09/21 15:11:13 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6627lb # [ 238.900093] niks3-server[1238]: 2026/09/21 15:11:13 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6628lb # [ 238.905873] niks3-server[1238]: 2026/09/21 15:11:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6629lb # [ 238.909478] niks3-server[1238]: 2026/09/21 15:11:13 INFO Signed narinfos id=46 count=16630worker1 # [ 238.868553] nix-grpc-daemon[1634]: time=2026-09-21T15:11:13.710Z level=INFO msg="Uploading 1 narinfos"6631lb # [ 238.914868] niks3-server[1238]: 2026/09/21 15:11:13 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6632lb # [ 238.924407] niks3-server[1238]: 2026/09/21 15:11:13 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6633lb # [ 238.927502] niks3-server[1238]: 2026/09/21 15:11:13 INFO Completed upload id=466634lb # [ 238.929826] niks3-server[1238]: 2026/09/21 15:11:13 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6635worker1 # [ 238.885190] nix-grpc-daemon[1634]: time=2026-09-21T15:11:13.727Z level=INFO msg="Upload complete. (332ms)"6636worker1 # [ 238.891231] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16637worker2 # [ 238.935697] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:13Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06638worker2 # [ 238.938237] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:13Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16639worker1 # [ 238.902944] nix-daemon[868]: reaped child process 1720, status = succeeded6640worker1 # [ 238.939166] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386641worker1 # [ 238.952387] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335386642worker1 # [ 238.967208] nix-grpc-daemon[1634]: time=2026-09-21T15:11:13.809Z level=INFO msg="All 1 paths already cached"6643client # copying 1 paths to 'lb:50051'...6644worker1 # [ 238.975650] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335386645client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6646worker1 # [ 238.995140] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6647worker1 # [ 239.216897] nix-daemon[868]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6648worker1 # [ 239.228958] nix-daemon[868]: reaped child process 1748, status = succeeded6649worker1 # [ 239.441848] nix-daemon[868]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6650worker1 # [ 239.453604] nix-daemon[868]: reaped child process 1753, status = succeeded6651lb # [ 239.520302] niks3-server[1238]: 2026/09/21 15:11:13 INFO Received uploads request method=POST path=/api/pending_closures6652worker1 # [ 239.485855] nix-grpc-daemon[1634]: time=2026-09-21T15:11:14.328Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6653worker1 # [ 239.487290] nix-grpc-daemon[1634]: time=2026-09-21T15:11:14.328Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6654lb # [ 239.547145] niks3-server[1238]: 2026/09/21 15:11:13 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6655lb # [ 239.559100] niks3-server[1238]: 2026/09/21 15:11:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6656worker1 # [ 239.521728] nix-grpc-daemon[1634]: time=2026-09-21T15:11:14.364Z level=INFO msg="Uploading 1 narinfos"6657lb # [ 239.565374] niks3-server[1238]: 2026/09/21 15:11:13 INFO Signed narinfos id=47 count=16658lb # [ 239.569976] niks3-server[1238]: 2026/09/21 15:11:13 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6659lb # [ 239.584062] niks3-server[1238]: 2026/09/21 15:11:14 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6660lb # [ 239.588762] niks3-server[1238]: 2026/09/21 15:11:14 INFO Completed upload id=476661worker1 # [ 239.547110] nix-grpc-daemon[1634]: time=2026-09-21T15:11:14.389Z level=INFO msg="Upload complete. (539ms)"6662lb # [ 239.592279] niks3-server[1238]: 2026/09/21 15:11:14 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6663worker1 # [ 239.549563] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=6406664worker1 # [ 239.554872] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386665worker1 # [ 239.564138] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6666worker1 # [ 239.572267] nix-daemon[868]: reaped child process 1743, status = succeeded6667client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6668worker1 # [ 239.578342] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6669client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6670client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6671client # farm-b> LOG-farm-b-during-drain6672worker1 # [ 240.601335] nix-daemon[868]: reaped child process 1757, status = succeeded6673worker1 # [ 240.812569] nix-daemon[868]: accepted connection from pid 1773, user nix-grpc-daemon (trusted)6674worker1 # [ 240.823348] nix-daemon[868]: reaped child process 1776, status = succeeded6675lb # [ 240.890876] niks3-server[1238]: 2026/09/21 15:11:15 INFO Received uploads request method=POST path=/api/pending_closures6676worker1 # [ 240.856278] nix-grpc-daemon[1634]: time=2026-09-21T15:11:15.698Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6677worker1 # [ 240.858630] nix-grpc-daemon[1634]: time=2026-09-21T15:11:15.700Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6678lb # [ 240.923138] niks3-server[1238]: 2026/09/21 15:11:15 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6679lb # [ 240.938581] niks3-server[1238]: 2026/09/21 15:11:15 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6680lb # [ 240.944494] niks3-server[1238]: 2026/09/21 15:11:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6681lb # [ 240.948629] niks3-server[1238]: 2026/09/21 15:11:15 INFO Signed narinfos id=48 count=16682worker1 # [ 240.907909] nix-grpc-daemon[1634]: time=2026-09-21T15:11:15.750Z level=INFO msg="Uploading 1 narinfos"6683lb # [ 240.958720] niks3-server[1238]: 2026/09/21 15:11:15 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6684lb # [ 240.966118] niks3-server[1238]: 2026/09/21 15:11:15 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6685lb # [ 240.970305] niks3-server[1238]: 2026/09/21 15:11:15 INFO Completed upload id=486686worker1 # [ 240.926648] nix-grpc-daemon[1634]: time=2026-09-21T15:11:15.769Z level=INFO msg="Upload complete. (342ms)"6687lb # [ 240.972365] niks3-server[1238]: 2026/09/21 15:11:15 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6688worker1 # [ 240.931772] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16689worker2 # [ 240.977774] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:15Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06690worker2 # [ 240.980294] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:15Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16691worker1 # [ 240.946590] nix-daemon[868]: reaped child process 1755, status = succeeded6692worker1 # [ 240.981279] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386693worker1 # [ 240.991673] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335386694worker1 # [ 241.004865] nix-grpc-daemon[1634]: time=2026-09-21T15:11:15.847Z level=INFO msg="All 2 paths already cached"6695client # copying 1 paths to 'lb:50051'...6696worker1 # [ 241.011969] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335386697client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6698worker1 # [ 241.023163] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6699worker1 # [ 241.246662] nix-daemon[868]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6700worker1 # [ 241.258951] nix-daemon[868]: reaped child process 1783, status = succeeded6701worker1 # [ 241.481132] nix-daemon[868]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6702worker1 # [ 241.492916] nix-daemon[868]: reaped child process 1788, status = succeeded6703lb # [ 241.560255] niks3-server[1238]: 2026/09/21 15:11:15 INFO Received uploads request method=POST path=/api/pending_closures6704worker1 # [ 241.525411] nix-grpc-daemon[1634]: time=2026-09-21T15:11:16.367Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6705worker1 # [ 241.526828] nix-grpc-daemon[1634]: time=2026-09-21T15:11:16.367Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6706lb # [ 241.586230] niks3-server[1238]: 2026/09/21 15:11:16 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6707lb # [ 241.598715] niks3-server[1238]: 2026/09/21 15:11:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6708worker1 # [ 241.559192] nix-grpc-daemon[1634]: time=2026-09-21T15:11:16.401Z level=INFO msg="Uploading 1 narinfos"6709lb # [ 241.602892] niks3-server[1238]: 2026/09/21 15:11:16 INFO Signed narinfos id=49 count=16710lb # [ 241.613884] niks3-server[1238]: 2026/09/21 15:11:16 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6711lb # [ 241.620400] niks3-server[1238]: 2026/09/21 15:11:16 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6712lb # [ 241.624083] niks3-server[1238]: 2026/09/21 15:11:16 INFO Completed upload id=496713worker1 # [ 241.582487] nix-grpc-daemon[1634]: time=2026-09-21T15:11:16.425Z level=INFO msg="Upload complete. (546ms)"6714worker1 # [ 241.586434] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=7686715worker1 # [ 241.591259] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386716worker1 # [ 241.599492] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6717worker1 # [ 241.603666] nix-daemon[868]: reaped child process 1778, status = succeeded6718client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6719client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6720worker1 # [ 241.609078] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6721client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6722client # farm-top> LOG-farm-top-during-drain6723worker1 # [ 242.622455] nix-daemon[868]: reaped child process 1792, status = succeeded6724worker1 # [ 242.822326] nix-daemon[868]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6725worker1 # [ 242.834366] nix-daemon[868]: reaped child process 1811, status = succeeded6726lb # [ 242.902447] niks3-server[1238]: 2026/09/21 15:11:17 INFO Received uploads request method=POST path=/api/pending_closures6727worker1 # [ 242.867967] nix-grpc-daemon[1634]: time=2026-09-21T15:11:17.710Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6728worker1 # [ 242.869684] nix-grpc-daemon[1634]: time=2026-09-21T15:11:17.712Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6729lb # [ 242.935682] niks3-server[1238]: 2026/09/21 15:11:17 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6730lb # [ 242.947873] niks3-server[1238]: 2026/09/21 15:11:17 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6731lb # [ 242.958956] niks3-server[1238]: 2026/09/21 15:11:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6732lb # [ 242.964091] niks3-server[1238]: 2026/09/21 15:11:17 INFO Signed narinfos id=50 count=16733worker1 # [ 242.923071] nix-grpc-daemon[1634]: time=2026-09-21T15:11:17.765Z level=INFO msg="Uploading 1 narinfos"6734lb # [ 242.970487] niks3-server[1238]: 2026/09/21 15:11:17 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6735lb # [ 242.981964] niks3-server[1238]: 2026/09/21 15:11:17 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6736lb # [ 242.984907] niks3-server[1238]: 2026/09/21 15:11:17 INFO Completed upload id=506737lb # [ 242.986636] niks3-server[1238]: 2026/09/21 15:11:17 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6738worker1 # [ 242.941361] nix-grpc-daemon[1634]: time=2026-09-21T15:11:17.783Z level=INFO msg="Upload complete. (325ms)"6739worker1 # [ 242.947292] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16740worker1 # [ 242.960304] nix-daemon[868]: reaped child process 1790, status = succeeded6741client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.14 seconds)6742client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6743client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6744worker2: must succeed: pkill -f 'read -t [9]0 x'6745worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6746client: waiting for success: ! systemctl is-active sw6747worker2 # [ 243.397115] nix-daemon[857]: reaped child process 1627, status = succeeded6748worker2 # [ 243.602469] nix-daemon[857]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6749worker2 # [ 243.613196] nix-daemon[857]: reaped child process 1812, status = succeeded6750lb # [ 243.638174] niks3-server[1238]: 2026/09/21 15:11:18 INFO Received uploads request method=POST path=/api/pending_closures6751worker2 # [ 243.644676] nix-grpc-daemon[1560]: time=2026-09-21T15:11:18.541Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6752worker2 # [ 243.646305] nix-grpc-daemon[1560]: time=2026-09-21T15:11:18.543Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6753lb # [ 243.675176] niks3-server[1238]: 2026/09/21 15:11:18 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6754lb # [ 243.680371] niks3-server[1238]: 2026/09/21 15:11:18 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6755lb # [ 243.690354] niks3-server[1238]: 2026/09/21 15:11:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6756lb # [ 243.694780] niks3-server[1238]: 2026/09/21 15:11:18 INFO Signed narinfos id=51 count=16757worker2 # [ 243.692891] nix-grpc-daemon[1560]: time=2026-09-21T15:11:18.588Z level=INFO msg="Uploading 1 narinfos"6758lb # [ 243.699131] niks3-server[1238]: 2026/09/21 15:11:18 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6759lb # [ 243.706307] niks3-server[1238]: 2026/09/21 15:11:18 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6760lb # [ 243.708877] niks3-server[1238]: 2026/09/21 15:11:18 INFO Completed upload id=516761worker2 # [ 243.706362] nix-grpc-daemon[1560]: time=2026-09-21T15:11:18.603Z level=INFO msg="Upload complete. (323ms)"6762lb # [ 243.710618] niks3-server[1238]: 2026/09/21 15:11:18 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6763worker2 # [ 243.712858] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45076 duration_s=13 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16764worker2 # [ 243.723836] nix-daemon[857]: reaped child process 1625, status = succeeded6765client # [ 243.761787] systemd[1]: sw.service: Deactivated successfully.6766client # [ 243.765664] systemd[1]: sw.service: Consumed 119ms CPU time over 14.479s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6767client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6768client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6769client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6770worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6771worker2 # [ 244.469820] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:19Z level=info event=signal_exit6772worker2 # [ 244.508312] systemd[1]: Starting scheduler-token.service...6773worker2 # [ 244.590885] systemd[1]: scheduler-token.service: Deactivated successfully.6774worker2 # [ 244.592081] systemd[1]: Finished scheduler-token.service.6775worker2 # [ 244.593683] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6776lb # [ 244.753601] niks3-server[1238]: 2026/09/21 15:11:19 INFO lead: released remote=192.168.1.4:458046777worker2 # [ 245.751159] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:20Z level=info event=scheduler_restarting6778worker1 # [ 245.710862] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:20Z level=info event=scheduler_restarting addr=lb:500516779worker2 # [ 245.758556] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:450666780worker2 # [ 245.762616] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:20Z level=info event=worker_gone addr=192.168.1.3:500516781worker2 # [ 245.766216] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:20Z level=info event=scheduler_restarting addr=lb:500516782worker2 # [ 245.769485] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:450666783worker2 # [ 245.773148] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:20Z level=info event=worker_gone addr=192.168.1.4:500516784worker2 # [ 245.790279] nix-daemon[857]: reaped child process 1526, status = succeeded6785worker1 # [ 245.910992] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:20Z level=info event=scheduler_connected addr=lb:500516786worker1 # [ 245.915089] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6787worker2 # [ 245.967970] nix-grpc-daemon[1452]: ts=2026-09-21T15:11:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6788worker1 # [ 246.115741] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:20Z level=info event=scheduler_connected addr=lb:500516789worker1 # [ 246.119690] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6790worker2 # [ 246.273423] nix-grpc-daemon[1452]: E0000 00:00:1790003481.169994 1497 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6791worker2 # [ 246.278416] nix-grpc-daemon[1452]: E0000 00:00:1790003481.170120 1497 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6792worker1 # [ 246.320554] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_connected addr=lb:500516793worker1 # [ 246.324568] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6794worker1 # [ 246.525545] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_connected addr=lb:500516795worker1 # [ 246.529422] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6796worker1 # [ 246.731280] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_connected addr=lb:500516797worker1 # [ 246.735064] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6798worker1 # [ 246.935996] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_connected addr=lb:500516799worker1 # [ 246.939933] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6800worker1 # [ 247.140389] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_connected addr=lb:500516801worker1 # [ 247.144274] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6802worker1 # [ 247.345182] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:22Z level=info event=scheduler_connected addr=lb:500516803worker1 # [ 247.349213] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6804worker2 # [ 247.580959] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6805worker2 # [ 247.584564] systemd[1]: nix-grpc-daemon.service: Consumed 920ms CPU time over 2min 4.373s wall clock time, 33M memory peak, 28.7K incoming IP traffic, 42.6K outgoing IP traffic.6806worker1 # [ 247.550161] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:22Z level=info event=scheduler_connected addr=lb:500516807worker1 # [ 247.554193] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6808worker2 # [ 247.601176] systemd[1]: Starting Nix worker-protocol over gRPC...6809worker1 # [ 247.755685] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:22Z level=info event=scheduler_connected addr=lb:500516810worker1 # [ 247.759502] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6811worker2 # [ 247.807372] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6812worker2 # [ 247.809888] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6813worker2 # [ 247.813665] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16814worker2 # [ 247.822066] nix-grpc-daemon[1829]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6815worker2 # [ 247.823531] nix-grpc-daemon[1829]: E0000 00:00:1790003482.717908 1829 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6816worker2 # [ 247.829070] nix-grpc-daemon[1829]: E0000 00:00:1790003482.723173 1829 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6817worker2 # [ 247.832176] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6818worker2 # [ 247.835264] systemd[1]: Started Nix worker-protocol over gRPC.6819worker2 # [ 247.840061] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=scheduler_take_over6820lb # [ 247.843394] niks3-server[1238]: 2026/09/21 15:11:22 INFO lead: acquired remote=192.168.1.4:518646821worker2 # [ 247.842206] nix-grpc-daemon[1829]: E0000 00:00:1790003482.738997 1871 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6822worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.42 seconds)6823lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6824lb: must succeed: curl -sf localhost:9901/clusters6825worker2 # [ 247.855138] nix-grpc-daemon[1829]: E0000 00:00:1790003482.751872 1871 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6826worker2 # [ 247.860056] nix-grpc-daemon[1829]: E0000 00:00:1790003482.754244 1871 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6827worker2 # [ 247.869383] nix-grpc-daemon[1829]: E0000 00:00:1790003482.766132 1871 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6828worker2 # [ 247.880586] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=scheduler_connected addr=lb:500516829worker2 # [ 247.882121] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6830lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6831worker1 # [ 247.960511] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:22Z level=info event=scheduler_connected addr=lb:500516832worker1 # [ 247.964450] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6833worker2 # [ 248.083335] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=scheduler_connected addr=lb:500516834worker2 # [ 248.087336] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6835worker1 # [ 248.165232] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:23Z level=info event=scheduler_connected addr=lb:500516836worker1 # [ 248.169155] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6837worker2 # [ 248.288387] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=scheduler_connected addr=lb:500516838worker2 # [ 248.292280] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6839worker1 # [ 248.370120] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:23Z level=info event=scheduler_connected addr=lb:500516840worker1 # [ 248.374159] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6841lb # [ 248.471945] envoy[979]: conn peer=[2001:db8:1::1]:42498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9628 tx=4078 ms=59236842lb # [ 248.475724] envoy[979]: conn peer=[2001:db8:1::1]:42902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=1430 ms=142516843lb # [ 248.479239] envoy[979]: conn peer=[2001:db8:1::4]:34092 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8235 tx=797 ms=1224846844worker2 # [ 248.493250] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=scheduler_connected addr=lb:500516845worker2 # [ 248.497149] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6846worker1 # [ 248.575080] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:23Z level=info event=scheduler_connected addr=lb:500516847worker2 # [ 248.620551] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:510446848worker2 # [ 248.625517] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06849worker2 # [ 248.698159] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=scheduler_connected addr=lb:500516850worker2 # [ 248.708140] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=oidc_jwks provider=mock keys=16851worker2 # [ 248.711649] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:510446852worker2 # [ 248.715527] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:23Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06853lb: must succeed: curl -sf localhost:9901/clusters6854lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6855lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)6856lb: must succeed: curl -sf localhost:9901/clusters6857lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6858worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6859worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6860(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 19.88 seconds)6861subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6862client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16863worker1 # [ 249.255812] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:335386864worker2 # [ 249.309504] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510486865worker2 # [ 249.321120] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6866worker2 # [ 249.338897] nix-grpc-daemon[1829]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)6867worker2 # [ 249.393893] nix-grpc-daemon[1829]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5)6868worker2 # [ 249.420731] nix-grpc-daemon[1829]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 246 ms (attempt 3/5)6869worker2 # [ 249.672314] nix-grpc-daemon[1829]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 598 ms (attempt 4/5)6870worker2 # [ 250.276576] nix-grpc-daemon[1829]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6871worker2 # [ 250.280250] nix-grpc-daemon[1829]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6872worker2 # [ 250.321973] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:25Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06873worker2 # [ 250.326125] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:25Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16874worker2 # [ 250.330360] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:25Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06875worker2 # [ 250.334166] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:25Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06876worker1 # [ 250.325312] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386877worker1 # [ 250.337368] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335386878worker1 # [ 250.348681] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335386879worker1 # [ 250.359984] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6880worker1 # [ 250.589459] nix-daemon[868]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6881worker1 # [ 250.601340] nix-daemon[868]: reaped child process 1818, status = succeeded6882worker1 # [ 250.826275] nix-daemon[868]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6883worker1 # [ 250.837951] nix-daemon[868]: reaped child process 1823, status = succeeded6884lb # [ 250.905703] niks3-server[1238]: 2026/09/21 15:11:25 INFO Received uploads request method=POST path=/api/pending_closures6885worker1 # [ 250.870287] nix-grpc-daemon[1634]: time=2026-09-21T15:11:25.712Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6886worker1 # [ 250.872122] nix-grpc-daemon[1634]: time=2026-09-21T15:11:25.714Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6887lb # [ 250.932772] niks3-server[1238]: 2026/09/21 15:11:25 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6888lb # [ 250.944475] niks3-server[1238]: 2026/09/21 15:11:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6889worker1 # [ 250.905351] nix-grpc-daemon[1634]: time=2026-09-21T15:11:25.747Z level=INFO msg="Uploading 1 narinfos"6890lb # [ 250.950398] niks3-server[1238]: 2026/09/21 15:11:25 INFO Signed narinfos id=52 count=16891lb # [ 250.954968] niks3-server[1238]: 2026/09/21 15:11:25 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6892lb # [ 250.965270] niks3-server[1238]: 2026/09/21 15:11:25 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6893worker1 # [ 250.927303] nix-grpc-daemon[1634]: time=2026-09-21T15:11:25.769Z level=INFO msg="Upload complete. (555ms)"6894lb # [ 250.971195] niks3-server[1238]: 2026/09/21 15:11:25 INFO Completed upload id=526895lb # [ 250.974596] niks3-server[1238]: 2026/09/21 15:11:25 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6896worker1 # [ 250.929433] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=5046897worker1 # [ 250.935079] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386898worker1 # [ 250.942238] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6899worker1 # [ 250.946895] nix-daemon[868]: reaped child process 1813, status = succeeded6900worker1 # [ 250.952087] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6901worker1 # [ 252.178447] nix-daemon[868]: reaped child process 1827, status = succeeded6902worker1 # [ 252.387146] nix-daemon[868]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6903worker1 # [ 252.398501] nix-daemon[868]: reaped child process 1846, status = succeeded6904lb # [ 252.464656] niks3-server[1238]: 2026/09/21 15:11:26 INFO Received uploads request method=POST path=/api/pending_closures6905worker1 # [ 252.429383] nix-grpc-daemon[1634]: time=2026-09-21T15:11:27.271Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6906worker1 # [ 252.434249] nix-grpc-daemon[1634]: time=2026-09-21T15:11:27.273Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6907lb # [ 252.497282] niks3-server[1238]: 2026/09/21 15:11:26 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6908lb # [ 252.513164] niks3-server[1238]: 2026/09/21 15:11:26 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6909lb # [ 252.522213] niks3-server[1238]: 2026/09/21 15:11:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6910lb # [ 252.526876] niks3-server[1238]: 2026/09/21 15:11:26 INFO Signed narinfos id=53 count=16911worker1 # [ 252.486390] nix-grpc-daemon[1634]: time=2026-09-21T15:11:27.328Z level=INFO msg="Uploading 1 narinfos"6912lb # [ 252.534110] niks3-server[1238]: 2026/09/21 15:11:26 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6913lb # [ 252.543847] niks3-server[1238]: 2026/09/21 15:11:26 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6914lb # [ 252.546869] niks3-server[1238]: 2026/09/21 15:11:26 INFO Completed upload id=536915lb # [ 252.548515] niks3-server[1238]: 2026/09/21 15:11:26 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6916worker1 # [ 252.504344] nix-grpc-daemon[1634]: time=2026-09-21T15:11:27.346Z level=INFO msg="Upload complete. (344ms)"6917worker1 # [ 252.509642] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16918worker2 # [ 252.557090] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:27Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16919worker2 # [ 252.562590] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:27Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06920worker1 # [ 252.524359] nix-daemon[868]: reaped child process 1825, status = succeeded6921worker1 # [ 252.557508] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386922worker1 # [ 252.570223] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335386923worker1 # [ 252.582332] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335386924worker1 # [ 252.591170] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6925worker1 # [ 252.810502] nix-daemon[868]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6926worker1 # [ 252.822436] nix-daemon[868]: reaped child process 1853, status = succeeded6927worker1 # [ 253.037229] nix-daemon[868]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6928worker1 # [ 253.049959] nix-daemon[868]: reaped child process 1858, status = succeeded6929lb # [ 253.113847] niks3-server[1238]: 2026/09/21 15:11:27 INFO Received uploads request method=POST path=/api/pending_closures6930worker1 # [ 253.079857] nix-grpc-daemon[1634]: time=2026-09-21T15:11:27.922Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6931worker1 # [ 253.081683] nix-grpc-daemon[1634]: time=2026-09-21T15:11:27.923Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6932lb # [ 253.142130] niks3-server[1238]: 2026/09/21 15:11:27 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6933lb # [ 253.154474] niks3-server[1238]: 2026/09/21 15:11:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6934worker1 # [ 253.116048] nix-grpc-daemon[1634]: time=2026-09-21T15:11:27.958Z level=INFO msg="Uploading 1 narinfos"6935lb # [ 253.160484] niks3-server[1238]: 2026/09/21 15:11:27 INFO Signed narinfos id=54 count=16936lb # [ 253.165320] niks3-server[1238]: 2026/09/21 15:11:27 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6937lb # [ 253.177215] niks3-server[1238]: 2026/09/21 15:11:27 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6938lb # [ 253.184619] niks3-server[1238]: 2026/09/21 15:11:27 INFO Completed upload id=546939worker1 # [ 253.140841] nix-grpc-daemon[1634]: time=2026-09-21T15:11:27.983Z level=INFO msg="Upload complete. (537ms)"6940lb # [ 253.186925] niks3-server[1238]: 2026/09/21 15:11:27 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6941worker1 # [ 253.145143] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=5046942worker1 # [ 253.149875] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386943worker1 # [ 253.159848] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6944worker1 # [ 253.163241] nix-daemon[868]: reaped child process 1848, status = succeeded6945worker1 # [ 253.168250] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6946worker1 # [ 253.992737] nix-daemon[868]: reaped child process 1862, status = succeeded6947worker1 # [ 254.201927] nix-daemon[868]: accepted connection from pid 1878, user nix-grpc-daemon (trusted)6948worker1 # [ 254.212362] nix-daemon[868]: reaped child process 1881, status = succeeded6949lb # [ 254.279219] niks3-server[1238]: 2026/09/21 15:11:28 INFO Received uploads request method=POST path=/api/pending_closures6950worker1 # [ 254.243384] nix-grpc-daemon[1634]: time=2026-09-21T15:11:29.085Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6951worker1 # [ 254.245039] nix-grpc-daemon[1634]: time=2026-09-21T15:11:29.087Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6952lb # [ 254.318147] niks3-server[1238]: 2026/09/21 15:11:28 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6953lb # [ 254.326305] niks3-server[1238]: 2026/09/21 15:11:28 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6954lb # [ 254.335982] niks3-server[1238]: 2026/09/21 15:11:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6955lb # [ 254.340693] niks3-server[1238]: 2026/09/21 15:11:28 INFO Signed narinfos id=55 count=16956worker1 # [ 254.299962] nix-grpc-daemon[1634]: time=2026-09-21T15:11:29.142Z level=INFO msg="Uploading 1 narinfos"6957lb # [ 254.348906] niks3-server[1238]: 2026/09/21 15:11:28 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6958lb # [ 254.356333] niks3-server[1238]: 2026/09/21 15:11:28 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6959lb # [ 254.362539] niks3-server[1238]: 2026/09/21 15:11:28 INFO Completed upload id=556960worker1 # [ 254.318383] nix-grpc-daemon[1634]: time=2026-09-21T15:11:29.160Z level=INFO msg="Upload complete. (343ms)"6961lb # [ 254.365150] niks3-server[1238]: 2026/09/21 15:11:28 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6962worker2 # [ 254.366702] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:29Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16963worker1 # [ 254.325750] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16964worker1 # [ 254.338645] nix-daemon[868]: reaped child process 1860, status = succeeded6965worker1 # [ 254.368406] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386966worker2 # [ 254.413143] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:29Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06967worker1 # [ 254.380509] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335386968worker1 # [ 254.391626] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335386969worker1 # [ 254.401173] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6970worker1 # [ 254.622511] nix-daemon[868]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6971worker1 # [ 254.635576] nix-daemon[868]: reaped child process 1888, status = succeeded6972worker1 # [ 254.856438] nix-daemon[868]: accepted connection from pid 1890, user nix-grpc-daemon (trusted)6973worker1 # [ 254.868252] nix-daemon[868]: reaped child process 1893, status = succeeded6974lb # [ 254.934628] niks3-server[1238]: 2026/09/21 15:11:29 INFO Received uploads request method=POST path=/api/pending_closures6975worker1 # [ 254.899968] nix-grpc-daemon[1634]: time=2026-09-21T15:11:29.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6976worker1 # [ 254.901498] nix-grpc-daemon[1634]: time=2026-09-21T15:11:29.744Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6977lb # [ 254.966721] niks3-server[1238]: 2026/09/21 15:11:29 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6978lb # [ 254.977751] niks3-server[1238]: 2026/09/21 15:11:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6979lb # [ 254.982097] niks3-server[1238]: 2026/09/21 15:11:29 INFO Signed narinfos id=56 count=16980worker1 # [ 254.940497] nix-grpc-daemon[1634]: time=2026-09-21T15:11:29.783Z level=INFO msg="Uploading 1 narinfos"6981lb # [ 254.987725] niks3-server[1238]: 2026/09/21 15:11:29 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6982lb # [ 254.997682] niks3-server[1238]: 2026/09/21 15:11:29 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6983lb # [ 255.003551] niks3-server[1238]: 2026/09/21 15:11:29 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6984lb # [ 255.006046] niks3-server[1238]: 2026/09/21 15:11:29 INFO Completed upload id=566985worker1 # [ 254.962408] nix-grpc-daemon[1634]: time=2026-09-21T15:11:29.804Z level=INFO msg="Upload complete. (549ms)"6986worker1 # [ 254.967142] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=5046987worker1 # [ 254.971513] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335386988worker1 # [ 254.979881] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6989worker1 # [ 254.984918] nix-daemon[868]: reaped child process 1883, status = succeeded6990worker1 # [ 254.991056] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6991worker1 # [ 255.813896] nix-daemon[868]: reaped child process 1897, status = succeeded6992worker1 # [ 256.021672] nix-daemon[868]: accepted connection from pid 1913, user nix-grpc-daemon (trusted)6993worker1 # [ 256.033365] nix-daemon[868]: reaped child process 1916, status = succeeded6994lb # [ 256.093530] niks3-server[1238]: 2026/09/21 15:11:30 INFO Received uploads request method=POST path=/api/pending_closures6995worker1 # [ 256.058343] nix-grpc-daemon[1634]: time=2026-09-21T15:11:30.900Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6996worker1 # [ 256.059839] nix-grpc-daemon[1634]: time=2026-09-21T15:11:30.902Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6997lb # [ 256.134157] niks3-server[1238]: 2026/09/21 15:11:30 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6998lb # [ 256.136252] niks3-server[1238]: 2026/09/21 15:11:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6999lb # [ 256.138329] niks3-server[1238]: 2026/09/21 15:11:30 INFO Signed narinfos id=57 count=17000worker1 # [ 256.094699] nix-grpc-daemon[1634]: time=2026-09-21T15:11:30.937Z level=INFO msg="Uploading 1 narinfos"7001lb # [ 256.141066] niks3-server[1238]: 2026/09/21 15:11:30 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7002lb # [ 256.146223] niks3-server[1238]: 2026/09/21 15:11:30 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete7003lb # [ 256.150620] niks3-server[1238]: 2026/09/21 15:11:30 INFO Completed upload id=577004worker1 # [ 256.106688] nix-grpc-daemon[1634]: time=2026-09-21T15:11:30.949Z level=INFO msg="Upload complete. (310ms)"7005lb # [ 256.152381] niks3-server[1238]: 2026/09/21 15:11:30 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7006worker1 # [ 256.109145] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=17007worker1 # [ 256.114655] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387008worker2 # [ 256.154266] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:31Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17009worker1 # [ 256.118871] nix-daemon[868]: reaped child process 1895, status = succeeded7010lb # [ 256.162283] niks3-server[1238]: 2026/09/21 15:11:30 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7011worker1 # [ 256.123829] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335387012worker2 # [ 256.164699] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:31Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07013worker1 # [ 256.130138] nix-grpc-daemon[1634]: time=2026-09-21T15:11:30.972Z level=INFO msg="All 1 paths already cached"7014worker1 # [ 256.133603] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335387015worker1 # [ 256.138296] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7016worker1 # [ 256.333882] nix-daemon[868]: accepted connection from pid 1920, user nix-grpc-daemon (trusted)7017worker1 # [ 256.346075] nix-daemon[868]: reaped child process 1923, status = succeeded7018worker1 # [ 256.555413] nix-daemon[868]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)7019worker1 # [ 256.567056] nix-daemon[868]: reaped child process 1928, status = succeeded7020lb # [ 256.630751] niks3-server[1238]: 2026/09/21 15:11:31 INFO Received uploads request method=POST path=/api/pending_closures7021worker1 # [ 256.598071] nix-grpc-daemon[1634]: time=2026-09-21T15:11:31.440Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7022worker1 # [ 256.599499] nix-grpc-daemon[1634]: time=2026-09-21T15:11:31.440Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7023lb # [ 256.661234] niks3-server[1238]: 2026/09/21 15:11:31 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7024lb # [ 256.670448] niks3-server[1238]: 2026/09/21 15:11:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign7025worker1 # [ 256.631755] nix-grpc-daemon[1634]: time=2026-09-21T15:11:31.474Z level=INFO msg="Uploading 1 narinfos"7026lb # [ 256.676943] niks3-server[1238]: 2026/09/21 15:11:31 INFO Signed narinfos id=58 count=17027lb # [ 256.682641] niks3-server[1238]: 2026/09/21 15:11:31 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7028lb # [ 256.692731] niks3-server[1238]: 2026/09/21 15:11:31 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete7029lb # [ 256.700240] niks3-server[1238]: 2026/09/21 15:11:31 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7030lb # [ 256.705736] niks3-server[1238]: 2026/09/21 15:11:31 INFO Completed upload id=587031worker1 # [ 256.661924] nix-grpc-daemon[1634]: time=2026-09-21T15:11:31.504Z level=INFO msg="Upload complete. (518ms)"7032worker1 # [ 256.665758] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=6247033worker1 # [ 256.672338] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387034worker1 # [ 256.681478] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7035worker1 # [ 256.690606] nix-daemon[868]: reaped child process 1918, status = succeeded7036worker1 # [ 256.697098] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7037worker1 # [ 258.122751] nix-daemon[868]: reaped child process 1932, status = succeeded7038worker1 # [ 258.327834] nix-daemon[868]: accepted connection from pid 1948, user nix-grpc-daemon (trusted)7039worker1 # [ 258.338412] nix-daemon[868]: reaped child process 1951, status = succeeded7040lb # [ 258.405677] niks3-server[1238]: 2026/09/21 15:11:32 INFO Received uploads request method=POST path=/api/pending_closures7041worker1 # [ 258.370900] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.213Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7042worker1 # [ 258.374043] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.214Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7043lb # [ 258.447725] niks3-server[1238]: 2026/09/21 15:11:32 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7044lb # [ 258.456583] niks3-server[1238]: 2026/09/21 15:11:32 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7045lb # [ 258.466177] niks3-server[1238]: 2026/09/21 15:11:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign7046lb # [ 258.471387] niks3-server[1238]: 2026/09/21 15:11:32 INFO Signed narinfos id=59 count=17047worker1 # [ 258.427712] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.270Z level=INFO msg="Uploading 1 narinfos"7048lb # [ 258.477699] niks3-server[1238]: 2026/09/21 15:11:32 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7049lb # [ 258.486215] niks3-server[1238]: 2026/09/21 15:11:32 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete7050lb # [ 258.492203] niks3-server[1238]: 2026/09/21 15:11:32 INFO Completed upload id=597051lb # [ 258.493848] niks3-server[1238]: 2026/09/21 15:11:32 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7052worker1 # [ 258.448474] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.291Z level=INFO msg="Upload complete. (342ms)"7053worker2 # [ 258.495900] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:33Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17054worker1 # [ 258.455798] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=17055worker1 # [ 258.462386] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387056worker2 # [ 258.506499] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:33Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07057worker1 # [ 258.473547] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335387058worker1 # [ 258.479774] nix-daemon[868]: reaped child process 1930, status = succeeded7059worker1 # [ 258.485187] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.327Z level=INFO msg="All 1 paths already cached"7060worker1 # [ 258.491059] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335387061worker1 # [ 258.497436] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7062worker1 # [ 258.695949] nix-daemon[868]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)7063worker1 # [ 258.708209] nix-daemon[868]: reaped child process 1958, status = succeeded7064worker1 # [ 258.925627] nix-daemon[868]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)7065worker1 # [ 258.937243] nix-daemon[868]: reaped child process 1963, status = succeeded7066lb # [ 259.003962] niks3-server[1238]: 2026/09/21 15:11:33 INFO Received uploads request method=POST path=/api/pending_closures7067worker1 # [ 258.968863] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.811Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7068worker1 # [ 258.970288] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.811Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7069lb # [ 259.030917] niks3-server[1238]: 2026/09/21 15:11:33 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7070lb # [ 259.041249] niks3-server[1238]: 2026/09/21 15:11:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign7071worker1 # [ 259.002817] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.845Z level=INFO msg="Uploading 1 narinfos"7072lb # [ 259.046681] niks3-server[1238]: 2026/09/21 15:11:33 INFO Signed narinfos id=60 count=17073lb # [ 259.050985] niks3-server[1238]: 2026/09/21 15:11:33 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7074lb # [ 259.063307] niks3-server[1238]: 2026/09/21 15:11:33 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete7075lb # [ 259.068114] niks3-server[1238]: 2026/09/21 15:11:33 INFO Completed upload id=607076worker1 # [ 259.026584] nix-grpc-daemon[1634]: time=2026-09-21T15:11:33.869Z level=INFO msg="Upload complete. (523ms)"7077lb # [ 259.071744] niks3-server[1238]: 2026/09/21 15:11:33 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7078worker1 # [ 259.029053] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=6247079worker1 # [ 259.034228] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387080worker1 # [ 259.044063] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7081worker1 # [ 259.048149] nix-daemon[868]: reaped child process 1953, status = succeeded7082worker1 # [ 259.053155] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7083worker1 # [ 260.278178] nix-daemon[868]: reaped child process 1967, status = succeeded7084worker1 # [ 260.460430] nix-daemon[868]: accepted connection from pid 1983, user nix-grpc-daemon (trusted)7085worker1 # [ 260.472185] nix-daemon[868]: reaped child process 1986, status = succeeded7086lb # [ 260.531910] niks3-server[1238]: 2026/09/21 15:11:34 INFO Received uploads request method=POST path=/api/pending_closures7087worker1 # [ 260.498189] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.340Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7088worker1 # [ 260.499733] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.342Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7089lb # [ 260.572288] niks3-server[1238]: 2026/09/21 15:11:34 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7090lb # [ 260.580625] niks3-server[1238]: 2026/09/21 15:11:35 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7091lb # [ 260.592765] niks3-server[1238]: 2026/09/21 15:11:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign7092worker1 # [ 260.556122] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.398Z level=INFO msg="Uploading 1 narinfos"7093lb # [ 260.600830] niks3-server[1238]: 2026/09/21 15:11:35 INFO Signed narinfos id=61 count=17094lb # [ 260.608706] niks3-server[1238]: 2026/09/21 15:11:35 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7095lb # [ 260.616595] niks3-server[1238]: 2026/09/21 15:11:35 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete7096lb # [ 260.621979] niks3-server[1238]: 2026/09/21 15:11:35 INFO Completed upload id=617097worker1 # [ 260.578372] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.420Z level=INFO msg="Upload complete. (317ms)"7098lb # [ 260.626421] niks3-server[1238]: 2026/09/21 15:11:35 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7099worker1 # [ 260.584630] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=17100worker2 # [ 260.627644] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:35Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17101worker1 # [ 260.598796] nix-daemon[868]: reaped child process 1965, status = succeeded7102worker2 # [ 260.637887] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:35Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07103worker1 # [ 260.629496] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387104worker1 # [ 260.641558] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335387105worker1 # [ 260.654827] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.497Z level=INFO msg="All 1 paths already cached"7106worker1 # [ 260.661598] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335387107worker1 # [ 260.672086] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7108worker1 # [ 260.859217] nix-daemon[868]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)7109worker1 # [ 260.870962] nix-daemon[868]: reaped child process 1993, status = succeeded7110worker1 # [ 261.033468] nix-daemon[868]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)7111worker1 # [ 261.044967] nix-daemon[868]: reaped child process 1998, status = succeeded7112lb # [ 261.105494] niks3-server[1238]: 2026/09/21 15:11:35 INFO Received uploads request method=POST path=/api/pending_closures7113worker1 # [ 261.072165] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7114worker1 # [ 261.073803] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.916Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7115lb # [ 261.137111] niks3-server[1238]: 2026/09/21 15:11:35 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7116lb # [ 261.150761] niks3-server[1238]: 2026/09/21 15:11:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7117worker1 # [ 261.114618] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.957Z level=INFO msg="Uploading 1 narinfos"7118lb # [ 261.158241] niks3-server[1238]: 2026/09/21 15:11:35 INFO Signed narinfos id=62 count=17119lb # [ 261.163484] niks3-server[1238]: 2026/09/21 15:11:35 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7120lb # [ 261.177487] niks3-server[1238]: 2026/09/21 15:11:35 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7121worker1 # [ 261.138550] nix-grpc-daemon[1634]: time=2026-09-21T15:11:35.981Z level=INFO msg="Upload complete. (454ms)"7122lb # [ 261.182711] niks3-server[1238]: 2026/09/21 15:11:35 INFO Completed upload id=627123lb # [ 261.186440] niks3-server[1238]: 2026/09/21 15:11:35 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7124worker1 # [ 261.141897] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=6247125worker1 # [ 261.146775] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387126worker1 # [ 261.158119] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7127worker1 # [ 261.162440] nix-daemon[868]: reaped child process 1988, status = succeeded7128worker1 # [ 261.167503] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7129worker1 # [ 262.780660] nix-daemon[868]: reaped child process 2002, status = succeeded7130worker1 # [ 262.966466] nix-daemon[868]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)7131worker1 # [ 262.978639] nix-daemon[868]: reaped child process 2021, status = succeeded7132lb # [ 263.043757] niks3-server[1238]: 2026/09/21 15:11:37 INFO Received uploads request method=POST path=/api/pending_closures7133worker1 # [ 263.008728] nix-grpc-daemon[1634]: time=2026-09-21T15:11:37.851Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7134worker1 # [ 263.010304] nix-grpc-daemon[1634]: time=2026-09-21T15:11:37.852Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7135lb # [ 263.080811] niks3-server[1238]: 2026/09/21 15:11:37 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7136lb # [ 263.088982] niks3-server[1238]: 2026/09/21 15:11:37 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7137lb # [ 263.096402] niks3-server[1238]: 2026/09/21 15:11:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7138lb # [ 263.100634] niks3-server[1238]: 2026/09/21 15:11:37 INFO Signed narinfos id=63 count=17139worker1 # [ 263.059767] nix-grpc-daemon[1634]: time=2026-09-21T15:11:37.902Z level=INFO msg="Uploading 1 narinfos"7140lb # [ 263.110545] niks3-server[1238]: 2026/09/21 15:11:37 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7141lb # [ 263.118497] niks3-server[1238]: 2026/09/21 15:11:37 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7142lb # [ 263.123988] niks3-server[1238]: 2026/09/21 15:11:37 INFO Completed upload id=637143worker1 # [ 263.080188] nix-grpc-daemon[1634]: time=2026-09-21T15:11:37.922Z level=INFO msg="Upload complete. (305ms)"7144lb # [ 263.126942] niks3-server[1238]: 2026/09/21 15:11:37 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7145worker1 # [ 263.087833] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=17146worker2 # [ 263.128539] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:38Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17147worker2 # [ 263.138347] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:38Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07148worker1 # [ 263.099916] nix-daemon[868]: reaped child process 2000, status = succeeded7149worker1 # [ 263.130854] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387150worker1 # [ 263.144703] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335387151worker1 # [ 263.158216] nix-grpc-daemon[1634]: time=2026-09-21T15:11:38.000Z level=INFO msg="All 2 paths already cached"7152worker1 # [ 263.165238] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335387153worker1 # [ 263.173437] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7154worker1 # [ 263.381650] nix-daemon[868]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)7155worker1 # [ 263.393672] nix-daemon[868]: reaped child process 2028, status = succeeded7156worker1 # [ 263.605158] nix-daemon[868]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)7157worker1 # [ 263.616320] nix-daemon[868]: reaped child process 2033, status = succeeded7158lb # [ 263.679354] niks3-server[1238]: 2026/09/21 15:11:38 INFO Received uploads request method=POST path=/api/pending_closures7159worker1 # [ 263.645701] nix-grpc-daemon[1634]: time=2026-09-21T15:11:38.488Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7160worker1 # [ 263.647248] nix-grpc-daemon[1634]: time=2026-09-21T15:11:38.489Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7161lb # [ 263.710723] niks3-server[1238]: 2026/09/21 15:11:38 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7162lb # [ 263.721601] niks3-server[1238]: 2026/09/21 15:11:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7163lb # [ 263.728474] niks3-server[1238]: 2026/09/21 15:11:38 INFO Signed narinfos id=64 count=17164worker1 # [ 263.688053] nix-grpc-daemon[1634]: time=2026-09-21T15:11:38.530Z level=INFO msg="Uploading 1 narinfos"7165lb # [ 263.733442] niks3-server[1238]: 2026/09/21 15:11:38 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7166lb # [ 263.746097] niks3-server[1238]: 2026/09/21 15:11:38 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7167lb # [ 263.751995] niks3-server[1238]: 2026/09/21 15:11:38 INFO Completed upload id=647168worker1 # [ 263.708281] nix-grpc-daemon[1634]: time=2026-09-21T15:11:38.550Z level=INFO msg="Upload complete. (522ms)"7169lb # [ 263.754213] niks3-server[1238]: 2026/09/21 15:11:38 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7170worker1 # [ 263.713141] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=7527171worker1 # [ 263.717980] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387172worker1 # [ 263.726846] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7173worker1 # [ 263.733673] nix-daemon[868]: reaped child process 2023, status = succeeded7174worker1 # [ 263.739869] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7175worker1 # [ 265.152471] nix-daemon[868]: reaped child process 2037, status = succeeded7176worker1 # [ 265.347659] nix-daemon[868]: accepted connection from pid 2053, user nix-grpc-daemon (trusted)7177worker1 # [ 265.359775] nix-daemon[868]: reaped child process 2056, status = succeeded7178lb # [ 265.425875] niks3-server[1238]: 2026/09/21 15:11:39 INFO Received uploads request method=POST path=/api/pending_closures7179worker1 # [ 265.392229] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.234Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7180worker1 # [ 265.394068] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.236Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7181lb # [ 265.467626] niks3-server[1238]: 2026/09/21 15:11:39 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7182lb # [ 265.476871] niks3-server[1238]: 2026/09/21 15:11:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7183lb # [ 265.481186] niks3-server[1238]: 2026/09/21 15:11:39 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7184worker1 # [ 265.444629] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.286Z level=INFO msg="Uploading 1 narinfos"7185lb # [ 265.488267] niks3-server[1238]: 2026/09/21 15:11:39 INFO Signed narinfos id=65 count=17186lb # [ 265.492701] niks3-server[1238]: 2026/09/21 15:11:39 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7187lb # [ 265.501759] niks3-server[1238]: 2026/09/21 15:11:39 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7188lb # [ 265.507174] niks3-server[1238]: 2026/09/21 15:11:39 INFO Completed upload id=657189worker1 # [ 265.462993] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.305Z level=INFO msg="Upload complete. (316ms)"7190lb # [ 265.509783] niks3-server[1238]: 2026/09/21 15:11:39 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7191worker1 # [ 265.469316] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=17192worker2 # [ 265.510766] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:40Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17193worker1 # [ 265.482954] nix-daemon[868]: reaped child process 2035, status = succeeded7194worker1 # [ 265.519194] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387195worker1 # [ 265.531707] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335387196worker1 # [ 265.544896] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.387Z level=INFO msg="All 2 paths already cached"7197worker1 # [ 265.551771] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335387198worker1 # [ 265.560372] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7199worker1 # [ 265.775361] nix-daemon[868]: accepted connection from pid 2060, user nix-grpc-daemon (trusted)7200worker1 # [ 265.787859] nix-daemon[868]: reaped child process 2063, status = succeeded7201worker1 # [ 266.001973] nix-daemon[868]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)7202worker1 # [ 266.013761] nix-daemon[868]: reaped child process 2068, status = succeeded7203lb # [ 266.078938] niks3-server[1238]: 2026/09/21 15:11:40 INFO Received uploads request method=POST path=/api/pending_closures7204worker1 # [ 266.044191] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.886Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7205worker1 # [ 266.046139] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.888Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7206lb # [ 266.109139] niks3-server[1238]: 2026/09/21 15:11:40 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7207lb # [ 266.117302] niks3-server[1238]: 2026/09/21 15:11:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7208lb # [ 266.121904] niks3-server[1238]: 2026/09/21 15:11:40 INFO Signed narinfos id=66 count=17209worker1 # [ 266.080949] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.923Z level=INFO msg="Uploading 1 narinfos"7210lb # [ 266.137764] niks3-server[1238]: 2026/09/21 15:11:40 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7211lb # [ 266.140819] niks3-server[1238]: 2026/09/21 15:11:40 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7212lb # [ 266.146698] niks3-server[1238]: 2026/09/21 15:11:40 INFO Completed upload id=667213worker1 # [ 266.102868] nix-grpc-daemon[1634]: time=2026-09-21T15:11:40.945Z level=INFO msg="Upload complete. (529ms)"7214lb # [ 266.149468] niks3-server[1238]: 2026/09/21 15:11:40 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7215worker1 # [ 266.106427] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=7527216worker1 # [ 266.111908] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387217worker1 # [ 266.122064] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7218worker1 # [ 266.125913] nix-daemon[868]: reaped child process 2058, status = succeeded7219worker1 # [ 266.131078] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7220worker1 # [ 267.344564] nix-daemon[868]: reaped child process 2072, status = succeeded7221worker1 # [ 267.535343] nix-daemon[868]: accepted connection from pid 2088, user nix-grpc-daemon (trusted)7222worker1 # [ 267.546068] nix-daemon[868]: reaped child process 2091, status = succeeded7223lb # [ 267.611633] niks3-server[1238]: 2026/09/21 15:11:42 INFO Received uploads request method=POST path=/api/pending_closures7224worker1 # [ 267.577706] nix-grpc-daemon[1634]: time=2026-09-21T15:11:42.420Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7225worker1 # [ 267.579281] nix-grpc-daemon[1634]: time=2026-09-21T15:11:42.421Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7226lb # [ 267.650255] niks3-server[1238]: 2026/09/21 15:11:42 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7227lb # [ 267.660224] niks3-server[1238]: 2026/09/21 15:11:42 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7228lb # [ 267.670854] niks3-server[1238]: 2026/09/21 15:11:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7229lb # [ 267.674518] niks3-server[1238]: 2026/09/21 15:11:42 INFO Signed narinfos id=67 count=17230worker1 # [ 267.630723] nix-grpc-daemon[1634]: time=2026-09-21T15:11:42.473Z level=INFO msg="Uploading 1 narinfos"7231lb # [ 267.679179] niks3-server[1238]: 2026/09/21 15:11:42 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7232lb # [ 267.689372] niks3-server[1238]: 2026/09/21 15:11:42 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7233lb # [ 267.692114] niks3-server[1238]: 2026/09/21 15:11:42 INFO Completed upload id=677234worker1 # [ 267.648387] nix-grpc-daemon[1634]: time=2026-09-21T15:11:42.490Z level=INFO msg="Upload complete. (308ms)"7235lb # [ 267.693799] niks3-server[1238]: 2026/09/21 15:11:42 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7236worker2 # [ 267.695924] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:42Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17237worker1 # [ 267.655242] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=17238worker1 # [ 267.667961] nix-daemon[868]: reaped child process 2070, status = succeeded7239worker1 # [ 267.704119] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387240worker1 # [ 267.716247] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335387241worker1 # [ 267.729205] nix-grpc-daemon[1634]: time=2026-09-21T15:11:42.571Z level=INFO msg="All 2 paths already cached"7242worker1 # [ 267.736320] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:335387243worker1 # [ 267.746165] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7244worker1 # [ 267.957778] nix-daemon[868]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)7245worker1 # [ 267.969898] nix-daemon[868]: reaped child process 2098, status = succeeded7246worker1 # [ 268.180523] nix-daemon[868]: accepted connection from pid 2100, user nix-grpc-daemon (trusted)7247worker1 # [ 268.193133] nix-daemon[868]: reaped child process 2103, status = succeeded7248lb # [ 268.258072] niks3-server[1238]: 2026/09/21 15:11:42 INFO Received uploads request method=POST path=/api/pending_closures7249worker1 # [ 268.222087] nix-grpc-daemon[1634]: time=2026-09-21T15:11:43.064Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7250worker1 # [ 268.223604] nix-grpc-daemon[1634]: time=2026-09-21T15:11:43.066Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7251lb # [ 268.286285] niks3-server[1238]: 2026/09/21 15:11:42 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7252lb # [ 268.299079] niks3-server[1238]: 2026/09/21 15:11:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7253worker1 # [ 268.261080] nix-grpc-daemon[1634]: time=2026-09-21T15:11:43.103Z level=INFO msg="Uploading 1 narinfos"7254lb # [ 268.304917] niks3-server[1238]: 2026/09/21 15:11:42 INFO Signed narinfos id=68 count=17255lb # [ 268.309441] niks3-server[1238]: 2026/09/21 15:11:42 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7256lb # [ 268.322620] niks3-server[1238]: 2026/09/21 15:11:42 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7257lb # [ 268.327565] niks3-server[1238]: 2026/09/21 15:11:42 INFO Completed upload id=687258worker1 # [ 268.283817] nix-grpc-daemon[1634]: time=2026-09-21T15:11:43.126Z level=INFO msg="Upload complete. (524ms)"7259lb # [ 268.330535] niks3-server[1238]: 2026/09/21 15:11:42 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7260worker1 # [ 268.287320] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=0 paths=1 nar_bytes_in=7527261worker1 # [ 268.292869] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:335387262worker1 # [ 268.303604] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7263worker1 # [ 268.306508] nix-daemon[868]: reaped child process 2093, status = succeeded7264worker1 # [ 268.311909] nix-daemon[868]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7265worker1 # [ 269.733441] nix-daemon[868]: reaped child process 2107, status = succeeded7266worker1 # [ 269.930098] nix-daemon[868]: accepted connection from pid 2123, user nix-grpc-daemon (trusted)7267worker1 # [ 269.942138] nix-daemon[868]: reaped child process 2126, status = succeeded7268lb # [ 270.007783] niks3-server[1238]: 2026/09/21 15:11:44 INFO Received uploads request method=POST path=/api/pending_closures7269worker1 # [ 269.972770] nix-grpc-daemon[1634]: time=2026-09-21T15:11:44.815Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7270worker1 # [ 269.974253] nix-grpc-daemon[1634]: time=2026-09-21T15:11:44.816Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7271lb # [ 270.045844] niks3-server[1238]: 2026/09/21 15:11:44 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7272lb # [ 270.054943] niks3-server[1238]: 2026/09/21 15:11:44 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7273lb # [ 270.065837] niks3-server[1238]: 2026/09/21 15:11:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7274worker1 # [ 270.026823] nix-grpc-daemon[1634]: time=2026-09-21T15:11:44.869Z level=INFO msg="Uploading 1 narinfos"7275lb # [ 270.070964] niks3-server[1238]: 2026/09/21 15:11:44 INFO Signed narinfos id=69 count=17276lb # [ 270.077135] niks3-server[1238]: 2026/09/21 15:11:44 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7277lb # [ 270.084486] niks3-server[1238]: 2026/09/21 15:11:44 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7278lb # [ 270.091052] niks3-server[1238]: 2026/09/21 15:11:44 INFO Completed upload id=697279lb # [ 270.092908] niks3-server[1238]: 2026/09/21 15:11:44 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7280worker1 # [ 270.048583] nix-grpc-daemon[1634]: time=2026-09-21T15:11:44.891Z level=INFO msg="Upload complete. (328ms)"7281worker1 # [ 270.055136] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33538 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=17282worker1 # [ 270.067872] nix-daemon[868]: reaped child process 2105, status = succeeded7283client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.02 seconds)7284worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7285worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7286client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'7287client # [ 270.255251] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7288client # Running as unit: gpu.service; invocation ID: cc4e713597944b81a932b5a2c90bf0fa7289client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.04 seconds)7290lb: must succeed: curl -sf localhost:9901/clusters7291lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7292lb: must succeed: curl -sf localhost:9901/clusters7293lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7294worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7295worker2 # [ 270.457302] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:510487296client # [ 270.471467] nix[1813]: don't know how to build these paths:7297client # [ 270.472343] nix[1813]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7298worker1 # [ 270.423284] nix-grpc-daemon[1558]: ts=2026-09-21T15:11:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335387299worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7300worker2 # [ 270.481888] nix-grpc-daemon[1829]: ts=2026-09-21T15:11:45Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07301client # [ 270.491635] nix[1813]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker7302lb: must succeed: curl -sf localhost:9901/clusters7303lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7304lb: must succeed: curl -sf localhost:9901/clusters7305lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7306worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7307worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7308client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7309lb # [ 278.476467] envoy[979]: conn peer=[2001:db8:1::1]:51750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27158 tx=11480 ms=208137310client # [ 300.492994] nix[1813]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}7311client # [ 300.504329] nix[1813]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7312client # [ 300.548252] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7313client # [ 300.549746] systemd[1]: gpu.service: Failed with result 'exit-code'.7314client # [ 300.552109] systemd[1]: gpu.service: Consumed 133ms CPU time over 30.296s wall clock time, 37.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7315client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.90 seconds)7316client: must succeed: journalctl -u gpu -o cat7317client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7318lb: must succeed: curl -sf localhost:9901/clusters7319lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7320lb: must succeed: curl -sf localhost:9901/clusters7321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7322worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7323worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7324(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 52.72 seconds)7325subtest: low disk drains a worker and builds go to the other7326worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7327worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.96 seconds)7328worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7329worker1 # [ 302.764734] nix-grpc-daemon[1558]: ts=2026-09-21T15:12:17Z level=info event=unhealthy reason=min_free7330worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.05 seconds)7331client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&27332worker1 # [ 303.061883] nix-grpc-daemon[1558]: ts=2026-09-21T15:12:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:335387333client # don't know how to build these paths:7334client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7335worker1 # [ 303.070927] nix-grpc-daemon[1558]: ts=2026-09-21T15:12:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:335387336worker2 # [ 303.137331] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:18Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07337worker2 # [ 303.141734] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:18Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17338worker2 # [ 303.179824] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510487339worker2 # [ 303.193074] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510487340client # copying 1 paths to 'lb:50051'...7341client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7342worker2 # [ 303.206618] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:510487343worker2 # [ 303.219143] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7344worker2 # [ 303.240068] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:18Z level=info event=niks3_push_started pid=19197345worker2 # [ 303.452590] nix-grpc-daemon[1919]: time=2026-09-21T15:12:18.348Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""7346worker2 # [ 303.629094] nix-daemon[857]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)7347worker2 # [ 303.640329] nix-daemon[857]: reaped child process 1935, status = succeeded7348worker2 # [ 303.824931] nix-daemon[857]: accepted connection from pid 1937, user nix-grpc-daemon (trusted)7349worker2 # [ 303.836975] nix-daemon[857]: reaped child process 1940, status = succeeded7350lb # [ 303.855780] niks3-server[1238]: 2026/09/21 15:12:18 INFO Received uploads request method=POST path=/api/pending_closures7351worker2 # [ 303.862952] nix-grpc-daemon[1919]: time=2026-09-21T15:12:18.759Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7352worker2 # [ 303.864489] nix-grpc-daemon[1919]: time=2026-09-21T15:12:18.761Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7353lb # [ 303.894402] niks3-server[1238]: 2026/09/21 15:12:18 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7354lb # [ 303.904772] niks3-server[1238]: 2026/09/21 15:12:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7355lb # [ 303.910117] niks3-server[1238]: 2026/09/21 15:12:18 INFO Signed narinfos id=70 count=17356worker2 # [ 303.910686] nix-grpc-daemon[1919]: time=2026-09-21T15:12:18.807Z level=INFO msg="Uploading 1 narinfos"7357lb # [ 303.919544] niks3-server[1238]: 2026/09/21 15:12:18 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7358lb # [ 303.928824] niks3-server[1238]: 2026/09/21 15:12:18 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7359lb # [ 303.937196] niks3-server[1238]: 2026/09/21 15:12:18 INFO Completed upload id=707360worker2 # [ 303.934574] nix-grpc-daemon[1919]: time=2026-09-21T15:12:18.831Z level=INFO msg="Upload complete. (478ms)"7361lb # [ 303.941399] niks3-server[1238]: 2026/09/21 15:12:18 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7362worker2 # [ 303.938772] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51048 duration_s=0 paths=1 nar_bytes_in=5127363worker2 # [ 303.944428] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510487364worker2 # [ 303.953706] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7365worker2 # [ 303.962441] nix-daemon[857]: reaped child process 1917, status = succeeded7366client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7367worker2 # [ 303.966511] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7368client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7369client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7370client # farm-a> LOG-farm-a-drain7371worker2 # [ 304.990322] nix-daemon[857]: reaped child process 1944, status = succeeded7372worker2 # [ 305.189679] nix-daemon[857]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)7373worker2 # [ 305.202451] nix-daemon[857]: reaped child process 1963, status = succeeded7374lb # [ 305.221370] niks3-server[1238]: 2026/09/21 15:12:19 INFO Received uploads request method=POST path=/api/pending_closures7375worker2 # [ 305.228771] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.125Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7376worker2 # [ 305.230381] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.127Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7377lb # [ 305.261774] niks3-server[1238]: 2026/09/21 15:12:19 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7378lb # [ 305.270581] niks3-server[1238]: 2026/09/21 15:12:19 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7379lb # [ 305.273708] niks3-server[1238]: 2026/09/21 15:12:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7380worker2 # [ 305.277339] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.173Z level=INFO msg="Uploading 1 narinfos"7381lb # [ 305.279558] niks3-server[1238]: 2026/09/21 15:12:19 INFO Signed narinfos id=71 count=17382lb # [ 305.285385] niks3-server[1238]: 2026/09/21 15:12:19 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7383lb # [ 305.289628] niks3-server[1238]: 2026/09/21 15:12:19 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7384lb # [ 305.293897] niks3-server[1238]: 2026/09/21 15:12:19 INFO Completed upload id=717385lb # [ 305.295689] niks3-server[1238]: 2026/09/21 15:12:19 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7386worker2 # [ 305.292940] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.189Z level=INFO msg="Upload complete. (318ms)"7387worker2 # [ 305.298517] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51048 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17388worker2 # [ 305.308444] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:20Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07389worker2 # [ 305.310411] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:20Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17390worker2 # [ 305.314964] nix-daemon[857]: reaped child process 1942, status = succeeded7391worker2 # [ 305.353525] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510487392worker2 # [ 305.368481] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510487393worker2 # [ 305.381975] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.278Z level=INFO msg="All 1 paths already cached"7394client # copying 1 paths to 'lb:50051'...7395client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7396worker2 # [ 305.390852] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:510487397worker2 # [ 305.399728] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7398worker2 # [ 305.590690] nix-daemon[857]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)7399worker2 # [ 305.602671] nix-daemon[857]: reaped child process 1972, status = succeeded7400worker2 # [ 305.790979] nix-daemon[857]: accepted connection from pid 1974, user nix-grpc-daemon (trusted)7401worker2 # [ 305.802927] nix-daemon[857]: reaped child process 1977, status = succeeded7402lb # [ 305.823167] niks3-server[1238]: 2026/09/21 15:12:20 INFO Received uploads request method=POST path=/api/pending_closures7403worker2 # [ 305.828440] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.725Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7404worker2 # [ 305.830388] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.726Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7405lb # [ 305.848679] niks3-server[1238]: 2026/09/21 15:12:20 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7406lb # [ 305.860687] niks3-server[1238]: 2026/09/21 15:12:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7407lb # [ 305.864816] niks3-server[1238]: 2026/09/21 15:12:20 INFO Signed narinfos id=72 count=17408worker2 # [ 305.865485] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.762Z level=INFO msg="Uploading 1 narinfos"7409lb # [ 305.872580] niks3-server[1238]: 2026/09/21 15:12:20 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7410lb # [ 305.881828] niks3-server[1238]: 2026/09/21 15:12:20 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7411worker2 # [ 305.883966] nix-grpc-daemon[1919]: time=2026-09-21T15:12:20.780Z level=INFO msg="Upload complete. (470ms)"7412lb # [ 305.887319] niks3-server[1238]: 2026/09/21 15:12:20 INFO Completed upload id=727413worker2 # [ 305.886285] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51048 duration_s=0 paths=1 nar_bytes_in=6247414lb # [ 305.893274] niks3-server[1238]: 2026/09/21 15:12:20 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7415worker2 # [ 305.891108] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510487416worker2 # [ 305.899154] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7417worker2 # [ 305.903815] nix-daemon[857]: reaped child process 1967, status = succeeded7418worker2 # [ 305.908904] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7419client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7420client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7421client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7422client # farm-b> LOG-farm-b-drain7423worker2 # [ 306.933504] nix-daemon[857]: reaped child process 1981, status = succeeded7424worker2 # [ 307.128921] nix-daemon[857]: accepted connection from pid 1997, user nix-grpc-daemon (trusted)7425worker2 # [ 307.140916] nix-daemon[857]: reaped child process 2000, status = succeeded7426lb # [ 307.164285] niks3-server[1238]: 2026/09/21 15:12:21 INFO Received uploads request method=POST path=/api/pending_closures7427worker2 # [ 307.170051] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.066Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7428worker2 # [ 307.171738] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.068Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7429lb # [ 307.204702] niks3-server[1238]: 2026/09/21 15:12:21 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7430lb # [ 307.210614] niks3-server[1238]: 2026/09/21 15:12:21 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7431lb # [ 307.219895] niks3-server[1238]: 2026/09/21 15:12:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7432worker2 # [ 307.223557] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.120Z level=INFO msg="Uploading 1 narinfos"7433lb # [ 307.225629] niks3-server[1238]: 2026/09/21 15:12:21 INFO Signed narinfos id=73 count=17434lb # [ 307.232382] niks3-server[1238]: 2026/09/21 15:12:21 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7435lb # [ 307.240877] niks3-server[1238]: 2026/09/21 15:12:21 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7436lb # [ 307.246123] niks3-server[1238]: 2026/09/21 15:12:21 INFO Completed upload id=737437worker2 # [ 307.243725] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.140Z level=INFO msg="Upload complete. (327ms)"7438lb # [ 307.249423] niks3-server[1238]: 2026/09/21 15:12:21 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7439worker2 # [ 307.251968] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51048 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17440worker2 # [ 307.262851] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:22Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07441worker2 # [ 307.266258] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:22Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17442worker2 # [ 307.275108] nix-daemon[857]: reaped child process 1979, status = succeeded7443worker2 # [ 307.306363] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510487444worker2 # [ 307.319586] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:510487445worker2 # [ 307.332707] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.229Z level=INFO msg="All 2 paths already cached"7446client # copying 1 paths to 'lb:50051'...7447client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7448worker2 # [ 307.341508] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:510487449worker2 # [ 307.350308] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7450worker2 # [ 307.555361] nix-daemon[857]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)7451worker2 # [ 307.567266] nix-daemon[857]: reaped child process 2010, status = succeeded7452worker2 # [ 307.757125] nix-daemon[857]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)7453worker2 # [ 307.768791] nix-daemon[857]: reaped child process 2015, status = succeeded7454lb # [ 307.790646] niks3-server[1238]: 2026/09/21 15:12:22 INFO Received uploads request method=POST path=/api/pending_closures7455worker2 # [ 307.796762] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.693Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7456worker2 # [ 307.798254] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.693Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7457lb # [ 307.817714] niks3-server[1238]: 2026/09/21 15:12:22 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7458lb # [ 307.828544] niks3-server[1238]: 2026/09/21 15:12:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7459lb # [ 307.834542] niks3-server[1238]: 2026/09/21 15:12:22 INFO Signed narinfos id=74 count=17460worker2 # [ 307.835076] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.731Z level=INFO msg="Uploading 1 narinfos"7461lb # [ 307.839412] niks3-server[1238]: 2026/09/21 15:12:22 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7462lb # [ 307.851545] niks3-server[1238]: 2026/09/21 15:12:22 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7463worker2 # [ 307.855646] nix-grpc-daemon[1919]: time=2026-09-21T15:12:22.752Z level=INFO msg="Upload complete. (492ms)"7464lb # [ 307.858618] niks3-server[1238]: 2026/09/21 15:12:22 INFO Completed upload id=747465lb # [ 307.862112] niks3-server[1238]: 2026/09/21 15:12:22 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7466worker2 # [ 307.858439] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51048 duration_s=0 paths=1 nar_bytes_in=7527467worker2 # [ 307.864122] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:510487468worker2 # [ 307.873261] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7469worker2 # [ 307.875739] nix-daemon[857]: reaped child process 2005, status = succeeded7470client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7471worker2 # [ 307.882126] nix-daemon[857]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)7472client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7473lb # [ 308.478425] envoy[979]: conn peer=[2001:db8:1::1]:49580 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300547474client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7475client # farm-top> LOG-farm-top-drain7476worker2 # [ 309.509171] nix-daemon[857]: reaped child process 2019, status = succeeded7477worker2 # [ 309.706143] nix-daemon[857]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7478worker2 # [ 309.718621] nix-daemon[857]: reaped child process 2038, status = succeeded7479lb # [ 309.740524] niks3-server[1238]: 2026/09/21 15:12:24 INFO Received uploads request method=POST path=/api/pending_closures7480worker2 # [ 309.746388] nix-grpc-daemon[1919]: time=2026-09-21T15:12:24.642Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7481worker2 # [ 309.747914] nix-grpc-daemon[1919]: time=2026-09-21T15:12:24.644Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7482lb # [ 309.772787] niks3-server[1238]: 2026/09/21 15:12:24 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7483lb # [ 309.786709] niks3-server[1238]: 2026/09/21 15:12:24 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7484lb # [ 309.796194] niks3-server[1238]: 2026/09/21 15:12:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7485worker2 # [ 309.798085] nix-grpc-daemon[1919]: time=2026-09-21T15:12:24.694Z level=INFO msg="Uploading 1 narinfos"7486lb # [ 309.802262] niks3-server[1238]: 2026/09/21 15:12:24 INFO Signed narinfos id=75 count=17487lb # [ 309.806465] niks3-server[1238]: 2026/09/21 15:12:24 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7488lb # [ 309.814442] niks3-server[1238]: 2026/09/21 15:12:24 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7489lb # [ 309.820321] niks3-server[1238]: 2026/09/21 15:12:24 INFO Completed upload id=757490worker2 # [ 309.817547] nix-grpc-daemon[1919]: time=2026-09-21T15:12:24.714Z level=INFO msg="Upload complete. (325ms)"7491lb # [ 309.822855] niks3-server[1238]: 2026/09/21 15:12:24 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7492worker2 # [ 309.823873] nix-grpc-daemon[1829]: ts=2026-09-21T15:12:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51048 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17493worker2 # [ 309.837460] nix-daemon[857]: reaped child process 2017, status = succeeded7494client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 7.03 seconds)7495client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7496client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.24 seconds)7497worker1: must succeed: rm /nix/.rw-store/fill7498worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.14 seconds)7499worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7500worker1 # [ 310.769513] nix-grpc-daemon[1558]: ts=2026-09-21T15:12:25Z level=info event=healthy reason=min_free7501worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)7502(finished: subtest: low disk drains a worker and builds go to the other, in 9.63 seconds)7503(finished: run the VM test script, in 312.36 seconds)7504test script finished in 312.66s7505cleanup7506kill QemuMachine (pid 49)7507client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7508kill QemuMachine (pid 50)7509lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7510kill QemuMachine (pid 51)7511worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7512kill QemuMachine (pid 52)7513worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7514(finished: cleanup, in 0.96 seconds)7515additionally exposed symbols:7516 client, lb, worker1, worker2,7517 vlan1,7518 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh