vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #345
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14lb: QEMU running (pid 49)15client: QEMU running (pid 50)16client # Formatting '/build/vm-state-client/tmp.DCoDeeY3wF', fmt=raw size=107374182417lb # Formatting '/build/vm-state-lb/tmp.Ymd3moCHBc', fmt=raw size=107374182418client # mke2fs 1.47.4 (6-Mar-2025)19lb # mke2fs 1.47.4 (6-Mar-2025)20client # Discarding device blocks: 0/262144 done21worker1 # Disk image does not exist, creating the virtualisation disk image...22lb # Discarding device blocks: 0/262144 done23worker1: QEMU running (pid 51)24client # Creating filesystem with 262144 4k blocks and 65536 inodes25worker1 # Formatting '/build/vm-state-worker1/tmp.1MuA6WDu1R', fmt=raw size=107374182426client # Filesystem UUID: 1b400cab-a219-4027-9bfd-228e923174a227worker1 # mke2fs 1.47.4 (6-Mar-2025)28client # Superblock backups stored on blocks:29worker1 # Discarding device blocks: 0/262144 done30client # 32768, 98304, 163840, 22937631worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes32client # 33worker1 # Filesystem UUID: 7712f232-32e5-4872-a836-3fc6d712be3134client # Allocating group tables: 0/8 done35worker1 # Superblock backups stored on blocks:36client # Writing inode tables: 0/8 done37worker1 # 32768, 98304, 163840, 22937638client # Creating journal (8192 blocks): done39worker1 # 40client # Writing superblocks and filesystem accounting information: 0/8 done41worker1 # Allocating group tables: 0/8 done42client # 43worker1 # Writing inode tables: 0/8 done44client # Virtualisation disk image created.45worker1 # Creating journal (8192 blocks): done46lb # Creating filesystem with 262144 4k blocks and 65536 inodes47worker1 # Writing superblocks and filesystem accounting information: 0/8 done48lb # Filesystem UUID: 543f34ef-f21d-4223-b782-c44207721c0749worker1 # 50worker2: QEMU running (pid 52)51worker2 # Disk image does not exist, creating the virtualisation disk image...52lb # Superblock backups stored on blocks:53worker2 # Formatting '/build/vm-state-worker2/tmp.KEYlplFi9c', fmt=raw size=107374182454worker1 # Virtualisation disk image created.55(finished: start all VMs, in 0.23 seconds)56lb # 32768, 98304, 163840, 22937657client: waiting for unit network-addresses-eth1.service58lb # 59client: waiting for the VM to finish booting60lb # Allocating group tables: 0/8 done61worker2 # mke2fs 1.47.4 (6-Mar-2025)62lb # Writing inode tables: 0/8 done63worker2 # Discarding device blocks: 0/262144 done64lb # Creating journal (8192 blocks): done65worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes66lb # Writing superblocks and filesystem accounting information: 0/8 done67worker2 # Filesystem UUID: e1bfb2a6-e55d-430a-b548-18e36d77a39968lb # 69worker2 # Superblock backups stored on blocks:70lb # Virtualisation disk image created.71worker2 # 32768, 98304, 163840, 22937672worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086client # 87client # 88lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...89lb # 90lb # 91lb # 92lb # 93client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0094lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0095client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...96client # 97client # 98client # 99client # 100lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...101lb # 102lb # 103client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00104client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105client # 106client # 107worker1 # 108worker1 # 109worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00110worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...111worker1 # 112worker1 # 113worker1 # 114worker1 # 115worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116worker2 # 117worker2 # 118worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...119worker1 # 120worker1 # 121worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123worker2 # 124worker2 # 125worker2 # 126worker2 # 127worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128lb # Booting from ROM...129worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...130worker2 # 131worker2 # 132client # Booting from ROM...133worker1 # Booting from ROM...134worker2 # Booting from ROM...135lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0137lb # [ 0.000000] BIOS-provided physical RAM map:138lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148lb # [ 0.000000] NX (Execute Disable) protection: active149lb # [ 0.000000] APIC: Static calls initialized150lb # [ 0.000000] SMBIOS 2.8 present.151lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014152lb # [ 0.000000] DMI: Memory slots populated: 1/1153lb # [ 0.000000] Hypervisor detected: KVM154lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000155lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00156lb # [ 0.000000] kvm-clock: using sched offset of 446953055 cycles157lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns158lb # [ 0.000005] tsc: Detected 2400.012 MHz processor159lb # [ 0.000848] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000160lb # [ 0.000876] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs161lb # [ 0.000878] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT162lb # [ 0.002772] found SMP MP-table at [mem 0x000f5450-0x000f545f]163lb # [ 0.002783] Using GB pages for direct mapping164lb # [ 0.002861] RAMDISK: [mem 0x5e367000-0x5ffcffff]165lb # [ 0.002882] ACPI: Early table checksum verification disabled166lb # [ 0.002885] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )167client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026168lb # [ 0.002890] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)169lb # [ 0.002894] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)170lb # [ 0.002901] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)171worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026172lb # [ 0.002903] ACPI: FACS 0x000000005FFE0000 000040173client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test/init regInfo=/nix/store/blpxkzm53swxlxffz2dn2ks4cp5hilgc-closure-info/registration console=ttyS0,115200n8 console=tty0174client # [ 0.000000] BIOS-provided physical RAM map:175lb # [ 0.002904] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)176client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable177lb # [ 0.002906] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)178client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved179lb # [ 0.002907] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)180client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved181worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ycsrgx65d5yfvzr743gfkyvj36s6578h-nixos-system-worker1-test/init regInfo=/nix/store/9kgibijjjb8lzklk71fyc6sa4ml0bv7g-closure-info/registration console=ttyS0,115200n8 console=tty0182worker1 # [ 0.000000] BIOS-provided physical RAM map:183lb # [ 0.002909] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)184client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable185worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable186lb # [ 0.002910] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]187client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved188worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved189lb # [ 0.002911] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]190client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved191worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved192lb # [ 0.002912] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]193client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved194worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable195lb # [ 0.002912] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]196client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved197worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved198lb # [ 0.002913] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]199client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved200worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved201lb # [ 0.002913] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]202client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved203client # [ 0.000000] NX (Execute Disable) protection: active204worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved205lb # [ 0.002914] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]206client # [ 0.000000] APIC: Static calls initialized207lb # [ 0.003142] No NUMA configuration found208worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved209client # [ 0.000000] SMBIOS 2.8 present.210lb # [ 0.003143] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]211worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved212client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014213lb # [ 0.003145] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]214lb # [ 0.005640] Zone ranges:215worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved216client # [ 0.000000] DMI: Memory slots populated: 1/1217worker1 # [ 0.000000] NX (Execute Disable) protection: active218client # [ 0.000000] Hypervisor detected: KVM219lb # [ 0.005641] DMA [mem 0x0000000000001000-0x0000000000ffffff]220worker1 # [ 0.000000] APIC: Static calls initialized221client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000222lb # [ 0.005643] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]223worker1 # [ 0.000000] SMBIOS 2.8 present.224lb # [ 0.005644] Normal empty225client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00226lb # [ 0.005644] Device empty227client # [ 0.000000] kvm-clock: using sched offset of 470713317 cycles228worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014229lb # [ 0.005645] Movable zone start for each node230lb # [ 0.005646] Early memory node ranges231worker1 # [ 0.000000] DMI: Memory slots populated: 1/1232worker1 # [ 0.000000] Hypervisor detected: KVM233lb # [ 0.005646] node 0: [mem 0x0000000000001000-0x000000000009efff]234client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns235worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000236client # [ 0.000005] tsc: Detected 2400.012 MHz processor237lb # [ 0.005647] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]238worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00239client # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000240lb # [ 0.005648] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]241worker1 # [ 0.000000] kvm-clock: using sched offset of 475014926 cycles242client # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs243lb # [ 0.005667] On node 0, zone DMA: 1 pages in unavailable ranges244lb # [ 0.005929] On node 0, zone DMA: 97 pages in unavailable ranges245worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns246client # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT247worker1 # [ 0.000004] tsc: Detected 2400.012 MHz processor248lb # [ 0.032866] On node 0, zone DMA32: 40 pages in unavailable ranges249client # [ 0.002736] found SMP MP-table at [mem 0x000f5450-0x000f545f]250lb # [ 0.033328] ACPI: PM-Timer IO Port: 0x608251worker1 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000252client # [ 0.002747] Using GB pages for direct mapping253client # [ 0.002906] RAMDISK: [mem 0x5e366000-0x5ffcffff]254lb # [ 0.033339] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])255worker1 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs256lb # [ 0.033364] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23257client # [ 0.002915] ACPI: Early table checksum verification disabled258worker1 # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT259client # [ 0.002918] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )260lb # [ 0.033367] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)261worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026262worker1 # [ 0.002739] found SMP MP-table at [mem 0x000f5450-0x000f545f]263worker1 # [ 0.002750] Using GB pages for direct mapping264lb # [ 0.033368] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)265client # [ 0.002922] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)266worker1 # [ 0.002880] RAMDISK: [mem 0x5e367000-0x5ffcffff]267lb # [ 0.033370] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)268client # [ 0.002926] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)269worker1 # [ 0.002889] ACPI: Early table checksum verification disabled270lb # [ 0.033371] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)271worker1 # [ 0.002892] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )272client # [ 0.002933] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)273lb # [ 0.033371] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)274client # [ 0.002935] ACPI: FACS 0x000000005FFE0000 000040275worker1 # [ 0.002895] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)276worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bhr46xja1kyki42v8lp87l0nsrargk1l-nixos-system-worker2-test/init regInfo=/nix/store/iw1fwr9xgy9i67hbc5f1wjqv08csn4jl-closure-info/registration console=ttyS0,115200n8 console=tty0277lb # [ 0.033374] ACPI: Using ACPI (MADT) for SMP configuration information278worker2 # [ 0.000000] BIOS-provided physical RAM map:279client # [ 0.002937] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)280lb # [ 0.033375] ACPI: HPET id: 0x8086a201 base: 0xfed00000281worker1 # [ 0.002900] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)282lb # [ 0.033379] TSC deadline timer available283worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable284client # [ 0.002938] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)285lb # [ 0.033383] CPU topo: Max. logical packages: 1286worker1 # [ 0.002907] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)287worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved288lb # [ 0.033383] CPU topo: Max. logical dies: 1289worker1 # [ 0.002909] ACPI: FACS 0x000000005FFE0000 000040290lb # [ 0.033384] CPU topo: Max. dies per package: 1291client # [ 0.002940] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)292worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved293lb # [ 0.033387] CPU topo: Max. threads per core: 1294worker1 # [ 0.002910] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)295client # [ 0.002941] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)296worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable297lb # [ 0.033388] CPU topo: Num. cores per package: 1298client # [ 0.002943] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]299lb # [ 0.033388] CPU topo: Num. threads per package: 1300worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved301worker1 # [ 0.002912] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)302lb # [ 0.033388] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs303worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved304client # [ 0.002944] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]305worker1 # [ 0.002913] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)306lb # [ 0.033405] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()307client # [ 0.002944] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]308worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved309worker1 # [ 0.002915] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)310worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved311lb # [ 0.033436] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]312client # [ 0.002945] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]313worker1 # [ 0.002916] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]314worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved315client # [ 0.002945] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]316lb # [ 0.033437] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]317worker1 # [ 0.002917] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]318lb # [ 0.033438] [mem 0x60000000-0xafffffff] available for PCI devices319worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved320client # [ 0.002946] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]321worker1 # [ 0.002918] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]322lb # [ 0.033440] Booting paravirtualized kernel on KVM323worker2 # [ 0.000000] NX (Execute Disable) protection: active324client # [ 0.002946] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]325worker2 # [ 0.000000] APIC: Static calls initialized326worker1 # [ 0.002918] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]327client # [ 0.003173] No NUMA configuration found328worker2 # [ 0.000000] SMBIOS 2.8 present.329lb # [ 0.033442] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns330worker1 # [ 0.002919] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]331client # [ 0.003174] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]332worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014333lb # [ 0.037884] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1334client # [ 0.003177] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]335worker1 # [ 0.002919] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]336client # [ 0.005618] Zone ranges:337worker2 # [ 0.000000] DMI: Memory slots populated: 1/1338lb # [ 0.040058] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152339worker2 # [ 0.000000] Hypervisor detected: KVM340worker1 # [ 0.002920] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]341client # [ 0.005619] DMA [mem 0x0000000000001000-0x0000000000ffffff]342lb # [ 0.040108] kvm-guest: PV spinlocks disabled, single CPU343worker1 # [ 0.003141] No NUMA configuration found344worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000345client # [ 0.005621] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]346worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00347worker1 # [ 0.003141] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]348client # [ 0.005622] Normal empty349client # [ 0.005623] Device empty350worker2 # [ 0.000000] kvm-clock: using sched offset of 493158213 cycles351worker1 # [ 0.003144] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]352client # [ 0.005623] Movable zone start for each node353worker1 # [ 0.005412] Zone ranges:354client # [ 0.005624] Early memory node ranges355worker1 # [ 0.005413] DMA [mem 0x0000000000001000-0x0000000000ffffff]356worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns357client # [ 0.005625] node 0: [mem 0x0000000000001000-0x000000000009efff]358worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor359worker1 # [ 0.005414] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]360lb # [ 0.040109] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0361client # [ 0.005626] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]362worker1 # [ 0.005415] Normal empty363worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000364worker1 # [ 0.005416] Device empty365worker1 # [ 0.005417] Movable zone start for each node366client # [ 0.005627] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]367worker2 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs368worker1 # [ 0.005417] Early memory node ranges369client # [ 0.005648] On node 0, zone DMA: 1 pages in unavailable ranges370lb # [ 0.040202] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.371worker2 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT372worker1 # [ 0.005418] node 0: [mem 0x0000000000001000-0x000000000009efff]373lb # [ 0.040214] random: crng init done374client # [ 0.005927] On node 0, zone DMA: 97 pages in unavailable ranges375worker2 # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]376worker1 # [ 0.005419] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]377lb # [ 0.040215] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes378client # [ 0.034783] On node 0, zone DMA32: 40 pages in unavailable ranges379worker2 # [ 0.002750] Using GB pages for direct mapping380client # [ 0.035249] ACPI: PM-Timer IO Port: 0x608381worker1 # [ 0.005420] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]382worker2 # [ 0.002903] RAMDISK: [mem 0x5e367000-0x5ffcffff]383lb # [ 0.042365] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)384client # [ 0.035262] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])385worker1 # [ 0.005439] On node 0, zone DMA: 1 pages in unavailable ranges386worker2 # [ 0.002912] ACPI: Early table checksum verification disabled387lb # [ 0.043449] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)388client # [ 0.035288] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23389worker1 # [ 0.005715] On node 0, zone DMA: 97 pages in unavailable ranges390worker2 # [ 0.002915] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )391lb # [ 0.043480] Fallback order for Node 0: 0392client # [ 0.035291] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)393worker1 # [ 0.033782] On node 0, zone DMA32: 40 pages in unavailable ranges394lb # [ 0.043482] Built 1 zonelists, mobility grouping on. Total pages: 393078395worker2 # [ 0.002919] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)396worker1 # [ 0.034234] ACPI: PM-Timer IO Port: 0x608397lb # [ 0.043484] Policy zone: DMA32398client # [ 0.035293] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)399worker1 # [ 0.034245] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])400worker2 # [ 0.002924] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)401lb # [ 0.046259] mem auto-init: stack:all(zero), heap alloc:on, heap free:off402client # [ 0.035294] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)403worker1 # [ 0.034272] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23404lb # [ 0.049401] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1405worker2 # [ 0.002931] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)406client # [ 0.035295] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)407worker1 # [ 0.034274] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)408lb # [ 0.052754] allocated 3145728 bytes of page_ext409worker2 # [ 0.002933] ACPI: FACS 0x000000005FFE0000 000040410client # [ 0.035296] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)411lb # [ 0.062725] ftrace: allocating 48731 entries in 192 pages412worker1 # [ 0.034276] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)413worker2 # [ 0.002934] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)414client # [ 0.035298] ACPI: Using ACPI (MADT) for SMP configuration information415lb # [ 0.062727] ftrace: allocated 192 pages with 2 groups416worker1 # [ 0.034277] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)417lb # [ 0.063563] Dynamic Preempt: lazy418client # [ 0.035299] ACPI: HPET id: 0x8086a201 base: 0xfed00000419worker2 # [ 0.002936] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)420client # [ 0.035304] TSC deadline timer available421worker1 # [ 0.034278] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)422lb # [ 0.063690] rcu: Preemptible hierarchical RCU implementation.423client # [ 0.035308] CPU topo: Max. logical packages: 1424lb # [ 0.063690] rcu: RCU event tracing is enabled.425worker1 # [ 0.034279] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)426worker2 # [ 0.002937] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)427client # [ 0.035309] CPU topo: Max. logical dies: 1428lb # [ 0.063691] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.429worker1 # [ 0.034281] ACPI: Using ACPI (MADT) for SMP configuration information430client # [ 0.035309] CPU topo: Max. dies per package: 1431worker2 # [ 0.002939] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)432lb # [ 0.063692] Trampoline variant of Tasks RCU enabled.433worker1 # [ 0.034282] ACPI: HPET id: 0x8086a201 base: 0xfed00000434client # [ 0.035313] CPU topo: Max. threads per core: 1435lb # [ 0.063693] Rude variant of Tasks RCU enabled.436worker2 # [ 0.002940] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]437worker1 # [ 0.034285] TSC deadline timer available438client # [ 0.035313] CPU topo: Num. cores per package: 1439lb # [ 0.063693] Tracing variant of Tasks RCU enabled.440worker1 # [ 0.034290] CPU topo: Max. logical packages: 1441client # [ 0.035314] CPU topo: Num. threads per package: 1442worker2 # [ 0.002941] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]443worker1 # [ 0.034290] CPU topo: Max. logical dies: 1444lb # [ 0.063694] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.445client # [ 0.035314] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs446worker2 # [ 0.002942] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]447worker1 # [ 0.034291] CPU topo: Max. dies per package: 1448lb # [ 0.063695] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1449worker1 # [ 0.034295] CPU topo: Max. threads per core: 1450client # [ 0.035336] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()451worker2 # [ 0.002942] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]452worker1 # [ 0.034296] CPU topo: Num. cores per package: 1453lb # [ 0.063745] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.454client # [ 0.035370] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]455worker2 # [ 0.002943] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]456worker1 # [ 0.034296] CPU topo: Num. threads per package: 1457worker2 # [ 0.002943] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]458worker1 # [ 0.034296] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs459client # [ 0.035372] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]460lb # [ 0.063747] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.461client # [ 0.035373] [mem 0x60000000-0xafffffff] available for PCI devices462worker2 # [ 0.002944] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]463worker1 # [ 0.034313] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()464lb # [ 0.063747] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.465worker2 # [ 0.003174] No NUMA configuration found466client # [ 0.035375] Booting paravirtualized kernel on KVM467worker1 # [ 0.034344] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]468lb # [ 0.068135] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16469worker2 # [ 0.003175] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]470worker1 # [ 0.034345] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]471client # [ 0.035379] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns472worker2 # [ 0.003177] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]473lb # [ 0.068426] rcu: srcu_init: Setting srcu_struct sizes based on contention.474worker2 # [ 0.005570] Zone ranges:475worker1 # [ 0.034347] [mem 0x60000000-0xafffffff] available for PCI devices476client # [ 0.039851] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1477worker2 # [ 0.005571] DMA [mem 0x0000000000001000-0x0000000000ffffff]478lb # [ 0.068432] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns479worker1 # [ 0.034348] Booting paravirtualized kernel on KVM480client # [ 0.042489] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152481worker2 # [ 0.005573] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]482worker2 # [ 0.005574] Normal empty483lb # [ 0.068537] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)484client # [ 0.042532] kvm-guest: PV spinlocks disabled, single CPU485worker1 # [ 0.034351] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns486worker2 # [ 0.005575] Device empty487lb # [ 0.072132] Console: colour VGA+ 80x25488worker2 # [ 0.005575] Movable zone start for each node489worker1 # [ 0.038824] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1490lb # [ 0.072135] printk: legacy console [tty0] enabled491worker2 # [ 0.005576] Early memory node ranges492lb # [ 0.101575] printk: legacy console [ttyS0] enabled493worker1 # [ 0.041577] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152494worker2 # [ 0.005577] node 0: [mem 0x0000000000001000-0x000000000009efff]495lb # [ 0.200499] ACPI: Core revision 20250807496worker1 # [ 0.041627] kvm-guest: PV spinlocks disabled, single CPU497worker2 # [ 0.005578] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]498worker2 # [ 0.005579] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]499client # [ 0.042534] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test/init regInfo=/nix/store/blpxkzm53swxlxffz2dn2ks4cp5hilgc-closure-info/registration console=ttyS0,115200n8 console=tty0500lb # [ 0.201325] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns501worker2 # [ 0.005598] On node 0, zone DMA: 1 pages in unavailable ranges502worker2 # [ 0.005870] On node 0, zone DMA: 97 pages in unavailable ranges503lb # [ 0.202797] APIC: Switch to symmetric I/O mode setup504client # [ 0.042632] Unknown kernel command line parameters "regInfo=/nix/store/blpxkzm53swxlxffz2dn2ks4cp5hilgc-closure-info/registration", will be passed to user space.505worker2 # [ 0.034289] On node 0, zone DMA32: 40 pages in unavailable ranges506lb # [ 0.203757] x2apic enabled507client # [ 0.042645] random: crng init done508worker2 # [ 0.034757] ACPI: PM-Timer IO Port: 0x608509worker1 # [ 0.041629] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ycsrgx65d5yfvzr743gfkyvj36s6578h-nixos-system-worker1-test/init regInfo=/nix/store/9kgibijjjb8lzklk71fyc6sa4ml0bv7g-closure-info/registration console=ttyS0,115200n8 console=tty0510worker2 # [ 0.034770] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])511client # [ 0.042646] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes512lb # [ 0.204489] APIC: Switched APIC routing to: physical x2apic513worker2 # [ 0.034801] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23514client # [ 0.044916] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)515worker1 # [ 0.041718] Unknown kernel command line parameters "regInfo=/nix/store/9kgibijjjb8lzklk71fyc6sa4ml0bv7g-closure-info/registration", will be passed to user space.516worker2 # [ 0.034803] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)517worker1 # [ 0.041730] random: crng init done518client # [ 0.046046] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)519lb # [ 0.206281] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1520worker2 # [ 0.034805] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)521client # [ 0.046078] Fallback order for Node 0: 0522worker1 # [ 0.041731] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes523worker2 # [ 0.034806] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)524client # [ 0.046081] Built 1 zonelists, mobility grouping on. Total pages: 393078525worker1 # [ 0.044413] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)526client # [ 0.046082] Policy zone: DMA32527lb # [ 0.207234] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns528worker2 # [ 0.034807] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)529worker1 # [ 0.046195] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)530client # [ 0.048742] mem auto-init: stack:all(zero), heap alloc:on, heap free:off531worker2 # [ 0.034808] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)532worker1 # [ 0.046231] Fallback order for Node 0: 0533lb # [ 0.208859] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)534client # [ 0.052239] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1535worker2 # [ 0.034810] ACPI: Using ACPI (MADT) for SMP configuration information536worker1 # [ 0.046233] Built 1 zonelists, mobility grouping on. Total pages: 393078537client # [ 0.056134] allocated 3145728 bytes of page_ext538worker1 # [ 0.046234] Policy zone: DMA32539lb # [ 0.209942] x86/cpu: User Mode Instruction Prevention (UMIP) activated540worker2 # [ 0.034811] ACPI: HPET id: 0x8086a201 base: 0xfed00000541client # [ 0.066389] ftrace: allocating 48731 entries in 192 pages542worker2 # [ 0.034815] TSC deadline timer available543worker1 # [ 0.049364] mem auto-init: stack:all(zero), heap alloc:on, heap free:off544client # [ 0.066391] ftrace: allocated 192 pages with 2 groups545worker2 # [ 0.034819] CPU topo: Max. logical packages: 1546lb # [ 0.211035] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127547client # [ 0.067259] Dynamic Preempt: lazy548worker1 # [ 0.052676] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1549worker2 # [ 0.034820] CPU topo: Max. logical dies: 1550worker1 # [ 0.056719] allocated 3145728 bytes of page_ext551client # [ 0.067390] rcu: Preemptible hierarchical RCU implementation.552lb # [ 0.211857] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0553worker2 # [ 0.034821] CPU topo: Max. dies per package: 1554client # [ 0.067391] rcu: RCU event tracing is enabled.555worker1 # [ 0.066750] ftrace: allocating 48731 entries in 192 pages556worker2 # [ 0.034824] CPU topo: Max. threads per core: 1557worker1 # [ 0.066752] ftrace: allocated 192 pages with 2 groups558worker2 # [ 0.034825] CPU topo: Num. cores per package: 1559client # [ 0.067392] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.560worker1 # [ 0.067641] Dynamic Preempt: lazy561lb # [ 0.212860] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto562worker2 # [ 0.034825] CPU topo: Num. threads per package: 1563client # [ 0.067393] Trampoline variant of Tasks RCU enabled.564worker1 # [ 0.067846] rcu: Preemptible hierarchical RCU implementation.565client # [ 0.067394] Rude variant of Tasks RCU enabled.566worker2 # [ 0.034825] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs567worker1 # [ 0.067846] rcu: RCU event tracing is enabled.568lb # [ 0.213857] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl569client # [ 0.067394] Tracing variant of Tasks RCU enabled.570worker2 # [ 0.034844] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()571worker1 # [ 0.067847] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.572client # [ 0.067395] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.573lb # [ 0.214857] Transient Scheduler Attacks: Vulnerable: No microcode574worker1 # [ 0.067848] Trampoline variant of Tasks RCU enabled.575worker2 # [ 0.034877] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]576worker1 # [ 0.067849] Rude variant of Tasks RCU enabled.577client # [ 0.067395] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1578lb # [ 0.215856] Spectre V2 : Mitigation: Enhanced / Automatic IBRS579worker2 # [ 0.034879] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]580worker1 # [ 0.067849] Tracing variant of Tasks RCU enabled.581client # [ 0.067455] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.582worker2 # [ 0.034880] [mem 0x60000000-0xafffffff] available for PCI devices583lb # [ 0.216856] Speculative Return Stack Overflow: Mitigation: Safe RET584worker1 # [ 0.067850] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.585worker2 # [ 0.034882] Booting paravirtualized kernel on KVM586client # [ 0.067457] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.587worker1 # [ 0.067851] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1588worker2 # [ 0.034885] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns589client # [ 0.067458] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.590worker1 # [ 0.067867] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.591worker2 # [ 0.039353] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1592client # [ 0.071899] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16593worker1 # [ 0.067869] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.594worker2 # [ 0.041647] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152595client # [ 0.072190] rcu: srcu_init: Setting srcu_struct sizes based on contention.596lb # [ 0.217856] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization597worker2 # [ 0.041696] kvm-guest: PV spinlocks disabled, single CPU598worker1 # [ 0.067869] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.599lb # [ 0.219864] Spectre V2 : Enabling IBPB for BPF600client # [ 0.072197] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns601worker1 # [ 0.072584] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16602worker1 # [ 0.072869] rcu: srcu_init: Setting srcu_struct sizes based on contention.603client # [ 0.072310] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)604lb # [ 0.220570] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier605client # [ 0.075944] Console: colour VGA+ 80x25606lb # [ 0.221857] active return thunk: srso_alias_return_thunk607worker1 # [ 0.072876] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns608client # [ 0.075947] printk: legacy console [tty0] enabled609client # [ 0.105482] printk: legacy console [ttyS0] enabled610worker2 # [ 0.041698] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bhr46xja1kyki42v8lp87l0nsrargk1l-nixos-system-worker2-test/init regInfo=/nix/store/iw1fwr9xgy9i67hbc5f1wjqv08csn4jl-closure-info/registration console=ttyS0,115200n8 console=tty0611worker1 # [ 0.072981] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)612lb # [ 0.222878] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'613client # [ 0.202866] ACPI: Core revision 20250807614worker1 # [ 0.076561] Console: colour VGA+ 80x25615worker1 # [ 0.076564] printk: legacy console [tty0] enabled616lb # [ 0.223856] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'617worker2 # [ 0.041794] Unknown kernel command line parameters "regInfo=/nix/store/iw1fwr9xgy9i67hbc5f1wjqv08csn4jl-closure-info/registration", will be passed to user space.618worker1 # [ 0.105992] printk: legacy console [ttyS0] enabled619worker2 # [ 0.041807] random: crng init done620client # [ 0.203746] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns621lb # [ 0.224856] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'622worker1 # [ 0.202042] ACPI: Core revision 20250807623worker2 # [ 0.041808] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes624client # [ 0.205240] APIC: Switch to symmetric I/O mode setup625lb # [ 0.225856] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'626worker2 # [ 0.044053] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)627client # [ 0.206212] x2apic enabled628worker1 # [ 0.202921] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns629lb # [ 0.226856] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'630worker2 # [ 0.045470] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)631client # [ 0.206987] APIC: Switched APIC routing to: physical x2apic632worker1 # [ 0.204411] APIC: Switch to symmetric I/O mode setup633worker2 # [ 0.045503] Fallback order for Node 0: 0634lb # [ 0.227856] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'635worker1 # [ 0.205339] x2apic enabled636worker2 # [ 0.045506] Built 1 zonelists, mobility grouping on. Total pages: 393078637worker2 # [ 0.045508] Policy zone: DMA32638lb # [ 0.228856] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'639client # [ 0.208773] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1640worker1 # [ 0.206090] APIC: Switched APIC routing to: physical x2apic641worker2 # [ 0.048151] mem auto-init: stack:all(zero), heap alloc:on, heap free:off642lb # [ 0.229857] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256643worker2 # [ 0.051574] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1644worker2 # [ 0.055094] allocated 3145728 bytes of page_ext645lb # [ 0.230856] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64646client # [ 0.209750] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns647worker1 # [ 0.207858] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1648worker2 # [ 0.065380] ftrace: allocating 48731 entries in 192 pages649lb # [ 0.231856] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512650worker2 # [ 0.065382] ftrace: allocated 192 pages with 2 groups651client # [ 0.211427] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)652worker2 # [ 0.066263] Dynamic Preempt: lazy653worker1 # [ 0.208810] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns654lb # [ 0.232856] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024655worker2 # [ 0.066460] rcu: Preemptible hierarchical RCU implementation.656client # [ 0.212513] x86/cpu: User Mode Instruction Prevention (UMIP) activated657worker2 # [ 0.066460] rcu: RCU event tracing is enabled.658lb # [ 0.233856] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8659worker1 # [ 0.210394] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)660worker2 # [ 0.066461] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.661client # [ 0.213558] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127662worker2 # [ 0.066463] Trampoline variant of Tasks RCU enabled.663lb # [ 0.234856] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.664worker1 # [ 0.212477] x86/cpu: User Mode Instruction Prevention (UMIP) activated665client # [ 0.214424] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0666worker2 # [ 0.066463] Rude variant of Tasks RCU enabled.667worker2 # [ 0.066464] Tracing variant of Tasks RCU enabled.668worker1 # [ 0.213514] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127669worker2 # [ 0.066464] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.670client # [ 0.215428] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto671worker1 # [ 0.214392] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0672worker2 # [ 0.066465] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1673client # [ 0.217425] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl674worker2 # [ 0.066482] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.675worker1 # [ 0.215395] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto676client # [ 0.218425] Transient Scheduler Attacks: Vulnerable: No microcode677worker2 # [ 0.066483] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.678worker1 # [ 0.216391] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl679client # [ 0.219424] Spectre V2 : Mitigation: Enhanced / Automatic IBRS680worker2 # [ 0.066484] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.681worker1 # [ 0.217392] Transient Scheduler Attacks: Vulnerable: No microcode682client # [ 0.220424] Speculative Return Stack Overflow: Mitigation: Safe RET683worker2 # [ 0.071240] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16684worker1 # [ 0.218391] Spectre V2 : Mitigation: Enhanced / Automatic IBRS685worker2 # [ 0.071523] rcu: srcu_init: Setting srcu_struct sizes based on contention.686worker1 # [ 0.219391] Speculative Return Stack Overflow: Mitigation: Safe RET687worker2 # [ 0.071530] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns688worker2 # [ 0.071640] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)689worker2 # [ 0.075176] Console: colour VGA+ 80x25690client # [ 0.221424] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization691worker2 # [ 0.075180] printk: legacy console [tty0] enabled692client # [ 0.223432] Spectre V2 : Enabling IBPB for BPF693worker2 # [ 0.104545] printk: legacy console [ttyS0] enabled694worker1 # [ 0.220391] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization695worker2 # [ 0.199904] ACPI: Core revision 20250807696worker1 # [ 0.222399] Spectre V2 : Enabling IBPB for BPF697client # [ 0.224425] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier698client # [ 0.225425] active return thunk: srso_alias_return_thunk699worker1 # [ 0.223908] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier700worker2 # [ 0.200789] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns701worker1 # [ 0.225391] active return thunk: srso_alias_return_thunk702worker2 # [ 0.202297] APIC: Switch to symmetric I/O mode setup703client # [ 0.227097] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'704worker2 # [ 0.203256] x2apic enabled705client # [ 0.228343] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'706worker1 # [ 0.226233] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'707worker2 # [ 0.204005] APIC: Switched APIC routing to: physical x2apic708worker1 # [ 0.227391] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'709client # [ 0.229424] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'710worker1 # [ 0.228327] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'711client # [ 0.230424] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'712worker2 # [ 0.205812] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1713worker1 # [ 0.229115] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'714client # [ 0.231424] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'715worker1 # [ 0.230129] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'716client # [ 0.232424] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'717worker2 # [ 0.206761] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns718worker1 # [ 0.231113] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'719client # [ 0.233424] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'720worker2 # [ 0.208400] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)721worker1 # [ 0.232151] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'722client # [ 0.234424] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256723worker2 # [ 0.209486] x86/cpu: User Mode Instruction Prevention (UMIP) activated724worker1 # [ 0.233392] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256725client # [ 0.235424] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64726worker2 # [ 0.210523] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127727worker1 # [ 0.234313] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64728client # [ 0.236424] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512729worker2 # [ 0.211397] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0730worker1 # [ 0.235091] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512731client # [ 0.237424] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024732worker1 # [ 0.236126] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024733worker2 # [ 0.212401] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto734client # [ 0.238424] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8735worker1 # [ 0.237097] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8736worker2 # [ 0.213397] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl737client # [ 0.239424] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.738worker1 # [ 0.237391] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.739worker2 # [ 0.214398] Transient Scheduler Attacks: Vulnerable: No microcode740worker2 # [ 0.215397] Spectre V2 : Mitigation: Enhanced / Automatic IBRS741worker2 # [ 0.216397] Speculative Return Stack Overflow: Mitigation: Safe RET742lb # [ 0.269595] Freeing SMP alternatives memory: 44K743worker2 # [ 0.217397] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization744lb # [ 0.269860] pid_max: default: 32768 minimum: 301745worker2 # [ 0.219404] Spectre V2 : Enabling IBPB for BPF746lb # [ 0.270943] LSM: initializing lsm=capability,landlock,yama,bpf,ima747worker2 # [ 0.220398] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier748lb # [ 0.271958] landlock: Up and running.749worker2 # [ 0.221397] active return thunk: srso_alias_return_thunk750lb # [ 0.272857] Yama: becoming mindful.751lb # [ 0.273684] LSM support for eBPF active752worker2 # [ 0.222418] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'753worker2 # [ 0.223397] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'754lb # [ 0.274467] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)755worker2 # [ 0.224397] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'756lb # [ 0.275804] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)757worker2 # [ 0.225397] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'758worker2 # [ 0.226397] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'759lb # [ 0.278098] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)760worker2 # [ 0.227397] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'761worker2 # [ 0.228397] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'762lb # [ 0.279423] Performance Events: Fam17h+ core perfctr, AMD PMU driver.763worker2 # [ 0.229397] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256764lb # [ 0.279865] ... version: 2765lb # [ 0.280576] ... bit width: 48766worker2 # [ 0.231115] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64767lb # [ 0.280859] ... generic counters: 6768worker2 # [ 0.231397] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512769lb # [ 0.281582] ... generic bitmap: 000000000000003f770worker2 # [ 0.232397] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024771lb # [ 0.281858] ... fixed-purpose counters: 0772worker2 # [ 0.233397] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8773lb # [ 0.282572] ... fixed-purpose bitmap: 0000000000000000774lb # [ 0.282859] ... value mask: 0000ffffffffffff775worker2 # [ 0.234397] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.776lb # [ 0.283747] ... max period: 00007fffffffffff777lb # [ 0.284559] ... global_ctrl mask: 000000000000003f778lb # [ 0.284955] signal: max sigframe size: 3376779lb # [ 0.285783] rcu: Hierarchical SRCU implementation.780lb # [ 0.286503] rcu: Max phase no-delay instances is 400.781lb # [ 0.291327] smp: Bringing up secondary CPUs ...782lb # [ 0.291873] smp: Brought up 1 node, 1 CPU783worker1 # [ 0.272569] Freeing SMP alternatives memory: 44K784client # [ 0.274884] Freeing SMP alternatives memory: 44K785lb # [ 0.292614] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)786worker1 # [ 0.273374] pid_max: default: 32768 minimum: 301787client # [ 0.275427] pid_max: default: 32768 minimum: 301788worker1 # [ 0.274009] LSM: initializing lsm=capability,landlock,yama,bpf,ima789client # [ 0.276518] LSM: initializing lsm=capability,landlock,yama,bpf,ima790worker1 # [ 0.275198] landlock: Up and running.791lb # [ 0.293008] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)792client # [ 0.277569] landlock: Up and running.793worker1 # [ 0.275817] Yama: becoming mindful.794client # [ 0.278190] Yama: becoming mindful.795lb # [ 0.294370] devtmpfs: initialized796worker1 # [ 0.276613] LSM support for eBPF active797client # [ 0.279082] LSM support for eBPF active798lb # [ 0.295008] x86/mm: Memory block size: 128MB799worker1 # [ 0.277392] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)800client # [ 0.279576] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)801worker1 # [ 0.278435] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)802lb # [ 0.296725] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)803client # [ 0.280465] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)804lb # [ 0.297760] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).805worker1 # [ 0.280797] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)806lb # [ 0.298957] pinctrl core: initialized pinctrl subsystem807client # [ 0.283652] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)808lb # [ 0.300037] PM: RTC time: 12:42:40, date: 2026-09-20809worker1 # [ 0.281969] Performance Events: Fam17h+ core perfctr, AMD PMU driver.810worker1 # [ 0.282401] ... version: 2811client # [ 0.285009] Performance Events: Fam17h+ core perfctr, AMD PMU driver.812worker1 # [ 0.283099] ... bit width: 48813client # [ 0.285433] ... version: 2814worker1 # [ 0.283394] ... generic counters: 6815client # [ 0.286159] ... bit width: 48816worker1 # [ 0.284098] ... generic bitmap: 000000000000003f817client # [ 0.286426] ... generic counters: 6818lb # [ 0.303611] NET: Registered PF_NETLINK/PF_ROUTE protocol family819worker1 # [ 0.284394] ... fixed-purpose counters: 0820client # [ 0.287132] ... generic bitmap: 000000000000003f821worker1 # [ 0.285083] ... fixed-purpose bitmap: 0000000000000000822client # [ 0.287426] ... fixed-purpose counters: 0823lb # [ 0.304399] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations824worker1 # [ 0.285393] ... value mask: 0000ffffffffffff825client # [ 0.288154] ... fixed-purpose bitmap: 0000000000000000826worker1 # [ 0.286232] ... max period: 00007fffffffffff827lb # [ 0.304879] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations828client # [ 0.288426] ... value mask: 0000ffffffffffff829worker1 # [ 0.287051] ... global_ctrl mask: 000000000000003f830client # [ 0.289327] ... max period: 00007fffffffffff831worker1 # [ 0.287502] signal: max sigframe size: 3376832lb # [ 0.306135] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations833client # [ 0.290101] ... global_ctrl mask: 000000000000003f834worker1 # [ 0.288271] rcu: Hierarchical SRCU implementation.835lb # [ 0.306868] audit: initializing netlink subsys (disabled)836client # [ 0.290535] signal: max sigframe size: 3376837worker1 # [ 0.288965] rcu: Max phase no-delay instances is 400.838lb # [ 0.307975] thermal_sys: Registered thermal governor 'fair_share'839client # [ 0.291327] rcu: Hierarchical SRCU implementation.840client # [ 0.292050] rcu: Max phase no-delay instances is 400.841lb # [ 0.307977] thermal_sys: Registered thermal governor 'bang_bang'842lb # [ 0.308860] thermal_sys: Registered thermal governor 'step_wise'843lb # [ 0.309834] thermal_sys: Registered thermal governor 'user_space'844worker2 # [ 0.269036] Freeing SMP alternatives memory: 44K845worker2 # [ 0.269400] pid_max: default: 32768 minimum: 301846lb # [ 0.310644] audit: type=2000 audit(1789908161.330:1): state=initialized audit_enabled=0 res=1847worker1 # [ 0.293839] smp: Bringing up secondary CPUs ...848worker2 # [ 0.270493] LSM: initializing lsm=capability,landlock,yama,bpf,ima849worker1 # [ 0.294415] smp: Brought up 1 node, 1 CPU850lb # [ 0.312862] thermal_sys: Registered thermal governor 'power_allocator'851worker2 # [ 0.271497] landlock: Up and running.852lb # [ 0.312876] cpuidle: using governor menu853client # [ 0.297027] smp: Bringing up secondary CPUs ...854worker2 # [ 0.272398] Yama: becoming mindful.855worker1 # [ 0.295049] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)856client # [ 0.297441] smp: Brought up 1 node, 1 CPU857worker2 # [ 0.273183] LSM support for eBPF active858lb # [ 0.315056] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5859client # [ 0.298143] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)860worker2 # [ 0.273997] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)861worker1 # [ 0.295558] Memory: 1454548K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)862lb # [ 0.316137] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]863worker1 # [ 0.296628] devtmpfs: initialized864worker2 # [ 0.274438] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)865lb # [ 0.316862] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry866worker1 # [ 0.297375] x86/mm: Memory block size: 128MB867client # [ 0.298655] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)868lb # [ 0.317863] PCI: Using configuration type 1 for base access869client # [ 0.299690] devtmpfs: initialized870client # [ 0.300520] x86/mm: Memory block size: 128MB871worker2 # [ 0.277579] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)872worker1 # [ 0.298938] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)873lb # [ 0.318987] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.874worker1 # [ 0.299420] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).875worker2 # [ 0.279004] Performance Events: Fam17h+ core perfctr, AMD PMU driver.876client # [ 0.302308] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)877worker2 # [ 0.279406] ... version: 2878worker1 # [ 0.300476] pinctrl core: initialized pinctrl subsystem879worker2 # [ 0.280078] ... bit width: 48880worker1 # [ 0.301571] PM: RTC time: 12:42:40, date: 2026-09-20881client # [ 0.303309] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).882worker2 # [ 0.280399] ... generic counters: 6883client # [ 0.304521] pinctrl core: initialized pinctrl subsystem884worker2 # [ 0.281095] ... generic bitmap: 000000000000003f885worker2 # [ 0.281400] ... fixed-purpose counters: 0886client # [ 0.305690] PM: RTC time: 12:42:40, date: 2026-09-20887lb # [ 0.324167] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages888worker2 # [ 0.282075] ... fixed-purpose bitmap: 0000000000000000889lb # [ 0.324860] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page890worker2 # [ 0.282399] ... value mask: 0000ffffffffffff891worker1 # [ 0.305158] NET: Registered PF_NETLINK/PF_ROUTE protocol family892worker2 # [ 0.283262] ... max period: 00007fffffffffff893worker2 # [ 0.284077] ... global_ctrl mask: 000000000000003f894worker1 # [ 0.306580] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations895worker2 # [ 0.284509] signal: max sigframe size: 3376896client # [ 0.309722] NET: Registered PF_NETLINK/PF_ROUTE protocol family897worker1 # [ 0.307420] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations898worker2 # [ 0.285323] rcu: Hierarchical SRCU implementation.899worker2 # [ 0.285987] rcu: Max phase no-delay instances is 400.900client # [ 0.310961] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations901worker1 # [ 0.308672] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations902lb # [ 0.329860] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages903worker1 # [ 0.309404] audit: initializing netlink subsys (disabled)904client # [ 0.311451] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations905lb # [ 0.330859] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page906worker1 # [ 0.310496] thermal_sys: Registered thermal governor 'fair_share'907client # [ 0.312719] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations908worker1 # [ 0.310498] thermal_sys: Registered thermal governor 'bang_bang'909client # [ 0.313446] audit: initializing netlink subsys (disabled)910worker1 # [ 0.311395] thermal_sys: Registered thermal governor 'step_wise'911worker1 # [ 0.312315] thermal_sys: Registered thermal governor 'user_space'912client # [ 0.314593] thermal_sys: Registered thermal governor 'fair_share'913worker2 # [ 0.291017] smp: Bringing up secondary CPUs ...914worker2 # [ 0.291414] smp: Brought up 1 node, 1 CPU915client # [ 0.314595] thermal_sys: Registered thermal governor 'bang_bang'916worker1 # [ 0.313110] audit: type=2000 audit(1789908161.362:1): state=initialized audit_enabled=0 res=1917client # [ 0.315427] thermal_sys: Registered thermal governor 'step_wise'918worker2 # [ 0.292087] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)919worker1 # [ 0.314396] thermal_sys: Registered thermal governor 'power_allocator'920client # [ 0.316374] thermal_sys: Registered thermal governor 'user_space'921worker1 # [ 0.314419] cpuidle: using governor menu922worker2 # [ 0.292600] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)923client # [ 0.317164] audit: type=2000 audit(1789908161.362:1): state=initialized audit_enabled=0 res=1924worker1 # [ 0.316585] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5925worker2 # [ 0.293669] devtmpfs: initialized926client # [ 0.318430] thermal_sys: Registered thermal governor 'power_allocator'927client # [ 0.318445] cpuidle: using governor menu928worker2 # [ 0.294517] x86/mm: Memory block size: 128MB929worker1 # [ 0.317626] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]930lb # [ 0.341208] ACPI: Added _OSI(Module Device)931worker1 # [ 0.318397] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry932lb # [ 0.341860] ACPI: Added _OSI(Processor Device)933client # [ 0.320644] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5934worker1 # [ 0.319405] PCI: Using configuration type 1 for base access935lb # [ 0.342593] ACPI: Added _OSI(Processor Aggregator Device)936worker2 # [ 0.296334] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)937client # [ 0.321749] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]938worker1 # [ 0.320471] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.939worker2 # [ 0.297284] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).940client # [ 0.322430] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry941worker2 # [ 0.298495] pinctrl core: initialized pinctrl subsystem942client # [ 0.323439] PCI: Using configuration type 1 for base access943worker2 # [ 0.299594] PM: RTC time: 12:42:40, date: 2026-09-20944client # [ 0.324581] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.945worker1 # [ 0.325485] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages946worker2 # [ 0.303271] NET: Registered PF_NETLINK/PF_ROUTE protocol family947worker1 # [ 0.326395] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page948worker1 # [ 0.327377] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages949worker2 # [ 0.304618] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations950lb # [ 0.351008] ACPI: 1 ACPI AML tables successfully acquired and loaded951worker2 # [ 0.305424] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations952client # [ 0.329719] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages953client # [ 0.330428] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page954worker2 # [ 0.306694] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations955worker2 # [ 0.307410] audit: initializing netlink subsys (disabled)956lb # [ 0.355262] ACPI: Interpreter enabled957worker1 # [ 0.332394] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page958lb # [ 0.355874] ACPI: PM: (supports S0 S3 S4 S5)959worker2 # [ 0.308554] thermal_sys: Registered thermal governor 'fair_share'960lb # [ 0.356567] ACPI: Using IOAPIC for interrupt routing961worker2 # [ 0.308557] thermal_sys: Registered thermal governor 'bang_bang'962worker2 # [ 0.309401] thermal_sys: Registered thermal governor 'step_wise'963client # [ 0.335428] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages964worker2 # [ 0.310339] thermal_sys: Registered thermal governor 'user_space'965client # [ 0.336426] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page966worker2 # [ 0.311122] audit: type=2000 audit(1789908161.385:1): state=initialized audit_enabled=0 res=1967lb # [ 0.358929] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug968lb # [ 0.359860] PCI: Using E820 reservations for host bridge windows969worker2 # [ 0.312403] thermal_sys: Registered thermal governor 'power_allocator'970worker1 # [ 0.338272] ACPI: Added _OSI(Module Device)971worker2 # [ 0.312418] cpuidle: using governor menu972worker1 # [ 0.338874] ACPI: Added _OSI(Processor Device)973worker1 # [ 0.339394] ACPI: Added _OSI(Processor Aggregator Device)974worker2 # [ 0.314629] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5975lb # [ 0.362956] ACPI: Enabled 2 GPEs in block 00 to 3F976worker2 # [ 0.315633] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]977worker2 # [ 0.316403] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry978worker2 # [ 0.317412] PCI: Using configuration type 1 for base access979worker2 # [ 0.318501] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.980client # [ 0.346832] ACPI: Added _OSI(Module Device)981client # [ 0.347429] ACPI: Added _OSI(Processor Device)982client # [ 0.348191] ACPI: Added _OSI(Processor Aggregator Device)983lb # [ 0.370463] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])984lb # [ 0.370865] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]985worker2 # [ 0.323645] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages986lb # [ 0.371938] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]987worker2 # [ 0.324401] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page988lb # [ 0.372981] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]989worker1 # [ 0.352799] ACPI: 1 ACPI AML tables successfully acquired and loaded990lb # [ 0.374188] PCI host bridge to bus 0000:00991worker1 # [ 0.354286] ACPI: Interpreter enabled992lb # [ 0.374864] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]993worker1 # [ 0.354854] ACPI: PM: (supports S0 S3 S4 S5)994lb # [ 0.375859] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]995worker1 # [ 0.355394] ACPI: Using IOAPIC for interrupt routing996worker2 # [ 0.329400] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages997lb # [ 0.376859] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]998client # [ 0.356694] ACPI: 1 ACPI AML tables successfully acquired and loaded999worker2 # [ 0.330400] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1000worker1 # [ 0.356238] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1001lb # [ 0.377859] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1002lb # [ 0.378859] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1003lb # [ 0.379859] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1004client # [ 0.360838] ACPI: Interpreter enabled1005lb # [ 0.380863] pci_bus 0000:00: root bus resource [bus 00-ff]1006client # [ 0.361451] ACPI: PM: (supports S0 S3 S4 S5)1007worker1 # [ 0.361396] PCI: Using E820 reservations for host bridge windows1008client # [ 0.362138] ACPI: Using IOAPIC for interrupt routing1009lb # [ 0.381903] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1010worker1 # [ 0.362463] ACPI: Enabled 2 GPEs in block 00 to 3F1011client # [ 0.362461] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1012lb # [ 0.383340] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1013client # [ 0.363427] PCI: Using E820 reservations for host bridge windows1014worker2 # [ 0.340801] ACPI: Added _OSI(Module Device)1015client # [ 0.366564] ACPI: Enabled 2 GPEs in block 00 to 3F1016worker2 # [ 0.341402] ACPI: Added _OSI(Processor Device)1017worker2 # [ 0.342137] ACPI: Added _OSI(Processor Aggregator Device)1018worker1 # [ 0.371005] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1019lb # [ 0.386006] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1020worker1 # [ 0.372731] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1021lb # [ 0.386872] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1022worker1 # [ 0.373472] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1023lb # [ 0.387843] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1024worker1 # [ 0.374515] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1025client # [ 0.375288] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1026lb # [ 0.389012] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1027worker1 # [ 0.375753] PCI host bridge to bus 0000:001028client # [ 0.376185] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1029worker2 # [ 0.350999] ACPI: 1 ACPI AML tables successfully acquired and loaded1030worker1 # [ 0.376399] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1031lb # [ 0.390554] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1032worker1 # [ 0.377394] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1033client # [ 0.378375] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1034worker1 # [ 0.378394] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1035client # [ 0.379453] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1036lb # [ 0.392888] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1037worker1 # [ 0.379394] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1038worker2 # [ 0.354837] ACPI: Interpreter enabled1039client # [ 0.380789] PCI host bridge to bus 0000:001040lb # [ 0.393735] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1041worker1 # [ 0.380394] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1042worker2 # [ 0.355416] ACPI: PM: (supports S0 S3 S4 S5)1043client # [ 0.381432] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1044lb # [ 0.394588] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1045worker2 # [ 0.356086] ACPI: Using IOAPIC for interrupt routing1046worker1 # [ 0.381394] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1047client # [ 0.382427] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1048lb # [ 0.394865] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1049worker1 # [ 0.382399] pci_bus 0000:00: root bus resource [bus 00-ff]1050worker2 # [ 0.356448] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1051client # [ 0.383427] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1052worker2 # [ 0.357400] PCI: Using E820 reservations for host bridge windows1053worker1 # [ 0.383298] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1054lb # [ 0.396650] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1055client # [ 0.384427] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1056worker1 # [ 0.384822] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1057client # [ 0.385427] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1058worker2 # [ 0.360439] ACPI: Enabled 2 GPEs in block 00 to 3F1059lb # [ 0.398887] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1060client # [ 0.386426] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1061lb # [ 0.399767] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1062client # [ 0.387427] pci_bus 0000:00: root bus resource [bus 00-ff]1063lb # [ 0.400638] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1064client # [ 0.388415] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1065lb # [ 0.402286] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1066client # [ 0.389865] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1067worker1 # [ 0.387458] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1068lb # [ 0.405490] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1069worker2 # [ 0.369070] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1070worker1 # [ 0.388406] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1071lb # [ 0.405866] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1072worker1 # [ 0.389311] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1073worker2 # [ 0.369407] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1074lb # [ 0.406829] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1075worker1 # [ 0.390485] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1076worker2 # [ 0.371090] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1077lb # [ 0.408261] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1078worker2 # [ 0.372357] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1079worker1 # [ 0.392098] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1080client # [ 0.392538] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1081worker2 # [ 0.373603] PCI host bridge to bus 0000:001082client # [ 0.393439] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1083lb # [ 0.409866] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1084worker2 # [ 0.374360] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1085lb # [ 0.410866] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1086client # [ 0.394402] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1087worker2 # [ 0.375215] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1088worker1 # [ 0.395401] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1089lb # [ 0.411831] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1090worker1 # [ 0.396218] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1091client # [ 0.395763] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1092worker2 # [ 0.376202] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1093worker1 # [ 0.397098] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1094worker2 # [ 0.377280] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1095lb # [ 0.413236] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1096worker1 # [ 0.397400] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1097client # [ 0.397149] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1098worker2 # [ 0.378273] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1099lb # [ 0.415885] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1100worker2 # [ 0.379281] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1101worker1 # [ 0.399370] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1102lb # [ 0.416746] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1103worker2 # [ 0.380306] pci_bus 0000:00: root bus resource [bus 00-ff]1104client # [ 0.400454] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1105lb # [ 0.417601] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1106worker2 # [ 0.381167] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1107client # [ 0.401280] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1108worker1 # [ 0.401428] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1109client # [ 0.402185] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1110lb # [ 0.418424] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1111worker2 # [ 0.382862] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1112worker1 # [ 0.402323] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1113client # [ 0.402433] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1114worker1 # [ 0.403099] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1115lb # [ 0.420646] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1116client # [ 0.403983] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1117worker1 # [ 0.403968] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118lb # [ 0.420866] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1119lb # [ 0.421883] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1120client # [ 0.406455] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1121client # [ 0.407356] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1122lb # [ 0.423479] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1123worker1 # [ 0.406402] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1124client # [ 0.408198] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1125worker1 # [ 0.407407] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1126lb # [ 0.424866] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1127worker2 # [ 0.385487] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1128worker1 # [ 0.408374] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1129client # [ 0.409861] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1130lb # [ 0.425695] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1131worker2 # [ 0.386412] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1132lb # [ 0.426605] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1133worker1 # [ 0.409763] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1134worker2 # [ 0.387354] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1135lb # [ 0.426865] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1136worker2 # [ 0.388572] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1137client # [ 0.412434] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1138worker1 # [ 0.411401] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1139lb # [ 0.428409] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1140client # [ 0.413286] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1141worker1 # [ 0.412226] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1142worker2 # [ 0.390112] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1143client # [ 0.414162] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1144worker1 # [ 0.413116] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1145lb # [ 0.429703] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1146lb # [ 0.430592] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1147client # [ 0.414995] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1148worker1 # [ 0.413962] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1149worker2 # [ 0.393427] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1150lb # [ 0.431409] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1151worker2 # [ 0.394287] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1152client # [ 0.416453] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1153worker1 # [ 0.416424] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1154worker2 # [ 0.395139] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1155worker1 # [ 0.417265] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1156client # [ 0.417296] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1157lb # [ 0.432866] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1158worker2 # [ 0.395406] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1159worker1 # [ 0.418091] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1160client # [ 0.418151] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1161lb # [ 0.433746] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1162lb # [ 0.434586] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1163worker2 # [ 0.396976] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1164worker1 # [ 0.418990] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1165client # [ 0.419706] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1166lb # [ 0.435394] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1167worker2 # [ 0.398436] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1168client # [ 0.421454] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1169worker1 # [ 0.421409] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1170lb # [ 0.436867] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1171worker2 # [ 0.399379] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1172client # [ 0.422310] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1173worker1 # [ 0.422400] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1174worker2 # [ 0.400126] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1175lb # [ 0.437693] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1176client # [ 0.423158] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1177worker1 # [ 0.423333] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1178lb # [ 0.438566] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1179worker2 # [ 0.400974] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1180client # [ 0.423990] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1181lb # [ 0.439441] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1182worker1 # [ 0.424829] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1183lb # [ 0.440432] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1184worker1 # [ 0.426401] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1185worker1 # [ 0.427294] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1186worker2 # [ 0.403430] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1187lb # [ 0.441063] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1188client # [ 0.426683] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1189worker2 # [ 0.404265] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1190worker1 # [ 0.428101] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1191client # [ 0.427453] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1192lb # [ 0.442415] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1193worker2 # [ 0.405131] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1194worker1 # [ 0.428401] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1195client # [ 0.428375] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1196worker2 # [ 0.405961] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1197client # [ 0.429849] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1198worker1 # [ 0.429875] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1199worker1 # [ 0.431217] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1200client # [ 0.431452] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1201worker2 # [ 0.407430] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1202worker1 # [ 0.432149] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1203client # [ 0.432284] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1204worker2 # [ 0.408268] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1205client # [ 0.433129] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1206worker1 # [ 0.432949] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1207client # [ 0.433433] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1208worker2 # [ 0.409111] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1209worker2 # [ 0.409972] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1210worker1 # [ 0.434402] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1211client # [ 0.435547] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1212worker1 # [ 0.435266] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1213worker1 # [ 0.436146] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1214client # [ 0.437251] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1215worker2 # [ 0.411424] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1216lb # [ 0.443048] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1217client # [ 0.438195] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1218worker2 # [ 0.412252] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1219lb # [ 0.445412] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1220worker1 # [ 0.438285] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1221worker2 # [ 0.413147] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1222client # [ 0.439835] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1223lb # [ 0.446096] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1224worker1 # [ 0.440401] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1225worker2 # [ 0.413989] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1226lb # [ 0.447406] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1227worker1 # [ 0.441263] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1228client # [ 0.441435] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1229worker1 # [ 0.442131] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1230client # [ 0.442301] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1231lb # [ 0.448128] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1232worker1 # [ 0.442940] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1233client # [ 0.443147] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1234lb # [ 0.449430] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1235worker2 # [ 0.416409] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1236worker2 # [ 0.418033] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1237worker1 # [ 0.443992] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1238client # [ 0.443977] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1239lb # [ 0.450100] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1240worker2 # [ 0.418422] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1241worker1 # [ 0.444587] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1242lb # [ 0.451629] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1243client # [ 0.445454] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1244worker2 # [ 0.420000] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1245lb # [ 0.451867] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1246client # [ 0.446270] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1247worker1 # [ 0.445929] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1248client # [ 0.447135] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1249lb # [ 0.453110] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1250worker2 # [ 0.421425] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1251lb # [ 0.454401] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1252worker2 # [ 0.422406] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1253client # [ 0.448047] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1254worker2 # [ 0.423338] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1255client # [ 0.449026] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1256worker2 # [ 0.424265] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1257client # [ 0.449661] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1258lb # [ 0.458181] ACPI: PCI: Interrupt link LNKA configured for IRQ 101259worker2 # [ 0.426287] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1260client # [ 0.451002] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1261lb # [ 0.458964] ACPI: PCI: Interrupt link LNKB configured for IRQ 101262lb # [ 0.459963] ACPI: PCI: Interrupt link LNKC configured for IRQ 111263worker2 # [ 0.428412] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1264lb # [ 0.460956] ACPI: PCI: Interrupt link LNKD configured for IRQ 111265worker2 # [ 0.429410] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1266lb # [ 0.461958] ACPI: PCI: Interrupt link LNKE configured for IRQ 101267lb # [ 0.462956] ACPI: PCI: Interrupt link LNKF configured for IRQ 101268worker1 # [ 0.446592] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1269worker2 # [ 0.430983] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1270lb # [ 0.463960] ACPI: PCI: Interrupt link LNKG configured for IRQ 111271worker1 # [ 0.448929] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1272lb # [ 0.464957] ACPI: PCI: Interrupt link LNKH configured for IRQ 111273worker1 # [ 0.449642] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1274worker2 # [ 0.432434] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1275lb # [ 0.465897] ACPI: PCI: Interrupt link GSIA configured for IRQ 161276lb # [ 0.466871] ACPI: PCI: Interrupt link GSIB configured for IRQ 171277worker1 # [ 0.450903] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1278worker2 # [ 0.433298] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1279lb # [ 0.467865] ACPI: PCI: Interrupt link GSIC configured for IRQ 181280worker1 # [ 0.451667] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1281worker2 # [ 0.434423] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1282lb # [ 0.468861] ACPI: PCI: Interrupt link GSID configured for IRQ 191283lb # [ 0.469781] ACPI: PCI: Interrupt link GSIE configured for IRQ 201284worker2 # [ 0.435945] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1285worker1 # [ 0.452980] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1286client # [ 0.451625] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1287lb # [ 0.470607] ACPI: PCI: Interrupt link GSIF configured for IRQ 211288lb # [ 0.470870] ACPI: PCI: Interrupt link GSIG configured for IRQ 221289worker1 # [ 0.453638] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1290client # [ 0.453999] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1291lb # [ 0.471831] ACPI: PCI: Interrupt link GSIH configured for IRQ 231292worker2 # [ 0.437407] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1293worker1 # [ 0.455194] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1294client # [ 0.454659] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1295worker2 # [ 0.438309] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1296lb # [ 0.473630] iommu: Default domain type: Translated1297worker1 # [ 0.456028] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1298worker2 # [ 0.439127] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1299client # [ 0.455939] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1300lb # [ 0.473872] iommu: DMA domain TLB invalidation policy: lazy mode1301worker1 # [ 0.456703] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1302lb # [ 0.475091] ACPI: bus type USB registered1303worker2 # [ 0.439979] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1304client # [ 0.456695] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1305lb # [ 0.475827] usbcore: registered new interface driver usbfs1306worker1 # [ 0.457928] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1307lb # [ 0.476575] usbcore: registered new interface driver hub1308client # [ 0.457737] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1309lb # [ 0.476869] usbcore: registered new device driver usb1310worker2 # [ 0.442143] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1311client # [ 0.458674] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1312lb # [ 0.478670] NetLabel: Initializing1313worker2 # [ 0.442625] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1314lb # [ 0.478866] NetLabel: domain hash size = 1281315worker1 # [ 0.461626] ACPI: PCI: Interrupt link LNKA configured for IRQ 101316client # [ 0.460478] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1317lb # [ 0.479607] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1318worker2 # [ 0.444057] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1319worker1 # [ 0.462496] ACPI: PCI: Interrupt link LNKB configured for IRQ 101320client # [ 0.461322] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1321lb # [ 0.479905] NetLabel: unlabeled traffic allowed by default1322worker1 # [ 0.463494] ACPI: PCI: Interrupt link LNKC configured for IRQ 111323lb # [ 0.480804] PCI: Using ACPI for IRQ routing1324client # [ 0.462474] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1325worker1 # [ 0.464504] ACPI: PCI: Interrupt link LNKD configured for IRQ 111326worker1 # [ 0.465493] ACPI: PCI: Interrupt link LNKE configured for IRQ 101327client # [ 0.464391] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1328worker1 # [ 0.466490] ACPI: PCI: Interrupt link LNKF configured for IRQ 101329worker1 # [ 0.467493] ACPI: PCI: Interrupt link LNKG configured for IRQ 111330worker1 # [ 0.468492] ACPI: PCI: Interrupt link LNKH configured for IRQ 111331worker1 # [ 0.469437] ACPI: PCI: Interrupt link GSIA configured for IRQ 161332client # [ 0.468170] ACPI: PCI: Interrupt link LNKA configured for IRQ 101333worker1 # [ 0.470407] ACPI: PCI: Interrupt link GSIB configured for IRQ 171334client # [ 0.468534] ACPI: PCI: Interrupt link LNKB configured for IRQ 101335worker1 # [ 0.471404] ACPI: PCI: Interrupt link GSIC configured for IRQ 181336client # [ 0.469526] ACPI: PCI: Interrupt link LNKC configured for IRQ 111337worker1 # [ 0.472387] ACPI: PCI: Interrupt link GSID configured for IRQ 191338client # [ 0.470525] ACPI: PCI: Interrupt link LNKD configured for IRQ 111339worker1 # [ 0.473151] ACPI: PCI: Interrupt link GSIE configured for IRQ 201340client # [ 0.471526] ACPI: PCI: Interrupt link LNKE configured for IRQ 101341worker1 # [ 0.473409] ACPI: PCI: Interrupt link GSIF configured for IRQ 211342worker2 # [ 0.444593] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1343client # [ 0.472523] ACPI: PCI: Interrupt link LNKF configured for IRQ 101344worker1 # [ 0.474320] ACPI: PCI: Interrupt link GSIG configured for IRQ 221345worker2 # [ 0.447044] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1346worker1 # [ 0.475130] ACPI: PCI: Interrupt link GSIH configured for IRQ 231347client # [ 0.473739] ACPI: PCI: Interrupt link LNKG configured for IRQ 111348client # [ 0.474536] ACPI: PCI: Interrupt link LNKH configured for IRQ 111349worker2 # [ 0.447650] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1350worker1 # [ 0.476546] iommu: Default domain type: Translated1351client # [ 0.475467] ACPI: PCI: Interrupt link GSIA configured for IRQ 161352worker1 # [ 0.477314] iommu: DMA domain TLB invalidation policy: lazy mode1353client # [ 0.476442] ACPI: PCI: Interrupt link GSIB configured for IRQ 171354worker2 # [ 0.449373] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1355worker1 # [ 0.478366] ACPI: bus type USB registered1356client # [ 0.477382] ACPI: PCI: Interrupt link GSIC configured for IRQ 181357worker1 # [ 0.478934] usbcore: registered new interface driver usbfs1358client # [ 0.478185] ACPI: PCI: Interrupt link GSID configured for IRQ 191359worker2 # [ 0.450413] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1360worker1 # [ 0.479406] usbcore: registered new interface driver hub1361client # [ 0.478438] ACPI: PCI: Interrupt link GSIE configured for IRQ 201362worker1 # [ 0.480281] usbcore: registered new device driver usb1363worker2 # [ 0.451719] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1364client # [ 0.479428] ACPI: PCI: Interrupt link GSIF configured for IRQ 211365worker1 # [ 0.481877] NetLabel: Initializing1366client # [ 0.480438] ACPI: PCI: Interrupt link GSIG configured for IRQ 221367worker2 # [ 0.452665] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1368worker1 # [ 0.482395] NetLabel: domain hash size = 1281369client # [ 0.481406] ACPI: PCI: Interrupt link GSIH configured for IRQ 231370worker1 # [ 0.483114] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1371worker2 # [ 0.454338] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1372worker1 # [ 0.483441] NetLabel: unlabeled traffic allowed by default1373client # [ 0.483222] iommu: Default domain type: Translated1374worker2 # [ 0.455050] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1375worker1 # [ 0.484350] PCI: Using ACPI for IRQ routing1376client # [ 0.483999] iommu: DMA domain TLB invalidation policy: lazy mode1377client # [ 0.484703] ACPI: bus type USB registered1378worker2 # [ 0.455747] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1379client # [ 0.485476] usbcore: registered new interface driver usbfs1380worker2 # [ 0.457085] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1381client # [ 0.486406] usbcore: registered new interface driver hub1382client # [ 0.487082] usbcore: registered new device driver usb1383client # [ 0.488319] NetLabel: Initializing1384client # [ 0.488895] NetLabel: domain hash size = 1281385worker2 # [ 0.460360] ACPI: PCI: Interrupt link LNKA configured for IRQ 101386client # [ 0.489426] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1387worker2 # [ 0.461284] ACPI: PCI: Interrupt link LNKB configured for IRQ 101388client # [ 0.490428] NetLabel: unlabeled traffic allowed by default1389client # [ 0.491353] PCI: Using ACPI for IRQ routing1390worker2 # [ 0.462270] ACPI: PCI: Interrupt link LNKC configured for IRQ 111391worker2 # [ 0.463273] ACPI: PCI: Interrupt link LNKD configured for IRQ 111392worker2 # [ 0.464263] ACPI: PCI: Interrupt link LNKE configured for IRQ 101393worker2 # [ 0.465492] ACPI: PCI: Interrupt link LNKF configured for IRQ 101394worker2 # [ 0.466501] ACPI: PCI: Interrupt link LNKG configured for IRQ 111395worker2 # [ 0.467496] ACPI: PCI: Interrupt link LNKH configured for IRQ 111396worker2 # [ 0.468472] ACPI: PCI: Interrupt link GSIA configured for IRQ 161397worker2 # [ 0.469420] ACPI: PCI: Interrupt link GSIB configured for IRQ 171398worker2 # [ 0.470414] ACPI: PCI: Interrupt link GSIC configured for IRQ 181399lb # [ 0.524231] pci 0000:00:01.0: vgaarb: setting as boot VGA device1400worker2 # [ 0.471415] ACPI: PCI: Interrupt link GSID configured for IRQ 191401lb # [ 0.524855] pci 0000:00:01.0: vgaarb: bridge control possible1402worker2 # [ 0.472379] ACPI: PCI: Interrupt link GSIE configured for IRQ 201403lb # [ 0.524855] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1404lb # [ 0.524863] vgaarb: loaded1405worker2 # [ 0.473413] ACPI: PCI: Interrupt link GSIF configured for IRQ 211406lb # [ 0.525584] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01407worker2 # [ 0.474376] ACPI: PCI: Interrupt link GSIG configured for IRQ 221408lb # [ 0.525861] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1409worker2 # [ 0.475166] ACPI: PCI: Interrupt link GSIH configured for IRQ 231410worker2 # [ 0.476586] iommu: Default domain type: Translated1411worker2 # [ 0.477413] iommu: DMA domain TLB invalidation policy: lazy mode1412worker2 # [ 0.478672] ACPI: bus type USB registered1413worker2 # [ 0.479439] usbcore: registered new interface driver usbfs1414worker2 # [ 0.480371] usbcore: registered new interface driver hub1415lb # [ 0.531945] clocksource: Switched to clocksource kvm-clock1416worker2 # [ 0.481070] usbcore: registered new device driver usb1417worker2 # [ 0.482316] NetLabel: Initializing1418worker2 # [ 0.482895] NetLabel: domain hash size = 1281419lb # [ 0.535395] VFS: Disk quotas dquot_6.6.01420worker2 # [ 0.483399] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1421lb # [ 0.536103] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1422worker2 # [ 0.484392] NetLabel: unlabeled traffic allowed by default1423worker2 # [ 0.485360] PCI: Using ACPI for IRQ routing1424lb # [ 0.537474] pnp: PnP ACPI init1425lb # [ 0.538254] ACPI: IRQ 4 override to edge(!), high(!)1426lb # [ 0.539293] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1427lb # [ 0.540870] pnp: PnP ACPI: found 6 devices1428worker1 # [ 0.528262] pci 0000:00:01.0: vgaarb: setting as boot VGA device1429worker1 # [ 0.528390] pci 0000:00:01.0: vgaarb: bridge control possible1430worker1 # [ 0.528390] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1431worker1 # [ 0.528399] vgaarb: loaded1432worker1 # [ 0.529145] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01433lb # [ 0.548464] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1434worker1 # [ 0.529394] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1435lb # [ 0.549896] clocksource: Switched to clocksource acpi_pm1436lb # [ 0.550886] NET: Registered PF_INET protocol family1437lb # [ 0.552088] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1438worker1 # [ 0.532485] clocksource: Switched to clocksource kvm-clock1439client # [ 0.536393] pci 0000:00:01.0: vgaarb: setting as boot VGA device1440client # [ 0.536423] pci 0000:00:01.0: vgaarb: bridge control possible1441worker1 # [ 0.535975] VFS: Disk quotas dquot_6.6.01442client # [ 0.536423] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1443worker1 # [ 0.536732] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1444client # [ 0.536432] vgaarb: loaded1445worker1 # [ 0.538052] pnp: PnP ACPI init1446client # [ 0.537200] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01447worker1 # [ 0.538863] ACPI: IRQ 4 override to edge(!), high(!)1448client # [ 0.537427] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1449worker1 # [ 0.539845] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1450worker1 # [ 0.541395] pnp: PnP ACPI: found 6 devices1451client # [ 0.541514] clocksource: Switched to clocksource kvm-clock1452client # [ 0.545117] VFS: Disk quotas dquot_6.6.01453lb # [ 0.568606] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1454client # [ 0.545812] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1455lb # [ 0.570023] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1456worker1 # [ 0.549038] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1457client # [ 0.547135] pnp: PnP ACPI init1458lb # [ 0.571255] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1459worker1 # [ 0.550449] clocksource: Switched to clocksource acpi_pm1460client # [ 0.547926] ACPI: IRQ 4 override to edge(!), high(!)1461worker1 # [ 0.551458] NET: Registered PF_INET protocol family1462lb # [ 0.572577] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1463client # [ 0.548931] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1464lb # [ 0.573776] TCP: Hash tables configured (established 16384 bind 16384)1465worker1 # [ 0.552634] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1466client # [ 0.550480] pnp: PnP ACPI: found 6 devices1467lb # [ 0.574852] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1468lb # [ 0.576141] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1469lb # [ 0.577258] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1470lb # [ 0.578532] NET: Registered PF_UNIX/PF_LOCAL protocol family1471lb # [ 0.579521] NET: Registered PF_XDP protocol family1472lb # [ 0.580359] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1473worker2 # [ 0.529070] pci 0000:00:01.0: vgaarb: setting as boot VGA device1474lb # [ 0.581405] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1475worker2 # [ 0.529396] pci 0000:00:01.0: vgaarb: bridge control possible1476lb # [ 0.582373] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1477client # [ 0.558141] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1478lb # [ 0.583460] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1479worker2 # [ 0.529396] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1480client # [ 0.559575] clocksource: Switched to clocksource acpi_pm1481worker2 # [ 0.529405] vgaarb: loaded1482client # [ 0.560544] NET: Registered PF_INET protocol family1483lb # [ 0.584514] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1484worker2 # [ 0.530140] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01485lb # [ 0.585601] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1486worker2 # [ 0.530401] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1487client # [ 0.561750] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1488lb # [ 0.587252] ACPI: \_SB_.GSIA: Enabled at IRQ 161489lb # [ 0.589467] ACPI: \_SB_.GSIB: Enabled at IRQ 171490worker1 # [ 0.569038] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1491worker2 # [ 0.534517] clocksource: Switched to clocksource kvm-clock1492lb # [ 0.591614] ACPI: \_SB_.GSIC: Enabled at IRQ 181493worker1 # [ 0.570618] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1494lb # [ 0.593570] ACPI: \_SB_.GSID: Enabled at IRQ 191495worker1 # [ 0.571958] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1496worker1 # [ 0.573340] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1497worker2 # [ 0.538346] VFS: Disk quotas dquot_6.6.01498lb # [ 0.595223] PCI: CLS 0 bytes, default 641499worker1 # [ 0.574626] TCP: Hash tables configured (established 16384 bind 16384)1500worker2 # [ 0.539081] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1501lb # [ 0.596093] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1502worker2 # [ 0.540457] pnp: PnP ACPI init1503worker1 # [ 0.575731] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1504lb # [ 0.597808] Trying to unpack rootfs image as initramfs...1505worker2 # [ 0.541289] ACPI: IRQ 4 override to edge(!), high(!)1506worker1 # [ 0.577006] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1507worker2 # [ 0.542265] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1508worker1 # [ 0.578125] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1509worker2 # [ 0.543771] pnp: PnP ACPI: found 6 devices1510worker1 # [ 0.579355] NET: Registered PF_UNIX/PF_LOCAL protocol family1511worker1 # [ 0.580367] NET: Registered PF_XDP protocol family1512worker1 # [ 0.581248] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1513client # [ 0.578111] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1514worker1 # [ 0.582259] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1515client # [ 0.579677] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1516worker1 # [ 0.583292] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1517client # [ 0.580978] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1518worker1 # [ 0.584400] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1519client # [ 0.582314] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1520worker1 # [ 0.585552] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1521client # [ 0.583521] TCP: Hash tables configured (established 16384 bind 16384)1522worker1 # [ 0.586696] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1523worker2 # [ 0.551514] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1524client # [ 0.584683] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1525worker1 # [ 0.588384] ACPI: \_SB_.GSIA: Enabled at IRQ 161526worker2 # [ 0.552977] clocksource: Switched to clocksource acpi_pm1527client # [ 0.585969] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1528worker2 # [ 0.553951] NET: Registered PF_INET protocol family1529client # [ 0.587069] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1530worker1 # [ 0.590572] ACPI: \_SB_.GSIB: Enabled at IRQ 171531worker2 # [ 0.555141] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1532client # [ 0.588256] NET: Registered PF_UNIX/PF_LOCAL protocol family1533client # [ 0.589148] NET: Registered PF_XDP protocol family1534worker1 # [ 0.592583] ACPI: \_SB_.GSIC: Enabled at IRQ 181535client # [ 0.589958] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1536client # [ 0.590918] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1537worker1 # [ 0.594540] ACPI: \_SB_.GSID: Enabled at IRQ 191538client # [ 0.591874] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1539worker1 # [ 0.596204] PCI: CLS 0 bytes, default 641540client # [ 0.592933] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1541client # [ 0.593974] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1542worker1 # [ 0.597058] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1543client # [ 0.595017] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1544worker1 # [ 0.598827] Trying to unpack rootfs image as initramfs...1545client # [ 0.596677] ACPI: \_SB_.GSIA: Enabled at IRQ 161546client # [ 0.599189] ACPI: \_SB_.GSIB: Enabled at IRQ 171547client # [ 0.601163] ACPI: \_SB_.GSIC: Enabled at IRQ 181548client # [ 0.603156] ACPI: \_SB_.GSID: Enabled at IRQ 191549client # [ 0.604817] PCI: CLS 0 bytes, default 641550worker2 # [ 0.572414] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1551client # [ 0.605684] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1552worker2 # [ 0.573969] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1553client # [ 0.607368] Trying to unpack rootfs image as initramfs...1554worker2 # [ 0.575223] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1555worker2 # [ 0.576501] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1556worker2 # [ 0.577725] TCP: Hash tables configured (established 16384 bind 16384)1557worker2 # [ 0.578812] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1558worker2 # [ 0.580045] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1559worker2 # [ 0.581086] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1560worker2 # [ 0.582284] NET: Registered PF_UNIX/PF_LOCAL protocol family1561worker2 # [ 0.583227] NET: Registered PF_XDP protocol family1562worker2 # [ 0.584068] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1563lb # [ 0.641845] Initialise system trusted keyrings1564worker2 # [ 0.585116] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1565worker2 # [ 0.586116] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1566worker2 # [ 0.587270] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1567worker2 # [ 0.588412] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1568lb # [ 0.645491] workingset: timestamp_bits=40 max_order=19 bucket_order=01569worker2 # [ 0.589530] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1570worker2 # [ 0.591407] ACPI: \_SB_.GSIA: Enabled at IRQ 161571worker2 # [ 0.593976] ACPI: \_SB_.GSIB: Enabled at IRQ 171572worker2 # [ 0.595934] ACPI: \_SB_.GSIC: Enabled at IRQ 181573worker2 # [ 0.597911] ACPI: \_SB_.GSID: Enabled at IRQ 191574worker2 # [ 0.599606] PCI: CLS 0 bytes, default 641575worker2 # [ 0.600514] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1576worker2 # [ 0.602317] Trying to unpack rootfs image as initramfs...1577worker1 # [ 0.644959] Initialise system trusted keyrings1578lb # [ 0.666718] Key type asymmetric registered1579worker1 # [ 0.645904] workingset: timestamp_bits=40 max_order=19 bucket_order=01580lb # [ 0.667412] Asymmetric key parser 'x509' registered1581lb # [ 0.668277] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582lb # [ 0.673598] io scheduler mq-deadline registered1583lb # [ 0.674406] io scheduler kyber registered1584client # [ 0.652322] Initialise system trusted keyrings1585lb # [ 0.677962] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1586lb # [ 0.679224] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1587client # [ 0.655313] workingset: timestamp_bits=40 max_order=19 bucket_order=01588lb # [ 0.683203] Linux agpgart interface v0.1031589lb # [ 0.684006] ACPI: bus type drm_connector registered1590lb # [ 0.686887] usbcore: registered new interface driver usbserial_generic1591lb # [ 0.687983] usbserial: USB Serial support registered for generic1592worker1 # [ 0.667611] Key type asymmetric registered1593worker1 # [ 0.668345] Asymmetric key parser 'x509' registered1594worker1 # [ 0.669243] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1595lb # [ 0.691461] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1596lb # [ 0.691461] Please enable it if your BIOS has the CPPC option.1597lb # [ 0.693747] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1598worker1 # [ 0.674831] io scheduler mq-deadline registered1599worker1 # [ 0.675648] io scheduler kyber registered1600lb # [ 0.697610] drop_monitor: Initializing network drop monitor service1601lb # [ 0.698810] NET: Registered PF_INET6 protocol family1602worker1 # [ 0.679061] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1603client # [ 0.677366] Key type asymmetric registered1604lb # [ 0.701877] Segment Routing with IPv61605worker1 # [ 0.680383] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1606client # [ 0.678065] Asymmetric key parser 'x509' registered1607client # [ 0.678910] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1608worker2 # [ 0.647397] Initialise system trusted keyrings1609lb # [ 0.704474] In-situ OAM (IOAM) with IPv61610lb # [ 0.705520] IPI shorthand broadcast: enabled1611worker1 # [ 0.685807] Linux agpgart interface v0.1031612worker1 # [ 0.686621] ACPI: bus type drm_connector registered1613worker2 # [ 0.650945] workingset: timestamp_bits=40 max_order=19 bucket_order=01614client # [ 0.685380] io scheduler mq-deadline registered1615client # [ 0.686163] io scheduler kyber registered1616worker1 # [ 0.689874] usbcore: registered new interface driver usbserial_generic1617worker1 # [ 0.690945] usbserial: USB Serial support registered for generic1618client # [ 0.688800] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1619lb # [ 0.713814] sched_clock: Marking stable (577014408, 136423308)->(787139504, -73701788)1620client # [ 0.690117] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1621worker1 # [ 0.693555] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1622worker1 # [ 0.693555] Please enable it if your BIOS has the CPPC option.1623worker1 # [ 0.695825] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1624lb # [ 0.719529] registered taskstats version 11625client # [ 0.695547] Linux agpgart interface v0.1031626client # [ 0.696326] ACPI: bus type drm_connector registered1627lb # [ 0.720489] Loading compiled-in X.509 certificates1628worker1 # [ 0.699725] drop_monitor: Initializing network drop monitor service1629worker1 # [ 0.700906] NET: Registered PF_INET6 protocol family1630client # [ 0.699619] usbcore: registered new interface driver usbserial_generic1631worker1 # [ 0.703992] Segment Routing with IPv61632client # [ 0.700728] usbserial: USB Serial support registered for generic1633worker1 # [ 0.706560] In-situ OAM (IOAM) with IPv61634worker1 # [ 0.707691] IPI shorthand broadcast: enabled1635worker2 # [ 0.672283] Key type asymmetric registered1636client # [ 0.704278] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1637client # [ 0.704278] Please enable it if your BIOS has the CPPC option.1638client # [ 0.706471] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1639worker2 # [ 0.674921] Asymmetric key parser 'x509' registered1640worker2 # [ 0.675798] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1641client # [ 0.710434] drop_monitor: Initializing network drop monitor service1642worker2 # [ 0.679028] io scheduler mq-deadline registered1643client # [ 0.711642] NET: Registered PF_INET6 protocol family1644worker2 # [ 0.679789] io scheduler kyber registered1645lb # [ 0.737057] Demotion targets for Node 0: null1646worker1 # [ 0.715843] sched_clock: Marking stable (582014091, 133514423)->(783944280, -68415766)1647client # [ 0.714765] Segment Routing with IPv61648lb # [ 0.739493] Key type .fscrypt registered1649worker2 # [ 0.682623] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1650lb # [ 0.740217] Key type fscrypt-provisioning registered1651client # [ 0.717293] In-situ OAM (IOAM) with IPv61652lb # [ 0.741216] ima: No TPM chip found, activating TPM-bypass!1653worker1 # [ 0.720783] registered taskstats version 11654client # [ 0.718386] IPI shorthand broadcast: enabled1655worker1 # [ 0.721766] Loading compiled-in X.509 certificates1656worker2 # [ 0.685896] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1657lb # [ 0.743457] ima: Allocated hash algorithm: sha11658lb # [ 0.744236] ima: No architecture policies found1659worker2 # [ 0.689558] Linux agpgart interface v0.1031660worker2 # [ 0.690359] ACPI: bus type drm_connector registered1661lb # [ 0.748458] PM: Magic number: 10:478:7341662lb # [ 0.750106] RAS: Correctable Errors collector initialized.1663worker2 # [ 0.693418] usbcore: registered new interface driver usbserial_generic1664client # [ 0.726514] sched_clock: Marking stable (591013214, 135226134)->(800435138, -74195790)1665worker2 # [ 0.694470] usbserial: USB Serial support registered for generic1666client # [ 0.730346] registered taskstats version 11667worker2 # [ 0.697908] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1668client # [ 0.731339] Loading compiled-in X.509 certificates1669worker2 # [ 0.697908] Please enable it if your BIOS has the CPPC option.1670worker2 # [ 0.700122] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1671lb # [ 0.758075] clk: Disabling unused clocks1672worker1 # [ 0.739544] Demotion targets for Node 0: null1673worker2 # [ 0.704039] drop_monitor: Initializing network drop monitor service1674worker1 # [ 0.740429] Key type .fscrypt registered1675worker2 # [ 0.705254] NET: Registered PF_INET6 protocol family1676lb # [ 0.762455] PM: genpd: Disabling unused power domains1677worker1 # [ 0.742549] Key type fscrypt-provisioning registered1678worker1 # [ 0.743548] ima: No TPM chip found, activating TPM-bypass!1679worker2 # [ 0.708620] Segment Routing with IPv61680worker1 # [ 0.744487] ima: Allocated hash algorithm: sha11681worker2 # [ 0.710942] In-situ OAM (IOAM) with IPv61682worker1 # [ 0.748576] ima: No architecture policies found1683worker2 # [ 0.713310] IPI shorthand broadcast: enabled1684worker1 # [ 0.749563] PM: Magic number: 10:478:7341685worker1 # [ 0.751213] RAS: Correctable Errors collector initialized.1686client # [ 0.749276] Demotion targets for Node 0: null1687client # [ 0.751299] Key type .fscrypt registered1688client # [ 0.751982] Key type fscrypt-provisioning registered1689client # [ 0.752949] ima: No TPM chip found, activating TPM-bypass!1690worker2 # [ 0.721899] sched_clock: Marking stable (589014575, 132866695)->(801631232, -79749962)1691worker1 # [ 0.760204] clk: Disabling unused clocks1692client # [ 0.757265] ima: Allocated hash algorithm: sha11693client # [ 0.758020] ima: No architecture policies found1694worker2 # [ 0.727206] registered taskstats version 11695client # [ 0.760273] PM: Magic number: 10:478:7341696worker2 # [ 0.728252] Loading compiled-in X.509 certificates1697worker1 # [ 0.764541] PM: genpd: Disabling unused power domains1698client # [ 0.761895] RAS: Correctable Errors collector initialized.1699client # [ 0.770949] clk: Disabling unused clocks1700client # [ 0.775274] PM: genpd: Disabling unused power domains1701worker2 # [ 0.747093] Demotion targets for Node 0: null1702worker2 # [ 0.748044] Key type .fscrypt registered1703worker2 # [ 0.748708] Key type fscrypt-provisioning registered1704worker2 # [ 0.752032] ima: No TPM chip found, activating TPM-bypass!1705worker2 # [ 0.752944] ima: Allocated hash algorithm: sha11706worker2 # [ 0.753721] ima: No architecture policies found1707worker2 # [ 0.756075] PM: Magic number: 10:478:7341708worker2 # [ 0.759933] RAS: Correctable Errors collector initialized.1709worker2 # [ 0.768949] clk: Disabling unused clocks1710worker2 # [ 0.769652] PM: genpd: Disabling unused power domains1711lb # [ 0.924812] Freeing initrd memory: 29092K1712lb # [ 0.927960] Freeing unused decrypted memory: 2028K1713lb # [ 0.930604] Freeing unused kernel image (initmem) memory: 3644K1714lb # [ 0.931716] Write protecting the kernel read-only data: 32768k1715lb # [ 0.933623] Freeing unused kernel image (text/rodata gap) memory: 1208K1716lb # [ 0.935170] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker1 # [ 0.925633] Freeing initrd memory: 29092K1718worker1 # [ 0.928740] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.931377] Freeing unused kernel image (initmem) memory: 3644K1720worker1 # [ 0.932483] Write protecting the kernel read-only data: 32768k1721worker1 # [ 0.934370] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker1 # [ 0.935941] Freeing unused kernel image (rodata/data gap) memory: 752K1723client # [ 0.941319] Freeing initrd memory: 29096K1724client # [ 0.944583] Freeing unused decrypted memory: 2028K1725client # [ 0.947332] Freeing unused kernel image (initmem) memory: 3644K1726client # [ 0.948459] Write protecting the kernel read-only data: 32768k1727client # [ 0.950421] Freeing unused kernel image (text/rodata gap) memory: 1208K1728client # [ 0.951996] Freeing unused kernel image (rodata/data gap) memory: 752K1729lb # [ 0.986454] x86/mm: Checked W+X mappings: passed, no W+X pages found.1730lb # [ 0.987552] Run /init as init process1731lb # [ 0.997928] systemd[1]: Inserted module 'autofs4'1732worker1 # [ 0.986879] x86/mm: Checked W+X mappings: passed, no W+X pages found.1733worker1 # [ 0.987916] Run /init as init process1734lb # [ 1.014127] fuse: init (API version 7.45)1735worker1 # [ 0.998045] systemd[1]: Inserted module 'autofs4'1736lb # [ 1.020020] ACPI: \_SB_.GSIG: Enabled at IRQ 221737lb # [ 1.022494] ACPI: \_SB_.GSIH: Enabled at IRQ 231738lb # [ 1.025220] ACPI: \_SB_.GSIE: Enabled at IRQ 201739client # [ 1.003060] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740lb # [ 1.027529] ACPI: \_SB_.GSIF: Enabled at IRQ 211741client # [ 1.004135] Run /init as init process1742worker1 # [ 1.014266] fuse: init (API version 7.45)1743client # [ 1.015034] systemd[1]: Inserted module 'autofs4'1744worker1 # [ 1.020064] ACPI: \_SB_.GSIG: Enabled at IRQ 221745worker1 # [ 1.022502] ACPI: \_SB_.GSIH: Enabled at IRQ 231746worker1 # [ 1.025343] ACPI: \_SB_.GSIE: Enabled at IRQ 201747worker1 # [ 1.027473] ACPI: \_SB_.GSIF: Enabled at IRQ 211748worker2 # [ 0.991919] Freeing initrd memory: 29092K1749worker2 # [ 0.995437] Freeing unused decrypted memory: 2028K1750worker2 # [ 0.998414] Freeing unused kernel image (initmem) memory: 3644K1751client # [ 1.031400] fuse: init (API version 7.45)1752lb # [ 1.055614] systemd[1]: Successfully made /usr/ read-only.1753worker2 # [ 0.999528] Write protecting the kernel read-only data: 32768k1754worker2 # [ 1.001511] Freeing unused kernel image (text/rodata gap) memory: 1208K1755worker2 # [ 1.003155] Freeing unused kernel image (rodata/data gap) memory: 752K1756client # [ 1.037175] ACPI: \_SB_.GSIG: Enabled at IRQ 221757client # [ 1.039847] ACPI: \_SB_.GSIH: Enabled at IRQ 231758client # [ 1.043208] ACPI: \_SB_.GSIE: Enabled at IRQ 201759client # [ 1.045728] ACPI: \_SB_.GSIF: Enabled at IRQ 211760worker1 # [ 1.055755] systemd[1]: Successfully made /usr/ read-only.1761client # [ 1.073165] systemd[1]: Successfully made /usr/ read-only.1762worker2 # [ 1.055071] x86/mm: Checked W+X mappings: passed, no W+X pages found.1763worker2 # [ 1.056180] Run /init as init process1764worker2 # [ 1.067931] systemd[1]: Inserted module 'autofs4'1765worker2 # [ 1.084950] fuse: init (API version 7.45)1766worker2 # [ 1.090898] ACPI: \_SB_.GSIG: Enabled at IRQ 221767worker2 # [ 1.093380] ACPI: \_SB_.GSIH: Enabled at IRQ 231768worker2 # [ 1.096427] ACPI: \_SB_.GSIE: Enabled at IRQ 201769worker2 # [ 1.099066] ACPI: \_SB_.GSIF: Enabled at IRQ 211770worker2 # [ 1.127072] systemd[1]: Successfully made /usr/ read-only.1771lb # [ 1.391245] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772lb # [ 1.395988] systemd[1]: Detected virtualization kvm.1773lb # [ 1.396810] systemd[1]: Detected architecture x86-64.1774lb # [ 1.397653] systemd[1]: Running in initrd.1775lb # [ 1.398872] systemd[1]: Initializing machine ID from random generator.1776lb # [ 1.400102] systemd[1]: Hostname set to <lb>.1777worker1 # [ 1.391398] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker1 # [ 1.397139] systemd[1]: Detected virtualization kvm.1779worker1 # [ 1.399417] systemd[1]: Detected architecture x86-64.1780worker1 # [ 1.401603] systemd[1]: Running in initrd.1781worker1 # [ 1.404205] systemd[1]: Initializing machine ID from random generator.1782worker1 # [ 1.407079] systemd[1]: Hostname set to <worker1>.1783client # [ 1.408949] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784client # [ 1.414358] systemd[1]: Detected virtualization kvm.1785client # [ 1.415308] systemd[1]: Detected architecture x86-64.1786client # [ 1.416313] systemd[1]: Running in initrd.1787client # [ 1.417557] systemd[1]: Initializing machine ID from random generator.1788client # [ 1.418956] systemd[1]: Hostname set to <client>.1789worker2 # [ 1.462720] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker2 # [ 1.467518] systemd[1]: Detected virtualization kvm.1791worker2 # [ 1.468506] systemd[1]: Detected architecture x86-64.1792worker2 # [ 1.469374] systemd[1]: Running in initrd.1793worker2 # [ 1.470533] systemd[1]: Initializing machine ID from random generator.1794worker2 # [ 1.471721] systemd[1]: Hostname set to <worker2>.1795lb # [ 1.591963] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796lb # [ 1.632235] systemd[1]: Queued start job for default target Initrd Default Target.1797lb # [ 1.636874] systemd[1]: Created slice Slice /system/modprobe.1798lb # [ 1.638052] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1799lb # [ 1.639412] systemd[1]: Expecting device /dev/disk/by-label/nixos...1800lb # [ 1.640505] systemd[1]: Reached target Path Units.1801lb # [ 1.641318] systemd[1]: Reached target Slice Units.1802lb # [ 1.642173] systemd[1]: Reached target Swaps.1803lb # [ 1.642937] systemd[1]: Reached target Timer Units.1804lb # [ 1.643905] systemd[1]: Listening on D-Bus System Message Bus Socket.1805lb # [ 1.645056] systemd[1]: Listening on Journal Socket (/dev/log).1806lb # [ 1.646172] systemd[1]: Listening on Journal Sockets.1807lb # [ 1.647173] systemd[1]: Listening on udev Control Socket.1808lb # [ 1.648140] systemd[1]: Listening on udev Kernel Socket.1809lb # [ 1.649039] systemd[1]: Reached target Socket Units.1810lb # [ 1.650854] systemd[1]: Starting Create List of Static Device Nodes...1811worker1 # [ 1.630447] systemd[1]: bpf-restrict-fs: LSM BPF program attached1812lb # [ 1.654511] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1813client # [ 1.632803] systemd[1]: bpf-restrict-fs: LSM BPF program attached1814lb # [ 1.665177] systemd[1]: Starting Load Kernel Module configfs...1815lb # [ 1.678068] systemd[1]: Starting Journal Service...1816lb # [ 1.689550] systemd[1]: Starting Load Kernel Modules...1817worker1 # [ 1.685826] systemd[1]: Queued start job for default target Initrd Default Target.1818lb # [ 1.696520] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1819client # [ 1.686526] systemd[1]: Queued start job for default target Initrd Default Target.1820worker1 # [ 1.690094] systemd[1]: Created slice Slice /system/modprobe.1821lb # [ 1.712270] netfs: FS-Cache loaded1822worker1 # [ 1.691278] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1823worker1 # [ 1.692734] systemd[1]: Expecting device /dev/disk/by-label/nixos...1824client # [ 1.690690] systemd[1]: Created slice Slice /system/modprobe.1825worker1 # [ 1.693895] systemd[1]: Reached target Path Units.1826worker1 # [ 1.694806] systemd[1]: Reached target Slice Units.1827client # [ 1.691870] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1828worker1 # [ 1.695708] systemd[1]: Reached target Swaps.1829lb # [ 1.717102] 9pnet: Installing 9P2000 support1830client # [ 1.693262] systemd[1]: Expecting device /dev/disk/by-label/nixos...1831worker1 # [ 1.696496] systemd[1]: Reached target Timer Units.1832client # [ 1.694333] systemd[1]: Reached target Path Units.1833worker1 # [ 1.697583] systemd[1]: Listening on D-Bus System Message Bus Socket.1834client # [ 1.695301] systemd[1]: Reached target Slice Units.1835worker1 # [ 1.698819] systemd[1]: Listening on Journal Socket (/dev/log).1836client # [ 1.696168] systemd[1]: Reached target Swaps.1837client # [ 1.696975] systemd[1]: Reached target Timer Units.1838worker1 # [ 1.699970] systemd[1]: Listening on Journal Sockets.1839worker1 # [ 1.700939] systemd[1]: Listening on udev Control Socket.1840client # [ 1.697921] systemd[1]: Listening on D-Bus System Message Bus Socket.1841worker1 # [ 1.701969] systemd[1]: Listening on udev Kernel Socket.1842client # [ 1.699192] systemd[1]: Listening on Journal Socket (/dev/log).1843worker1 # [ 1.702914] systemd[1]: Reached target Socket Units.1844client # [ 1.700424] systemd[1]: Listening on Journal Sockets.1845client # [ 1.701463] systemd[1]: Listening on udev Control Socket.1846worker1 # [ 1.704786] systemd[1]: Starting Create List of Static Device Nodes...1847client # [ 1.702473] systemd[1]: Listening on udev Kernel Socket.1848client # [ 1.703436] systemd[1]: Reached target Socket Units.1849lb # [ 1.728520] systemd[1]: Starting Coldplug All udev Devices...1850client # [ 1.705324] systemd[1]: Starting Create List of Static Device Nodes...1851worker1 # [ 1.708597] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1852client # [ 1.709114] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1853worker2 # [ 1.677042] systemd[1]: bpf-restrict-fs: LSM BPF program attached1854worker1 # [ 1.716606] systemd[1]: Starting Load Kernel Module configfs...1855client # [ 1.718360] systemd[1]: Starting Load Kernel Module configfs...1856lb # [ 1.743588] systemd[1]: Finished Create List of Static Device Nodes.1857lb # [ 1.750073] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1858worker1 # [ 1.732008] systemd[1]: Starting Journal Service...1859client # [ 1.731448] systemd[1]: Starting Journal Service...1860lb # [ 1.757857] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1861lb # [ 1.762902] systemd[1]: modprobe@configfs.service: Deactivated successfully.1862lb # [ 1.765876] systemd-journald[67]: Collecting audit messages is disabled.1863worker1 # [ 1.745405] systemd[1]: Starting Load Kernel Modules...1864client # [ 1.744826] systemd[1]: Starting Load Kernel Modules...1865lb # [ 1.769835] systemd[1]: Finished Load Kernel Module configfs.1866lb # [ 1.774697] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1867lb # [ 1.778409] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1868worker1 # [ 1.751618] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1869lb # [ 1.787508] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1870worker2 # [ 1.730854] systemd[1]: Queued start job for default target Initrd Default Target.1871worker1 # [ 1.769198] netfs: FS-Cache loaded1872client # [ 1.754351] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1873lb # [ 1.791566] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1874worker2 # [ 1.735327] systemd[1]: Created slice Slice /system/modprobe.1875client # [ 1.769755] netfs: FS-Cache loaded1876worker2 # [ 1.736496] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1877worker2 # [ 1.737946] systemd[1]: Expecting device /dev/disk/by-label/nixos...1878worker1 # [ 1.773968] 9pnet: Installing 9P2000 support1879worker2 # [ 1.738991] systemd[1]: Reached target Path Units.1880worker2 # [ 1.739801] systemd[1]: Reached target Slice Units.1881worker2 # [ 1.740637] systemd[1]: Reached target Swaps.1882worker2 # [ 1.741378] systemd[1]: Reached target Timer Units.1883worker2 # [ 1.742333] systemd[1]: Listening on D-Bus System Message Bus Socket.1884client # [ 1.775976] 9pnet: Installing 9P2000 support1885worker2 # [ 1.743496] systemd[1]: Listening on Journal Socket (/dev/log).1886worker2 # [ 1.744623] systemd[1]: Listening on Journal Sockets.1887worker2 # [ 1.745558] systemd[1]: Listening on udev Control Socket.1888worker2 # [ 1.746513] systemd[1]: Listening on udev Kernel Socket.1889worker2 # [ 1.747458] systemd[1]: Reached target Socket Units.1890worker1 # [ 1.783361] systemd[1]: Starting Coldplug All udev Devices...1891worker2 # [ 1.749346] systemd[1]: Starting Create List of Static Device Nodes...1892worker2 # [ 1.752951] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1893client # [ 1.792890] systemd[1]: Starting Coldplug All udev Devices...1894worker2 # [ 1.760481] systemd[1]: Starting Load Kernel Module configfs...1895worker1 # [ 1.799695] systemd[1]: Finished Create List of Static Device Nodes.1896lb # [ 1.821975] systemd[1]: Finished Load Kernel Modules.1897client # [ 1.801320] systemd-journald[67]: Collecting audit messages is disabled.1898worker1 # [ 1.805196] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1899lb # [ 1.827620] systemd[1]: Starting Apply Kernel Variables...1900worker2 # [ 1.775956] systemd[1]: Starting Journal Service...1901client # [ 1.808737] systemd[1]: Finished Create List of Static Device Nodes.1902worker1 # [ 1.812929] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1903lb # [ 1.838551] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1904worker1 # [ 1.817966] systemd[1]: modprobe@configfs.service: Deactivated successfully.1905client # [ 1.815945] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1906worker1 # [ 1.822820] systemd-journald[67]: Collecting audit messages is disabled.1907worker1 # [ 1.824298] systemd[1]: Finished Load Kernel Module configfs.1908worker2 # [ 1.790012] systemd[1]: Starting Load Kernel Modules...1909lb # [ 1.846823] systemd[1]: Starting Create Static Device Nodes in /dev...1910client # [ 1.823669] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1911client # [ 1.828709] systemd[1]: modprobe@configfs.service: Deactivated successfully.1912worker1 # [ 1.832013] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1913worker1 # [ 1.836555] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1914client # [ 1.836650] systemd[1]: Finished Load Kernel Module configfs.1915worker1 # [ 1.841737] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1916worker2 # [ 1.796979] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1917worker1 # [ 1.844668] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1918client # [ 1.841707] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1919worker2 # [ 1.813090] netfs: FS-Cache loaded1920client # [ 1.847371] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1921lb # [ 1.872601] systemd[1]: Finished Apply Kernel Variables.1922worker2 # [ 1.817987] 9pnet: Installing 9P2000 support1923client # [ 1.854502] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1924lb # [ 1.880656] systemd[1]: Started Journal Service.1925client # [ 1.857460] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1926lb # [ 1.745251] systemd-modules-load[68]: Inserted module 'dm_mod'1927worker2 # [ 1.827101] systemd[1]: Starting Coldplug All udev Devices...1928lb # [ 1.748413] systemd-modules-load[68]: Inserted module 'virtio_balloon'1929lb # [ 1.750572] systemd-modules-load[68]: Inserted module 'virtio_gpu'1930worker1 # [ 1.876110] systemd[1]: Finished Load Kernel Modules.1931worker2 # [ 1.843221] systemd[1]: Finished Create List of Static Device Nodes.1932worker2 # [ 1.847699] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1933worker1 # [ 1.883737] systemd[1]: Starting Apply Kernel Variables...1934lb # [ 1.770157] systemd[1]: Finished Create Static Device Nodes in /dev.1935lb # [ 1.771211] systemd[1]: Reached target Preparation for Local File Systems.1936lb # [ 1.772228] systemd[1]: Reached target Local File Systems.1937worker2 # [ 1.855292] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1938lb # [ 1.776066] systemd[1]: Starting Create System Files and Directories...1939worker2 # [ 1.857222] systemd-journald[67]: Collecting audit messages is disabled.1940client # [ 1.891829] systemd[1]: Finished Load Kernel Modules.1941worker1 # [ 1.896604] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1942worker2 # [ 1.862325] systemd[1]: modprobe@configfs.service: Deactivated successfully.1943lb # [ 1.784069] systemd[1]: Starting Rule-based Manager for Device Events and Files...1944client # [ 1.900469] systemd[1]: Starting Apply Kernel Variables...1945worker1 # [ 1.903743] systemd[1]: Starting Create Static Device Nodes in /dev...1946worker2 # [ 1.869257] systemd[1]: Finished Load Kernel Module configfs.1947worker2 # [ 1.873359] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1948worker2 # [ 1.875740] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1949client # [ 1.911400] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1950worker2 # [ 1.883045] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1951worker2 # [ 1.887098] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1952worker1 # [ 1.924799] systemd[1]: Finished Apply Kernel Variables.1953lb # [ 1.811776] systemd[1]: Finished Create System Files and Directories.1954client # [ 1.925480] systemd[1]: Starting Create Static Device Nodes in /dev...1955client # [ 1.797223] systemd-modules-load[68]: Inserted module 'dm_mod'1956client # [ 1.799205] systemd-modules-load[68]: Inserted module 'virtio_balloon'1957worker1 # [ 1.937459] systemd[1]: Started Journal Service.1958worker1 # [ 1.806274] systemd-modules-load[68]: Inserted module 'dm_mod'1959client # [ 1.803334] systemd-modules-load[68]: Inserted module 'virtio_gpu'1960worker1 # [ 1.808389] systemd-modules-load[68]: Inserted module 'virtio_balloon'1961client # [ 1.940530] systemd[1]: Started Journal Service.1962worker1 # [ 1.809981] systemd-modules-load[68]: Inserted module 'virtio_gpu'1963worker1 # [ 1.813116] systemd[1]: Finished Create Static Device Nodes in /dev.1964worker1 # [ 1.814262] systemd[1]: Reached target Preparation for Local File Systems.1965worker1 # [ 1.816307] systemd[1]: Reached target Local File Systems.1966worker2 # [ 1.914443] systemd[1]: Finished Load Kernel Modules.1967client # [ 1.812571] systemd[1]: Finished Apply Kernel Variables.1968lb # [ 1.836297] systemd-udevd[83]: Using default interface naming scheme 'v261'.1969worker1 # [ 1.821069] systemd[1]: Starting Create System Files and Directories...1970worker2 # [ 1.922081] systemd[1]: Starting Apply Kernel Variables...1971worker1 # [ 1.824332] systemd[1]: Starting Rule-based Manager for Device Events and Files...1972client # [ 1.833145] systemd[1]: Finished Create Static Device Nodes in /dev.1973client # [ 1.834211] systemd[1]: Reached target Preparation for Local File Systems.1974client # [ 1.835176] systemd[1]: Reached target Local File Systems.1975worker2 # [ 1.939994] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1976client # [ 1.838075] systemd[1]: Starting Create System Files and Directories...1977lb # [ 1.863072] systemd[1]: Started Rule-based Manager for Device Events and Files.1978client # [ 1.842796] systemd[1]: Starting Rule-based Manager for Device Events and Files...1979worker2 # [ 1.948092] systemd[1]: Starting Create Static Device Nodes in /dev...1980worker1 # [ 1.857943] systemd[1]: Finished Create System Files and Directories.1981worker2 # [ 1.960986] systemd[1]: Finished Apply Kernel Variables.1982client # [ 1.867448] systemd[1]: Finished Create System Files and Directories.1983worker2 # [ 1.978164] systemd[1]: Started Journal Service.1984lb # [ 1.899257] systemd[1]: Finished Coldplug All udev Devices.1985worker1 # [ 1.881345] systemd-udevd[78]: Using default interface naming scheme 'v261'.1986worker2 # [ 1.846351] systemd-modules-load[68]: Inserted module 'dm_mod'1987lb # [ 1.901896] systemd[1]: Reached target System Initialization.1988worker2 # [ 1.849312] systemd-modules-load[68]: Inserted module 'virtio_balloon'1989lb # [ 1.902738] systemd[1]: Reached target Basic System.1990worker2 # [ 1.851304] systemd-modules-load[68]: Inserted module 'virtio_gpu'1991worker2 # [ 1.857178] systemd[1]: Finished Create Static Device Nodes in /dev.1992client # [ 1.893340] systemd-udevd[83]: Using default interface naming scheme 'v261'.1993worker2 # [ 1.863865] systemd[1]: Reached target Preparation for Local File Systems.1994worker2 # [ 1.865056] systemd[1]: Reached target Local File Systems.1995worker2 # [ 1.869081] systemd[1]: Starting Create System Files and Directories...1996worker1 # [ 1.908365] systemd[1]: Started Rule-based Manager for Device Events and Files.1997worker2 # [ 1.876073] systemd[1]: Starting Rule-based Manager for Device Events and Files...1998client # [ 1.918155] systemd[1]: Started Rule-based Manager for Device Events and Files.1999worker2 # [ 1.902138] systemd[1]: Finished Create System Files and Directories.2000worker1 # [ 1.952746] systemd[1]: Finished Coldplug All udev Devices.
loading 2,023 hidden lines…
4024lb # [ 17.643158] envoy[921]: [2026-09-20 12:42:57.833][921][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4025lb # [ 17.645683] envoy[921]: [2026-09-20 12:42:57.833][921][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4026lb # [ 17.649313] envoy[921]: [2026-09-20 12:42:57.833][921][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4027lb # [ 17.651688] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4028lb # [ 17.655198] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4029lb # [ 17.660604] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4030lb # [ 17.664474] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4031lb # [ 17.668143] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4032lb # [ 17.670467] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4033lb # [ 17.673486] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4034lb # [ 17.675213] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4035lb # [ 17.676982] envoy[921]: [2026-09-20 12:42:57.836][921][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4036lb # [ 17.684155] envoy[921]: [2026-09-20 12:42:57.839][921][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4037lb # [ 17.686457] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4038lb # [ 17.693700] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4039lb # [ 17.696175] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4040lb # [ 17.700569] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4041lb # [ 17.702550] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4042lb # [ 17.704950] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4043lb # [ 17.710028] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4044lb # [ 17.712039] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4045lb # [ 17.713803] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4046lb # [ 17.715609] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4047lb # [ 17.717538] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4048lb # [ 17.720618] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4049lb # [ 17.723514] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4050lb # [ 17.725558] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4051lb # [ 17.731627] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4052lb # [ 17.733754] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4053lb # [ 17.738235] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4054lb # [ 17.740663] envoy[921]: [2026-09-20 12:42:57.842][921][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4055lb # [ 17.743763] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4056lb # [ 17.750936] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4057lb # [ 17.753750] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4058lb # [ 17.755924] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4059lb # [ 17.761324] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4060lb # [ 17.763548] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4061lb # [ 17.766271] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4062lb # [ 17.768589] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4063lb # [ 17.770671] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4064lb # [ 17.773126] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4065lb # [ 17.776478] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4066lb # [ 17.783638] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84067lb # [ 17.786223] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4068lb # [ 17.791312] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4069lb # [ 17.820186] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4070lb # [ 17.822836] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4071lb # [ 17.825364] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4072lb # [ 17.837210] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24073lb # [ 17.838994] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4074lb # [ 17.840738] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4075lb # [ 17.842826] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4076lb # [ 17.845225] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4077lb # [ 17.847537] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14078lb # [ 17.849672] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4079lb # [ 17.851388] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4080lb # [ 17.853394] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4081lb # [ 17.855906] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4082lb # [ 17.859061] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4083lb # [ 17.863560] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24084lb # [ 17.865810] envoy[921]: [2026-09-20 12:42:57.847][921][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4085lb # [ 17.868087] envoy[921]: [2026-09-20 12:42:57.848][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4086lb # [ 17.870103] envoy[921]: [2026-09-20 12:42:57.848][921][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4087lb # [ 17.871898] envoy[921]: [2026-09-20 12:42:57.848][921][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4088lb # [ 17.874189] envoy[921]: [2026-09-20 12:42:57.861][921][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4089lb # [ 17.876979] envoy[921]: [2026-09-20 12:42:57.861][921][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4090lb # [ 17.881542] envoy[921]: [2026-09-20 12:42:57.861][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4091lb # [ 17.884467] envoy[921]: [2026-09-20 12:42:57.861][921][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4092lb # [ 17.890450] envoy[921]: [2026-09-20 12:42:57.861][921][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4093lb # [ 17.892546] envoy[921]: [2026-09-20 12:42:57.861][921][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4094lb # [ 17.894817] envoy[921]: [2026-09-20 12:42:57.863][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4095lb # [ 17.896491] envoy[921]: [2026-09-20 12:42:58.024][921][info][main] [source/server/server.cc:531] HTTP header map info:4096lb # [ 17.898097] envoy[921]: [2026-09-20 12:42:58.077][921][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4097lb # [ 17.912410] envoy[921]: [2026-09-20 12:42:58.077][921][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4098lb # [ 17.914079] envoy[921]: [2026-09-20 12:42:58.077][921][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4099lb # [ 17.922675] envoy[921]: [2026-09-20 12:42:58.077][921][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4100lb # [ 17.965291] envoy[921]: [2026-09-20 12:42:58.266][921][info][main] [source/server/server.cc:968] runtime: {}4101lb # [ 17.974367] envoy[921]: [2026-09-20 12:42:58.275][921][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4102lb # [ 17.976743] envoy[921]: [2026-09-20 12:42:58.278][921][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014103lb # [ 17.980061] envoy[921]: [2026-09-20 12:42:58.281][921][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014104lb # [ 17.983461] envoy[921]: [2026-09-20 12:42:58.284][921][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4105lb # [ 17.985594] envoy[921]: [2026-09-20 12:42:58.286][921][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4106lb # [ 17.987559] envoy[921]: [2026-09-20 12:42:58.286][921][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4107worker1 # [ 18.018628] systemd[1]: Starting Nix worker-protocol over gRPC...4108lb # [ 18.036777] envoy[921]: [2026-09-20 12:42:58.338][921][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4109worker2 # [ 17.985689] systemd[1]: Reached target Network is Online.4110worker2 # [ 17.991733] systemd[1]: Starting scheduler-token.service...4111lb # [ 18.059300] envoy[921]: [2026-09-20 12:42:58.360][921][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4112lb # [ 18.061824] envoy[921]: [2026-09-20 12:42:58.363][921][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4113lb # [ 18.064727] envoy[921]: [2026-09-20 12:42:58.363][921][info][main] [source/server/server.cc:1077] starting main dispatch loop4114worker2 # [ 18.094421] systemd[1]: scheduler-token.service: Deactivated successfully.4115worker2 # [ 18.096306] systemd[1]: Finished scheduler-token.service.4116worker2 # [ 18.097823] systemd[1]: scheduler-token.service: Consumed 31ms CPU time over 104ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4117worker2 # [ 18.104077] systemd[1]: Starting Nix worker-protocol over gRPC...4118worker1 # [ 18.199551] nix-grpc-daemon[786]: ts=2026-09-20T12:42:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4119worker1 # [ 18.208718] nix-grpc-daemon[786]: ts=2026-09-20T12:42:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4120worker1 # [ 18.215777] nix-grpc-daemon[786]: ts=2026-09-20T12:42:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14121worker1 # [ 18.257407] nix-grpc-daemon[786]: ts=2026-09-20T12:42:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4122worker1 # [ 18.264379] systemd[1]: Started Nix worker-protocol over gRPC.4123worker2 # [ 18.269469] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4124worker2 # [ 18.279416] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4125worker1 # [ 18.315742] nix-grpc-daemon[786]: ts=2026-09-20T12:42:59Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4126lb # [ 18.334632] envoy[921]: [2026-09-20 12:42:58.635][921][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4127lb # [ 18.339200] envoy[921]: [2026-09-20 12:42:58.639][921][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4128worker2 # [ 18.288354] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14129lb # [ 18.343708] envoy[921]: [2026-09-20 12:42:58.640][921][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4130lb # [ 18.348204] envoy[921]: [2026-09-20 12:42:58.640][921][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4131worker2 # [ 18.325538] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4132worker2 # [ 18.330640] systemd[1]: Started Nix worker-protocol over gRPC.4133worker2 # [ 18.379658] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=scheduler_connected addr=lb:500514134worker2 # [ 18.389288] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker1 # [ 18.516720] nix-grpc-daemon[786]: ts=2026-09-20T12:42:59Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4136worker2 # [ 18.591321] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 18.593462] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker2 # [ 18.794328] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 18.796061] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker1 # [ 18.917490] nix-grpc-daemon[786]: ts=2026-09-20T12:43:00Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4141worker2 # [ 18.997187] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=scheduler_connected addr=lb:500514142worker2 # [ 18.999450] nix-grpc-daemon[854]: ts=2026-09-20T12:42:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker2 # [ 19.200688] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_connected addr=lb:500514144worker2 # [ 19.203216] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 19.405458] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 19.407315] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 19.606637] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 19.608980] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker1 # [ 19.721932] nix-grpc-daemon[786]: ts=2026-09-20T12:43:00Z level=info event=scheduler_connected addr=lb:500514150worker1 # [ 19.726936] nix-grpc-daemon[786]: ts=2026-09-20T12:43:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker2 # [ 19.810366] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_connected addr=lb:500514152worker2 # [ 19.812710] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker1 # [ 19.927849] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514154worker1 # [ 19.929886] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker2 # [ 20.013606] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 20.015989] nix-grpc-daemon[854]: ts=2026-09-20T12:43:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker1 # [ 20.130615] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514158worker1 # [ 20.132790] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker2 # [ 20.217254] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 20.219589] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 20.333615] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514162worker1 # [ 20.337656] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker2 # [ 20.420713] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514164worker2 # [ 20.422687] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker1 # [ 20.538333] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514166worker1 # [ 20.540392] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker2 # [ 20.623575] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 20.625683] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 20.741496] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514170worker1 # [ 20.744559] nix-grpc-daemon[786]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker2 # [ 20.826609] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514172worker2 # [ 20.828731] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 20.944604] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 20.946702] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 21.029954] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 21.032105] nix-grpc-daemon[854]: ts=2026-09-20T12:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 21.147326] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 21.149623] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 21.232836] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 21.234658] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 21.350165] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 21.352244] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 21.435969] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 21.437772] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 21.552908] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 21.555485] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker2 # [ 21.638668] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514188worker2 # [ 21.640690] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker1 # [ 21.757367] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 21.759325] nix-grpc-daemon[786]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 21.841717] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514192worker2 # [ 21.843955] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker1 # [ 21.960162] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 21.962688] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker2 # [ 22.044909] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_connected addr=lb:500514196worker2 # [ 22.047200] nix-grpc-daemon[854]: ts=2026-09-20T12:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker1 # [ 22.163695] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514198worker1 # [ 22.166138] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker2 # [ 22.248177] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514200worker2 # [ 22.250332] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker1 # [ 22.366683] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514202worker1 # [ 22.368803] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker2 # [ 22.452048] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514204worker2 # [ 22.453685] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 22.569611] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514206worker1 # [ 22.571752] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker2 # [ 22.653565] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 22.657628] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker1 # [ 22.774070] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514210worker1 # [ 22.777302] nix-grpc-daemon[786]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker2 # [ 22.858297] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 22.860238] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker1 # [ 22.974407] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514214worker1 # [ 22.976290] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker2 # [ 23.061133] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_connected addr=lb:500514216worker2 # [ 23.062924] nix-grpc-daemon[854]: ts=2026-09-20T12:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker1 # [ 23.177036] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 23.179609] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker2 # [ 23.263772] nix-grpc-daemon[854]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514220worker2 # [ 23.266198] nix-grpc-daemon[854]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker1 # [ 23.380846] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 23.385768] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker2 # [ 23.468485] nix-grpc-daemon[854]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514224worker2 # [ 23.471664] nix-grpc-daemon[854]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker1 # [ 23.585946] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514226worker1 # [ 23.588217] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker2 # [ 23.668865] nix-grpc-daemon[854]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514228worker2 # [ 23.670581] nix-grpc-daemon[854]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker1 # [ 23.789212] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514230worker1 # [ 23.790962] nix-grpc-daemon[786]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker2 # [ 23.871936] nix-grpc-daemon[854]: ts=2026-09-20T12:43:04Z level=info event=scheduler_connected addr=lb:500514232worker2 # [ 23.873884] nix-grpc-daemon[854]: ts=2026-09-20T12:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 23.991597] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514234worker1 # [ 23.993344] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker2 # [ 24.074913] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514236worker2 # [ 24.077415] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker1 # [ 24.193970] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514238worker1 # [ 24.195751] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker2 # [ 24.278515] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514240worker2 # [ 24.280713] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 24.396781] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514242worker1 # [ 24.400755] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker2 # [ 24.482219] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514244worker2 # [ 24.484371] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker1 # [ 24.601216] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514246worker1 # [ 24.602971] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker2 # [ 24.685612] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514248worker2 # [ 24.687786] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker1 # [ 24.804372] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514250worker1 # [ 24.806690] nix-grpc-daemon[786]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker2 # [ 24.888639] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_connected addr=lb:500514252worker2 # [ 24.890436] nix-grpc-daemon[854]: ts=2026-09-20T12:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 25.007752] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514254worker1 # [ 25.011729] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker2 # [ 25.091360] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514256worker2 # [ 25.093155] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker1 # [ 25.212386] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514258worker1 # [ 25.214705] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker2 # [ 25.293868] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514260worker2 # [ 25.295570] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker1 # [ 25.415594] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514262worker1 # [ 25.417860] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker2 # [ 25.497135] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514264worker2 # [ 25.500293] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker1 # [ 25.618749] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514266worker1 # [ 25.620712] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker2 # [ 25.701045] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514268worker2 # [ 25.702765] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker1 # [ 25.822250] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514270worker1 # [ 25.824235] nix-grpc-daemon[786]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker2 # [ 25.903627] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_connected addr=lb:500514272worker2 # [ 25.905701] nix-grpc-daemon[854]: ts=2026-09-20T12:43:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker1 # [ 26.024762] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514274worker1 # [ 26.026980] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker2 # [ 26.106904] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514276worker2 # [ 26.110883] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker1 # [ 26.228255] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514278worker1 # [ 26.231898] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker2 # [ 26.311870] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 26.313975] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 26.432520] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514282worker1 # [ 26.435181] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker2 # [ 26.515405] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514284worker2 # [ 26.517513] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker1 # [ 26.636626] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514286worker1 # [ 26.638849] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker2 # [ 26.718954] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 26.723832] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 26.840220] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514290worker1 # [ 26.842513] nix-grpc-daemon[786]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker2 # [ 26.924915] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_connected addr=lb:500514292worker2 # [ 26.929275] nix-grpc-daemon[854]: ts=2026-09-20T12:43:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker1 # [ 27.043355] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514294worker1 # [ 27.045765] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker2 # [ 27.129919] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514296worker2 # [ 27.132475] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker1 # [ 27.247258] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514298worker1 # [ 27.251483] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker2 # [ 27.333230] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514300worker2 # [ 27.335295] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker1 # [ 27.452425] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514302worker1 # [ 27.456498] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker2 # [ 27.536913] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514304worker2 # [ 27.538858] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker1 # [ 27.657686] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 27.661506] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 27.739855] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 27.741844] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 27.864184] nix-grpc-daemon[786]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 27.867339] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 27.942940] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 27.945037] nix-grpc-daemon[854]: ts=2026-09-20T12:43:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker1 # [ 28.065141] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514314worker1 # [ 28.067475] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4315worker2 # [ 28.145961] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514316worker2 # [ 28.148165] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker1 # [ 28.267931] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514318worker1 # [ 28.269656] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker2 # [ 28.349044] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514320worker2 # [ 28.351405] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker1 # [ 28.470867] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514322worker1 # [ 28.475292] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker2 # [ 28.553239] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514324worker2 # [ 28.555283] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker1 # [ 28.676197] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514326worker1 # [ 28.678249] nix-grpc-daemon[786]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker2 # [ 28.756569] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514328worker2 # [ 28.758921] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker1 # [ 28.880631] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514330worker1 # [ 28.883839] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker2 # [ 28.960074] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_connected addr=lb:500514332worker2 # [ 28.962465] nix-grpc-daemon[854]: ts=2026-09-20T12:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker1 # [ 29.081848] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514334worker1 # [ 29.085875] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker2 # [ 29.163880] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514336worker2 # [ 29.167837] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker1 # [ 29.286855] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514338worker1 # [ 29.291155] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4339worker2 # [ 29.368553] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514340worker2 # [ 29.370973] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker1 # [ 29.491750] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514342worker1 # [ 29.493753] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker2 # [ 29.572674] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514344worker2 # [ 29.574758] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4345worker1 # [ 29.694462] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514346worker1 # [ 29.696874] nix-grpc-daemon[786]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4347worker2 # [ 29.775564] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514348worker2 # [ 29.777510] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349worker1 # [ 29.898437] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514350worker1 # [ 29.900454] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4351worker2 # [ 29.978624] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_connected addr=lb:500514352worker2 # [ 29.981145] nix-grpc-daemon[854]: ts=2026-09-20T12:43:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4353lb # [ 30.043765] rustfs-bucket-start[1060]: mb s3://farm4354lb # [ 30.049257] systemd[1]: Finished rustfs-bucket.service.4355lb # [ 30.056711] systemd[1]: Starting niks3 server...4356worker1 # [ 30.101452] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514357worker1 # [ 30.103949] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4358worker2 # [ 30.182100] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514359worker2 # [ 30.184333] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4360lb # [ 30.282560] postgres[1074]: [1074] ERROR: relation "goose_db_version" does not exist at character 364361lb # [ 30.283989] postgres[1074]: [1074] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4362worker1 # [ 30.304609] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514363lb # [ 30.325328] niks3-server[1069]: 2026/09/20 12:43:10 OK 20241026095416_initial_model.sql (22.26ms)4364worker1 # [ 30.306967] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4365lb # [ 30.329982] niks3-server[1069]: 2026/09/20 12:43:10 OK 20251210153512_drop_unused_gin_index.sql (2.86ms)4366lb # [ 30.335459] niks3-server[1069]: 2026/09/20 12:43:10 OK 20251218171726_add_pins.sql (5.41ms)4367lb # [ 30.341384] niks3-server[1069]: 2026/09/20 12:43:10 OK 20260628120000_add_object_size_and_stats.sql (6.01ms)4368lb # [ 30.345969] niks3-server[1069]: 2026/09/20 12:43:10 OK 20260905000000_add_claims.sql (3.08ms)4369lb # [ 30.350335] niks3-server[1069]: 2026/09/20 12:43:10 OK 20260920000000_drop_claims.sql (4.46ms)4370lb # [ 30.352306] niks3-server[1069]: 2026/09/20 12:43:10 goose: successfully migrated database to version: 202609200000004371lb # [ 30.358436] niks3-server[1069]: 2026/09/20 12:43:10 OK 1_commit_pending_closure.sql (6.42ms)4372lb # [ 30.361632] niks3-server[1069]: 2026/09/20 12:43:10 OK 2_object_stats_trigger.sql (1.79ms)4373lb # [ 30.362955] niks3-server[1069]: 2026/09/20 12:43:10 goose: up to current file version: 24374lb # [ 30.369201] niks3-server[1069]: 2026/09/20 12:43:10 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4375worker2 # [ 30.386421] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514376worker2 # [ 30.387975] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4377lb # [ 30.455653] niks3-server[1069]: 2026/09/20 12:43:10 INFO Read proxy enabled — serving cache objects from S34378lb # [ 30.457628] niks3-server[1069]: 2026/09/20 12:43:10 INFO Using socket-activated listener address=0.0.0.0:57514379lb # [ 30.464422] systemd[1]: Started niks3 server.4380lb # [ 30.465279] niks3-server[1069]: 2026/09/20 12:43:10 INFO systemd watchdog enabled interval=15s4381lb # [ 30.466588] systemd[1]: Reached target Multi-User System.4382lb # [ 30.469215] systemd[1]: Startup finished in 854ms (kernel) + 3.923s (initrd) + 25.690s (userspace) = 30.469s.4383lb # [ 30.471339] niks3-server[1069]: 2026/09/20 12:43:10 INFO Starting HTTP server address=0.0.0.0:57514384lb # [ 30.479310] niks3-server[1069]: 2026/09/20 12:43:10 INFO lead: acquired remote=192.168.1.4:547764385worker2 # [ 30.432080] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_take_over4386worker1 # [ 30.507692] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514387worker1 # [ 30.510034] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4388worker2 # [ 30.590187] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514389worker2 # [ 30.594153] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4390worker1 # [ 30.710910] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514391worker1 # [ 30.713398] nix-grpc-daemon[786]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4392worker2 # [ 30.795448] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514393worker2 # [ 30.799344] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4394worker1 # [ 30.914714] nix-grpc-daemon[786]: ts=2026-09-20T12:43:12Z level=info event=scheduler_connected addr=lb:500514395worker1 # [ 30.917082] nix-grpc-daemon[786]: ts=2026-09-20T12:43:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4396lb: (finished: waiting for unit niks3.service, in 19.42 seconds)4397lb: waiting for TCP port 5751 on localhost4398worker2 # [ 31.000536] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_connected addr=lb:500514399worker2 # [ 31.004437] nix-grpc-daemon[854]: ts=2026-09-20T12:43:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4400lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4401lb: (finished: waiting for TCP port 5751 on localhost, in 0.06 seconds)4402lb: waiting for unit envoy.service4403lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4404lb: waiting for TCP port 50051 on localhost4405worker1 # [ 31.117642] nix-grpc-daemon[786]: ts=2026-09-20T12:43:12Z level=info event=scheduler_connected addr=lb:500514406worker1 # [ 31.119851] nix-grpc-daemon[786]: ts=2026-09-20T12:43:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4407lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4408lb # [ 31.166194] envoy[921]: conn peer=[::1]:39094 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34409lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4410worker2 # [ 31.205361] nix-grpc-daemon[854]: ts=2026-09-20T12:43:12Z level=info event=scheduler_connected addr=lb:500514411worker2 # [ 31.208298] nix-grpc-daemon[854]: ts=2026-09-20T12:43:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4412worker2 # [ 31.228556] systemd[1]: Starting scheduler-token.service...4413worker1 # [ 31.320592] nix-grpc-daemon[786]: ts=2026-09-20T12:43:12Z level=info event=scheduler_connected addr=lb:500514414worker1 # [ 31.322912] nix-grpc-daemon[786]: ts=2026-09-20T12:43:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4415worker2 # [ 31.311475] systemd[1]: scheduler-token.service: Deactivated successfully.4416worker2 # [ 31.313942] systemd[1]: Finished scheduler-token.service.4417worker2 # [ 31.316466] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 85ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4418lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4419lb: must succeed: curl -sf localhost:9901/clusters4420lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4421lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4422subtest: one node schedules and both builders hold a WorkerSession on it4423lb: must succeed: curl -sf localhost:9901/clusters4424worker2 # [ 31.408889] nix-grpc-daemon[854]: ts=2026-09-20T12:43:12Z level=info event=scheduler_connected addr=lb:500514425worker2 # [ 31.434792] nix-grpc-daemon[854]: ts=2026-09-20T12:43:12Z level=info event=oidc_jwks provider=mock keys=14426worker2 # [ 31.437935] nix-grpc-daemon[854]: ts=2026-09-20T12:43:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:573964427worker2 # [ 31.441086] nix-grpc-daemon[854]: ts=2026-09-20T12:43:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04428lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4429worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4430worker1 # [ 31.523611] nix-grpc-daemon[786]: ts=2026-09-20T12:43:12Z level=info event=scheduler_connected addr=lb:500514431worker2 # [ 31.490989] nix-grpc-daemon[854]: ts=2026-09-20T12:43:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:573964432worker2 # [ 31.493635] nix-grpc-daemon[854]: ts=2026-09-20T12:43:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04433worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4434lb: must succeed: curl -sf localhost:9901/clusters4435lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4436lb: must succeed: curl -sf localhost:9901/clusters4437lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4438worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4439worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)4440(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.26 seconds)4441subtest: balancer auth: client cert, bearer token, nothing, unknown CN4442worker1 # [ 31.993100] nix-grpc-daemon[786]: ts=2026-09-20T12:43:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585684443worker1 # [ 32.002325] systemd[1]: Started Nix Daemon.4444worker1 # [ 32.220239] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4445worker1 # [ 32.300924] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)4446worker1 # [ 32.340124] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 2/5)4447worker1 # [ 32.357882] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 3/5)4448worker1 # [ 32.394589] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 489 ms (attempt 4/5)4449worker1 # [ 32.889613] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4450worker1 # [ 32.922454] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:585684451worker1 # [ 32.937169] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=path_info_miss path=4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test error="error: path '/nix/store/4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test' is required, but there is no substituter that can build it"4452worker1 # [ 32.942607] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:585684453worker1 # [ 32.947781] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=path_info_miss path=4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test error="error: path '/nix/store/4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test' is required, but there is no substituter that can build it"4454client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4455client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds)4456client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4457client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4458worker1 # [ 33.343116] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=info event=oidc_jwks provider=mock keys=14459worker1 # [ 33.345208] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:585684460worker1 # [ 33.355384] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:585684461worker1 # [ 33.361848] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=path_info_miss path=4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test error="error: path '/nix/store/4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test' is required, but there is no substituter that can build it"4462worker1 # [ 33.367449] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:585684463worker1 # [ 33.372911] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=debug event=path_info_miss path=4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test error="error: path '/nix/store/4mswv1w7crl5njwzp9d41jjig5l256xq-nixos-system-client-test' is required, but there is no substituter that can build it"4464client: must succeed: rm -r /var/lib/nix-grpc-store4465client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4466worker1 # [ 33.638629] nix-grpc-daemon[786]: ts=2026-09-20T12:43:14Z level=info event=denied method=QueryMissing cn=- role=none4467worker2 # [ 33.849057] nix-grpc-daemon[854]: ts=2026-09-20T12:43:14Z level=info event=denied method=QueryMissing cn=stranger role=none4468(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.21 seconds)4469subtest: DAG build is scheduled across workers and lands in the cache4470client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24471worker2 # [ 34.118534] nix-grpc-daemon[854]: ts=2026-09-20T12:43:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334804472worker2 # [ 34.129487] systemd[1]: Started Nix Daemon.4473worker2 # [ 34.351359] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4474client # don't know how to build these paths:4475client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4476worker1 # [ 34.414740] nix-grpc-daemon[786]: ts=2026-09-20T12:43:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585684477worker2 # [ 34.407982] nix-grpc-daemon[854]: ts=2026-09-20T12:43:15Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04478worker2 # [ 34.410215] nix-grpc-daemon[854]: ts=2026-09-20T12:43:15Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14479worker2 # [ 34.465786] nix-grpc-daemon[854]: ts=2026-09-20T12:43:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804480worker2 # [ 34.509959] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)4481worker2 # [ 34.584301] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 2/5)4482worker2 # [ 34.684791] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 3/5)4483worker2 # [ 34.704031] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 4/5)4484worker2 # [ 34.716431] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4485worker2 # [ 34.740688] nix-grpc-daemon[854]: ts=2026-09-20T12:43:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804486client # copying 1 paths to 'lb:50051'...4487worker2 # [ 34.753108] nix-grpc-daemon[854]: ts=2026-09-20T12:43:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334804488client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4489worker2 # [ 34.775161] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4490worker2 # [ 34.795570] nix-grpc-daemon[854]: ts=2026-09-20T12:43:15Z level=info event=niks3_push_started pid=9354491worker2 # [ 35.140151] nix-grpc-daemon[935]: time=2026-09-20T12:43:16.068Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4492worker2 # [ 35.371908] nix-daemon[924]: accepted connection from pid 948, user nix-grpc-daemon (trusted)4493worker2 # [ 35.392183] nix-daemon[924]: reaped child process 951, status = succeeded4494worker2 # [ 35.589629] nix-daemon[924]: accepted connection from pid 953, user nix-grpc-daemon (trusted)4495worker2 # [ 35.608516] nix-daemon[924]: reaped child process 956, status = succeeded4496lb # [ 35.675348] niks3-server[1069]: 2026/09/20 12:43:15 INFO Received uploads request method=POST path=/api/pending_closures4497worker2 # [ 35.641992] nix-grpc-daemon[935]: time=2026-09-20T12:43:16.575Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4498worker2 # [ 35.645239] nix-grpc-daemon[935]: time=2026-09-20T12:43:16.578Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4499lb # [ 35.730985] niks3-server[1069]: 2026/09/20 12:43:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4500lb # [ 35.751728] niks3-server[1069]: 2026/09/20 12:43:16 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4501lb # [ 35.759102] niks3-server[1069]: 2026/09/20 12:43:16 INFO Signed narinfos id=1 count=14502worker2 # [ 35.708527] nix-grpc-daemon[935]: time=2026-09-20T12:43:16.641Z level=INFO msg="Uploading 1 narinfos"4503lb # [ 35.776192] niks3-server[1069]: 2026/09/20 12:43:16 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4504lb # [ 35.779114] niks3-server[1069]: 2026/09/20 12:43:16 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4505lb # [ 35.792128] niks3-server[1069]: 2026/09/20 12:43:16 INFO Completed upload id=14506worker2 # [ 35.739298] nix-grpc-daemon[935]: time=2026-09-20T12:43:16.672Z level=INFO msg="Upload complete. (573ms)"4507worker2 # [ 35.742789] nix-grpc-daemon[854]: ts=2026-09-20T12:43:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_in=5044508lb # [ 35.800074] niks3-server[1069]: 2026/09/20 12:43:16 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4509worker2 # [ 35.747939] nix-grpc-daemon[854]: ts=2026-09-20T12:43:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804510worker2 # [ 35.757622] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4511worker2 # [ 35.761231] nix-daemon[924]: reaped child process 933, status = succeeded4512client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4513worker2 # [ 35.769046] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4514client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4515client # farm-a> LOG-farm-a-t14516worker2 # [ 37.009815] nix-daemon[924]: reaped child process 960, status = succeeded4517worker2 # [ 37.171210] nix-daemon[924]: accepted connection from pid 976, user nix-grpc-daemon (trusted)4518worker2 # [ 37.186522] nix-daemon[924]: reaped child process 979, status = succeeded4519lb # [ 37.253061] niks3-server[1069]: 2026/09/20 12:43:17 INFO Received uploads request method=POST path=/api/pending_closures4520worker2 # [ 37.208910] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.141Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4521worker2 # [ 37.210696] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.143Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4522lb # [ 37.294860] niks3-server[1069]: 2026/09/20 12:43:17 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4523lb # [ 37.304784] niks3-server[1069]: 2026/09/20 12:43:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4524lb # [ 37.309165] niks3-server[1069]: 2026/09/20 12:43:17 INFO Signed narinfos id=2 count=14525lb # [ 37.311210] niks3-server[1069]: 2026/09/20 12:43:17 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4526worker2 # [ 37.258616] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.191Z level=INFO msg="Uploading 1 narinfos"4527lb # [ 37.319474] niks3-server[1069]: 2026/09/20 12:43:17 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4528lb # [ 37.327190] niks3-server[1069]: 2026/09/20 12:43:17 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4529lb # [ 37.333122] niks3-server[1069]: 2026/09/20 12:43:17 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4530lb # [ 37.337930] niks3-server[1069]: 2026/09/20 12:43:17 INFO Completed upload id=24531worker2 # [ 37.285213] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.218Z level=INFO msg="Upload complete. (301ms)"4532worker2 # [ 37.293906] nix-grpc-daemon[854]: ts=2026-09-20T12:43:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14533worker2 # [ 37.310333] nix-daemon[924]: reaped child process 958, status = succeeded4534worker2 # [ 37.313556] nix-grpc-daemon[854]: ts=2026-09-20T12:43:18Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04535worker2 # [ 37.316533] nix-grpc-daemon[854]: ts=2026-09-20T12:43:18Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14536worker2 # [ 37.358561] nix-grpc-daemon[854]: ts=2026-09-20T12:43:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804537worker2 # [ 37.367599] nix-grpc-daemon[854]: ts=2026-09-20T12:43:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804538worker2 # [ 37.376758] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.309Z level=INFO msg="All 1 paths already cached"4539client # copying 1 paths to 'lb:50051'...4540client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4541worker2 # [ 37.382829] nix-grpc-daemon[854]: ts=2026-09-20T12:43:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334804542worker2 # [ 37.389126] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4543worker2 # [ 37.593065] nix-daemon[924]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4544worker2 # [ 37.614633] nix-daemon[924]: reaped child process 988, status = succeeded4545worker2 # [ 37.775467] nix-daemon[924]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4546worker2 # [ 37.791153] nix-daemon[924]: reaped child process 993, status = succeeded4547lb # [ 37.857339] niks3-server[1069]: 2026/09/20 12:43:18 INFO Received uploads request method=POST path=/api/pending_closures4548worker2 # [ 37.825697] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.758Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4549worker2 # [ 37.827289] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.758Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4550lb # [ 37.901556] niks3-server[1069]: 2026/09/20 12:43:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4551worker2 # [ 37.855178] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.787Z level=INFO msg="Uploading 1 narinfos"4552lb # [ 37.908150] niks3-server[1069]: 2026/09/20 12:43:18 INFO Signed narinfos id=3 count=14553lb # [ 37.913683] niks3-server[1069]: 2026/09/20 12:43:18 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4554lb # [ 37.929323] niks3-server[1069]: 2026/09/20 12:43:18 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4555lb # [ 37.936446] niks3-server[1069]: 2026/09/20 12:43:18 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4556lb # [ 37.946380] niks3-server[1069]: 2026/09/20 12:43:18 INFO Completed upload id=34557worker2 # [ 37.893660] nix-grpc-daemon[935]: time=2026-09-20T12:43:18.826Z level=INFO msg="Upload complete. (491ms)"4558worker2 # [ 37.899254] nix-grpc-daemon[854]: ts=2026-09-20T12:43:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_in=6244559lb # [ 37.958083] niks3-server[1069]: 2026/09/20 12:43:18 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4560worker2 # [ 37.905643] nix-grpc-daemon[854]: ts=2026-09-20T12:43:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804561worker2 # [ 37.917587] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4562worker2 # [ 37.935144] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4563client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4564worker2 # [ 37.938256] nix-daemon[924]: reaped child process 983, status = succeeded4565client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4566lb # [ 38.045197] envoy[921]: conn peer=[2001:db8:1::1]:34312 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=364 ms=10124567lb # [ 38.047102] envoy[921]: conn peer=[2001:db8:1::1]:48706 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=362 ms=554568lb # [ 38.050259] envoy[921]: conn peer=[2001:db8:1::1]:48716 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114569lb # [ 38.054076] envoy[921]: conn peer=[2001:db8:1::1]:48718 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=182 ms=184570client # farm-b> LOG-farm-b-t14571worker2 # [ 38.759930] nix-daemon[924]: reaped child process 997, status = succeeded4572worker2 # [ 38.919958] nix-daemon[924]: accepted connection from pid 1012, user nix-grpc-daemon (trusted)4573worker2 # [ 38.935382] nix-daemon[924]: reaped child process 1015, status = succeeded4574lb # [ 38.999239] niks3-server[1069]: 2026/09/20 12:43:19 INFO Received uploads request method=POST path=/api/pending_closures4575worker2 # [ 38.954413] nix-grpc-daemon[935]: time=2026-09-20T12:43:19.887Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4576worker2 # [ 38.956566] nix-grpc-daemon[935]: time=2026-09-20T12:43:19.889Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4577lb # [ 39.026376] niks3-server[1069]: 2026/09/20 12:43:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4578lb # [ 39.029268] niks3-server[1069]: 2026/09/20 12:43:19 INFO Signed narinfos id=4 count=14579worker2 # [ 38.976063] nix-grpc-daemon[935]: time=2026-09-20T12:43:19.908Z level=INFO msg="Uploading 1 narinfos"4580lb # [ 39.031526] niks3-server[1069]: 2026/09/20 12:43:19 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4581lb # [ 39.034594] niks3-server[1069]: 2026/09/20 12:43:19 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4582lb # [ 39.040064] niks3-server[1069]: 2026/09/20 12:43:19 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4583lb # [ 39.044330] niks3-server[1069]: 2026/09/20 12:43:19 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4584lb # [ 39.047813] niks3-server[1069]: 2026/09/20 12:43:19 INFO Completed upload id=44585worker2 # [ 38.996060] nix-grpc-daemon[935]: time=2026-09-20T12:43:19.929Z level=INFO msg="Upload complete. (249ms)"4586lb # [ 39.050163] niks3-server[1069]: 2026/09/20 12:43:19 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4587worker2 # [ 38.998863] nix-grpc-daemon[854]: ts=2026-09-20T12:43:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14588worker2 # [ 39.007126] nix-grpc-daemon[854]: ts=2026-09-20T12:43:19Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04589worker2 # [ 39.009308] nix-grpc-daemon[854]: ts=2026-09-20T12:43:19Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14590worker2 # [ 39.014184] nix-daemon[924]: reaped child process 995, status = succeeded4591worker2 # [ 39.053187] nix-grpc-daemon[854]: ts=2026-09-20T12:43:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804592worker2 # [ 39.066982] nix-grpc-daemon[854]: ts=2026-09-20T12:43:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804593worker2 # [ 39.081120] nix-grpc-daemon[935]: time=2026-09-20T12:43:20.014Z level=INFO msg="All 2 paths already cached"4594client # copying 1 paths to 'lb:50051'...4595worker2 # [ 39.090482] nix-grpc-daemon[854]: ts=2026-09-20T12:43:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334804596client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4597worker2 # [ 39.112258] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4598worker2 # [ 39.274240] nix-daemon[924]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4599worker2 # [ 39.291915] nix-daemon[924]: reaped child process 1024, status = succeeded4600worker2 # [ 39.469112] nix-daemon[924]: accepted connection from pid 1026, user nix-grpc-daemon (trusted)4601worker2 # [ 39.484514] nix-daemon[924]: reaped child process 1029, status = succeeded4602lb # [ 39.549211] niks3-server[1069]: 2026/09/20 12:43:19 INFO Received uploads request method=POST path=/api/pending_closures4603worker2 # [ 39.501650] nix-grpc-daemon[935]: time=2026-09-20T12:43:20.434Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4604worker2 # [ 39.503560] nix-grpc-daemon[935]: time=2026-09-20T12:43:20.436Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4605lb # [ 39.573161] niks3-server[1069]: 2026/09/20 12:43:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4606lb # [ 39.576714] niks3-server[1069]: 2026/09/20 12:43:19 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4607lb # [ 39.579831] niks3-server[1069]: 2026/09/20 12:43:19 INFO Signed narinfos id=5 count=14608worker2 # [ 39.528410] nix-grpc-daemon[935]: time=2026-09-20T12:43:20.461Z level=INFO msg="Uploading 1 narinfos"4609lb # [ 39.584393] niks3-server[1069]: 2026/09/20 12:43:19 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4610lb # [ 39.592325] niks3-server[1069]: 2026/09/20 12:43:19 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4611lb # [ 39.595925] niks3-server[1069]: 2026/09/20 12:43:19 INFO Completed upload id=54612worker2 # [ 39.542936] nix-grpc-daemon[935]: time=2026-09-20T12:43:20.476Z level=INFO msg="Upload complete. (420ms)"4613worker2 # [ 39.545247] nix-grpc-daemon[854]: ts=2026-09-20T12:43:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_in=7524614lb # [ 39.599674] niks3-server[1069]: 2026/09/20 12:43:19 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4615worker2 # [ 39.549531] nix-grpc-daemon[854]: ts=2026-09-20T12:43:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804616worker2 # [ 39.557700] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4617worker2 # [ 39.561317] nix-daemon[924]: reaped child process 1019, status = succeeded4618client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4619worker2 # [ 39.567990] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4620client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4621client # farm-top> LOG-farm-top-t14622worker2 # [ 40.400960] nix-daemon[924]: reaped child process 1033, status = succeeded4623worker2 # [ 40.543107] nix-daemon[924]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)4624worker2 # [ 40.557730] nix-daemon[924]: reaped child process 1052, status = succeeded4625lb # [ 40.624759] niks3-server[1069]: 2026/09/20 12:43:20 INFO Received uploads request method=POST path=/api/pending_closures4626worker2 # [ 40.584281] nix-grpc-daemon[935]: time=2026-09-20T12:43:21.517Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4627worker2 # [ 40.588032] nix-grpc-daemon[935]: time=2026-09-20T12:43:21.518Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4628lb # [ 40.656703] niks3-server[1069]: 2026/09/20 12:43:20 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4629lb # [ 40.666517] niks3-server[1069]: 2026/09/20 12:43:20 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4630lb # [ 40.670402] niks3-server[1069]: 2026/09/20 12:43:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4631lb # [ 40.674245] niks3-server[1069]: 2026/09/20 12:43:20 INFO Signed narinfos id=6 count=14632worker2 # [ 40.621479] nix-grpc-daemon[935]: time=2026-09-20T12:43:21.554Z level=INFO msg="Uploading 1 narinfos"4633lb # [ 40.680339] niks3-server[1069]: 2026/09/20 12:43:20 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4634lb # [ 40.684162] niks3-server[1069]: 2026/09/20 12:43:20 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4635lb # [ 40.692349] niks3-server[1069]: 2026/09/20 12:43:20 INFO Completed upload id=64636worker2 # [ 40.639320] nix-grpc-daemon[935]: time=2026-09-20T12:43:21.572Z level=INFO msg="Upload complete. (262ms)"4637lb # [ 40.694649] niks3-server[1069]: 2026/09/20 12:43:20 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4638worker2 # [ 40.643826] nix-grpc-daemon[854]: ts=2026-09-20T12:43:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14639worker2 # [ 40.656346] nix-daemon[924]: reaped child process 1031, status = succeeded4640client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.85 seconds)4641client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4642client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4643client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4644client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4645client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4646client # copying 3 paths...4647client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4648client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4649client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4650client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4651(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.34 seconds)4652subtest: repeat is answered Cached by the scheduler without building4653worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4654worker1 # finding garbage collector roots...4655worker1 # deleting specified paths...4656worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4657worker1 # deleting unused links...4658worker1 # note: hard linking is currently saving 0.0 KiB4659worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4660worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4661worker2 # finding garbage collector roots...4662worker2 # deleting specified paths...4663worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4664worker2 # deleting unused links...4665worker2 # note: hard linking is currently saving 0.0 KiB4666worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4667client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24668worker1 # [ 41.950865] nix-grpc-daemon[786]: ts=2026-09-20T12:43:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585684669client # don't know how to build these paths:4670client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4671worker2 # [ 41.927838] nix-grpc-daemon[854]: ts=2026-09-20T12:43:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804672worker2 # [ 41.952433] nix-grpc-daemon[935]: time=2026-09-20T12:43:22.885Z level=INFO msg="All 2 paths already cached"4673worker2 # [ 41.966089] nix-grpc-daemon[854]: ts=2026-09-20T12:43:22Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14674client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.28 seconds)4675client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4676client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4677lb: must succeed: curl -sf localhost:9901/clusters4678lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4679lb: must succeed: curl -sf localhost:9901/clusters4680lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4681worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4682worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4683(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.18 seconds)4684subtest: inputs already in the cache are not sent to the scheduler4685lb: must succeed: curl -sf localhost:9901/clusters4686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4687lb: must succeed: curl -sf localhost:9901/clusters4688lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4689worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4690worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4691client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24692worker1 # [ 42.800759] nix-grpc-daemon[786]: ts=2026-09-20T12:43:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585684693client # don't know how to build these paths:4694client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4695worker2 # [ 42.775888] nix-grpc-daemon[854]: ts=2026-09-20T12:43:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804696worker2 # [ 42.784982] nix-grpc-daemon[935]: time=2026-09-20T12:43:23.717Z level=INFO msg="All 2 paths already cached"4697worker2 # [ 42.798956] nix-grpc-daemon[854]: ts=2026-09-20T12:43:23Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04698worker2 # [ 42.805150] nix-grpc-daemon[854]: ts=2026-09-20T12:43:23Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14699worker2 # [ 42.812304] nix-grpc-daemon[854]: ts=2026-09-20T12:43:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804700worker2 # [ 42.826347] nix-grpc-daemon[854]: ts=2026-09-20T12:43:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804701worker2 # [ 42.840027] nix-grpc-daemon[935]: time=2026-09-20T12:43:23.773Z level=INFO msg="All 2 paths already cached"4702client # copying 1 paths to 'lb:50051'...4703client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4704worker2 # [ 42.845870] nix-grpc-daemon[854]: ts=2026-09-20T12:43:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334804705worker2 # [ 42.852572] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4706worker2 # [ 43.022380] nix-daemon[924]: accepted connection from pid 1081, user nix-grpc-daemon (trusted)4707worker2 # [ 43.043938] nix-daemon[924]: reaped child process 1084, status = succeeded4708worker2 # [ 43.229354] nix-daemon[924]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)4709worker2 # [ 43.255833] nix-daemon[924]: reaped child process 1089, status = succeeded4710lb # [ 43.316831] niks3-server[1069]: 2026/09/20 12:43:23 INFO Received uploads request method=POST path=/api/pending_closures4711worker2 # [ 43.272599] nix-grpc-daemon[935]: time=2026-09-20T12:43:24.205Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4712worker2 # [ 43.274247] nix-grpc-daemon[935]: time=2026-09-20T12:43:24.205Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4713lb # [ 43.342576] niks3-server[1069]: 2026/09/20 12:43:23 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4714lb # [ 43.349336] niks3-server[1069]: 2026/09/20 12:43:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4715lb # [ 43.352321] niks3-server[1069]: 2026/09/20 12:43:23 INFO Signed narinfos id=7 count=14716worker2 # [ 43.299837] nix-grpc-daemon[935]: time=2026-09-20T12:43:24.232Z level=INFO msg="Uploading 1 narinfos"4717lb # [ 43.357819] niks3-server[1069]: 2026/09/20 12:43:23 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4718lb # [ 43.368284] niks3-server[1069]: 2026/09/20 12:43:23 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4719lb # [ 43.414722] niks3-server[1069]: 2026/09/20 12:43:23 INFO Completed upload id=74720worker2 # [ 43.361994] nix-grpc-daemon[935]: time=2026-09-20T12:43:24.294Z level=INFO msg="Upload complete. (502ms)"4721lb # [ 43.419051] niks3-server[1069]: 2026/09/20 12:43:23 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4722worker2 # [ 43.368726] nix-grpc-daemon[854]: ts=2026-09-20T12:43:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_in=7524723worker2 # [ 43.376155] nix-grpc-daemon[854]: ts=2026-09-20T12:43:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804724worker2 # [ 43.386486] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4725worker2 # [ 43.399981] nix-daemon[924]: reaped child process 1079, status = succeeded4726client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4727client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4728worker2 # [ 43.412530] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4729client # farm-top> LOG-farm-top-t1b4730worker2 # [ 44.433938] nix-daemon[924]: reaped child process 1094, status = succeeded4731worker2 # [ 44.601182] nix-daemon[924]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4732worker2 # [ 44.626402] nix-daemon[924]: reaped child process 1112, status = succeeded4733lb # [ 44.694832] niks3-server[1069]: 2026/09/20 12:43:24 INFO Received uploads request method=POST path=/api/pending_closures4734worker2 # [ 44.656987] nix-grpc-daemon[935]: time=2026-09-20T12:43:25.589Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4735worker2 # [ 44.666618] nix-grpc-daemon[935]: time=2026-09-20T12:43:25.592Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4736lb # [ 44.746330] niks3-server[1069]: 2026/09/20 12:43:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4737lb # [ 44.749487] niks3-server[1069]: 2026/09/20 12:43:25 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4738lb # [ 44.754975] niks3-server[1069]: 2026/09/20 12:43:25 INFO Signed narinfos id=8 count=14739worker2 # [ 44.702214] nix-grpc-daemon[935]: time=2026-09-20T12:43:25.634Z level=INFO msg="Uploading 1 narinfos"4740lb # [ 44.758777] niks3-server[1069]: 2026/09/20 12:43:25 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4741lb # [ 44.768334] niks3-server[1069]: 2026/09/20 12:43:25 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4742lb # [ 44.770933] niks3-server[1069]: 2026/09/20 12:43:25 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4743lb # [ 44.816393] niks3-server[1069]: 2026/09/20 12:43:25 INFO Completed upload id=84744worker2 # [ 44.763601] nix-grpc-daemon[935]: time=2026-09-20T12:43:25.696Z level=INFO msg="Upload complete. (337ms)"4745worker2 # [ 44.768266] nix-grpc-daemon[854]: ts=2026-09-20T12:43:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14746lb # [ 44.823721] niks3-server[1069]: 2026/09/20 12:43:25 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4747worker2 # [ 44.776603] nix-daemon[924]: reaped child process 1092, status = succeeded4748client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.28 seconds)4749client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4750client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4751lb: must succeed: curl -sf localhost:9901/clusters4752lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4753lb: must succeed: curl -sf localhost:9901/clusters4754lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4755worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4756worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4757(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.78 seconds)4758subtest: two clients wanting the same drv build it once4759client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4760client # Running as unit: dedup1.service; invocation ID: 068c3baceaad420391150fb764522fe84761client # [ 45.248677] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4762client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.05 seconds)4763client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4764worker1 # [ 45.478745] nix-grpc-daemon[786]: ts=2026-09-20T12:43:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585684765client # [ 45.480343] nix[887]: don't know how to build these paths:4766client # [ 45.481427] nix[887]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4767worker2 # [ 45.456905] nix-grpc-daemon[854]: ts=2026-09-20T12:43:26Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04768worker2 # [ 45.459798] nix-grpc-daemon[854]: ts=2026-09-20T12:43:26Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14769worker1 # [ 45.497640] nix-grpc-daemon[786]: ts=2026-09-20T12:43:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585684770worker1 # [ 45.511976] nix-grpc-daemon[786]: ts=2026-09-20T12:43:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585684771client # [ 45.516209] nix[887]: copying 1 paths to 'lb:50051'...4772client # [ 45.517133] nix[887]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4773worker1 # [ 45.521760] nix-grpc-daemon[786]: ts=2026-09-20T12:43:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585684774worker1 # [ 45.533767] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4775worker1 # [ 45.555410] nix-grpc-daemon[786]: ts=2026-09-20T12:43:26Z level=info event=niks3_push_started pid=8704776worker1 # [ 45.868716] nix-grpc-daemon[870]: time=2026-09-20T12:43:27.001Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4777worker1 # [ 46.078399] nix-daemon[844]: accepted connection from pid 883, user nix-grpc-daemon (trusted)4778worker1 # [ 46.100177] nix-daemon[844]: reaped child process 886, status = succeeded4779worker1 # [ 46.253057] nix-daemon[844]: accepted connection from pid 888, user nix-grpc-daemon (trusted)4780worker1 # [ 46.271127] nix-daemon[844]: reaped child process 891, status = succeeded4781lb # [ 46.301587] niks3-server[1069]: 2026/09/20 12:43:26 INFO Received uploads request method=POST path=/api/pending_closures4782worker1 # [ 46.290343] nix-grpc-daemon[870]: time=2026-09-20T12:43:27.427Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4783worker1 # [ 46.293393] nix-grpc-daemon[870]: time=2026-09-20T12:43:27.430Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4784lb # [ 46.337886] niks3-server[1069]: 2026/09/20 12:43:26 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4785client # [ 46.317920] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4786lb # [ 46.344311] niks3-server[1069]: 2026/09/20 12:43:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4787client # Running as unit: dedup2.service; invocation ID: 0ec61c1f8c544f59a0e981945790a72f4788lb # [ 46.348395] niks3-server[1069]: 2026/09/20 12:43:26 INFO Signed narinfos id=9 count=14789worker1 # [ 46.332494] nix-grpc-daemon[870]: time=2026-09-20T12:43:27.469Z level=INFO msg="Uploading 1 narinfos"4790client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.07 seconds)4791lb # [ 46.355729] niks3-server[1069]: 2026/09/20 12:43:26 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4792lb # [ 46.366128] niks3-server[1069]: 2026/09/20 12:43:26 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4793lb # [ 46.372687] niks3-server[1069]: 2026/09/20 12:43:26 INFO Completed upload id=94794worker1 # [ 46.357092] nix-grpc-daemon[870]: time=2026-09-20T12:43:27.493Z level=INFO msg="Upload complete. (462ms)"4795lb # [ 46.375038] niks3-server[1069]: 2026/09/20 12:43:26 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4796worker1 # [ 46.360090] nix-grpc-daemon[786]: ts=2026-09-20T12:43:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=0 paths=1 nar_bytes_in=4324797worker1 # [ 46.363459] nix-grpc-daemon[786]: ts=2026-09-20T12:43:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585684798worker1 # [ 46.373649] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4799worker1 # [ 46.384654] nix-daemon[844]: reaped child process 868, status = succeeded4800client # [ 46.384231] nix[887]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4801worker1 # [ 46.390064] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4802worker2 # [ 46.555586] nix-grpc-daemon[854]: ts=2026-09-20T12:43:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334804803client # [ 46.590747] nix[903]: don't know how to build these paths:4804client # [ 46.593309] nix[903]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4805worker2 # [ 46.570170] nix-grpc-daemon[854]: ts=2026-09-20T12:43:27Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04806worker1 # [ 46.612637] nix-grpc-daemon[786]: ts=2026-09-20T12:43:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585684807client # [ 46.612146] nix[903]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4808lb # [ 48.045679] envoy[921]: conn peer=[2001:db8:1::1]:48732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9472 tx=3820 ms=66254809lb # [ 48.047639] envoy[921]: conn peer=[2001:db8:1::1]:48736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=674810lb # [ 48.049478] envoy[921]: conn peer=[2001:db8:1::1]:60512 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4152 tx=1712 ms=20214811client: must succeed: sleep 24812client: (finished: must succeed: sleep 2, in 2.04 seconds)4813client: waiting for success: ! systemctl is-active dedup1 dedup24814worker1 # [ 51.014311] nix-daemon[844]: reaped child process 899, status = succeeded4815worker1 # [ 51.181046] nix-daemon[844]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4816worker1 # [ 51.195112] nix-daemon[844]: reaped child process 931, status = succeeded4817lb # [ 51.225693] niks3-server[1069]: 2026/09/20 12:43:31 INFO Received uploads request method=POST path=/api/pending_closures4818worker1 # [ 51.223430] nix-grpc-daemon[870]: time=2026-09-20T12:43:32.360Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4819worker1 # [ 51.225287] nix-grpc-daemon[870]: time=2026-09-20T12:43:32.362Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4820lb # [ 51.280521] niks3-server[1069]: 2026/09/20 12:43:31 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4821lb # [ 51.290158] niks3-server[1069]: 2026/09/20 12:43:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4822lb # [ 51.295219] niks3-server[1069]: 2026/09/20 12:43:31 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4823worker1 # [ 51.285711] nix-grpc-daemon[870]: time=2026-09-20T12:43:32.421Z level=INFO msg="Uploading 1 narinfos"4824lb # [ 51.303817] niks3-server[1069]: 2026/09/20 12:43:31 INFO Signed narinfos id=10 count=14825lb # [ 51.316268] niks3-server[1069]: 2026/09/20 12:43:31 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4826lb # [ 51.326133] niks3-server[1069]: 2026/09/20 12:43:31 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4827lb # [ 51.329794] niks3-server[1069]: 2026/09/20 12:43:31 INFO Completed upload id=104828worker1 # [ 51.311957] nix-grpc-daemon[870]: time=2026-09-20T12:43:32.448Z level=INFO msg="Upload complete. (304ms)"4829lb # [ 51.332382] niks3-server[1069]: 2026/09/20 12:43:31 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4830worker1 # [ 51.319575] nix-grpc-daemon[786]: ts=2026-09-20T12:43:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14831worker1 # [ 51.324550] nix-grpc-daemon[786]: ts=2026-09-20T12:43:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14832worker1 # [ 51.335893] nix-daemon[844]: reaped child process 897, status = succeeded4833client # [ 51.351440] systemd[1]: dedup2.service: Deactivated successfully.4834client # [ 51.353951] systemd[1]: dedup2.service: Consumed 152ms CPU time over 5.032s wall clock time, 31.1M memory peak, 7.4K incoming IP traffic, 6.5K outgoing IP traffic.4835client # [ 51.390120] systemd[1]: dedup1.service: Deactivated successfully.4836client # [ 51.392470] systemd[1]: dedup1.service: Consumed 143ms CPU time over 6.148s wall clock time, 39.3M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.4837client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4838client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4839client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4840worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4841worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4842worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4843worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4844worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4845worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4846worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4847worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4848(finished: subtest: two clients wanting the same drv build it once, in 6.99 seconds)4849subtest: build hook: nix-daemon with builders = grpc://lb4850client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4851client # [ 52.362709] systemd[1]: Started Nix Daemon.4852client # [ 52.514515] nix-daemon[945]: accepted connection from pid 942, user root (trusted)4853worker2 # [ 52.667981] nix-grpc-daemon[854]: ts=2026-09-20T12:43:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334804854worker2 # [ 52.674517] nix-grpc-daemon[854]: ts=2026-09-20T12:43:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804855worker2 # [ 52.688278] nix-grpc-daemon[854]: ts=2026-09-20T12:43:33Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04856worker2 # [ 52.691415] nix-grpc-daemon[854]: ts=2026-09-20T12:43:33Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14857worker2 # [ 52.733551] nix-grpc-daemon[854]: ts=2026-09-20T12:43:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804858worker2 # [ 52.743262] nix-grpc-daemon[854]: ts=2026-09-20T12:43:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804859worker2 # [ 52.752739] nix-grpc-daemon[854]: ts=2026-09-20T12:43:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334804860worker2 # [ 52.762070] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4861worker2 # [ 52.955661] nix-daemon[924]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)4862worker2 # [ 52.977786] nix-daemon[924]: reaped child process 1155, status = succeeded4863worker2 # [ 53.132341] nix-daemon[924]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4864worker2 # [ 53.151677] nix-daemon[924]: reaped child process 1160, status = succeeded4865lb # [ 53.215596] niks3-server[1069]: 2026/09/20 12:43:33 INFO Received uploads request method=POST path=/api/pending_closures4866worker2 # [ 53.167760] nix-grpc-daemon[935]: time=2026-09-20T12:43:34.100Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4867worker2 # [ 53.169986] nix-grpc-daemon[935]: time=2026-09-20T12:43:34.102Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4868lb # [ 53.235632] niks3-server[1069]: 2026/09/20 12:43:33 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4869lb # [ 53.240061] niks3-server[1069]: 2026/09/20 12:43:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4870lb # [ 53.242915] niks3-server[1069]: 2026/09/20 12:43:33 INFO Signed narinfos id=11 count=14871worker2 # [ 53.190283] nix-grpc-daemon[935]: time=2026-09-20T12:43:34.123Z level=INFO msg="Uploading 1 narinfos"4872lb # [ 53.246134] niks3-server[1069]: 2026/09/20 12:43:33 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4873lb # [ 53.251737] niks3-server[1069]: 2026/09/20 12:43:33 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4874lb # [ 53.254917] niks3-server[1069]: 2026/09/20 12:43:33 INFO Completed upload id=114875worker2 # [ 53.201960] nix-grpc-daemon[935]: time=2026-09-20T12:43:34.135Z level=INFO msg="Upload complete. (430ms)"4876lb # [ 53.258045] niks3-server[1069]: 2026/09/20 12:43:33 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4877worker2 # [ 53.205108] nix-grpc-daemon[854]: ts=2026-09-20T12:43:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_in=5124878worker2 # [ 53.208104] nix-grpc-daemon[854]: ts=2026-09-20T12:43:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804879worker2 # [ 53.217087] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4880worker2 # [ 53.224250] nix-daemon[924]: reaped child process 1150, status = succeeded4881worker2 # [ 53.229482] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4882worker2 # [ 54.248667] nix-daemon[924]: reaped child process 1164, status = succeeded4883worker2 # [ 54.383728] nix-daemon[924]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)4884worker2 # [ 54.400466] nix-daemon[924]: reaped child process 1182, status = succeeded4885lb # [ 54.464233] niks3-server[1069]: 2026/09/20 12:43:34 INFO Received uploads request method=POST path=/api/pending_closures4886worker2 # [ 54.416890] nix-grpc-daemon[935]: time=2026-09-20T12:43:35.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4887worker2 # [ 54.418645] nix-grpc-daemon[935]: time=2026-09-20T12:43:35.351Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4888lb # [ 54.491411] niks3-server[1069]: 2026/09/20 12:43:34 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4889lb # [ 54.498693] niks3-server[1069]: 2026/09/20 12:43:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4890lb # [ 54.501451] niks3-server[1069]: 2026/09/20 12:43:34 INFO Signed narinfos id=12 count=14891worker2 # [ 54.451882] nix-grpc-daemon[935]: time=2026-09-20T12:43:35.384Z level=INFO msg="Uploading 1 narinfos"4892lb # [ 54.506291] niks3-server[1069]: 2026/09/20 12:43:34 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4893lb # [ 54.509406] niks3-server[1069]: 2026/09/20 12:43:34 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4894lb # [ 54.514953] niks3-server[1069]: 2026/09/20 12:43:34 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4895lb # [ 54.519416] niks3-server[1069]: 2026/09/20 12:43:34 INFO Completed upload id=124896worker2 # [ 54.467821] nix-grpc-daemon[935]: time=2026-09-20T12:43:35.400Z level=INFO msg="Upload complete. (229ms)"4897lb # [ 54.522125] niks3-server[1069]: 2026/09/20 12:43:34 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4898worker2 # [ 54.470732] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14899worker1 # [ 54.512196] nix-grpc-daemon[786]: ts=2026-09-20T12:43:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:585684900worker2 # [ 54.479661] nix-daemon[924]: reaped child process 1162, status = succeeded4901worker1 # [ 54.521242] nix-grpc-daemon[786]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4902worker2 # [ 54.510987] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334804903worker2 # [ 54.514141] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_out=1284904worker1 # [ 54.771223] nix-grpc-daemon[786]: ts=2026-09-20T12:43:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:585684905worker2 # [ 54.739269] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804906worker2 # [ 54.745492] nix-grpc-daemon[935]: time=2026-09-20T12:43:35.678Z level=INFO msg="All 1 paths already cached"4907worker2 # [ 54.755863] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04908worker2 # [ 54.758934] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14909worker2 # [ 54.761360] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804910worker2 # [ 54.768577] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804911worker2 # [ 54.776090] nix-grpc-daemon[935]: time=2026-09-20T12:43:35.709Z level=INFO msg="All 1 paths already cached"4912worker2 # [ 54.780715] nix-grpc-daemon[854]: ts=2026-09-20T12:43:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334804913worker2 # [ 54.785082] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4914worker2 # [ 54.932819] nix-daemon[924]: accepted connection from pid 1188, user nix-grpc-daemon (trusted)4915worker2 # [ 54.948375] nix-daemon[924]: reaped child process 1191, status = succeeded4916worker2 # [ 55.102555] nix-daemon[924]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)4917worker2 # [ 55.120475] nix-daemon[924]: reaped child process 1196, status = succeeded4918lb # [ 55.183148] niks3-server[1069]: 2026/09/20 12:43:35 INFO Received uploads request method=POST path=/api/pending_closures4919worker2 # [ 55.144421] nix-grpc-daemon[935]: time=2026-09-20T12:43:36.077Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4920worker2 # [ 55.145967] nix-grpc-daemon[935]: time=2026-09-20T12:43:36.077Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4921lb # [ 55.213170] niks3-server[1069]: 2026/09/20 12:43:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4922lb # [ 55.216795] niks3-server[1069]: 2026/09/20 12:43:35 INFO Signed narinfos id=13 count=14923worker2 # [ 55.163708] nix-grpc-daemon[935]: time=2026-09-20T12:43:36.096Z level=INFO msg="Uploading 1 narinfos"4924lb # [ 55.223357] niks3-server[1069]: 2026/09/20 12:43:35 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4925lb # [ 55.684945] niks3-server[1069]: 2026/09/20 12:43:35 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4926lb # [ 55.713799] niks3-server[1069]: 2026/09/20 12:43:36 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4927lb # [ 55.741202] niks3-server[1069]: 2026/09/20 12:43:36 INFO Completed upload id=134928worker2 # [ 55.688891] nix-grpc-daemon[935]: time=2026-09-20T12:43:36.621Z level=INFO msg="Upload complete. (895ms)"4929worker2 # [ 55.692513] nix-grpc-daemon[854]: ts=2026-09-20T12:43:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_in=6244930worker2 # [ 55.697103] nix-grpc-daemon[854]: ts=2026-09-20T12:43:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804931lb # [ 55.755340] niks3-server[1069]: 2026/09/20 12:43:36 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4932worker2 # [ 55.706331] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4933worker2 # [ 55.709269] nix-daemon[924]: reaped child process 1186, status = succeeded4934worker2 # [ 55.715340] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4935worker2 # [ 56.934766] nix-daemon[924]: reaped child process 1200, status = succeeded4936worker2 # [ 57.073254] nix-daemon[924]: accepted connection from pid 1215, user nix-grpc-daemon (trusted)4937worker2 # [ 57.089985] nix-daemon[924]: reaped child process 1218, status = succeeded4938lb # [ 57.157623] niks3-server[1069]: 2026/09/20 12:43:37 INFO Received uploads request method=POST path=/api/pending_closures4939worker2 # [ 57.179595] nix-grpc-daemon[935]: time=2026-09-20T12:43:38.112Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4940worker2 # [ 57.185149] nix-grpc-daemon[935]: time=2026-09-20T12:43:38.115Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4941lb # [ 57.255613] niks3-server[1069]: 2026/09/20 12:43:37 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4942lb # [ 57.271343] niks3-server[1069]: 2026/09/20 12:43:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4943worker2 # [ 57.222991] nix-grpc-daemon[935]: time=2026-09-20T12:43:38.155Z level=INFO msg="Uploading 1 narinfos"4944lb # [ 57.276147] niks3-server[1069]: 2026/09/20 12:43:37 INFO Signed narinfos id=14 count=14945lb # [ 57.283678] niks3-server[1069]: 2026/09/20 12:43:37 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4946lb # [ 57.291113] niks3-server[1069]: 2026/09/20 12:43:37 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4947lb # [ 57.297852] niks3-server[1069]: 2026/09/20 12:43:37 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4948lb # [ 57.301101] niks3-server[1069]: 2026/09/20 12:43:37 INFO Completed upload id=144949worker2 # [ 57.249615] nix-grpc-daemon[935]: time=2026-09-20T12:43:38.182Z level=INFO msg="Upload complete. (325ms)"4950lb # [ 57.303128] niks3-server[1069]: 2026/09/20 12:43:37 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4951worker2 # [ 57.255275] nix-grpc-daemon[854]: ts=2026-09-20T12:43:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14952worker1 # [ 57.292822] nix-grpc-daemon[786]: ts=2026-09-20T12:43:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:585684953worker1 # [ 57.298471] nix-grpc-daemon[786]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4954worker2 # [ 57.265735] nix-daemon[924]: reaped child process 1198, status = succeeded4955worker1 # [ 57.315909] nix-grpc-daemon[786]: ts=2026-09-20T12:43:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:585684956worker1 # [ 57.318848] nix-grpc-daemon[786]: ts=2026-09-20T12:43:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=0 paths=1 nar_bytes_out=1764957worker1 # [ 57.558844] nix-grpc-daemon[786]: ts=2026-09-20T12:43:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:585684958worker2 # [ 57.527553] nix-grpc-daemon[854]: ts=2026-09-20T12:43:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804959worker2 # [ 57.534434] nix-grpc-daemon[935]: time=2026-09-20T12:43:38.467Z level=INFO msg="All 2 paths already cached"4960worker2 # [ 57.545914] nix-grpc-daemon[854]: ts=2026-09-20T12:43:38Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04961worker2 # [ 57.548971] nix-grpc-daemon[854]: ts=2026-09-20T12:43:38Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14962worker2 # [ 57.590861] nix-grpc-daemon[854]: ts=2026-09-20T12:43:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804963worker2 # [ 57.601705] nix-grpc-daemon[854]: ts=2026-09-20T12:43:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334804964worker2 # [ 57.631229] nix-grpc-daemon[935]: time=2026-09-20T12:43:38.564Z level=INFO msg="All 2 paths already cached"4965worker2 # [ 57.637893] nix-grpc-daemon[854]: ts=2026-09-20T12:43:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334804966worker2 # [ 57.645106] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4967worker2 # [ 57.810242] nix-daemon[924]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)4968worker2 # [ 57.828299] nix-daemon[924]: reaped child process 1227, status = succeeded4969worker2 # [ 57.987821] nix-daemon[924]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)4970lb # [ 58.046203] envoy[921]: conn peer=[2001:db8:1::1]:60540 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1082 ms=47454971lb # [ 58.047834] envoy[921]: conn peer=[2001:db8:1::1]:60524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2985 tx=1370 ms=58994972lb # [ 58.049402] envoy[921]: conn peer=[2001:db8:1::1]:60542 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3390 tx=1774 ms=18584973lb # [ 58.050865] envoy[921]: conn peer=[2001:db8:1::1]:50842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=144974lb # [ 58.052258] envoy[921]: conn peer=[2001:db8:1::1]:50848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3880 tx=1989 ms=26014975lb # [ 58.053749] envoy[921]: conn peer=[2001:db8:1::1]:50852 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=534976worker2 # [ 58.007310] nix-daemon[924]: reaped child process 1232, status = succeeded4977lb # [ 58.068780] niks3-server[1069]: 2026/09/20 12:43:38 INFO Received uploads request method=POST path=/api/pending_closures4978worker2 # [ 58.294293] nix-grpc-daemon[935]: time=2026-09-20T12:43:39.226Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4979worker2 # [ 58.297445] nix-grpc-daemon[935]: time=2026-09-20T12:43:39.226Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4980lb # [ 58.362177] niks3-server[1069]: 2026/09/20 12:43:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4981lb # [ 58.366678] niks3-server[1069]: 2026/09/20 12:43:38 INFO Signed narinfos id=15 count=14982worker2 # [ 58.314446] nix-grpc-daemon[935]: time=2026-09-20T12:43:39.247Z level=INFO msg="Uploading 1 narinfos"4983lb # [ 58.375258] niks3-server[1069]: 2026/09/20 12:43:38 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4984lb # [ 58.394621] niks3-server[1069]: 2026/09/20 12:43:38 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4985lb # [ 58.411601] niks3-server[1069]: 2026/09/20 12:43:38 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4986lb # [ 58.428084] niks3-server[1069]: 2026/09/20 12:43:38 INFO Completed upload id=154987worker2 # [ 58.375181] nix-grpc-daemon[935]: time=2026-09-20T12:43:39.308Z level=INFO msg="Upload complete. (720ms)"4988worker2 # [ 58.379455] nix-grpc-daemon[854]: ts=2026-09-20T12:43:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_in=7524989worker2 # [ 58.382429] nix-grpc-daemon[854]: ts=2026-09-20T12:43:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334804990worker2 # [ 58.389406] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4991lb # [ 58.443531] niks3-server[1069]: 2026/09/20 12:43:38 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4992worker2 # [ 58.397824] nix-daemon[924]: reaped child process 1222, status = succeeded4993worker2 # [ 58.404080] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4994worker2 # [ 59.625887] nix-daemon[924]: reaped child process 1236, status = succeeded4995worker2 # [ 59.766462] nix-daemon[924]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)4996worker2 # [ 59.786958] nix-daemon[924]: reaped child process 1255, status = succeeded4997lb # [ 59.850372] niks3-server[1069]: 2026/09/20 12:43:40 INFO Received uploads request method=POST path=/api/pending_closures4998worker2 # [ 59.818909] nix-grpc-daemon[935]: time=2026-09-20T12:43:40.751Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4999worker2 # [ 59.826904] nix-grpc-daemon[935]: time=2026-09-20T12:43:40.753Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5000lb # [ 59.898577] niks3-server[1069]: 2026/09/20 12:43:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5001lb # [ 59.901048] niks3-server[1069]: 2026/09/20 12:43:40 INFO Signed narinfos id=16 count=15002worker2 # [ 59.849731] nix-grpc-daemon[935]: time=2026-09-20T12:43:40.782Z level=INFO msg="Uploading 1 narinfos"5003lb # [ 59.908841] niks3-server[1069]: 2026/09/20 12:43:40 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5004lb # [ 59.910840] niks3-server[1069]: 2026/09/20 12:43:40 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5005lb # [ 59.924915] niks3-server[1069]: 2026/09/20 12:43:40 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5006lb # [ 59.929887] niks3-server[1069]: 2026/09/20 12:43:40 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5007lb # [ 59.933296] niks3-server[1069]: 2026/09/20 12:43:40 INFO Completed upload id=165008worker2 # [ 59.882041] nix-grpc-daemon[935]: time=2026-09-20T12:43:40.814Z level=INFO msg="Upload complete. (268ms)"5009lb # [ 59.935508] niks3-server[1069]: 2026/09/20 12:43:40 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5010worker2 # [ 59.886440] nix-grpc-daemon[854]: ts=2026-09-20T12:43:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15011worker2 # [ 59.895699] nix-daemon[924]: reaped child process 1234, status = succeeded5012worker1 # [ 59.967834] nix-grpc-daemon[786]: ts=2026-09-20T12:43:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:585685013worker1 # [ 59.975812] nix-grpc-daemon[786]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5014worker1 # [ 60.002093] nix-grpc-daemon[786]: ts=2026-09-20T12:43:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:585685015worker1 # [ 60.004869] nix-grpc-daemon[786]: ts=2026-09-20T12:43:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=0 paths=1 nar_bytes_out=2325016client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.91 seconds)5017client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5018client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5019client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5020client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)5021client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5022client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5023(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.99 seconds)5024subtest: an input only one worker has still reaches the builder5025client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5026client # [ 60.204266] nix-daemon[945]: reaped child process 947, status = succeeded5027client # this derivation will be built:5028client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5029client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5030client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.40 seconds)5031client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5032client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds)5033worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5034worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)5035worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5036worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5037client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25038client # [ 61.199143] nix-daemon[945]: accepted connection from pid 1027, user root (trusted)5039client # this derivation will be built:5040client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5041worker1 # [ 61.400043] nix-grpc-daemon[786]: ts=2026-09-20T12:43:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:585685042worker2 # [ 61.368805] nix-grpc-daemon[854]: ts=2026-09-20T12:43:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334805043client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5044client # copying 1 paths...5045client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5046worker1 # [ 61.414437] nix-grpc-daemon[786]: ts=2026-09-20T12:43:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585685047worker1 # [ 61.423717] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5048worker1 # [ 61.580932] nix-daemon[844]: accepted connection from pid 952, user nix-grpc-daemon (trusted)5049worker1 # [ 61.596192] nix-daemon[844]: reaped child process 955, status = succeeded5050lb # [ 61.628868] niks3-server[1069]: 2026/09/20 12:43:41 INFO Received uploads request method=POST path=/api/pending_closures5051worker1 # [ 61.927456] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.063Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5052worker1 # [ 61.929604] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.063Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5053lb # [ 61.960512] niks3-server[1069]: 2026/09/20 12:43:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5054lb # [ 61.965103] niks3-server[1069]: 2026/09/20 12:43:42 INFO Signed narinfos id=17 count=15055worker1 # [ 61.947262] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.083Z level=INFO msg="Uploading 1 narinfos"5056lb # [ 61.970898] niks3-server[1069]: 2026/09/20 12:43:42 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5057lb # [ 61.976255] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5058lb # [ 61.987497] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5059lb # [ 61.998037] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5060lb # [ 62.009216] niks3-server[1069]: 2026/09/20 12:43:42 INFO Completed upload id=175061worker1 # [ 61.991359] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.128Z level=INFO msg="Upload complete. (559ms)"5062worker1 # [ 61.994460] nix-grpc-daemon[786]: ts=2026-09-20T12:43:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=0 paths=1 nar_bytes_in=1205063worker1 # [ 62.003647] nix-daemon[844]: reaped child process 950, status = succeeded5064worker2 # [ 61.971995] nix-grpc-daemon[854]: ts=2026-09-20T12:43:42Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05065worker2 # [ 61.974989] nix-grpc-daemon[854]: ts=2026-09-20T12:43:42Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15066worker1 # [ 62.051502] nix-grpc-daemon[786]: ts=2026-09-20T12:43:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585685067worker1 # [ 62.060479] nix-grpc-daemon[786]: ts=2026-09-20T12:43:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585685068client # copying 2 paths to 'lb:50051'...5069client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5070worker1 # [ 62.070361] nix-grpc-daemon[786]: ts=2026-09-20T12:43:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585685071client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5072worker1 # [ 62.079092] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5073worker1 # [ 62.248697] nix-daemon[844]: accepted connection from pid 959, user nix-grpc-daemon (trusted)5074worker1 # [ 62.275168] nix-daemon[844]: reaped child process 962, status = succeeded5075worker1 # [ 62.447888] nix-daemon[844]: accepted connection from pid 964, user nix-grpc-daemon (trusted)5076worker1 # [ 62.465522] nix-daemon[844]: reaped child process 967, status = succeeded5077lb # [ 62.492103] niks3-server[1069]: 2026/09/20 12:43:42 INFO Received uploads request method=POST path=/api/pending_closures5078lb # [ 62.500929] niks3-server[1069]: 2026/09/20 12:43:42 INFO Received uploads request method=POST path=/api/pending_closures5079worker1 # [ 62.490100] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.626Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5080worker1 # [ 62.491604] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.626Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5081worker1 # [ 62.493255] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.626Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5082lb # [ 62.532222] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5083lb # [ 62.535824] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5084lb # [ 62.543501] niks3-server[1069]: 2026/09/20 12:43:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5085lb # [ 62.547497] niks3-server[1069]: 2026/09/20 12:43:42 INFO Signed narinfos id=18 count=15086lb # [ 62.549603] niks3-server[1069]: 2026/09/20 12:43:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5087lb # [ 62.551840] niks3-server[1069]: 2026/09/20 12:43:42 INFO Signed narinfos id=19 count=25088worker1 # [ 62.536213] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.672Z level=INFO msg="Uploading 3 narinfos"5089lb # [ 62.568099] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5090lb # [ 62.571924] niks3-server[1069]: 2026/09/20 12:43:42 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5091lb # [ 62.575599] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5092lb # [ 62.578141] niks3-server[1069]: 2026/09/20 12:43:42 INFO Completed upload id=185093lb # [ 62.579742] niks3-server[1069]: 2026/09/20 12:43:42 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5094lb # [ 62.583747] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5095lb # [ 62.586582] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5096lb # [ 62.588922] niks3-server[1069]: 2026/09/20 12:43:42 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5097lb # [ 62.591365] niks3-server[1069]: 2026/09/20 12:43:42 INFO Completed upload id=195098worker1 # [ 62.575183] nix-grpc-daemon[870]: time=2026-09-20T12:43:43.712Z level=INFO msg="Upload complete. (486ms)"5099worker1 # [ 62.579489] nix-grpc-daemon[786]: ts=2026-09-20T12:43:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=0 paths=2 nar_bytes_in=9765100worker1 # [ 62.582992] nix-grpc-daemon[786]: ts=2026-09-20T12:43:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585685101worker1 # [ 62.589926] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5102worker1 # [ 62.599122] nix-daemon[844]: reaped child process 957, status = succeeded5103worker1 # [ 62.606478] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5104client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5105client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5106worker1 # [ 63.825440] nix-daemon[844]: reaped child process 971, status = succeeded5107worker1 # [ 63.968441] nix-daemon[844]: accepted connection from pid 986, user nix-grpc-daemon (trusted)5108worker1 # [ 63.985446] nix-daemon[844]: reaped child process 989, status = succeeded5109lb # [ 64.015309] niks3-server[1069]: 2026/09/20 12:43:44 INFO Received uploads request method=POST path=/api/pending_closures5110worker1 # [ 64.002687] nix-grpc-daemon[870]: time=2026-09-20T12:43:45.139Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5111lb # [ 64.035249] niks3-server[1069]: 2026/09/20 12:43:44 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5112lb # [ 64.039325] niks3-server[1069]: 2026/09/20 12:43:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5113lb # [ 64.041517] niks3-server[1069]: 2026/09/20 12:43:44 INFO Signed narinfos id=20 count=15114worker1 # [ 64.025310] nix-grpc-daemon[870]: time=2026-09-20T12:43:45.162Z level=INFO msg="Uploading 1 narinfos"5115lb # [ 64.050125] niks3-server[1069]: 2026/09/20 12:43:44 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5116lb # [ 64.052946] niks3-server[1069]: 2026/09/20 12:43:44 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5117lb # [ 64.059223] niks3-server[1069]: 2026/09/20 12:43:44 INFO Completed upload id=205118worker1 # [ 64.041391] nix-grpc-daemon[870]: time=2026-09-20T12:43:45.178Z level=INFO msg="Upload complete. (224ms)"5119lb # [ 64.061354] niks3-server[1069]: 2026/09/20 12:43:44 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5120worker1 # [ 64.045390] nix-grpc-daemon[786]: ts=2026-09-20T12:43:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15121worker1 # [ 64.052844] nix-daemon[844]: reaped child process 969, status = succeeded5122client # copying 1 paths...5123worker2 # [ 64.057828] nix-grpc-daemon[854]: ts=2026-09-20T12:43:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334805124worker2 # [ 64.071843] nix-grpc-daemon[854]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5125client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5126worker2 # [ 64.097290] nix-grpc-daemon[854]: ts=2026-09-20T12:43:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334805127worker2 # [ 64.099934] nix-grpc-daemon[854]: ts=2026-09-20T12:43:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_out=1205128client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.22 seconds)5129client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25130client # [ 64.344169] nix-daemon[945]: reaped child process 1030, status = succeeded5131client # [ 64.410616] nix-daemon[945]: accepted connection from pid 1049, user root (trusted)5132client # this derivation will be built:5133client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5134worker2 # [ 64.984800] nix-grpc-daemon[854]: ts=2026-09-20T12:43:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334805135worker1 # [ 65.023341] nix-grpc-daemon[786]: ts=2026-09-20T12:43:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585685136client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5137worker1 # [ 65.029962] nix-grpc-daemon[870]: time=2026-09-20T12:43:46.166Z level=INFO msg="All 1 paths already cached"5138worker2 # [ 65.010302] nix-grpc-daemon[854]: ts=2026-09-20T12:43:45Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05139worker2 # [ 65.015393] nix-grpc-daemon[854]: ts=2026-09-20T12:43:45Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15140worker2 # [ 65.066902] nix-grpc-daemon[854]: ts=2026-09-20T12:43:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334805141worker2 # [ 65.080162] nix-grpc-daemon[854]: ts=2026-09-20T12:43:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334805142client # copying 1 paths to 'lb:50051'...5143client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5144worker2 # [ 65.095733] nix-grpc-daemon[854]: ts=2026-09-20T12:43:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334805145worker2 # [ 65.108419] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5146worker2 # [ 65.125320] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5)5147worker2 # [ 65.136440] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 2/5)5148worker2 # [ 65.234674] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 177 ms (attempt 3/5)5149worker2 # [ 65.418168] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 765 ms (attempt 4/5)5150worker2 # [ 66.188272] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5151worker2 # [ 66.192056] nix-grpc-daemon[854]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5152worker2 # [ 66.398144] nix-daemon[924]: accepted connection from pid 1271, user nix-grpc-daemon (trusted)5153worker2 # [ 66.417618] nix-daemon[924]: reaped child process 1274, status = succeeded5154worker2 # [ 66.592482] nix-daemon[924]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5155worker2 # [ 66.610409] nix-daemon[924]: reaped child process 1279, status = succeeded5156lb # [ 66.676577] niks3-server[1069]: 2026/09/20 12:43:46 INFO Received uploads request method=POST path=/api/pending_closures5157worker2 # [ 66.629815] nix-grpc-daemon[935]: time=2026-09-20T12:43:47.561Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5158worker2 # [ 66.631840] nix-grpc-daemon[935]: time=2026-09-20T12:43:47.562Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5159lb # [ 66.697990] niks3-server[1069]: 2026/09/20 12:43:46 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5160lb # [ 66.709172] niks3-server[1069]: 2026/09/20 12:43:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5161lb # [ 66.713821] niks3-server[1069]: 2026/09/20 12:43:47 INFO Signed narinfos id=21 count=15162worker2 # [ 66.661489] nix-grpc-daemon[935]: time=2026-09-20T12:43:47.594Z level=INFO msg="Uploading 1 narinfos"5163lb # [ 66.718813] niks3-server[1069]: 2026/09/20 12:43:47 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5164lb # [ 66.726190] niks3-server[1069]: 2026/09/20 12:43:47 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5165lb # [ 66.732057] niks3-server[1069]: 2026/09/20 12:43:47 INFO Completed upload id=215166worker2 # [ 66.679295] nix-grpc-daemon[935]: time=2026-09-20T12:43:47.612Z level=INFO msg="Upload complete. (472ms)"5167lb # [ 66.736142] niks3-server[1069]: 2026/09/20 12:43:47 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5168worker2 # [ 66.682619] nix-grpc-daemon[854]: ts=2026-09-20T12:43:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 paths=1 nar_bytes_in=5605169worker2 # [ 66.686936] nix-grpc-daemon[854]: ts=2026-09-20T12:43:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334805170worker2 # [ 66.697126] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5171worker2 # [ 66.706291] nix-daemon[924]: reaped child process 1265, status = succeeded5172worker2 # [ 66.718085] nix-grpc-daemon[854]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5173client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5174client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5175worker2 # [ 66.731692] nix-daemon[924]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5176worker2 # [ 67.756978] nix-daemon[924]: reaped child process 1284, status = succeeded5177worker2 # [ 67.946566] nix-daemon[924]: accepted connection from pid 1299, user nix-grpc-daemon (trusted)5178worker2 # [ 67.969356] nix-daemon[924]: reaped child process 1302, status = succeeded5179lb # [ 68.034852] niks3-server[1069]: 2026/09/20 12:43:48 INFO Received uploads request method=POST path=/api/pending_closures5180worker2 # [ 67.987106] nix-grpc-daemon[935]: time=2026-09-20T12:43:48.919Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5181lb # [ 68.047681] envoy[921]: conn peer=[2001:db8:1::1]:50856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2223 ms=25005182lb # [ 68.050551] envoy[921]: conn peer=[2001:db8:1::1]:50862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=555183lb # [ 68.053546] envoy[921]: conn peer=[2001:db8:1::1]:50864 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4199 tx=1861 ms=27875184lb # [ 68.056520] envoy[921]: conn peer=[2001:db8:1::1]:57366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=535185lb # [ 68.070120] niks3-server[1069]: 2026/09/20 12:43:48 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5186lb # [ 68.072422] niks3-server[1069]: 2026/09/20 12:43:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5187lb # [ 68.074657] niks3-server[1069]: 2026/09/20 12:43:48 INFO Signed narinfos id=22 count=15188worker2 # [ 68.023489] nix-grpc-daemon[935]: time=2026-09-20T12:43:48.956Z level=INFO msg="Uploading 1 narinfos"5189lb # [ 68.080807] niks3-server[1069]: 2026/09/20 12:43:48 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5190lb # [ 68.089629] niks3-server[1069]: 2026/09/20 12:43:48 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5191lb # [ 68.095277] niks3-server[1069]: 2026/09/20 12:43:48 INFO Completed upload id=225192worker2 # [ 68.042382] nix-grpc-daemon[935]: time=2026-09-20T12:43:48.975Z level=INFO msg="Upload complete. (297ms)"5193lb # [ 68.097818] niks3-server[1069]: 2026/09/20 12:43:48 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5194worker2 # [ 68.045950] nix-grpc-daemon[854]: ts=2026-09-20T12:43:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15195worker2 # [ 68.055070] nix-daemon[924]: reaped child process 1281, status = succeeded5196client # copying 1 paths...5197worker1 # [ 68.129944] nix-grpc-daemon[786]: ts=2026-09-20T12:43:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:585685198worker1 # [ 68.137362] nix-grpc-daemon[786]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5199client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5200worker2 # [ 68.128557] nix-grpc-daemon[854]: ts=2026-09-20T12:43:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334805201worker2 # [ 68.130480] nix-grpc-daemon[854]: ts=2026-09-20T12:43:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33480 duration_s=0 paths=1 nar_bytes_out=1205202client # [ 68.228273] nix-daemon[945]: reaped child process 1052, status = succeeded5203client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 4.00 seconds)5204client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25205client # [ 68.391765] nix-daemon[945]: accepted connection from pid 1071, user root (trusted)5206client # this derivation will be built:5207client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5208worker2 # [ 68.549621] nix-grpc-daemon[854]: ts=2026-09-20T12:43:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334805209client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5210worker2 # [ 68.558889] nix-grpc-daemon[854]: ts=2026-09-20T12:43:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334805211worker2 # [ 68.566468] nix-grpc-daemon[935]: time=2026-09-20T12:43:49.499Z level=INFO msg="All 1 paths already cached"5212worker2 # [ 68.581447] nix-grpc-daemon[854]: ts=2026-09-20T12:43:49Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05213worker1 # [ 68.619177] nix-grpc-daemon[786]: ts=2026-09-20T12:43:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585685214worker2 # [ 68.584610] nix-grpc-daemon[854]: ts=2026-09-20T12:43:49Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15215worker1 # [ 68.631676] nix-grpc-daemon[786]: ts=2026-09-20T12:43:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585685216worker1 # [ 68.643674] nix-grpc-daemon[870]: time=2026-09-20T12:43:49.780Z level=INFO msg="All 1 paths already cached"5217client # copying 1 paths to 'lb:50051'...5218client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5219worker1 # [ 68.650319] nix-grpc-daemon[786]: ts=2026-09-20T12:43:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585685220worker1 # [ 68.657803] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5221worker1 # [ 68.826522] nix-daemon[844]: accepted connection from pid 994, user nix-grpc-daemon (trusted)5222worker1 # [ 68.843921] nix-daemon[844]: reaped child process 997, status = succeeded5223worker1 # [ 68.996286] nix-daemon[844]: accepted connection from pid 999, user nix-grpc-daemon (trusted)5224worker1 # [ 69.012226] nix-daemon[844]: reaped child process 1002, status = succeeded5225lb # [ 69.043577] niks3-server[1069]: 2026/09/20 12:43:49 INFO Received uploads request method=POST path=/api/pending_closures5226worker1 # [ 69.033103] nix-grpc-daemon[870]: time=2026-09-20T12:43:50.169Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5227worker1 # [ 69.034635] nix-grpc-daemon[870]: time=2026-09-20T12:43:50.169Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5228lb # [ 69.071195] niks3-server[1069]: 2026/09/20 12:43:49 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5229lb # [ 69.081735] niks3-server[1069]: 2026/09/20 12:43:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5230lb # [ 69.085991] niks3-server[1069]: 2026/09/20 12:43:49 INFO Signed narinfos id=23 count=15231worker1 # [ 69.068380] nix-grpc-daemon[870]: time=2026-09-20T12:43:50.205Z level=INFO msg="Uploading 1 narinfos"5232lb # [ 69.092695] niks3-server[1069]: 2026/09/20 12:43:49 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5233lb # [ 69.103676] niks3-server[1069]: 2026/09/20 12:43:49 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5234worker1 # [ 69.092401] nix-grpc-daemon[870]: time=2026-09-20T12:43:50.229Z level=INFO msg="Upload complete. (427ms)"5235lb # [ 69.110780] niks3-server[1069]: 2026/09/20 12:43:49 INFO Completed upload id=235236worker1 # [ 69.096109] nix-grpc-daemon[786]: ts=2026-09-20T12:43:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=0 paths=1 nar_bytes_in=5605237lb # [ 69.114615] niks3-server[1069]: 2026/09/20 12:43:49 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5238worker1 # [ 69.100102] nix-grpc-daemon[786]: ts=2026-09-20T12:43:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585685239worker1 # [ 69.107255] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5240worker1 # [ 69.110217] nix-daemon[844]: reaped child process 992, status = succeeded5241client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5242worker1 # [ 69.118044] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5243client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5244worker1 # [ 70.134683] nix-daemon[844]: reaped child process 1006, status = succeeded5245worker1 # [ 70.288766] nix-daemon[844]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)5246worker1 # [ 70.305288] nix-daemon[844]: reaped child process 1024, status = succeeded5247lb # [ 70.336418] niks3-server[1069]: 2026/09/20 12:43:50 INFO Received uploads request method=POST path=/api/pending_closures5248worker1 # [ 70.333685] nix-grpc-daemon[870]: time=2026-09-20T12:43:51.470Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5249lb # [ 70.371934] niks3-server[1069]: 2026/09/20 12:43:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5250lb # [ 70.378172] niks3-server[1069]: 2026/09/20 12:43:50 INFO Signed narinfos id=24 count=15251worker1 # [ 70.362914] nix-grpc-daemon[870]: time=2026-09-20T12:43:51.499Z level=INFO msg="Uploading 1 narinfos"5252lb # [ 70.386831] niks3-server[1069]: 2026/09/20 12:43:50 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5253lb # [ 70.390852] niks3-server[1069]: 2026/09/20 12:43:50 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5254lb # [ 70.403056] niks3-server[1069]: 2026/09/20 12:43:50 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5255lb # [ 70.409631] niks3-server[1069]: 2026/09/20 12:43:50 INFO Completed upload id=245256worker1 # [ 70.391826] nix-grpc-daemon[870]: time=2026-09-20T12:43:51.528Z level=INFO msg="Upload complete. (267ms)"5257worker1 # [ 70.395894] nix-grpc-daemon[786]: ts=2026-09-20T12:43:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15258lb # [ 70.415576] niks3-server[1069]: 2026/09/20 12:43:50 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5259worker1 # [ 70.407126] nix-daemon[844]: reaped child process 1004, status = succeeded5260client # copying 1 paths...5261worker2 # [ 70.407641] nix-grpc-daemon[854]: ts=2026-09-20T12:43:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334805262worker2 # [ 70.417533] nix-grpc-daemon[854]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5263client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5264worker1 # [ 70.482900] nix-grpc-daemon[786]: ts=2026-09-20T12:43:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:585685265worker1 # [ 70.486420] nix-grpc-daemon[786]: ts=2026-09-20T12:43:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58568 duration_s=0 paths=1 nar_bytes_out=1205266client # [ 70.539613] nix-daemon[945]: reaped child process 1074, status = succeeded5267client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.31 seconds)5268(finished: subtest: an input only one worker has still reaches the builder, in 10.37 seconds)5269subtest: upload whose reference the worker lacks is completed from the cache5270client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5271client # this derivation will be built:5272client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5273client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5274client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.42 seconds)5275client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5276client # this derivation will be built:5277client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5278client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5279client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)5280client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5281client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5282worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5283worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)5284worker2 # [ 71.841991] systemd[1]: Stopping Nix worker-protocol over gRPC...5285worker2 # [ 72.354208] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=signal_exit5286worker2 # [ 72.595155] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=scheduler_yield5287lb # [ 72.650771] niks3-server[1069]: 2026/09/20 12:43:52 INFO lead: released remote=192.168.1.4:547765288worker1 # [ 72.638053] nix-grpc-daemon[786]: ts=2026-09-20T12:43:53Z level=info event=scheduler_restarting addr=lb:500515289worker2 # [ 72.606486] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573965290worker2 # [ 72.610308] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=worker_gone addr=192.168.1.3:500515291worker2 # [ 72.612296] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=scheduler_restarting addr=lb:500515292worker2 # [ 72.614239] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573965293worker2 # [ 72.616357] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=worker_gone addr=192.168.1.4:500515294worker1 # [ 72.838453] nix-grpc-daemon[786]: ts=2026-09-20T12:43:53Z level=info event=scheduler_connected addr=lb:500515295worker1 # [ 72.841116] nix-grpc-daemon[786]: ts=2026-09-20T12:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5296worker2 # [ 72.810455] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=scheduler_connected addr=lb:500515297worker2 # [ 72.812883] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5298worker1 # [ 73.041813] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_connected addr=lb:500515299worker1 # [ 73.044230] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5300worker2 # [ 73.014197] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=scheduler_connected addr=lb:500515301worker2 # [ 73.016858] nix-grpc-daemon[854]: ts=2026-09-20T12:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5302worker1 # [ 73.244835] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_connected addr=lb:500515303worker1 # [ 73.249349] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker2 # [ 73.218083] nix-grpc-daemon[854]: ts=2026-09-20T12:43:54Z level=info event=scheduler_connected addr=lb:500515305worker2 # [ 73.221578] nix-grpc-daemon[854]: ts=2026-09-20T12:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5306worker1 # [ 73.450395] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_connected addr=lb:500515307worker1 # [ 73.452889] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5308worker2 # [ 73.423398] nix-grpc-daemon[854]: ts=2026-09-20T12:43:54Z level=info event=scheduler_connected addr=lb:500515309worker2 # [ 73.426055] nix-grpc-daemon[854]: ts=2026-09-20T12:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5310worker2 # [ 73.598391] nix-grpc-daemon[854]: ts=2026-09-20T12:43:54Z level=info event=scheduler_restarting5311worker1 # [ 73.654410] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_connected addr=lb:500515312worker2 # [ 73.622159] nix-daemon[924]: reaped child process 926, status = succeeded5313worker1 # [ 73.656890] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5314worker2 # [ 73.637580] nix-grpc-daemon[854]: ts=2026-09-20T12:43:54Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5315worker1 # [ 73.855710] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_connected addr=lb:500515316worker1 # [ 73.858471] nix-grpc-daemon[786]: ts=2026-09-20T12:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker1 # [ 74.059204] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=info event=scheduler_connected addr=lb:500515318worker1 # [ 74.061640] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5319worker2 # [ 74.204484] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5320worker2 # [ 74.207533] systemd[1]: Stopped Nix worker-protocol over gRPC.5321worker2 # [ 74.208904] systemd[1]: nix-grpc-daemon.service: Consumed 3.150s CPU time over 56.100s wall clock time, 61.5M memory peak, 95.1K incoming IP traffic, 228.9K outgoing IP traffic.5322worker2 # [ 74.215988] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5323worker2 # [ 74.218733] systemd[1]: Closed Nix worker-protocol over gRPC.5324worker2 # [ 74.220868] systemd[1]: nix-grpc-daemon.socket: Consumed 436us CPU time over 1min 4.300s wall clock time, 256K memory peak, 87.5K incoming IP traffic, 71.9K outgoing IP traffic.5325lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5326worker1 # [ 74.262627] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=info event=scheduler_connected addr=lb:500515327lb: must succeed: curl -sf localhost:9901/clusters5328worker1 # [ 74.265440] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5330lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5331client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25332worker1 # [ 74.466369] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=info event=scheduler_connected addr=lb:500515333worker1 # [ 74.469101] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5334worker1 # [ 74.546512] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585685335worker1 # [ 74.552651] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:585685336client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5337client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.26 seconds)5338worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5339worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5340worker1 # [ 74.670521] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=info event=scheduler_connected addr=lb:500515341worker1 # [ 74.672889] nix-grpc-daemon[786]: ts=2026-09-20T12:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5342worker2 # [ 74.641142] systemd[1]: Listening on Nix worker-protocol over gRPC.5343worker1 # [ 74.729391] systemd[1]: Stopping Nix worker-protocol over gRPC...5344worker1 # [ 74.873539] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 74.875953] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker1 # [ 75.076616] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_connected addr=lb:500515347worker1 # [ 75.079546] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5348worker1 # [ 75.280125] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_connected addr=lb:500515349worker1 # [ 75.282377] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5350worker1 # [ 75.284647] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=signal_exit5351lb # [ 75.476360] niks3-server[1069]: 2026/09/20 12:43:55 INFO lead: acquired remote=192.168.1.3:371125352worker1 # [ 75.460607] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_take_over5353lb # [ 75.480430] niks3-server[1069]: 2026/09/20 12:43:55 INFO lead: released remote=192.168.1.3:371125354worker1 # [ 75.462724] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_yield5355worker1 # [ 75.482959] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_connected addr=lb:500515356worker1 # [ 75.485288] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357worker2 # [ 75.483413] systemd[1]: Starting scheduler-token.service...5358worker2 # [ 75.559731] systemd[1]: scheduler-token.service: Deactivated successfully.5359worker2 # [ 75.562866] systemd[1]: Finished scheduler-token.service.5360worker2 # [ 75.565488] systemd[1]: scheduler-token.service: Consumed 31ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5361worker2 # [ 75.574683] systemd[1]: Starting Nix worker-protocol over gRPC...5362worker1 # [ 75.686556] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_connected addr=lb:500515363worker1 # [ 75.688435] nix-grpc-daemon[786]: ts=2026-09-20T12:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker2 # [ 75.757584] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5365worker2 # [ 75.760324] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5366worker2 # [ 75.764838] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15367worker2 # [ 75.775110] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5368worker2 # [ 75.779370] systemd[1]: Started Nix worker-protocol over gRPC.5369lb # [ 75.843183] niks3-server[1069]: 2026/09/20 12:43:56 INFO lead: acquired remote=192.168.1.4:330865370worker2 # [ 75.790279] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=scheduler_take_over5371worker2 # [ 75.798671] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=scheduler_connected addr=lb:500515372worker2 # [ 75.806724] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=oidc_jwks provider=mock keys=15373worker2 # [ 75.808509] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:360365374worker2 # [ 75.810344] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05375worker1 # [ 75.889357] nix-grpc-daemon[786]: ts=2026-09-20T12:43:57Z level=info event=scheduler_connected addr=lb:500515376worker2 # [ 75.857612] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:360365377worker2 # [ 75.860580] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05378worker1 # [ 76.461234] nix-grpc-daemon[786]: ts=2026-09-20T12:43:57Z level=info event=scheduler_restarting5379worker2 # [ 76.433729] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:360365380worker2 # [ 76.436305] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:57Z level=info event=worker_gone addr=192.168.1.3:500515381worker1 # [ 76.478632] nix-grpc-daemon[786]: ts=2026-09-20T12:43:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5382worker1 # [ 76.493580] nix-daemon[844]: reaped child process 846, status = succeeded5383worker1 # [ 77.425530] nix-grpc-daemon[786]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5384worker1 # [ 77.427829] nix-grpc-daemon[786]: E0000 00:00:1789908238.561317 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5385worker1 # [ 77.431389] nix-grpc-daemon[786]: E0000 00:00:1789908238.561915 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5386lb # [ 78.047393] envoy[921]: conn peer=[2001:db8:1::1]:57382 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3729 tx=1959 ms=31575387lb # [ 78.049948] envoy[921]: conn peer=[2001:db8:1::1]:57394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=125388lb # [ 78.053139] envoy[921]: conn peer=[2001:db8:1::1]:57408 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3714 tx=1917 ms=19135389lb # [ 78.055258] envoy[921]: conn peer=[2001:db8:1::1]:57412 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=165390lb # [ 78.057418] envoy[921]: conn peer=[2001:db8:1::4]:52548 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=58902 tx=3322 ms=552685391lb # [ 78.059719] envoy[921]: conn peer=[2001:db8:1::1]:51190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=205392lb # [ 78.062066] envoy[921]: conn peer=[2001:db8:1::3]:48508 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15906 tx=2923 ms=572815393worker1 # [ 78.200595] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5394worker1 # [ 78.203478] systemd[1]: Stopped Nix worker-protocol over gRPC.5395worker1 # [ 78.204755] systemd[1]: nix-grpc-daemon.service: Consumed 1.664s CPU time over 1min 179ms wall clock time, 47.7M memory peak, 56.6K incoming IP traffic, 104.2K outgoing IP traffic.5396worker1 # [ 78.213298] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5397worker1 # [ 78.215859] systemd[1]: Closed Nix worker-protocol over gRPC.5398worker1 # [ 78.218455] systemd[1]: nix-grpc-daemon.socket: Consumed 382us CPU time over 1min 7.866s wall clock time, 256K memory peak, 62.5K incoming IP traffic, 55.2K outgoing IP traffic.5399lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5400lb: must succeed: curl -sf localhost:9901/clusters5401lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5402lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5403client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25404worker2 # [ 78.521625] nix-grpc-daemon[1329]: ts=2026-09-20T12:43:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360485405worker2 # [ 78.531181] nix-daemon[924]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5406worker2 # [ 78.544351] nix-grpc-daemon[1329]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)5407worker2 # [ 78.598452] nix-grpc-daemon[1329]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)5408worker2 # [ 78.717432] nix-grpc-daemon[1329]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 3/5)5409worker2 # [ 78.743873] nix-grpc-daemon[1329]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 392 ms (attempt 4/5)5410worker2 # [ 79.140943] nix-grpc-daemon[1329]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5411client # copying 2 paths...5412client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5413client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5414worker2 # [ 79.157072] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360485415worker2 # [ 79.166153] nix-daemon[924]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5416worker2 # [ 79.183469] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:00Z level=info event=niks3_push_started pid=13855417worker2 # [ 79.386337] nix-grpc-daemon[1385]: time=2026-09-20T12:44:00.318Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5418worker2 # [ 79.570804] nix-daemon[924]: accepted connection from pid 1398, user nix-grpc-daemon (trusted)5419worker2 # [ 79.598202] nix-daemon[924]: reaped child process 1401, status = succeeded5420lb # [ 79.666871] niks3-server[1069]: 2026/09/20 12:43:59 INFO Received uploads request method=POST path=/api/pending_closures5421lb # [ 79.673591] niks3-server[1069]: 2026/09/20 12:43:59 INFO Received uploads request method=POST path=/api/pending_closures5422worker2 # [ 79.626336] nix-grpc-daemon[1385]: time=2026-09-20T12:44:00.559Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5423worker2 # [ 79.634887] nix-grpc-daemon[1385]: time=2026-09-20T12:44:00.561Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5424worker2 # [ 79.638309] nix-grpc-daemon[1385]: time=2026-09-20T12:44:00.562Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5425lb # [ 79.736161] niks3-server[1069]: 2026/09/20 12:44:00 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5426lb # [ 79.738703] niks3-server[1069]: 2026/09/20 12:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5427lb # [ 79.742707] niks3-server[1069]: 2026/09/20 12:44:00 INFO Signed narinfos id=25 count=15428lb # [ 79.749901] niks3-server[1069]: 2026/09/20 12:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5429lb # [ 79.760823] niks3-server[1069]: 2026/09/20 12:44:00 INFO Signed narinfos id=26 count=25430worker2 # [ 79.707136] nix-grpc-daemon[1385]: time=2026-09-20T12:44:00.639Z level=INFO msg="Uploading 3 narinfos"5431lb # [ 79.773563] niks3-server[1069]: 2026/09/20 12:44:00 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5432lb # [ 79.785058] niks3-server[1069]: 2026/09/20 12:44:00 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5433lb # [ 79.797719] niks3-server[1069]: 2026/09/20 12:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5434lb # [ 79.800055] niks3-server[1069]: 2026/09/20 12:44:00 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5435lb # [ 79.811427] niks3-server[1069]: 2026/09/20 12:44:00 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5436lb # [ 79.817776] niks3-server[1069]: 2026/09/20 12:44:00 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5437lb # [ 79.821231] niks3-server[1069]: 2026/09/20 12:44:00 INFO Completed upload id=255438lb # [ 79.823045] niks3-server[1069]: 2026/09/20 12:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5439lb # [ 79.825894] niks3-server[1069]: 2026/09/20 12:44:00 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5440lb # [ 79.828458] niks3-server[1069]: 2026/09/20 12:44:00 INFO Completed upload id=265441worker2 # [ 79.777169] nix-grpc-daemon[1385]: time=2026-09-20T12:44:00.710Z level=INFO msg="Upload complete. (386ms)"5442worker2 # [ 79.782615] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36048 duration_s=0 paths=2 nar_bytes_in=3125443worker2 # [ 79.792587] nix-daemon[924]: reaped child process 1383, status = succeeded5444client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.56 seconds)5445worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5446worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5447worker1 # [ 79.912628] systemd[1]: Listening on Nix worker-protocol over gRPC.5448worker1 # [ 79.916511] systemd[1]: Starting Nix worker-protocol over gRPC...5449worker1 # [ 80.073408] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5450worker1 # [ 80.075788] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5451worker1 # [ 80.079836] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15452worker1 # [ 80.087577] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5453worker1 # [ 80.092384] systemd[1]: Started Nix worker-protocol over gRPC.5454lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5455lb: must succeed: curl -sf localhost:9901/clusters5456worker1 # [ 80.103544] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:01Z level=info event=scheduler_connected addr=lb:500515457worker2 # [ 80.069929] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:360365458worker2 # [ 80.071921] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05459lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5460lb: must succeed: curl -sf localhost:9901/clusters5461lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5462lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.13 seconds)5463(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.67 seconds)5464subtest: scheduler down: the other node takes the lock and keeps it5465lb: must succeed: curl -sf localhost:9901/clusters5466lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5467lb: must succeed: curl -sf localhost:9901/clusters5468lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5469lb: must succeed: curl -sf localhost:9901/clusters5470lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5471lb: must succeed: curl -sf localhost:9901/clusters5472lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5473worker2 # [ 81.440317] systemd[1]: Stopping Nix worker-protocol over gRPC...5474worker2 # [ 81.781559] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=signal_exit5475worker2 # [ 81.794725] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=scheduler_yield5476lb # [ 81.850492] niks3-server[1069]: 2026/09/20 12:44:02 INFO lead: released remote=192.168.1.4:330865477worker1 # [ 81.835682] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:02Z level=info event=scheduler_restarting addr=lb:500515478worker2 # [ 81.801264] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:360365479worker2 # [ 81.808462] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=worker_gone addr=192.168.1.3:500515480worker2 # [ 81.814838] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=scheduler_restarting addr=lb:500515481worker2 # [ 81.817930] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:360365482worker2 # [ 81.821259] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=worker_gone addr=192.168.1.4:500515483worker1 # [ 82.035683] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_connected addr=lb:500515484worker1 # [ 82.037655] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5485worker2 # [ 82.014664] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=scheduler_connected addr=lb:500515486worker2 # [ 82.016916] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5487worker1 # [ 82.238253] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_connected addr=lb:500515488worker1 # [ 82.240232] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5489worker2 # [ 82.218033] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:03Z level=info event=scheduler_connected addr=lb:500515490worker2 # [ 82.220374] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5491worker1 # [ 82.440655] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_connected addr=lb:500515492worker1 # [ 82.442639] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5493worker2 # [ 82.421167] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:03Z level=info event=scheduler_connected addr=lb:500515494worker2 # [ 82.423482] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5495worker1 # [ 82.643277] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_connected addr=lb:500515496worker1 # [ 82.645542] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker2 # [ 82.624650] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:03Z level=info event=scheduler_connected addr=lb:500515498worker2 # [ 82.626821] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker2 # [ 82.803443] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:03Z level=info event=scheduler_restarting5500worker2 # [ 82.809302] nix-grpc-daemon[1329]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5501worker2 # [ 82.811260] nix-grpc-daemon[1329]: E0000 00:00:1789908243.742037 1372 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5502worker1 # [ 82.847203] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_connected addr=lb:500515503worker1 # [ 82.848722] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5504worker2 # [ 82.813503] nix-grpc-daemon[1329]: E0000 00:00:1789908243.744249 1372 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5505worker2 # [ 82.824262] nix-daemon[924]: reaped child process 1378, status = succeeded5506worker2 # [ 82.831916] nix-grpc-daemon[1329]: ts=2026-09-20T12:44:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5507worker1 # [ 83.048299] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_connected addr=lb:500515508worker1 # [ 83.050531] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5509worker1 # [ 83.251082] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_connected addr=lb:500515510worker1 # [ 83.253389] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5511worker1 # [ 83.454185] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_connected addr=lb:500515512worker1 # [ 83.456432] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5513worker1 # [ 83.657046] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_connected addr=lb:500515514worker1 # [ 83.659077] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5515worker2 # [ 83.804929] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5516worker2 # [ 83.808383] systemd[1]: Stopped Nix worker-protocol over gRPC.5517worker2 # [ 83.809597] systemd[1]: nix-grpc-daemon.service: Consumed 444ms CPU time over 8.229s wall clock time, 30.6M memory peak, 19.7K incoming IP traffic, 26.4K outgoing IP traffic.5518worker2 # [ 83.817229] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5519worker2 # [ 83.819685] systemd[1]: Closed Nix worker-protocol over gRPC.5520worker2 # [ 83.821762] systemd[1]: nix-grpc-daemon.socket: Consumed 903us CPU time over 9.187s wall clock time, 256K memory peak, 20K incoming IP traffic, 14.7K outgoing IP traffic.5521worker1 # [ 83.860085] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_connected addr=lb:500515522worker1 # [ 83.861770] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5523worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5524worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5525worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5526worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5527worker1 # [ 84.062840] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:05Z level=info event=scheduler_connected addr=lb:500515528worker1 # [ 84.065465] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5529worker1 # [ 84.266326] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:05Z level=info event=scheduler_connected addr=lb:500515530worker1 # [ 84.268950] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5531worker1 # [ 84.469928] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:05Z level=info event=scheduler_connected addr=lb:500515532worker1 # [ 84.472452] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5533worker1 # [ 84.673623] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:05Z level=info event=scheduler_connected addr=lb:500515534worker1 # [ 84.676481] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5535worker1 # [ 84.879174] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:06Z level=info event=scheduler_connected addr=lb:500515536worker1 # [ 84.881258] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5537worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5538worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5539worker1 # [ 85.080834] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:06Z level=info event=scheduler_connected addr=lb:500515540worker1 # [ 85.083816] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5541lb # [ 85.115584] niks3-server[1069]: 2026/09/20 12:44:05 INFO lead: acquired remote=192.168.1.3:592665542worker1 # [ 85.097493] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:06Z level=info event=scheduler_take_over5543worker1 # [ 85.284515] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:06Z level=info event=scheduler_connected addr=lb:500515544worker1 # [ 85.288889] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:570425545worker1 # [ 85.290935] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05546worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5547worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5548client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25549worker1 # [ 86.292879] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570585550worker1 # [ 86.298681] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5551client # don't know how to build these paths:5552client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5553worker1 # [ 86.310233] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570585554worker1 # [ 86.316640] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5)5555worker1 # [ 86.359500] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 2/5)5556worker1 # [ 86.530200] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 283 ms (attempt 3/5)5557worker1 # [ 86.816248] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 451 ms (attempt 4/5)5558worker1 # [ 87.271184] nix-grpc-daemon[1047]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5559worker1 # [ 87.288424] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:08Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05560worker1 # [ 87.291207] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:08Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15561worker1 # [ 87.334959] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570585562worker1 # [ 87.343554] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570585563client # copying 1 paths to 'lb:50051'...5564client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5565worker1 # [ 87.351822] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570585566worker1 # [ 87.358643] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5567worker1 # [ 87.371872] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:08Z level=info event=niks3_push_started pid=11205568worker1 # [ 87.551678] nix-grpc-daemon[1120]: time=2026-09-20T12:44:08.688Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5569worker1 # [ 87.713601] nix-daemon[844]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5570worker1 # [ 87.728367] nix-daemon[844]: reaped child process 1136, status = succeeded5571worker1 # [ 87.881437] nix-daemon[844]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)5572worker1 # [ 87.899286] nix-daemon[844]: reaped child process 1141, status = succeeded5573lb # [ 87.933333] niks3-server[1069]: 2026/09/20 12:44:08 INFO Received uploads request method=POST path=/api/pending_closures5574worker1 # [ 87.923213] nix-grpc-daemon[1120]: time=2026-09-20T12:44:09.060Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5575worker1 # [ 87.925829] nix-grpc-daemon[1120]: time=2026-09-20T12:44:09.062Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5576lb # [ 87.967417] niks3-server[1069]: 2026/09/20 12:44:08 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5577lb # [ 87.972825] niks3-server[1069]: 2026/09/20 12:44:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5578lb # [ 87.978073] niks3-server[1069]: 2026/09/20 12:44:08 INFO Signed narinfos id=27 count=15579worker1 # [ 87.959757] nix-grpc-daemon[1120]: time=2026-09-20T12:44:09.096Z level=INFO msg="Uploading 1 narinfos"5580lb # [ 87.992958] niks3-server[1069]: 2026/09/20 12:44:08 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5581lb # [ 87.996285] niks3-server[1069]: 2026/09/20 12:44:08 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5582lb # [ 88.005413] niks3-server[1069]: 2026/09/20 12:44:08 INFO Completed upload id=275583worker1 # [ 87.987425] nix-grpc-daemon[1120]: time=2026-09-20T12:44:09.124Z level=INFO msg="Upload complete. (431ms)"5584worker1 # [ 87.990098] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57058 duration_s=0 paths=1 nar_bytes_in=5205585lb # [ 88.010452] niks3-server[1069]: 2026/09/20 12:44:08 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5586worker1 # [ 87.995977] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570585587worker1 # [ 88.006062] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5588worker1 # [ 88.010079] nix-daemon[844]: reaped child process 1118, status = succeeded5589client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5590client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5591worker1 # [ 88.018715] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5592lb # [ 88.047625] envoy[921]: conn peer=[2001:db8:1::1]:51200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=281 ms=12735593lb # [ 88.050902] envoy[921]: conn peer=[2001:db8:1::4]:41494 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4478 tx=349 ms=70355594client # farm-a> LOG-farm-a-on-standby5595worker1 # [ 88.844541] nix-daemon[844]: reaped child process 1145, status = succeeded5596worker1 # [ 88.987694] nix-daemon[844]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5597worker1 # [ 89.001388] nix-daemon[844]: reaped child process 1163, status = succeeded5598lb # [ 89.034268] niks3-server[1069]: 2026/09/20 12:44:09 INFO Received uploads request method=POST path=/api/pending_closures5599worker1 # [ 89.023514] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5600worker1 # [ 89.027559] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.162Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5601lb # [ 89.063815] niks3-server[1069]: 2026/09/20 12:44:09 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5602lb # [ 89.074440] niks3-server[1069]: 2026/09/20 12:44:09 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5603lb # [ 89.083178] niks3-server[1069]: 2026/09/20 12:44:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5604lb # [ 89.087645] niks3-server[1069]: 2026/09/20 12:44:09 INFO Signed narinfos id=28 count=15605worker1 # [ 89.070426] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.206Z level=INFO msg="Uploading 1 narinfos"5606lb # [ 89.090083] niks3-server[1069]: 2026/09/20 12:44:09 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5607lb # [ 89.101519] niks3-server[1069]: 2026/09/20 12:44:09 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5608lb # [ 89.105243] niks3-server[1069]: 2026/09/20 12:44:09 INFO Completed upload id=285609lb # [ 89.107735] niks3-server[1069]: 2026/09/20 12:44:09 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5610worker1 # [ 89.089290] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.226Z level=INFO msg="Upload complete. (257ms)"5611worker1 # [ 89.100790] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57058 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15612worker1 # [ 89.114154] nix-daemon[844]: reaped child process 1143, status = succeeded5613worker1 # [ 89.116916] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:10Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05614worker1 # [ 89.120514] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:10Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15615worker1 # [ 89.162962] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570585616worker1 # [ 89.172803] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570585617worker1 # [ 89.182685] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.319Z level=INFO msg="All 1 paths already cached"5618client # copying 1 paths to 'lb:50051'...5619client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5620worker1 # [ 89.189201] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570585621worker1 # [ 89.197107] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5622worker1 # [ 89.344747] nix-daemon[844]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5623worker1 # [ 89.361945] nix-daemon[844]: reaped child process 1172, status = succeeded5624worker1 # [ 89.516312] nix-daemon[844]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5625worker1 # [ 89.532501] nix-daemon[844]: reaped child process 1177, status = succeeded5626lb # [ 89.561212] niks3-server[1069]: 2026/09/20 12:44:09 INFO Received uploads request method=POST path=/api/pending_closures5627worker1 # [ 89.548318] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.685Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5628worker1 # [ 89.550241] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.687Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5629lb # [ 89.586083] niks3-server[1069]: 2026/09/20 12:44:09 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5630lb # [ 89.591864] niks3-server[1069]: 2026/09/20 12:44:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5631lb # [ 89.596355] niks3-server[1069]: 2026/09/20 12:44:09 INFO Signed narinfos id=29 count=15632worker1 # [ 89.578525] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.715Z level=INFO msg="Uploading 1 narinfos"5633lb # [ 89.598656] niks3-server[1069]: 2026/09/20 12:44:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5634lb # [ 89.608488] niks3-server[1069]: 2026/09/20 12:44:09 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5635lb # [ 89.612524] niks3-server[1069]: 2026/09/20 12:44:09 INFO Completed upload id=295636worker1 # [ 89.594609] nix-grpc-daemon[1120]: time=2026-09-20T12:44:10.731Z level=INFO msg="Upload complete. (388ms)"5637lb # [ 89.616430] niks3-server[1069]: 2026/09/20 12:44:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5638worker1 # [ 89.598144] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57058 duration_s=0 paths=1 nar_bytes_in=6405639worker1 # [ 89.601400] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570585640worker1 # [ 89.610890] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5641worker1 # [ 89.615408] nix-daemon[844]: reaped child process 1167, status = succeeded5642client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5643worker1 # [ 89.623221] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5644client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5645client # farm-b> LOG-farm-b-on-standby5646worker1 # [ 90.844431] nix-daemon[844]: reaped child process 1181, status = succeeded5647worker1 # [ 90.974302] nix-daemon[844]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5648worker1 # [ 90.989266] nix-daemon[844]: reaped child process 1199, status = succeeded5649lb # [ 91.018547] niks3-server[1069]: 2026/09/20 12:44:11 INFO Received uploads request method=POST path=/api/pending_closures5650worker1 # [ 91.006463] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.143Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5651worker1 # [ 91.008089] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.145Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5652lb # [ 91.042955] niks3-server[1069]: 2026/09/20 12:44:11 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5653lb # [ 91.046100] niks3-server[1069]: 2026/09/20 12:44:11 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5654lb # [ 91.049224] niks3-server[1069]: 2026/09/20 12:44:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5655lb # [ 91.051404] niks3-server[1069]: 2026/09/20 12:44:11 INFO Signed narinfos id=30 count=15656worker1 # [ 91.033430] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.170Z level=INFO msg="Uploading 1 narinfos"5657lb # [ 91.056620] niks3-server[1069]: 2026/09/20 12:44:11 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5658lb # [ 91.061530] niks3-server[1069]: 2026/09/20 12:44:11 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5659lb # [ 91.066791] niks3-server[1069]: 2026/09/20 12:44:11 INFO Completed upload id=305660worker1 # [ 91.048709] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.185Z level=INFO msg="Upload complete. (214ms)"5661lb # [ 91.068185] niks3-server[1069]: 2026/09/20 12:44:11 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5662worker1 # [ 91.051396] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57058 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15663worker1 # [ 91.058600] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:12Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05664worker1 # [ 91.060889] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:12Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15665worker1 # [ 91.065933] nix-daemon[844]: reaped child process 1179, status = succeeded5666worker1 # [ 91.102393] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570585667worker1 # [ 91.110572] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570585668worker1 # [ 91.118550] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.255Z level=INFO msg="All 2 paths already cached"5669client # copying 1 paths to 'lb:50051'...5670client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5671worker1 # [ 91.124348] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570585672worker1 # [ 91.130098] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5673worker1 # [ 91.318792] nix-daemon[844]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5674worker1 # [ 91.335979] nix-daemon[844]: reaped child process 1208, status = succeeded5675worker1 # [ 91.469385] nix-daemon[844]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)5676worker1 # [ 91.485209] nix-daemon[844]: reaped child process 1213, status = succeeded5677lb # [ 91.513214] niks3-server[1069]: 2026/09/20 12:44:11 INFO Received uploads request method=POST path=/api/pending_closures5678worker1 # [ 91.504502] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.641Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5679worker1 # [ 91.505985] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.641Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5680lb # [ 91.538752] niks3-server[1069]: 2026/09/20 12:44:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5681lb # [ 91.541219] niks3-server[1069]: 2026/09/20 12:44:11 INFO Signed narinfos id=31 count=15682worker1 # [ 91.523306] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.660Z level=INFO msg="Uploading 1 narinfos"5683lb # [ 91.545525] niks3-server[1069]: 2026/09/20 12:44:11 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5684lb # [ 91.550875] niks3-server[1069]: 2026/09/20 12:44:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5685lb # [ 91.554263] niks3-server[1069]: 2026/09/20 12:44:11 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5686lb # [ 91.557077] niks3-server[1069]: 2026/09/20 12:44:11 INFO Completed upload id=315687worker1 # [ 91.539094] nix-grpc-daemon[1120]: time=2026-09-20T12:44:12.676Z level=INFO msg="Upload complete. (402ms)"5688lb # [ 91.560177] niks3-server[1069]: 2026/09/20 12:44:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5689worker1 # [ 91.542118] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57058 duration_s=0 paths=1 nar_bytes_in=7685690worker1 # [ 91.544964] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570585691worker1 # [ 91.554099] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5692worker1 # [ 91.560826] nix-daemon[844]: reaped child process 1203, status = succeeded5693client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5694worker1 # [ 91.568138] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5695client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5696client # farm-top> LOG-farm-top-on-standby5697worker1 # [ 92.787555] nix-daemon[844]: reaped child process 1217, status = succeeded5698worker1 # [ 92.929110] nix-daemon[844]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5699worker1 # [ 92.944197] nix-daemon[844]: reaped child process 1235, status = succeeded5700lb # [ 92.972281] niks3-server[1069]: 2026/09/20 12:44:13 INFO Received uploads request method=POST path=/api/pending_closures5701worker1 # [ 92.959401] nix-grpc-daemon[1120]: time=2026-09-20T12:44:14.096Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5702worker1 # [ 92.964636] nix-grpc-daemon[1120]: time=2026-09-20T12:44:14.098Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5703lb # [ 92.992707] niks3-server[1069]: 2026/09/20 12:44:13 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5704lb # [ 92.996266] niks3-server[1069]: 2026/09/20 12:44:13 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5705lb # [ 93.001060] niks3-server[1069]: 2026/09/20 12:44:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5706lb # [ 93.002783] niks3-server[1069]: 2026/09/20 12:44:13 INFO Signed narinfos id=32 count=15707worker1 # [ 92.986346] nix-grpc-daemon[1120]: time=2026-09-20T12:44:14.123Z level=INFO msg="Uploading 1 narinfos"5708lb # [ 93.007380] niks3-server[1069]: 2026/09/20 12:44:13 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5709lb # [ 93.011883] niks3-server[1069]: 2026/09/20 12:44:13 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5710lb # [ 93.016094] niks3-server[1069]: 2026/09/20 12:44:13 INFO Completed upload id=325711lb # [ 93.017506] niks3-server[1069]: 2026/09/20 12:44:13 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5712worker1 # [ 92.999302] nix-grpc-daemon[1120]: time=2026-09-20T12:44:14.136Z level=INFO msg="Upload complete. (221ms)"5713worker1 # [ 93.003421] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57058 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15714worker1 # [ 93.013222] nix-daemon[844]: reaped child process 1215, status = succeeded5715client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.95 seconds)5716client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5717client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5718worker2 # [ 93.249317] systemd[1]: Listening on Nix worker-protocol over gRPC.5719worker2 # [ 93.254427] systemd[1]: Starting scheduler-token.service...5720worker2 # [ 93.316825] systemd[1]: scheduler-token.service: Deactivated successfully.5721worker2 # [ 93.318094] systemd[1]: Finished scheduler-token.service.5722worker2 # [ 93.319851] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 62ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5723worker2 # [ 93.326104] systemd[1]: Starting Nix worker-protocol over gRPC...5724worker2 # [ 93.465374] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:14Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5725worker2 # [ 93.468934] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5726worker2 # [ 93.472097] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15727worker2 # [ 93.478072] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:14Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5728worker2 # [ 93.484539] systemd[1]: Started Nix worker-protocol over gRPC.5729worker2 # [ 93.495941] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:14Z level=info event=scheduler_connected addr=lb:500515730lb: must succeed: curl -sf localhost:9901/clusters5731worker1 # [ 93.536125] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:14Z level=info event=oidc_jwks provider=mock keys=15732worker1 # [ 93.538797] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:14Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:570425733worker1 # [ 93.540963] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:14Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05734lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5735worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5736worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5737lb: must succeed: curl -sf localhost:9901/clusters5738lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5739lb: must succeed: curl -sf localhost:9901/clusters5740lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5741worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5742worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5743(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.53 seconds)5744subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5745client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5746client # [ 93.793754] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5747client # Running as unit: rs.service; invocation ID: b027647cd5744f0584deae931d085da45748client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.05 seconds)5749worker1 # [ 94.015256] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570585750client # [ 94.014885] nix[1174]: don't know how to build these paths:5751client # [ 94.016574] nix[1174]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5752worker1 # [ 94.027470] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:15Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05753worker1 # [ 94.030566] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:15Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15754worker1 # [ 94.071302] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570585755worker1 # [ 94.079984] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570585756client # [ 94.082186] nix[1174]: copying 1 paths to 'lb:50051'...5757client # [ 94.083060] nix[1174]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5758worker1 # [ 94.087326] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570585759worker1 # [ 94.095722] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5760worker1 # [ 94.291237] nix-daemon[844]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)5761worker1 # [ 94.314630] nix-daemon[844]: reaped child process 1251, status = succeeded5762worker1 # [ 94.490515] nix-daemon[844]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5763worker1 # [ 94.506363] nix-daemon[844]: reaped child process 1256, status = succeeded5764lb # [ 94.537472] niks3-server[1069]: 2026/09/20 12:44:14 INFO Received uploads request method=POST path=/api/pending_closures5765worker1 # [ 94.532058] nix-grpc-daemon[1120]: time=2026-09-20T12:44:15.668Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5766worker1 # [ 94.534392] nix-grpc-daemon[1120]: time=2026-09-20T12:44:15.670Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5767lb # [ 94.562903] niks3-server[1069]: 2026/09/20 12:44:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5768lb # [ 94.566985] niks3-server[1069]: 2026/09/20 12:44:14 INFO Signed narinfos id=33 count=15769worker1 # [ 94.549230] nix-grpc-daemon[1120]: time=2026-09-20T12:44:15.686Z level=INFO msg="Uploading 1 narinfos"5770lb # [ 94.572296] niks3-server[1069]: 2026/09/20 12:44:14 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5771lb # [ 94.575541] niks3-server[1069]: 2026/09/20 12:44:14 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5772lb # [ 94.581045] niks3-server[1069]: 2026/09/20 12:44:14 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5773lb # [ 94.584698] niks3-server[1069]: 2026/09/20 12:44:14 INFO Completed upload id=335774worker1 # [ 94.568094] nix-grpc-daemon[1120]: time=2026-09-20T12:44:15.705Z level=INFO msg="Upload complete. (464ms)"5775lb # [ 94.587327] niks3-server[1069]: 2026/09/20 12:44:14 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5776worker1 # [ 94.571093] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57058 duration_s=0 paths=1 nar_bytes_in=4165777worker1 # [ 94.574089] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570585778worker1 # [ 94.583739] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5779worker1 # [ 94.585739] nix-daemon[844]: reaped child process 1246, status = succeeded5780worker1 # [ 94.591646] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5781client # [ 94.586745] nix[1174]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5782lb: must succeed: curl -sf localhost:9901/clusters5783lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5784lb: must succeed: curl -sf localhost:9901/clusters5785lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5786lb: must succeed: curl -sf localhost:9901/clusters5787lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5788lb: must succeed: curl -sf localhost:9901/clusters5789lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5790worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5791worker2 # [ 97.594790] systemd-journald[286]: Received client request to rotate journal, rotating.5792worker2 # [ 97.607213] systemd-journald[286]: Vacuuming done, freed 0B of archived journals from /var/log/journal/164135244615496691440f92e0c1b55c.5793worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.04 seconds)5794worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5795worker1 # [ 97.550836] systemd[1]: Stopping Nix worker-protocol over gRPC...5796worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5797client: must succeed: sleep 5.05798lb # [ 98.047688] envoy[921]: conn peer=[2001:db8:1::1]:35480 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9577 tx=3819 ms=67255799worker1 # [ 98.095611] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:19Z level=info event=draining5800client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5801worker1 # [ 102.818182] nix-daemon[844]: reaped child process 1260, status = succeeded5802worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5803worker1 # [ 102.997987] nix-daemon[844]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5804worker1 # [ 103.020410] nix-daemon[844]: reaped child process 1307, status = succeeded5805lb # [ 103.052663] niks3-server[1069]: 2026/09/20 12:44:23 INFO Received uploads request method=POST path=/api/pending_closures5806worker1 # [ 103.045881] nix-grpc-daemon[1120]: time=2026-09-20T12:44:24.182Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5807worker1 # [ 103.055446] nix-grpc-daemon[1120]: time=2026-09-20T12:44:24.185Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5808lb # [ 103.096472] niks3-server[1069]: 2026/09/20 12:44:23 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5809lb # [ 103.104151] niks3-server[1069]: 2026/09/20 12:44:23 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5810lb # [ 103.106033] niks3-server[1069]: 2026/09/20 12:44:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5811lb # [ 103.109283] niks3-server[1069]: 2026/09/20 12:44:23 INFO Signed narinfos id=34 count=15812worker1 # [ 103.092064] nix-grpc-daemon[1120]: time=2026-09-20T12:44:24.228Z level=INFO msg="Uploading 1 narinfos"5813lb # [ 103.114387] niks3-server[1069]: 2026/09/20 12:44:23 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5814lb # [ 103.120943] niks3-server[1069]: 2026/09/20 12:44:23 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5815lb # [ 103.127389] niks3-server[1069]: 2026/09/20 12:44:23 INFO Completed upload id=345816worker1 # [ 103.109529] nix-grpc-daemon[1120]: time=2026-09-20T12:44:24.246Z level=INFO msg="Upload complete. (301ms)"5817lb # [ 103.130781] niks3-server[1069]: 2026/09/20 12:44:23 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5818worker1 # [ 103.115171] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57058 duration_s=8 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15819worker1 # [ 103.125158] nix-daemon[844]: reaped child process 1258, status = succeeded5820client # [ 103.144169] systemd[1]: rs.service: Deactivated successfully.5821client # [ 103.145966] systemd[1]: rs.service: Consumed 118ms CPU time over 9.351s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.5822worker1 # [ 104.097394] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=signal_exit5823worker1 # [ 104.317696] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=scheduler_yield5824worker1 # [ 104.319703] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=scheduler_restarting addr=lb:500515825lb # [ 104.338416] niks3-server[1069]: 2026/09/20 12:44:24 INFO lead: released remote=192.168.1.3:592665826worker2 # [ 104.285361] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:25Z level=info event=scheduler_restarting addr=lb:500515827worker1 # [ 104.322837] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570425828worker1 # [ 104.324342] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=worker_gone addr=192.168.1.4:500515829worker1 # [ 104.325895] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570425830worker1 # [ 104.327333] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=worker_gone addr=192.168.1.3:500515831worker1 # [ 104.520461] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=scheduler_connected addr=lb:500515832worker2 # [ 104.485902] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:25Z level=info event=scheduler_connected addr=lb:500515833worker1 # [ 104.522782] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5834worker2 # [ 104.488764] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5835worker1 # [ 104.723801] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=scheduler_connected addr=lb:500515836worker2 # [ 104.689633] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:25Z level=info event=scheduler_connected addr=lb:500515837worker1 # [ 104.726040] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5838worker2 # [ 104.692208] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5839worker1 # [ 104.926598] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:26Z level=info event=scheduler_connected addr=lb:500515840worker1 # [ 104.929156] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5841worker2 # [ 104.894588] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5842worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.16 seconds)5843client: waiting for success: ! systemctl is-active rs5844client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)5845client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5846client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5847client: must fail: journalctl -u rs | grep -q 'reconnecting'5848client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5849lb: must succeed: curl -sf localhost:9901/clusters5850worker1 # [ 105.129550] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:26Z level=info event=scheduler_connected addr=lb:500515851worker1 # [ 105.131523] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5852lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5853worker1 # [ 105.320521] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:26Z level=info event=scheduler_restarting5854worker2 # [ 105.296296] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:26Z level=info event=scheduler_connected addr=lb:500515855worker2 # [ 105.298537] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5856worker1 # [ 105.338619] nix-grpc-daemon[1047]: ts=2026-09-20T12:44:26Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5857worker1 # [ 105.345190] nix-daemon[844]: reaped child process 1111, status = succeeded5858worker2 # [ 105.499452] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:26Z level=info event=scheduler_connected addr=lb:500515859worker2 # [ 105.501663] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5860worker1 # [ 105.653423] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5861worker1 # [ 105.655926] systemd[1]: Stopped Nix worker-protocol over gRPC.5862worker1 # [ 105.658592] systemd[1]: nix-grpc-daemon.service: Consumed 1.541s CPU time over 25.736s wall clock time, 33.5M memory peak, 49.2K incoming IP traffic, 91.5K outgoing IP traffic.5863worker1 # [ 105.668611] systemd[1]: Starting Nix worker-protocol over gRPC...5864worker2 # [ 105.702985] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:26Z level=info event=scheduler_connected addr=lb:500515865worker2 # [ 105.705955] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5866worker1 # [ 105.864520] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5867worker1 # [ 105.867541] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5868worker1 # [ 105.872578] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15869worker1 # [ 105.882106] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5870worker1 # [ 105.889114] systemd[1]: Started Nix worker-protocol over gRPC.5871lb # [ 105.910357] niks3-server[1069]: 2026/09/20 12:44:26 INFO lead: acquired remote=192.168.1.3:361345872worker1 # [ 105.891973] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=scheduler_take_over5873worker1 # [ 105.901168] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=scheduler_connected addr=lb:500515874worker1 # [ 105.903463] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5875worker2 # [ 105.906881] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:26Z level=info event=scheduler_connected addr=lb:500515876worker2 # [ 105.909164] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5877worker1 # [ 106.102641] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=scheduler_connected addr=lb:500515878worker1 # [ 106.106929] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5879worker2 # [ 106.110414] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:27Z level=info event=scheduler_connected addr=lb:500515880worker2 # [ 106.113113] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5881lb: must succeed: curl -sf localhost:9901/clusters5882lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5883worker1 # [ 106.308177] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=scheduler_connected addr=lb:500515884worker1 # [ 106.314105] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:513125885worker1 # [ 106.316487] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05886worker2 # [ 106.314864] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:27Z level=info event=scheduler_connected addr=lb:500515887worker1 # [ 106.356149] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=oidc_jwks provider=mock keys=15888worker1 # [ 106.358352] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:513125889worker1 # [ 106.361123] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05890lb: must succeed: curl -sf localhost:9901/clusters5891lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5892worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5893worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5894(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.55 seconds)5895subtest: scheduler restart mid-build: build finishes, no double build5896client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5897client # [ 107.352589] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5898client # Running as unit: mid.service; invocation ID: acdd27d1b2584098918c4c42f0cf44ca5899client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.06 seconds)5900worker2 # [ 107.570271] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502385901worker2 # [ 107.578465] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)5902client # [ 107.618726] nix[1210]: don't know how to build these paths:5903client # [ 107.619754] nix[1210]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5904worker1 # [ 107.632507] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:28Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05905worker1 # [ 107.634329] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:28Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15906worker1 # [ 107.679203] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513165907worker1 # [ 107.685705] nix-daemon[844]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)5908worker1 # [ 107.695595] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 1/5)5909worker1 # [ 107.755642] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 2/5)5910worker1 # [ 107.779547] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 3/5)5911worker1 # [ 107.930437] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 244 ms (attempt 4/5)5912lb # [ 108.047946] envoy[921]: conn peer=[2001:db8:1::1]:48618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2917 tx=1328 ms=91135913lb # [ 108.050104] envoy[921]: conn peer=[2001:db8:1::3]:36058 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5568 tx=1292 ms=252335914worker1 # [ 108.178437] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5915worker1 # [ 108.190302] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:513165916client # [ 108.193330] nix[1210]: copying 1 paths to 'lb:50051'...5917client # [ 108.196124] nix[1210]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5918worker1 # [ 108.201120] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:513165919worker1 # [ 108.206711] nix-daemon[844]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)5920worker1 # [ 108.222923] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:29Z level=info event=niks3_push_started pid=13795921worker1 # [ 108.406242] nix-grpc-daemon[1379]: time=2026-09-20T12:44:29.542Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5922worker1 # [ 108.542825] nix-daemon[844]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)5923worker1 # [ 108.561838] nix-daemon[844]: reaped child process 1399, status = succeeded5924worker1 # [ 108.718625] nix-daemon[844]: accepted connection from pid 1401, user nix-grpc-daemon (trusted)5925worker1 # [ 108.733683] nix-daemon[844]: reaped child process 1404, status = succeeded5926lb # [ 108.764324] niks3-server[1069]: 2026/09/20 12:44:29 INFO Received uploads request method=POST path=/api/pending_closures5927worker1 # [ 108.755405] nix-grpc-daemon[1379]: time=2026-09-20T12:44:29.892Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5928worker1 # [ 108.757538] nix-grpc-daemon[1379]: time=2026-09-20T12:44:29.894Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5929lb # [ 108.791580] niks3-server[1069]: 2026/09/20 12:44:29 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5930lb # [ 108.796172] niks3-server[1069]: 2026/09/20 12:44:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5931lb # [ 108.800246] niks3-server[1069]: 2026/09/20 12:44:29 INFO Signed narinfos id=35 count=15932worker1 # [ 108.782548] nix-grpc-daemon[1379]: time=2026-09-20T12:44:29.919Z level=INFO msg="Uploading 1 narinfos"5933lb # [ 108.812354] niks3-server[1069]: 2026/09/20 12:44:29 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5934lb # [ 108.818519] niks3-server[1069]: 2026/09/20 12:44:29 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5935lb # [ 108.831634] niks3-server[1069]: 2026/09/20 12:44:29 INFO Completed upload id=355936worker1 # [ 108.813863] nix-grpc-daemon[1379]: time=2026-09-20T12:44:29.950Z level=INFO msg="Upload complete. (403ms)"5937worker1 # [ 108.817375] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51316 duration_s=0 paths=1 nar_bytes_in=4245938lb # [ 108.837740] niks3-server[1069]: 2026/09/20 12:44:29 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5939worker1 # [ 108.820679] nix-grpc-daemon[1314]: ts=2026-09-20T12:44:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513165940worker1 # [ 108.826730] nix-daemon[844]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)5941worker1 # [ 108.832526] nix-daemon[844]: reaped child process 1377, status = succeeded5942client # [ 108.834206] nix[1210]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5943worker1 # [ 108.840097] nix-daemon[844]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)5944lb: must succeed: curl -sf localhost:9901/clusters5945lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5946lb: must succeed: curl -sf localhost:9901/clusters5947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5948worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5949worker1 # [ 109.945081] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1314 (nix-grpc-daemon) on client request.5950worker1 # [ 109.948776] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5951lb # [ 109.976753] niks3-server[1069]: 2026/09/20 12:44:30 INFO lead: released remote=192.168.1.3:361345952worker2 # [ 109.930476] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:30Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5953client # [ 109.962292] nix[1210]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5954client # [ 109.965245] nix[1210]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5955worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5956client: must succeed: sleep 5.05957worker1 # [ 109.977327] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5958worker1 # [ 109.979237] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5959worker1 # [ 109.980855] systemd[1]: nix-grpc-daemon.service: Consumed 463ms CPU time over 4.308s wall clock time, 30.6M memory peak, 14K incoming IP traffic, 16.5K outgoing IP traffic.5960worker1 # [ 109.995424] nix-daemon[844]: reaped child process 1372, status = succeeded5961worker1 # [ 109.996751] nix-daemon[844]: reaped child process 1406, status = succeeded5962worker1 # [ 110.007348] nix-daemon[1408]: unexpected Nix daemon error: error: interrupted by the user5963worker1 # [ 110.012293] nix-daemon[844]: reaped child process 1408, status = failed with exit code 15964worker1 # [ 110.098375] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5965worker1 # [ 110.104513] systemd[1]: Starting Nix worker-protocol over gRPC...5966worker2 # [ 110.134424] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:31Z level=info event=scheduler_connected addr=lb:500515967worker1 # [ 110.231208] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5968worker1 # [ 110.233701] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5969worker1 # [ 110.237085] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15970worker1 # [ 110.244471] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5971worker1 # [ 110.247751] systemd[1]: Started Nix worker-protocol over gRPC.5972lb # [ 110.272438] niks3-server[1069]: 2026/09/20 12:44:30 INFO lead: acquired remote=192.168.1.3:362005973worker1 # [ 110.254754] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=scheduler_take_over5974worker2 # [ 110.222426] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5975worker1 # [ 110.262354] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:513225976worker1 # [ 110.264288] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=scheduler_connected addr=lb:500515977worker1 # [ 110.265853] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05978worker2 # [ 110.423541] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:31Z level=info event=scheduler_connected addr=lb:500515979worker1 # [ 110.465182] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=oidc_jwks provider=mock keys=15980worker1 # [ 110.467116] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:513225981worker1 # [ 110.469449] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05982worker1 # [ 110.478970] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05983worker1 # [ 110.482293] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15984worker1 # [ 110.526201] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513505985worker1 # [ 110.534136] nix-daemon[844]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)5986worker1 # [ 110.543590] nix-daemon[844]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)5987client # [ 110.548891] nix[1210]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5988worker1 # [ 110.554454] nix-daemon[844]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)5989client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5990client: waiting for success: ! systemctl is-active mid5991worker1 # [ 115.373836] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:36Z level=info event=niks3_push_started pid=15195992worker1 # [ 115.386068] nix-daemon[844]: reaped child process 1495, status = succeeded5993worker1 # [ 115.522520] nix-grpc-daemon[1519]: time=2026-09-20T12:44:36.658Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5994worker1 # [ 115.649336] nix-daemon[844]: accepted connection from pid 1532, user nix-grpc-daemon (trusted)5995worker1 # [ 115.663914] nix-daemon[844]: reaped child process 1535, status = succeeded5996lb # [ 115.691591] niks3-server[1069]: 2026/09/20 12:44:35 INFO Received uploads request method=POST path=/api/pending_closures5997worker1 # [ 115.679735] nix-grpc-daemon[1519]: time=2026-09-20T12:44:36.816Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5998worker1 # [ 115.681285] nix-grpc-daemon[1519]: time=2026-09-20T12:44:36.818Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5999lb # [ 115.721672] niks3-server[1069]: 2026/09/20 12:44:36 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6000lb # [ 115.732094] niks3-server[1069]: 2026/09/20 12:44:36 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6001lb # [ 115.734299] niks3-server[1069]: 2026/09/20 12:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6002lb # [ 115.737462] niks3-server[1069]: 2026/09/20 12:44:36 INFO Signed narinfos id=36 count=16003worker1 # [ 115.720992] nix-grpc-daemon[1519]: time=2026-09-20T12:44:36.858Z level=INFO msg="Uploading 1 narinfos"6004lb # [ 115.743379] niks3-server[1069]: 2026/09/20 12:44:36 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6005lb # [ 115.751721] niks3-server[1069]: 2026/09/20 12:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6006lb # [ 115.755627] niks3-server[1069]: 2026/09/20 12:44:36 INFO Completed upload id=366007worker1 # [ 115.739561] nix-grpc-daemon[1519]: time=2026-09-20T12:44:36.876Z level=INFO msg="Upload complete. (214ms)"6008lb # [ 115.758430] niks3-server[1069]: 2026/09/20 12:44:36 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6009worker1 # [ 115.742365] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51350 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16010worker1 # [ 115.752588] nix-daemon[844]: reaped child process 1493, status = succeeded6011client # [ 115.773326] systemd[1]: mid.service: Deactivated successfully.6012client # [ 115.775801] systemd[1]: mid.service: Consumed 162ms CPU time over 8.422s wall clock time, 37.3M memory peak, 9.3K incoming IP traffic, 10.6K outgoing IP traffic.6013client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6014client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6015client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6016(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.01 seconds)6017subtest: interrupting the client stops the build on the worker6018client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6019client # [ 116.348635] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6020client # Running as unit: intr.service; invocation ID: 7e866b7dd9e24a9a971c388c8efa159b6021client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6022worker2 # [ 116.491520] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502386023client # [ 116.525657] nix[1246]: don't know how to build these paths:6024client # [ 116.526606] nix[1246]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6025worker1 # [ 116.535165] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:37Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06026worker1 # [ 116.537132] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:37Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16027worker2 # [ 116.544512] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386028worker2 # [ 116.551985] nix-grpc-daemon[1425]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5)6029worker2 # [ 116.623190] nix-grpc-daemon[1425]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 2/5)6030worker2 # [ 116.712931] nix-grpc-daemon[1425]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 231 ms (attempt 3/5)6031worker2 # [ 116.946764] nix-grpc-daemon[1425]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 382 ms (attempt 4/5)6032worker2 # [ 117.331883] nix-grpc-daemon[1425]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6033worker2 # [ 117.342885] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502386034client # [ 117.379119] nix[1246]: copying 1 paths to 'lb:50051'...6035client # [ 117.380199] nix[1246]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6036worker2 # [ 117.349250] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502386037worker2 # [ 117.357335] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6038worker2 # [ 117.373293] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:38Z level=info event=niks3_push_started pid=15636039worker2 # [ 117.557834] nix-grpc-daemon[1563]: time=2026-09-20T12:44:38.490Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6040worker2 # [ 117.738099] nix-daemon[924]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)6041worker2 # [ 117.751125] nix-daemon[924]: reaped child process 1583, status = succeeded6042worker2 # [ 117.900415] nix-daemon[924]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6043worker2 # [ 117.915925] nix-daemon[924]: reaped child process 1588, status = succeeded6044lb # [ 117.979379] niks3-server[1069]: 2026/09/20 12:44:38 INFO Received uploads request method=POST path=/api/pending_closures6045worker2 # [ 117.942514] nix-grpc-daemon[1563]: time=2026-09-20T12:44:38.875Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6046worker2 # [ 117.946572] nix-grpc-daemon[1563]: time=2026-09-20T12:44:38.877Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6047lb # [ 118.022334] niks3-server[1069]: 2026/09/20 12:44:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6048lb # [ 118.026104] niks3-server[1069]: 2026/09/20 12:44:38 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6049lb # [ 118.033831] niks3-server[1069]: 2026/09/20 12:44:38 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6050lb # [ 118.040703] niks3-server[1069]: 2026/09/20 12:44:38 INFO Signed narinfos id=37 count=16051worker2 # [ 117.988360] nix-grpc-daemon[1563]: time=2026-09-20T12:44:38.920Z level=INFO msg="Uploading 1 narinfos"6052lb # [ 118.046920] niks3-server[1069]: 2026/09/20 12:44:38 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6053lb # [ 118.049468] envoy[921]: conn peer=[2001:db8:1::3]:45048 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=937 tx=355 ms=40676054lb # [ 118.050989] envoy[921]: conn peer=[2001:db8:1::1]:52414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3848 tx=1635 ms=81586055lb # [ 118.059400] niks3-server[1069]: 2026/09/20 12:44:38 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6056lb # [ 118.062726] niks3-server[1069]: 2026/09/20 12:44:38 INFO Completed upload id=376057worker2 # [ 118.009785] nix-grpc-daemon[1563]: time=2026-09-20T12:44:38.942Z level=INFO msg="Upload complete. (447ms)"6058worker2 # [ 118.013386] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=0 paths=1 nar_bytes_in=4246059worker2 # [ 118.016145] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386060worker2 # [ 118.023084] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6061worker2 # [ 118.029180] nix-daemon[924]: reaped child process 1561, status = succeeded6062worker2 # [ 118.032939] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6063client # [ 118.063108] nix[1246]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6064client: must succeed: systemctl kill -s INT intr6065client # [ 119.799081] systemd[1]: intr.service: Sent signal SIGINT to main process 1246 (nix) on client request.6066worker2 # [ 119.775515] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:40Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6067client # [ 119.805757] nix[1246]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6068client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6069client # [ 119.812822] nix[1246]: error: interrupted by the user6070client # [ 119.843333] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6071client # [ 119.845056] systemd[1]: intr.service: Failed with result 'exit-code'.6072client # [ 119.846710] systemd[1]: intr.service: Consumed 111ms CPU time over 3.496s wall clock time, 37.1M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.6073worker2 # [ 119.857435] nix-daemon[924]: reaped child process 1590, status = succeeded6074worker2 # [ 119.863316] nix-daemon[1592]: unexpected Nix daemon error: error: interrupted by the user6075worker2 # [ 119.870546] nix-daemon[924]: reaped child process 1592, status = failed with exit code 16076(finished: subtest: interrupting the client stops the build on the worker, in 3.61 seconds)6077subtest: a deploy mid-build drains: the build finishes, the new generation takes over6078client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6079client # [ 119.959818] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6080client # Running as unit: sw.service; invocation ID: 4861c7dfb7c94b95951539c35091382f6081client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6082worker2 # [ 120.100689] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502386083client # [ 120.132631] nix[1267]: don't know how to build these paths:6084client # [ 120.133564] nix[1267]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6085worker1 # [ 120.144480] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:41Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06086worker1 # [ 120.147606] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:41Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16087worker1 # [ 120.189234] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513506088worker1 # [ 120.195934] nix-grpc-daemon[1440]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5)6089worker1 # [ 120.214455] nix-grpc-daemon[1440]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 2/5)6090worker1 # [ 120.310062] nix-grpc-daemon[1440]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 3/5)6091worker1 # [ 120.424362] nix-grpc-daemon[1440]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 622 ms (attempt 4/5)6092worker1 # [ 121.046930] nix-grpc-daemon[1440]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6093worker1 # [ 121.061830] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:513506094client # [ 121.064138] nix[1267]: copying 1 paths to 'lb:50051'...6095client # [ 121.065238] nix[1267]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6096worker1 # [ 121.069418] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:513506097worker1 # [ 121.078093] nix-daemon[844]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)6098worker1 # [ 121.247921] nix-daemon[844]: accepted connection from pid 1574, user nix-grpc-daemon (trusted)6099worker1 # [ 121.267860] nix-daemon[844]: reaped child process 1577, status = succeeded6100worker1 # [ 121.402656] nix-daemon[844]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6101worker1 # [ 121.416183] nix-daemon[844]: reaped child process 1582, status = succeeded6102lb # [ 121.444550] niks3-server[1069]: 2026/09/20 12:44:41 INFO Received uploads request method=POST path=/api/pending_closures6103worker1 # [ 121.433546] nix-grpc-daemon[1519]: time=2026-09-20T12:44:42.570Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6104worker1 # [ 121.435932] nix-grpc-daemon[1519]: time=2026-09-20T12:44:42.572Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6105lb # [ 121.464887] niks3-server[1069]: 2026/09/20 12:44:41 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6106lb # [ 121.468059] niks3-server[1069]: 2026/09/20 12:44:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6107lb # [ 121.470724] niks3-server[1069]: 2026/09/20 12:44:41 INFO Signed narinfos id=38 count=16108worker1 # [ 121.452941] nix-grpc-daemon[1519]: time=2026-09-20T12:44:42.589Z level=INFO msg="Uploading 1 narinfos"6109lb # [ 121.476410] niks3-server[1069]: 2026/09/20 12:44:41 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6110lb # [ 121.478424] niks3-server[1069]: 2026/09/20 12:44:41 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6111lb # [ 121.482685] niks3-server[1069]: 2026/09/20 12:44:41 INFO Completed upload id=386112worker1 # [ 121.464615] nix-grpc-daemon[1519]: time=2026-09-20T12:44:42.601Z level=INFO msg="Upload complete. (377ms)"6113lb # [ 121.485869] niks3-server[1069]: 2026/09/20 12:44:41 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6114worker1 # [ 121.468085] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51350 duration_s=0 paths=1 nar_bytes_in=4166115worker1 # [ 121.470574] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:513506116worker1 # [ 121.476631] nix-daemon[844]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)6117worker1 # [ 121.480316] nix-daemon[844]: reaped child process 1571, status = succeeded6118client # [ 121.481382] nix[1267]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6119worker1 # [ 121.486170] nix-daemon[844]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)6120worker1: must succeed: systemctl reload nix-grpc-daemon.service6121worker1 # [ 123.466434] systemd[1]: Reloading Nix worker-protocol over gRPC...6122worker1 # [ 123.504907] systemd[1]: Reloaded Nix worker-protocol over gRPC.6123worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6124worker1: must succeed: pgrep -f 'read -t [9]0 x'6125worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6126client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26127worker2 # [ 123.668859] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502386128client # don't know how to build these paths:6129client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6130worker1 # [ 123.710523] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:513506131worker1 # [ 123.725302] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:44Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06132worker1 # [ 123.727035] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:44Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16133worker2 # [ 123.737344] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386134worker2 # [ 123.745091] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502386135client # copying 1 paths to 'lb:50051'...6136worker2 # [ 123.752279] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502386137client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6138worker2 # [ 123.763245] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6139worker2 # [ 123.916227] nix-daemon[924]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6140worker2 # [ 123.932056] nix-daemon[924]: reaped child process 1645, status = succeeded6141worker2 # [ 124.051972] nix-daemon[924]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6142worker2 # [ 124.065955] nix-daemon[924]: reaped child process 1650, status = succeeded6143lb # [ 124.130319] niks3-server[1069]: 2026/09/20 12:44:44 INFO Received uploads request method=POST path=/api/pending_closures6144worker2 # [ 124.172631] nix-grpc-daemon[1563]: time=2026-09-20T12:44:45.105Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6145worker2 # [ 124.175278] nix-grpc-daemon[1563]: time=2026-09-20T12:44:45.107Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6146lb # [ 124.252148] niks3-server[1069]: 2026/09/20 12:44:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6147lb # [ 124.259479] niks3-server[1069]: 2026/09/20 12:44:44 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6148worker2 # [ 124.212490] nix-grpc-daemon[1563]: time=2026-09-20T12:44:45.145Z level=INFO msg="Uploading 1 narinfos"6149worker1 # [ 124.248357] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:45Z level=info event=draining6150lb # [ 124.265151] niks3-server[1069]: 2026/09/20 12:44:44 INFO Signed narinfos id=39 count=16151lb # [ 124.273879] niks3-server[1069]: 2026/09/20 12:44:44 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6152lb # [ 124.280480] niks3-server[1069]: 2026/09/20 12:44:44 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6153lb # [ 124.295849] niks3-server[1069]: 2026/09/20 12:44:44 INFO Completed upload id=396154worker2 # [ 124.242632] nix-grpc-daemon[1563]: time=2026-09-20T12:44:45.175Z level=INFO msg="Upload complete. (472ms)"6155worker2 # [ 124.244795] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=0 paths=1 nar_bytes_in=5286156worker2 # [ 124.247852] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386157lb # [ 124.302457] niks3-server[1069]: 2026/09/20 12:44:44 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6158worker2 # [ 124.253738] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6159worker2 # [ 124.261153] nix-daemon[924]: reaped child process 1640, status = succeeded6160client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6161client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6162worker2 # [ 124.264850] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6163client # farm-a> LOG-farm-a-during-drain6164worker2 # [ 125.682804] nix-daemon[924]: reaped child process 1654, status = succeeded6165worker2 # [ 125.811129] nix-daemon[924]: accepted connection from pid 1669, user nix-grpc-daemon (trusted)6166worker2 # [ 125.828608] nix-daemon[924]: reaped child process 1672, status = succeeded6167lb # [ 125.896401] niks3-server[1069]: 2026/09/20 12:44:46 INFO Received uploads request method=POST path=/api/pending_closures6168worker2 # [ 125.858321] nix-grpc-daemon[1563]: time=2026-09-20T12:44:46.791Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6169worker2 # [ 125.861907] nix-grpc-daemon[1563]: time=2026-09-20T12:44:46.792Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6170lb # [ 125.947104] niks3-server[1069]: 2026/09/20 12:44:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6171lb # [ 125.950948] niks3-server[1069]: 2026/09/20 12:44:46 INFO Signed narinfos id=40 count=16172worker2 # [ 125.901085] nix-grpc-daemon[1563]: time=2026-09-20T12:44:46.833Z level=INFO msg="Uploading 1 narinfos"6173lb # [ 125.956428] niks3-server[1069]: 2026/09/20 12:44:46 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6174lb # [ 125.967583] niks3-server[1069]: 2026/09/20 12:44:46 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6175lb # [ 125.973184] niks3-server[1069]: 2026/09/20 12:44:46 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6176lb # [ 125.976402] niks3-server[1069]: 2026/09/20 12:44:46 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6177lb # [ 125.979315] niks3-server[1069]: 2026/09/20 12:44:46 INFO Completed upload id=406178worker2 # [ 125.927458] nix-grpc-daemon[1563]: time=2026-09-20T12:44:46.860Z level=INFO msg="Upload complete. (254ms)"6179lb # [ 125.981218] niks3-server[1069]: 2026/09/20 12:44:46 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6180worker1 # [ 125.971031] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:47Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06181worker2 # [ 125.933324] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16182worker1 # [ 125.972852] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:47Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16183worker2 # [ 125.945223] nix-daemon[924]: reaped child process 1652, status = succeeded6184worker2 # [ 125.980817] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386185worker2 # [ 125.992708] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502386186worker2 # [ 126.006340] nix-grpc-daemon[1563]: time=2026-09-20T12:44:46.939Z level=INFO msg="All 1 paths already cached"6187client # copying 1 paths to 'lb:50051'...6188worker2 # [ 126.014661] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502386189client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6190worker2 # [ 126.035408] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6191worker2 # [ 126.184136] nix-daemon[924]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)6192worker2 # [ 126.202425] nix-daemon[924]: reaped child process 1679, status = succeeded6193worker2 # [ 126.327680] nix-daemon[924]: accepted connection from pid 1681, user nix-grpc-daemon (trusted)6194worker2 # [ 126.343583] nix-daemon[924]: reaped child process 1684, status = succeeded6195lb # [ 126.405114] niks3-server[1069]: 2026/09/20 12:44:46 INFO Received uploads request method=POST path=/api/pending_closures6196worker2 # [ 126.495290] nix-grpc-daemon[1563]: time=2026-09-20T12:44:47.427Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6197worker2 # [ 126.500430] nix-grpc-daemon[1563]: time=2026-09-20T12:44:47.427Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6198lb # [ 126.563807] niks3-server[1069]: 2026/09/20 12:44:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6199lb # [ 126.566556] niks3-server[1069]: 2026/09/20 12:44:46 INFO Signed narinfos id=41 count=16200worker2 # [ 126.514073] nix-grpc-daemon[1563]: time=2026-09-20T12:44:47.446Z level=INFO msg="Uploading 1 narinfos"6201lb # [ 126.571713] niks3-server[1069]: 2026/09/20 12:44:46 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6202lb # [ 126.574186] niks3-server[1069]: 2026/09/20 12:44:46 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6203lb # [ 126.581668] niks3-server[1069]: 2026/09/20 12:44:46 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6204lb # [ 126.589092] niks3-server[1069]: 2026/09/20 12:44:46 INFO Completed upload id=416205worker2 # [ 126.536340] nix-grpc-daemon[1563]: time=2026-09-20T12:44:47.469Z level=INFO msg="Upload complete. (485ms)"6206lb # [ 126.595721] niks3-server[1069]: 2026/09/20 12:44:46 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6207worker2 # [ 126.543169] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=0 paths=1 nar_bytes_in=6406208worker2 # [ 126.550216] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386209worker2 # [ 126.564172] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6210worker2 # [ 126.574445] nix-daemon[924]: reaped child process 1674, status = succeeded6211client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6212client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6213worker2 # [ 126.584546] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6214client # farm-b> LOG-farm-b-during-drain6215worker2 # [ 127.602799] nix-daemon[924]: reaped child process 1688, status = succeeded6216worker2 # [ 127.732981] nix-daemon[924]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)6217worker2 # [ 127.746072] nix-daemon[924]: reaped child process 1706, status = succeeded6218lb # [ 127.812626] niks3-server[1069]: 2026/09/20 12:44:48 INFO Received uploads request method=POST path=/api/pending_closures6219worker2 # [ 127.767732] nix-grpc-daemon[1563]: time=2026-09-20T12:44:48.700Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6220worker2 # [ 127.769286] nix-grpc-daemon[1563]: time=2026-09-20T12:44:48.702Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6221lb # [ 127.849396] niks3-server[1069]: 2026/09/20 12:44:48 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6222lb # [ 127.856693] niks3-server[1069]: 2026/09/20 12:44:48 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6223lb # [ 127.867401] niks3-server[1069]: 2026/09/20 12:44:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6224lb # [ 127.871808] niks3-server[1069]: 2026/09/20 12:44:48 INFO Signed narinfos id=42 count=16225worker2 # [ 127.822058] nix-grpc-daemon[1563]: time=2026-09-20T12:44:48.754Z level=INFO msg="Uploading 1 narinfos"6226lb # [ 127.881541] niks3-server[1069]: 2026/09/20 12:44:48 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6227lb # [ 127.890239] niks3-server[1069]: 2026/09/20 12:44:48 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6228lb # [ 127.895818] niks3-server[1069]: 2026/09/20 12:44:48 INFO Completed upload id=426229worker2 # [ 127.842892] nix-grpc-daemon[1563]: time=2026-09-20T12:44:48.776Z level=INFO msg="Upload complete. (247ms)"6230lb # [ 127.899629] niks3-server[1069]: 2026/09/20 12:44:48 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6231worker2 # [ 127.845513] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16232worker1 # [ 127.886771] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:49Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06233worker2 # [ 127.853752] nix-daemon[924]: reaped child process 1686, status = succeeded6234worker1 # [ 127.889229] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:49Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16235worker2 # [ 127.894870] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386236worker2 # [ 127.903140] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502386237worker2 # [ 127.912652] nix-grpc-daemon[1563]: time=2026-09-20T12:44:48.845Z level=INFO msg="All 2 paths already cached"6238client # copying 1 paths to 'lb:50051'...6239client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6240worker2 # [ 127.917783] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502386241worker2 # [ 127.926497] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6242lb # [ 128.049343] envoy[921]: conn peer=[2001:db8:1::1]:42810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=732 ms=33036243worker2 # [ 128.087130] nix-daemon[924]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6244worker2 # [ 128.100935] nix-daemon[924]: reaped child process 1713, status = succeeded6245worker2 # [ 128.250584] nix-daemon[924]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6246worker2 # [ 128.263760] nix-daemon[924]: reaped child process 1718, status = succeeded6247lb # [ 128.329956] niks3-server[1069]: 2026/09/20 12:44:48 INFO Received uploads request method=POST path=/api/pending_closures6248worker2 # [ 128.293179] nix-grpc-daemon[1563]: time=2026-09-20T12:44:49.226Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6249worker2 # [ 128.294819] nix-grpc-daemon[1563]: time=2026-09-20T12:44:49.227Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6250lb # [ 128.361091] niks3-server[1069]: 2026/09/20 12:44:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6251lb # [ 128.363491] niks3-server[1069]: 2026/09/20 12:44:48 INFO Signed narinfos id=43 count=16252worker2 # [ 128.311995] nix-grpc-daemon[1563]: time=2026-09-20T12:44:49.244Z level=INFO msg="Uploading 1 narinfos"6253lb # [ 128.371797] niks3-server[1069]: 2026/09/20 12:44:48 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6254lb # [ 128.374661] niks3-server[1069]: 2026/09/20 12:44:48 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6255lb # [ 128.385814] niks3-server[1069]: 2026/09/20 12:44:48 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6256lb # [ 128.407219] niks3-server[1069]: 2026/09/20 12:44:48 INFO Completed upload id=436257worker2 # [ 128.354399] nix-grpc-daemon[1563]: time=2026-09-20T12:44:49.287Z level=INFO msg="Upload complete. (422ms)"6258worker2 # [ 128.358140] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=0 paths=1 nar_bytes_in=7686259worker2 # [ 128.360663] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386260lb # [ 128.418507] niks3-server[1069]: 2026/09/20 12:44:48 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6261worker2 # [ 128.367593] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6262worker2 # [ 128.371284] nix-daemon[924]: reaped child process 1708, status = succeeded6263client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6264worker2 # [ 128.376842] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6265client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6266client # farm-top> LOG-farm-top-during-drain6267worker2 # [ 129.404650] nix-daemon[924]: reaped child process 1722, status = succeeded6268worker2 # [ 129.546280] nix-daemon[924]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6269worker2 # [ 129.561674] nix-daemon[924]: reaped child process 1740, status = succeeded6270lb # [ 129.626952] niks3-server[1069]: 2026/09/20 12:44:49 INFO Received uploads request method=POST path=/api/pending_closures6271worker2 # [ 129.582276] nix-grpc-daemon[1563]: time=2026-09-20T12:44:50.515Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6272worker2 # [ 129.583795] nix-grpc-daemon[1563]: time=2026-09-20T12:44:50.516Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6273lb # [ 129.668390] niks3-server[1069]: 2026/09/20 12:44:49 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6274lb # [ 129.675863] niks3-server[1069]: 2026/09/20 12:44:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6275lb # [ 129.679810] niks3-server[1069]: 2026/09/20 12:44:49 INFO Signed narinfos id=44 count=16276worker2 # [ 129.629498] nix-grpc-daemon[1563]: time=2026-09-20T12:44:50.562Z level=INFO msg="Uploading 1 narinfos"6277lb # [ 129.686300] niks3-server[1069]: 2026/09/20 12:44:49 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6278lb # [ 129.692586] niks3-server[1069]: 2026/09/20 12:44:49 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6279lb # [ 129.701209] niks3-server[1069]: 2026/09/20 12:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6280lb # [ 129.704810] niks3-server[1069]: 2026/09/20 12:44:50 INFO Completed upload id=446281worker2 # [ 129.651819] nix-grpc-daemon[1563]: time=2026-09-20T12:44:50.584Z level=INFO msg="Upload complete. (267ms)"6282lb # [ 129.706336] niks3-server[1069]: 2026/09/20 12:44:50 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6283worker2 # [ 129.654471] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16284worker2 # [ 129.661995] nix-daemon[924]: reaped child process 1720, status = succeeded6285client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.18 seconds)6286client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6287client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.14 seconds)6288worker1: must succeed: pkill -f 'read -t [9]0 x'6289worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6290client: waiting for success: ! systemctl is-active sw6291worker1 # [ 130.110275] nix-daemon[844]: reaped child process 1586, status = succeeded6292worker1 # [ 130.261774] nix-daemon[844]: accepted connection from pid 1633, user nix-grpc-daemon (trusted)6293worker1 # [ 130.280120] nix-daemon[844]: reaped child process 1636, status = succeeded6294lb # [ 130.306679] niks3-server[1069]: 2026/09/20 12:44:50 INFO Received uploads request method=POST path=/api/pending_closures6295worker1 # [ 130.294734] nix-grpc-daemon[1519]: time=2026-09-20T12:44:51.431Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6296worker1 # [ 130.299437] nix-grpc-daemon[1519]: time=2026-09-20T12:44:51.434Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6297lb # [ 130.331344] niks3-server[1069]: 2026/09/20 12:44:50 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6298lb # [ 130.334897] niks3-server[1069]: 2026/09/20 12:44:50 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6299lb # [ 130.337293] niks3-server[1069]: 2026/09/20 12:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6300lb # [ 130.339308] niks3-server[1069]: 2026/09/20 12:44:50 INFO Signed narinfos id=45 count=16301worker1 # [ 130.323135] nix-grpc-daemon[1519]: time=2026-09-20T12:44:51.459Z level=INFO msg="Uploading 1 narinfos"6302lb # [ 130.342643] niks3-server[1069]: 2026/09/20 12:44:50 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6303lb # [ 130.348640] niks3-server[1069]: 2026/09/20 12:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6304lb # [ 130.351795] niks3-server[1069]: 2026/09/20 12:44:50 INFO Completed upload id=456305worker1 # [ 130.333733] nix-grpc-daemon[1519]: time=2026-09-20T12:44:51.470Z level=INFO msg="Upload complete. (231ms)"6306lb # [ 130.354760] niks3-server[1069]: 2026/09/20 12:44:50 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6307worker1 # [ 130.340307] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51350 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16308worker1 # [ 130.351107] nix-daemon[844]: reaped child process 1584, status = succeeded6309client # [ 130.362110] systemd[1]: sw.service: Deactivated successfully.6310client # [ 130.364166] systemd[1]: sw.service: Consumed 108ms CPU time over 10.403s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.6311client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6312client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6313client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6314worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6315worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.05 seconds)6316lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6317lb: must succeed: curl -sf localhost:9901/clusters6318lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6319worker1 # [ 131.252855] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=signal_exit6320worker1 # [ 131.474429] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=scheduler_yield6321lb # [ 131.494598] niks3-server[1069]: 2026/09/20 12:44:51 INFO lead: released remote=192.168.1.3:362006322worker2 # [ 131.444519] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:52Z level=info event=scheduler_restarting addr=lb:500516323worker1 # [ 131.479987] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:513226324worker1 # [ 131.485379] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=worker_gone addr=192.168.1.4:500516325worker1 # [ 131.488361] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=scheduler_restarting addr=lb:500516326worker1 # [ 131.492089] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:513226327worker1 # [ 131.495350] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=worker_gone addr=192.168.1.3:500516328worker2 # [ 131.645025] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:52Z level=info event=scheduler_connected addr=lb:500516329worker2 # [ 131.646825] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6330worker1 # [ 131.683196] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=scheduler_connected addr=lb:500516331worker1 # [ 131.687222] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6332worker2 # [ 131.848345] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:52Z level=info event=scheduler_connected addr=lb:500516333worker2 # [ 131.852266] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6334worker1 # [ 131.887939] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:53Z level=info event=scheduler_connected addr=lb:500516335worker1 # [ 131.891951] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6336worker2 # [ 132.052908] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:52Z level=info event=scheduler_connected addr=lb:500516337worker2 # [ 132.054702] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6338worker1 # [ 132.092718] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:53Z level=info event=scheduler_connected addr=lb:500516339worker1 # [ 132.094511] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6340lb: must succeed: curl -sf localhost:9901/clusters6341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6342worker2 # [ 132.255509] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:53Z level=info event=scheduler_connected addr=lb:500516343worker2 # [ 132.257407] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6344worker1 # [ 132.294136] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:53Z level=info event=scheduler_connected addr=lb:500516345worker1 # [ 132.296098] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6346worker1 # [ 132.478934] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:53Z level=info event=scheduler_restarting6347worker2 # [ 132.460815] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:53Z level=info event=scheduler_connected addr=lb:500516348worker2 # [ 132.464812] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6349worker1 # [ 132.507383] nix-daemon[844]: reaped child process 1491, status = succeeded6350worker1 # [ 132.509467] nix-grpc-daemon[1440]: ts=2026-09-20T12:44:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6351worker2 # [ 132.661231] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:53Z level=info event=scheduler_connected addr=lb:500516352worker2 # [ 132.662984] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6353worker2 # [ 132.864305] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:53Z level=info event=scheduler_connected addr=lb:500516354worker2 # [ 132.868300] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6355worker2 # [ 133.069823] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_connected addr=lb:500516356worker2 # [ 133.073476] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6357lb: must succeed: curl -sf localhost:9901/clusters6358lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6359worker1 # [ 133.302505] nix-grpc-daemon[1440]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6360worker1 # [ 133.304196] nix-grpc-daemon[1440]: E0000 00:00:1789908294.439370 1483 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6361worker1 # [ 133.306071] nix-grpc-daemon[1440]: E0000 00:00:1789908294.439528 1483 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6362worker2 # [ 133.274667] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_connected addr=lb:500516363worker2 # [ 133.278524] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6364worker2 # [ 133.479594] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_connected addr=lb:500516365worker2 # [ 133.481363] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6366lb # [ 133.542287] niks3-server[1069]: 2026/09/20 12:44:53 INFO lead: acquired remote=192.168.1.4:411566367worker2 # [ 133.491286] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_take_over6368worker2 # [ 133.682822] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_connected addr=lb:500516369worker2 # [ 133.686632] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6370worker2 # [ 133.887382] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_connected addr=lb:500516371worker2 # [ 133.890050] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6372worker2 # [ 134.090685] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:55Z level=info event=scheduler_connected addr=lb:500516373worker2 # [ 134.101552] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:55Z level=info event=oidc_jwks provider=mock keys=16374worker2 # [ 134.102940] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:557946375worker2 # [ 134.104473] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06376lb: must succeed: curl -sf localhost:9901/clusters6377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6378worker1 # [ 134.286544] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6379worker1 # [ 134.288815] systemd[1]: nix-grpc-daemon.service: Consumed 692ms CPU time over 24.308s wall clock time, 31.3M memory peak, 25.9K incoming IP traffic, 33.5K outgoing IP traffic.6380lb: must succeed: curl -sf localhost:9901/clusters6381worker1 # [ 135.277960] systemd[1]: Starting Nix worker-protocol over gRPC...6382lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6383worker1 # [ 135.442767] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6384worker1 # [ 135.445511] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6385worker1 # [ 135.448858] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16386worker1 # [ 135.457071] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6387worker1 # [ 135.460421] systemd[1]: Started Nix worker-protocol over gRPC.6388worker1 # [ 135.473623] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:56Z level=info event=scheduler_connected addr=lb:500516389worker2 # [ 135.441074] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:557946390worker2 # [ 135.445229] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06391lb: must succeed: curl -sf localhost:9901/clusters6392lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6393lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 5.27 seconds)6394lb: must succeed: curl -sf localhost:9901/clusters6395lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6396worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6397worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6398(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.47 seconds)6399subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6400client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16401worker1 # [ 136.564575] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470206402worker1 # [ 136.571723] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6403worker1 # [ 136.579309] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206404worker1 # [ 136.585135] nix-grpc-daemon[1648]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5)6405worker1 # [ 136.625488] nix-grpc-daemon[1648]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 2/5)6406worker1 # [ 136.652650] nix-grpc-daemon[1648]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 3/5)6407worker1 # [ 136.703483] nix-grpc-daemon[1648]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 695 ms (attempt 4/5)6408worker1 # [ 137.399283] nix-grpc-daemon[1648]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6409worker2 # [ 137.392058] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:58Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06410worker2 # [ 137.393886] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:58Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16411worker2 # [ 137.395529] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:58Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06412worker2 # [ 137.397064] nix-grpc-daemon[1425]: ts=2026-09-20T12:44:58Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06413worker1 # [ 137.470589] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206414worker1 # [ 137.479315] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206415worker1 # [ 137.485533] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206416worker1 # [ 137.489597] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6417worker1 # [ 137.503717] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:58Z level=info event=niks3_push_started pid=17046418worker1 # [ 137.661128] nix-grpc-daemon[1704]: time=2026-09-20T12:44:58.797Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6419worker1 # [ 137.825295] nix-daemon[844]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6420worker1 # [ 137.837320] nix-daemon[844]: reaped child process 1720, status = succeeded6421worker1 # [ 138.008390] nix-daemon[844]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6422worker1 # [ 138.021481] nix-daemon[844]: reaped child process 1725, status = succeeded6423lb # [ 138.049537] envoy[921]: conn peer=[2001:db8:1::1]:48736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9641 tx=3892 ms=60016424lb # [ 138.051599] envoy[921]: conn peer=[2001:db8:1::1]:42818 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1350 ms=102176425lb # [ 138.053636] envoy[921]: conn peer=[2001:db8:1::3]:45062 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1789 tx=621 ms=222436426lb # [ 138.056699] niks3-server[1069]: 2026/09/20 12:44:58 INFO Received uploads request method=POST path=/api/pending_closures6427worker1 # [ 138.044083] nix-grpc-daemon[1704]: time=2026-09-20T12:44:59.180Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6428worker1 # [ 138.045487] nix-grpc-daemon[1704]: time=2026-09-20T12:44:59.180Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6429lb # [ 138.080353] niks3-server[1069]: 2026/09/20 12:44:58 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6430lb # [ 138.085768] niks3-server[1069]: 2026/09/20 12:44:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6431lb # [ 138.089734] niks3-server[1069]: 2026/09/20 12:44:58 INFO Signed narinfos id=46 count=16432worker1 # [ 138.071805] nix-grpc-daemon[1704]: time=2026-09-20T12:44:59.208Z level=INFO msg="Uploading 1 narinfos"6433lb # [ 138.096052] niks3-server[1069]: 2026/09/20 12:44:58 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6434lb # [ 138.102199] niks3-server[1069]: 2026/09/20 12:44:58 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6435lb # [ 138.106599] niks3-server[1069]: 2026/09/20 12:44:58 INFO Completed upload id=466436worker1 # [ 138.088547] nix-grpc-daemon[1704]: time=2026-09-20T12:44:59.225Z level=INFO msg="Upload complete. (423ms)"6437lb # [ 138.110091] niks3-server[1069]: 2026/09/20 12:44:58 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6438worker1 # [ 138.092094] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=5046439worker1 # [ 138.095155] nix-grpc-daemon[1648]: ts=2026-09-20T12:44:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206440worker1 # [ 138.102252] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6441worker1 # [ 138.104838] nix-daemon[844]: reaped child process 1702, status = succeeded6442worker1 # [ 138.110278] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6443worker1 # [ 139.138691] nix-daemon[844]: reaped child process 1729, status = succeeded6444worker1 # [ 139.264782] nix-daemon[844]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6445worker1 # [ 139.277757] nix-daemon[844]: reaped child process 1747, status = succeeded6446lb # [ 139.306260] niks3-server[1069]: 2026/09/20 12:44:59 INFO Received uploads request method=POST path=/api/pending_closures6447worker1 # [ 139.293300] nix-grpc-daemon[1704]: time=2026-09-20T12:45:00.430Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6448worker1 # [ 139.294881] nix-grpc-daemon[1704]: time=2026-09-20T12:45:00.431Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6449lb # [ 139.330994] niks3-server[1069]: 2026/09/20 12:44:59 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6450lb # [ 139.336953] niks3-server[1069]: 2026/09/20 12:44:59 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6451lb # [ 139.340477] niks3-server[1069]: 2026/09/20 12:44:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6452lb # [ 139.343501] niks3-server[1069]: 2026/09/20 12:44:59 INFO Signed narinfos id=47 count=16453worker1 # [ 139.325833] nix-grpc-daemon[1704]: time=2026-09-20T12:45:00.462Z level=INFO msg="Uploading 1 narinfos"6454lb # [ 139.346828] niks3-server[1069]: 2026/09/20 12:44:59 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6455lb # [ 139.354892] niks3-server[1069]: 2026/09/20 12:44:59 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6456worker1 # [ 139.339824] nix-grpc-daemon[1704]: time=2026-09-20T12:45:00.476Z level=INFO msg="Upload complete. (221ms)"6457lb # [ 139.357956] niks3-server[1069]: 2026/09/20 12:44:59 INFO Completed upload id=476458lb # [ 139.360797] niks3-server[1069]: 2026/09/20 12:44:59 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6459worker1 # [ 139.342406] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16460worker1 # [ 139.349705] nix-daemon[844]: reaped child process 1727, status = succeeded6461worker2 # [ 139.314610] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:00Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16462worker2 # [ 139.320587] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:00Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06463worker1 # [ 139.390121] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206464worker1 # [ 139.400950] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206465worker1 # [ 139.410936] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206466worker1 # [ 139.418259] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6467worker1 # [ 139.553591] nix-daemon[844]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)6468worker1 # [ 139.565685] nix-daemon[844]: reaped child process 1754, status = succeeded6469worker1 # [ 139.715203] nix-daemon[844]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6470worker1 # [ 139.728899] nix-daemon[844]: reaped child process 1759, status = succeeded6471lb # [ 139.763129] niks3-server[1069]: 2026/09/20 12:45:00 INFO Received uploads request method=POST path=/api/pending_closures6472worker1 # [ 139.752339] nix-grpc-daemon[1704]: time=2026-09-20T12:45:00.889Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6473worker1 # [ 139.753936] nix-grpc-daemon[1704]: time=2026-09-20T12:45:00.891Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6474lb # [ 139.791103] niks3-server[1069]: 2026/09/20 12:45:00 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6475lb # [ 139.804204] niks3-server[1069]: 2026/09/20 12:45:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6476worker1 # [ 139.790980] nix-grpc-daemon[1704]: time=2026-09-20T12:45:00.927Z level=INFO msg="Uploading 1 narinfos"6477lb # [ 139.810131] niks3-server[1069]: 2026/09/20 12:45:00 INFO Signed narinfos id=48 count=16478lb # [ 139.815433] niks3-server[1069]: 2026/09/20 12:45:00 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6479lb # [ 139.829256] niks3-server[1069]: 2026/09/20 12:45:00 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6480worker1 # [ 139.816317] nix-grpc-daemon[1704]: time=2026-09-20T12:45:00.953Z level=INFO msg="Upload complete. (387ms)"6481lb # [ 139.834493] niks3-server[1069]: 2026/09/20 12:45:00 INFO Completed upload id=486482worker1 # [ 139.818517] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=5046483lb # [ 139.837752] niks3-server[1069]: 2026/09/20 12:45:00 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6484worker1 # [ 139.823409] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206485worker1 # [ 139.829194] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6486worker1 # [ 139.831500] nix-daemon[844]: reaped child process 1749, status = succeeded6487worker1 # [ 139.836887] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6488worker1 # [ 140.866647] nix-daemon[844]: reaped child process 1763, status = succeeded6489worker1 # [ 141.020691] nix-daemon[844]: accepted connection from pid 1778, user nix-grpc-daemon (trusted)6490worker1 # [ 141.032505] nix-daemon[844]: reaped child process 1781, status = succeeded6491lb # [ 141.068633] niks3-server[1069]: 2026/09/20 12:45:01 INFO Received uploads request method=POST path=/api/pending_closures6492worker1 # [ 141.057177] nix-grpc-daemon[1704]: time=2026-09-20T12:45:02.194Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6493worker1 # [ 141.058785] nix-grpc-daemon[1704]: time=2026-09-20T12:45:02.195Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6494lb # [ 141.101501] niks3-server[1069]: 2026/09/20 12:45:01 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6495lb # [ 141.114595] niks3-server[1069]: 2026/09/20 12:45:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6496lb # [ 141.121709] niks3-server[1069]: 2026/09/20 12:45:01 INFO Signed narinfos id=49 count=16497worker1 # [ 141.106644] nix-grpc-daemon[1704]: time=2026-09-20T12:45:02.243Z level=INFO msg="Uploading 1 narinfos"6498lb # [ 141.131531] niks3-server[1069]: 2026/09/20 12:45:01 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6499lb # [ 141.136103] niks3-server[1069]: 2026/09/20 12:45:01 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6500lb # [ 141.144792] niks3-server[1069]: 2026/09/20 12:45:01 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6501lb # [ 141.147335] niks3-server[1069]: 2026/09/20 12:45:01 INFO Completed upload id=496502worker1 # [ 141.129433] nix-grpc-daemon[1704]: time=2026-09-20T12:45:02.266Z level=INFO msg="Upload complete. (284ms)"6503lb # [ 141.149761] niks3-server[1069]: 2026/09/20 12:45:01 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6504worker2 # [ 141.100348] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:02Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16505worker1 # [ 141.137519] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16506worker2 # [ 141.107384] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:02Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06507worker1 # [ 141.142513] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206508worker1 # [ 141.151522] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206509worker1 # [ 141.155924] nix-daemon[844]: reaped child process 1761, status = succeeded6510worker1 # [ 141.157708] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206511worker1 # [ 141.162064] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6512worker1 # [ 141.311327] nix-daemon[844]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6513worker1 # [ 141.324175] nix-daemon[844]: reaped child process 1788, status = succeeded6514worker1 # [ 141.471586] nix-daemon[844]: accepted connection from pid 1790, user nix-grpc-daemon (trusted)6515worker1 # [ 141.485497] nix-daemon[844]: reaped child process 1793, status = succeeded6516lb # [ 141.512517] niks3-server[1069]: 2026/09/20 12:45:01 INFO Received uploads request method=POST path=/api/pending_closures6517worker1 # [ 141.501335] nix-grpc-daemon[1704]: time=2026-09-20T12:45:02.638Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6518worker1 # [ 141.502790] nix-grpc-daemon[1704]: time=2026-09-20T12:45:02.639Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6519lb # [ 141.538834] niks3-server[1069]: 2026/09/20 12:45:01 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6520lb # [ 141.549882] niks3-server[1069]: 2026/09/20 12:45:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6521lb # [ 141.554569] niks3-server[1069]: 2026/09/20 12:45:01 INFO Signed narinfos id=50 count=16522worker1 # [ 141.539212] nix-grpc-daemon[1704]: time=2026-09-20T12:45:02.676Z level=INFO msg="Uploading 1 narinfos"6523lb # [ 141.560604] niks3-server[1069]: 2026/09/20 12:45:01 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6524lb # [ 141.572395] niks3-server[1069]: 2026/09/20 12:45:01 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6525worker1 # [ 141.559499] nix-grpc-daemon[1704]: time=2026-09-20T12:45:02.696Z level=INFO msg="Upload complete. (392ms)"6526lb # [ 141.577882] niks3-server[1069]: 2026/09/20 12:45:01 INFO Completed upload id=506527worker1 # [ 141.561404] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=5046528lb # [ 141.581131] niks3-server[1069]: 2026/09/20 12:45:01 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6529worker1 # [ 141.566945] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206530worker1 # [ 141.573136] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6531worker1 # [ 141.576082] nix-daemon[844]: reaped child process 1783, status = succeeded6532worker1 # [ 141.580861] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6533worker1 # [ 142.594085] nix-daemon[844]: reaped child process 1797, status = succeeded6534worker1 # [ 142.731524] nix-daemon[844]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6535worker1 # [ 142.744240] nix-daemon[844]: reaped child process 1815, status = succeeded6536lb # [ 142.777305] niks3-server[1069]: 2026/09/20 12:45:03 INFO Received uploads request method=POST path=/api/pending_closures6537worker1 # [ 142.768314] nix-grpc-daemon[1704]: time=2026-09-20T12:45:03.904Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6538worker1 # [ 142.770793] nix-grpc-daemon[1704]: time=2026-09-20T12:45:03.904Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6539lb # [ 142.813149] niks3-server[1069]: 2026/09/20 12:45:03 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6540lb # [ 142.827080] niks3-server[1069]: 2026/09/20 12:45:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6541lb # [ 142.835131] niks3-server[1069]: 2026/09/20 12:45:03 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6542lb # [ 142.840198] niks3-server[1069]: 2026/09/20 12:45:03 INFO Signed narinfos id=51 count=16543worker1 # [ 142.824932] nix-grpc-daemon[1704]: time=2026-09-20T12:45:03.961Z level=INFO msg="Uploading 1 narinfos"6544lb # [ 142.856733] niks3-server[1069]: 2026/09/20 12:45:03 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6545lb # [ 142.859623] niks3-server[1069]: 2026/09/20 12:45:03 INFO Completed upload id=516546worker1 # [ 142.841692] nix-grpc-daemon[1704]: time=2026-09-20T12:45:03.978Z level=INFO msg="Upload complete. (254ms)"6547lb # [ 142.863056] niks3-server[1069]: 2026/09/20 12:45:03 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6548lb # [ 142.866331] niks3-server[1069]: 2026/09/20 12:45:03 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6549worker2 # [ 142.813943] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:03Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16550worker1 # [ 142.848804] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16551worker1 # [ 142.855803] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206552worker2 # [ 142.822293] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:03Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06553worker1 # [ 142.864311] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206554worker1 # [ 142.870431] nix-daemon[844]: reaped child process 1795, status = succeeded6555worker1 # [ 142.874993] nix-grpc-daemon[1704]: time=2026-09-20T12:45:04.012Z level=INFO msg="All 1 paths already cached"6556worker1 # [ 142.878497] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206557worker1 # [ 142.883491] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6558worker1 # [ 143.027635] nix-daemon[844]: accepted connection from pid 1819, user nix-grpc-daemon (trusted)6559worker1 # [ 143.041286] nix-daemon[844]: reaped child process 1822, status = succeeded6560worker1 # [ 143.185335] nix-daemon[844]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6561worker1 # [ 143.198341] nix-daemon[844]: reaped child process 1827, status = succeeded6562lb # [ 143.228931] niks3-server[1069]: 2026/09/20 12:45:03 INFO Received uploads request method=POST path=/api/pending_closures6563worker1 # [ 143.221419] nix-grpc-daemon[1704]: time=2026-09-20T12:45:04.358Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6564worker1 # [ 143.223073] nix-grpc-daemon[1704]: time=2026-09-20T12:45:04.360Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6565lb # [ 143.261151] niks3-server[1069]: 2026/09/20 12:45:03 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6566lb # [ 143.270480] niks3-server[1069]: 2026/09/20 12:45:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6567lb # [ 143.274828] niks3-server[1069]: 2026/09/20 12:45:03 INFO Signed narinfos id=52 count=16568worker1 # [ 143.260788] nix-grpc-daemon[1704]: time=2026-09-20T12:45:04.397Z level=INFO msg="Uploading 1 narinfos"6569lb # [ 143.283392] niks3-server[1069]: 2026/09/20 12:45:03 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6570lb # [ 143.295659] niks3-server[1069]: 2026/09/20 12:45:03 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6571lb # [ 143.302537] niks3-server[1069]: 2026/09/20 12:45:03 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6572lb # [ 143.307587] niks3-server[1069]: 2026/09/20 12:45:03 INFO Completed upload id=526573worker1 # [ 143.289605] nix-grpc-daemon[1704]: time=2026-09-20T12:45:04.426Z level=INFO msg="Upload complete. (401ms)"6574worker1 # [ 143.293619] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=6246575worker1 # [ 143.298353] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206576worker1 # [ 143.307251] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6577worker1 # [ 143.317055] nix-daemon[844]: reaped child process 1817, status = succeeded6578worker1 # [ 143.320397] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6579worker1 # [ 144.551296] nix-daemon[844]: reaped child process 1831, status = succeeded6580worker1 # [ 144.673587] nix-daemon[844]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6581worker1 # [ 144.687051] nix-daemon[844]: reaped child process 1849, status = succeeded6582lb # [ 144.717330] niks3-server[1069]: 2026/09/20 12:45:05 INFO Received uploads request method=POST path=/api/pending_closures6583worker1 # [ 144.705290] nix-grpc-daemon[1704]: time=2026-09-20T12:45:05.842Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6584worker1 # [ 144.707848] nix-grpc-daemon[1704]: time=2026-09-20T12:45:05.844Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6585lb # [ 144.749301] niks3-server[1069]: 2026/09/20 12:45:05 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6586lb # [ 144.756309] niks3-server[1069]: 2026/09/20 12:45:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6587lb # [ 144.758356] niks3-server[1069]: 2026/09/20 12:45:05 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6588lb # [ 144.760605] niks3-server[1069]: 2026/09/20 12:45:05 INFO Signed narinfos id=53 count=16589worker1 # [ 144.742592] nix-grpc-daemon[1704]: time=2026-09-20T12:45:05.879Z level=INFO msg="Uploading 1 narinfos"6590lb # [ 144.763701] niks3-server[1069]: 2026/09/20 12:45:05 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6591lb # [ 144.768148] niks3-server[1069]: 2026/09/20 12:45:05 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6592lb # [ 144.772598] niks3-server[1069]: 2026/09/20 12:45:05 INFO Completed upload id=536593worker1 # [ 144.755729] nix-grpc-daemon[1704]: time=2026-09-20T12:45:05.892Z level=INFO msg="Upload complete. (227ms)"6594lb # [ 144.774286] niks3-server[1069]: 2026/09/20 12:45:05 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6595worker1 # [ 144.758825] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16596worker2 # [ 144.724774] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:05Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16597worker1 # [ 144.767094] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206598worker1 # [ 144.770109] nix-daemon[844]: reaped child process 1829, status = succeeded6599worker2 # [ 144.734975] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:05Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06600worker1 # [ 144.776190] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206601worker1 # [ 144.782927] nix-grpc-daemon[1704]: time=2026-09-20T12:45:05.919Z level=INFO msg="All 1 paths already cached"6602worker1 # [ 144.787367] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206603worker1 # [ 144.792502] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6604worker1 # [ 144.922113] nix-daemon[844]: accepted connection from pid 1853, user nix-grpc-daemon (trusted)6605worker1 # [ 144.935808] nix-daemon[844]: reaped child process 1856, status = succeeded6606worker1 # [ 145.050992] nix-daemon[844]: accepted connection from pid 1858, user nix-grpc-daemon (trusted)6607worker1 # [ 145.065596] nix-daemon[844]: reaped child process 1861, status = succeeded6608lb # [ 145.091087] niks3-server[1069]: 2026/09/20 12:45:05 INFO Received uploads request method=POST path=/api/pending_closures6609worker1 # [ 145.076696] nix-grpc-daemon[1704]: time=2026-09-20T12:45:06.213Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6610worker1 # [ 145.078649] nix-grpc-daemon[1704]: time=2026-09-20T12:45:06.215Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6611lb # [ 145.105277] niks3-server[1069]: 2026/09/20 12:45:05 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6612lb # [ 145.109113] niks3-server[1069]: 2026/09/20 12:45:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6613lb # [ 145.110862] niks3-server[1069]: 2026/09/20 12:45:05 INFO Signed narinfos id=54 count=16614worker1 # [ 145.093054] nix-grpc-daemon[1704]: time=2026-09-20T12:45:06.229Z level=INFO msg="Uploading 1 narinfos"6615lb # [ 145.116659] niks3-server[1069]: 2026/09/20 12:45:05 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6616lb # [ 145.118939] niks3-server[1069]: 2026/09/20 12:45:05 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6617lb # [ 145.122165] niks3-server[1069]: 2026/09/20 12:45:05 INFO Completed upload id=546618worker1 # [ 145.103994] nix-grpc-daemon[1704]: time=2026-09-20T12:45:06.241Z level=INFO msg="Upload complete. (306ms)"6619lb # [ 145.124668] niks3-server[1069]: 2026/09/20 12:45:05 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6620worker1 # [ 145.105987] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=6246621worker1 # [ 145.109480] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206622worker1 # [ 145.117183] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6623worker1 # [ 145.119044] nix-daemon[844]: reaped child process 1851, status = succeeded6624worker1 # [ 145.124187] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6625worker1 # [ 145.938604] nix-daemon[844]: reaped child process 1865, status = succeeded6626worker1 # [ 146.079645] nix-daemon[844]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6627worker1 # [ 146.090701] nix-daemon[844]: reaped child process 1883, status = succeeded6628lb # [ 146.122625] niks3-server[1069]: 2026/09/20 12:45:06 INFO Received uploads request method=POST path=/api/pending_closures6629worker1 # [ 146.108499] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.245Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6630worker1 # [ 146.110048] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.247Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6631lb # [ 146.141567] niks3-server[1069]: 2026/09/20 12:45:06 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6632lb # [ 146.144379] niks3-server[1069]: 2026/09/20 12:45:06 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6633lb # [ 146.148566] niks3-server[1069]: 2026/09/20 12:45:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6634lb # [ 146.151585] niks3-server[1069]: 2026/09/20 12:45:06 INFO Signed narinfos id=55 count=16635worker1 # [ 146.133630] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.270Z level=INFO msg="Uploading 1 narinfos"6636lb # [ 146.154967] niks3-server[1069]: 2026/09/20 12:45:06 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6637lb # [ 146.159895] niks3-server[1069]: 2026/09/20 12:45:06 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6638lb # [ 146.162789] niks3-server[1069]: 2026/09/20 12:45:06 INFO Completed upload id=556639worker1 # [ 146.146053] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.283Z level=INFO msg="Upload complete. (215ms)"6640lb # [ 146.164632] niks3-server[1069]: 2026/09/20 12:45:06 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6641worker1 # [ 146.149224] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16642worker2 # [ 146.114729] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:07Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16643worker1 # [ 146.152381] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206644worker2 # [ 146.119407] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:07Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06645worker1 # [ 146.159333] nix-daemon[844]: reaped child process 1863, status = succeeded6646worker1 # [ 146.160671] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206647worker1 # [ 146.166267] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.303Z level=INFO msg="All 1 paths already cached"6648worker1 # [ 146.170937] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206649worker1 # [ 146.174546] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6650worker1 # [ 146.301149] nix-daemon[844]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6651worker1 # [ 146.314323] nix-daemon[844]: reaped child process 1890, status = succeeded6652worker1 # [ 146.430469] nix-daemon[844]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6653worker1 # [ 146.442567] nix-daemon[844]: reaped child process 1895, status = succeeded6654lb # [ 146.469947] niks3-server[1069]: 2026/09/20 12:45:06 INFO Received uploads request method=POST path=/api/pending_closures6655worker1 # [ 146.459206] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.595Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6656worker1 # [ 146.460590] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.596Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6657lb # [ 146.494401] niks3-server[1069]: 2026/09/20 12:45:06 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6658lb # [ 146.506403] niks3-server[1069]: 2026/09/20 12:45:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6659worker1 # [ 146.495398] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.632Z level=INFO msg="Uploading 1 narinfos"6660lb # [ 146.513595] niks3-server[1069]: 2026/09/20 12:45:06 INFO Signed narinfos id=56 count=16661lb # [ 146.517844] niks3-server[1069]: 2026/09/20 12:45:06 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6662lb # [ 146.528228] niks3-server[1069]: 2026/09/20 12:45:06 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6663worker1 # [ 146.516798] nix-grpc-daemon[1704]: time=2026-09-20T12:45:07.653Z level=INFO msg="Upload complete. (336ms)"6664lb # [ 146.535244] niks3-server[1069]: 2026/09/20 12:45:06 INFO Completed upload id=566665worker1 # [ 146.520075] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=6246666lb # [ 146.538902] niks3-server[1069]: 2026/09/20 12:45:06 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6667worker1 # [ 146.525391] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206668worker1 # [ 146.527200] nix-daemon[844]: reaped child process 1885, status = succeeded6669worker1 # [ 146.530161] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6670worker1 # [ 146.537126] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6671worker1 # [ 147.753146] nix-daemon[844]: reaped child process 1899, status = succeeded6672worker1 # [ 147.889917] nix-daemon[844]: accepted connection from pid 1914, user nix-grpc-daemon (trusted)6673worker1 # [ 147.907811] nix-daemon[844]: reaped child process 1917, status = succeeded6674lb # [ 147.934917] niks3-server[1069]: 2026/09/20 12:45:08 INFO Received uploads request method=POST path=/api/pending_closures6675worker1 # [ 147.947630] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.084Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6676worker1 # [ 147.949729] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.086Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6677lb # [ 147.981375] niks3-server[1069]: 2026/09/20 12:45:08 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6678lb # [ 147.984274] niks3-server[1069]: 2026/09/20 12:45:08 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6679lb # [ 147.988054] niks3-server[1069]: 2026/09/20 12:45:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6680lb # [ 147.990540] niks3-server[1069]: 2026/09/20 12:45:08 INFO Signed narinfos id=57 count=16681worker1 # [ 147.972497] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.109Z level=INFO msg="Uploading 1 narinfos"6682lb # [ 147.993972] niks3-server[1069]: 2026/09/20 12:45:08 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6683lb # [ 147.998252] niks3-server[1069]: 2026/09/20 12:45:08 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6684lb # [ 148.001301] niks3-server[1069]: 2026/09/20 12:45:08 INFO Completed upload id=576685worker1 # [ 147.983192] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.120Z level=INFO msg="Upload complete. (237ms)"6686lb # [ 148.002886] niks3-server[1069]: 2026/09/20 12:45:08 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6687worker2 # [ 147.952775] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:08Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16688worker1 # [ 147.986035] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16689worker1 # [ 147.990863] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206690worker2 # [ 147.958546] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:08Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06691worker1 # [ 147.999391] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206692worker1 # [ 148.003904] nix-daemon[844]: reaped child process 1897, status = succeeded6693worker1 # [ 148.006259] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.143Z level=INFO msg="All 2 paths already cached"6694worker1 # [ 148.010132] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206695worker1 # [ 148.015911] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6696worker1 # [ 148.143719] nix-daemon[844]: accepted connection from pid 1921, user nix-grpc-daemon (trusted)6697worker1 # [ 148.158081] nix-daemon[844]: reaped child process 1924, status = succeeded6698worker1 # [ 148.279656] nix-daemon[844]: accepted connection from pid 1926, user nix-grpc-daemon (trusted)6699worker1 # [ 148.293564] nix-daemon[844]: reaped child process 1929, status = succeeded6700lb # [ 148.319737] niks3-server[1069]: 2026/09/20 12:45:08 INFO Received uploads request method=POST path=/api/pending_closures6701worker1 # [ 148.305631] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.442Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6702worker1 # [ 148.307638] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.444Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6703lb # [ 148.334914] niks3-server[1069]: 2026/09/20 12:45:08 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6704lb # [ 148.336814] niks3-server[1069]: 2026/09/20 12:45:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6705lb # [ 148.339514] niks3-server[1069]: 2026/09/20 12:45:08 INFO Signed narinfos id=58 count=16706worker1 # [ 148.321261] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.458Z level=INFO msg="Uploading 1 narinfos"6707lb # [ 148.343263] niks3-server[1069]: 2026/09/20 12:45:08 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6708lb # [ 148.346881] niks3-server[1069]: 2026/09/20 12:45:08 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6709lb # [ 148.349449] niks3-server[1069]: 2026/09/20 12:45:08 INFO Completed upload id=586710worker1 # [ 148.331259] nix-grpc-daemon[1704]: time=2026-09-20T12:45:09.468Z level=INFO msg="Upload complete. (309ms)"6711lb # [ 148.351808] niks3-server[1069]: 2026/09/20 12:45:08 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6712worker1 # [ 148.333313] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=7526713worker1 # [ 148.336058] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206714worker1 # [ 148.342897] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6715worker1 # [ 148.347494] nix-daemon[844]: reaped child process 1919, status = succeeded6716worker1 # [ 148.352626] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6717worker1 # [ 149.379516] nix-daemon[844]: reaped child process 1933, status = succeeded6718worker1 # [ 149.531107] nix-daemon[844]: accepted connection from pid 1948, user nix-grpc-daemon (trusted)6719worker1 # [ 149.546300] nix-daemon[844]: reaped child process 1951, status = succeeded6720lb # [ 149.572926] niks3-server[1069]: 2026/09/20 12:45:09 INFO Received uploads request method=POST path=/api/pending_closures6721worker1 # [ 149.564295] nix-grpc-daemon[1704]: time=2026-09-20T12:45:10.701Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6722worker1 # [ 149.568046] nix-grpc-daemon[1704]: time=2026-09-20T12:45:10.703Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6723lb # [ 149.614400] niks3-server[1069]: 2026/09/20 12:45:09 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6724lb # [ 149.626997] niks3-server[1069]: 2026/09/20 12:45:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6725lb # [ 149.630782] niks3-server[1069]: 2026/09/20 12:45:09 INFO Signed narinfos id=59 count=16726worker1 # [ 149.615147] nix-grpc-daemon[1704]: time=2026-09-20T12:45:10.751Z level=INFO msg="Uploading 1 narinfos"6727lb # [ 149.633437] niks3-server[1069]: 2026/09/20 12:45:09 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6728lb # [ 149.644103] niks3-server[1069]: 2026/09/20 12:45:09 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6729worker1 # [ 149.633839] nix-grpc-daemon[1704]: time=2026-09-20T12:45:10.770Z level=INFO msg="Upload complete. (272ms)"6730lb # [ 149.652309] niks3-server[1069]: 2026/09/20 12:45:09 INFO Completed upload id=596731worker2 # [ 149.602893] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:10Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16732lb # [ 149.655441] niks3-server[1069]: 2026/09/20 12:45:09 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6733worker1 # [ 149.637434] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16734lb # [ 149.659035] niks3-server[1069]: 2026/09/20 12:45:09 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6735worker1 # [ 149.643219] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206736worker1 # [ 149.649910] nix-daemon[844]: reaped child process 1931, status = succeeded6737worker1 # [ 149.651497] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206738worker1 # [ 149.658132] nix-grpc-daemon[1704]: time=2026-09-20T12:45:10.795Z level=INFO msg="All 2 paths already cached"6739worker1 # [ 149.661961] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206740worker1 # [ 149.666502] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6741worker1 # [ 149.808606] nix-daemon[844]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)6742worker1 # [ 149.821058] nix-daemon[844]: reaped child process 1958, status = succeeded6743worker1 # [ 149.961864] nix-daemon[844]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)6744worker1 # [ 149.974902] nix-daemon[844]: reaped child process 1963, status = succeeded6745lb # [ 150.002358] niks3-server[1069]: 2026/09/20 12:45:10 INFO Received uploads request method=POST path=/api/pending_closures6746worker1 # [ 149.992721] nix-grpc-daemon[1704]: time=2026-09-20T12:45:11.129Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6747worker1 # [ 149.994132] nix-grpc-daemon[1704]: time=2026-09-20T12:45:11.129Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6748lb # [ 150.026211] niks3-server[1069]: 2026/09/20 12:45:10 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6749lb # [ 150.038190] niks3-server[1069]: 2026/09/20 12:45:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6750lb # [ 150.044353] niks3-server[1069]: 2026/09/20 12:45:10 INFO Signed narinfos id=60 count=16751worker1 # [ 150.029824] nix-grpc-daemon[1704]: time=2026-09-20T12:45:11.166Z level=INFO msg="Uploading 1 narinfos"6752lb # [ 150.051836] niks3-server[1069]: 2026/09/20 12:45:10 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6753lb # [ 150.061329] niks3-server[1069]: 2026/09/20 12:45:10 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6754worker1 # [ 150.049793] nix-grpc-daemon[1704]: time=2026-09-20T12:45:11.186Z level=INFO msg="Upload complete. (375ms)"6755lb # [ 150.067953] niks3-server[1069]: 2026/09/20 12:45:10 INFO Completed upload id=606756worker1 # [ 150.051769] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=7526757lb # [ 150.073290] niks3-server[1069]: 2026/09/20 12:45:10 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6758worker1 # [ 150.054921] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206759worker1 # [ 150.064683] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6760worker1 # [ 150.067197] nix-daemon[844]: reaped child process 1953, status = succeeded6761worker1 # [ 150.072926] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6762worker1 # [ 150.687378] nix-daemon[844]: reaped child process 1967, status = succeeded6763worker1 # [ 150.822777] nix-daemon[844]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6764worker1 # [ 150.835125] nix-daemon[844]: reaped child process 1985, status = succeeded6765lb # [ 150.863950] niks3-server[1069]: 2026/09/20 12:45:11 INFO Received uploads request method=POST path=/api/pending_closures6766worker1 # [ 150.851083] nix-grpc-daemon[1704]: time=2026-09-20T12:45:11.987Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6767worker1 # [ 150.852475] nix-grpc-daemon[1704]: time=2026-09-20T12:45:11.987Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6768lb # [ 150.878286] niks3-server[1069]: 2026/09/20 12:45:11 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6769lb # [ 150.884488] niks3-server[1069]: 2026/09/20 12:45:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6770lb # [ 150.886638] niks3-server[1069]: 2026/09/20 12:45:11 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6771lb # [ 150.888575] niks3-server[1069]: 2026/09/20 12:45:11 INFO Signed narinfos id=61 count=16772worker1 # [ 150.871783] nix-grpc-daemon[1704]: time=2026-09-20T12:45:12.008Z level=INFO msg="Uploading 1 narinfos"6773lb # [ 150.889895] niks3-server[1069]: 2026/09/20 12:45:11 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6774lb # [ 150.895567] niks3-server[1069]: 2026/09/20 12:45:11 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6775lb # [ 150.898908] niks3-server[1069]: 2026/09/20 12:45:11 INFO Completed upload id=616776worker1 # [ 150.880784] nix-grpc-daemon[1704]: time=2026-09-20T12:45:12.017Z level=INFO msg="Upload complete. (201ms)"6777lb # [ 150.900876] niks3-server[1069]: 2026/09/20 12:45:11 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6778worker1 # [ 150.883365] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16779worker1 # [ 150.890206] nix-daemon[844]: reaped child process 1965, status = succeeded6780worker2 # [ 150.851960] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:11Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16781worker1 # [ 150.929782] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206782worker1 # [ 150.938247] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206783worker1 # [ 150.946621] nix-grpc-daemon[1704]: time=2026-09-20T12:45:12.083Z level=INFO msg="All 2 paths already cached"6784worker1 # [ 150.951414] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:470206785worker1 # [ 150.957871] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6786worker1 # [ 151.080400] nix-daemon[844]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6787worker1 # [ 151.093321] nix-daemon[844]: reaped child process 1992, status = succeeded6788worker1 # [ 151.234329] nix-daemon[844]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6789worker1 # [ 151.246715] nix-daemon[844]: reaped child process 1997, status = succeeded6790lb # [ 151.277537] niks3-server[1069]: 2026/09/20 12:45:11 INFO Received uploads request method=POST path=/api/pending_closures6791worker1 # [ 151.263556] nix-grpc-daemon[1704]: time=2026-09-20T12:45:12.400Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6792worker1 # [ 151.265061] nix-grpc-daemon[1704]: time=2026-09-20T12:45:12.400Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6793lb # [ 151.289980] niks3-server[1069]: 2026/09/20 12:45:11 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6794lb # [ 151.292986] niks3-server[1069]: 2026/09/20 12:45:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6795lb # [ 151.295435] niks3-server[1069]: 2026/09/20 12:45:11 INFO Signed narinfos id=62 count=16796worker1 # [ 151.277523] nix-grpc-daemon[1704]: time=2026-09-20T12:45:12.414Z level=INFO msg="Uploading 1 narinfos"6797lb # [ 151.298496] niks3-server[1069]: 2026/09/20 12:45:11 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6798lb # [ 151.302641] niks3-server[1069]: 2026/09/20 12:45:11 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6799lb # [ 151.306310] niks3-server[1069]: 2026/09/20 12:45:11 INFO Completed upload id=626800worker1 # [ 151.288170] nix-grpc-daemon[1704]: time=2026-09-20T12:45:12.425Z level=INFO msg="Upload complete. (323ms)"6801lb # [ 151.307402] niks3-server[1069]: 2026/09/20 12:45:11 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6802worker1 # [ 151.290092] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=0 paths=1 nar_bytes_in=7526803worker1 # [ 151.293106] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:470206804worker1 # [ 151.298928] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6805worker1 # [ 151.302528] nix-daemon[844]: reaped child process 1987, status = succeeded6806worker1 # [ 151.307321] nix-daemon[844]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6807worker1 # [ 152.320858] nix-daemon[844]: reaped child process 2001, status = succeeded6808worker1 # [ 152.429988] nix-daemon[844]: accepted connection from pid 2016, user nix-grpc-daemon (trusted)6809worker1 # [ 152.443300] nix-daemon[844]: reaped child process 2019, status = succeeded6810lb # [ 152.476119] niks3-server[1069]: 2026/09/20 12:45:12 INFO Received uploads request method=POST path=/api/pending_closures6811worker1 # [ 152.463319] nix-grpc-daemon[1704]: time=2026-09-20T12:45:13.598Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6812worker1 # [ 152.468853] nix-grpc-daemon[1704]: time=2026-09-20T12:45:13.599Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6813lb # [ 152.498111] niks3-server[1069]: 2026/09/20 12:45:12 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6814lb # [ 152.502575] niks3-server[1069]: 2026/09/20 12:45:12 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6815lb # [ 152.504914] niks3-server[1069]: 2026/09/20 12:45:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6816lb # [ 152.507432] niks3-server[1069]: 2026/09/20 12:45:12 INFO Signed narinfos id=63 count=16817worker1 # [ 152.489495] nix-grpc-daemon[1704]: time=2026-09-20T12:45:13.626Z level=INFO msg="Uploading 1 narinfos"6818lb # [ 152.510268] niks3-server[1069]: 2026/09/20 12:45:12 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6819lb # [ 152.515159] niks3-server[1069]: 2026/09/20 12:45:12 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6820lb # [ 152.518096] niks3-server[1069]: 2026/09/20 12:45:12 INFO Completed upload id=636821worker1 # [ 152.501841] nix-grpc-daemon[1704]: time=2026-09-20T12:45:13.638Z level=INFO msg="Upload complete. (187ms)"6822lb # [ 152.519918] niks3-server[1069]: 2026/09/20 12:45:12 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6823worker1 # [ 152.504650] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47020 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16824worker1 # [ 152.511540] nix-daemon[844]: reaped child process 1999, status = succeeded6825client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.14 seconds)6826worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6827worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6828client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16829worker1 # [ 152.771455] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470206830worker2 # [ 152.742274] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502386831worker2 # [ 152.754637] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:13Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06832lb # [ 158.050555] envoy[921]: conn peer=[2001:db8:1::1]:32912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27046 tx=10641 ms=159546833client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.24 seconds)6834(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.42 seconds)6835subtest: low disk drains a worker and builds go to the other6836worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6837worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.97 seconds)6838worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6839worker1 # [ 184.476793] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:45Z level=info event=unhealthy reason=min_free6840worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.08 seconds)6841client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26842worker1 # [ 185.056377] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:470206843client # don't know how to build these paths:6844client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6845worker1 # [ 185.066248] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:470206846worker2 # [ 185.061738] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:45Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06847worker2 # [ 185.066148] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:45Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16848worker2 # [ 185.103899] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386849worker2 # [ 185.117109] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502386850client # copying 1 paths to 'lb:50051'...6851client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6852worker2 # [ 185.129902] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502386853worker2 # [ 185.142671] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6854worker2 # [ 185.325436] nix-daemon[924]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6855worker2 # [ 185.339720] nix-daemon[924]: reaped child process 1768, status = succeeded6856worker2 # [ 185.483395] nix-daemon[924]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6857worker2 # [ 185.496932] nix-daemon[924]: reaped child process 1773, status = succeeded6858lb # [ 185.560794] niks3-server[1069]: 2026/09/20 12:45:45 INFO Received uploads request method=POST path=/api/pending_closures6859worker2 # [ 185.515293] nix-grpc-daemon[1563]: time=2026-09-20T12:45:46.448Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6860worker2 # [ 185.516799] nix-grpc-daemon[1563]: time=2026-09-20T12:45:46.449Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6861lb # [ 185.583475] niks3-server[1069]: 2026/09/20 12:45:45 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6862lb # [ 185.588222] niks3-server[1069]: 2026/09/20 12:45:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6863lb # [ 185.590390] niks3-server[1069]: 2026/09/20 12:45:45 INFO Signed narinfos id=64 count=16864worker2 # [ 185.537511] nix-grpc-daemon[1563]: time=2026-09-20T12:45:46.470Z level=INFO msg="Uploading 1 narinfos"6865lb # [ 185.596072] niks3-server[1069]: 2026/09/20 12:45:45 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6866lb # [ 185.599680] niks3-server[1069]: 2026/09/20 12:45:45 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6867lb # [ 185.603125] niks3-server[1069]: 2026/09/20 12:45:45 INFO Completed upload id=646868worker2 # [ 185.551197] nix-grpc-daemon[1563]: time=2026-09-20T12:45:46.484Z level=INFO msg="Upload complete. (396ms)"6869lb # [ 185.605899] niks3-server[1069]: 2026/09/20 12:45:45 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6870worker2 # [ 185.553624] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=0 paths=1 nar_bytes_in=5126871worker2 # [ 185.557114] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386872worker2 # [ 185.563934] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6873worker2 # [ 185.566411] nix-daemon[924]: reaped child process 1763, status = succeeded6874client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6875worker2 # [ 185.572304] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6876client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6877worker2 # [ 186.586529] nix-daemon[924]: reaped child process 1777, status = succeeded6878client # farm-a> LOG-farm-a-drain6879worker2 # [ 186.712101] nix-daemon[924]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6880worker2 # [ 186.725314] nix-daemon[924]: reaped child process 1795, status = succeeded6881lb # [ 186.793864] niks3-server[1069]: 2026/09/20 12:45:47 INFO Received uploads request method=POST path=/api/pending_closures6882worker2 # [ 186.748297] nix-grpc-daemon[1563]: time=2026-09-20T12:45:47.681Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6883worker2 # [ 186.749870] nix-grpc-daemon[1563]: time=2026-09-20T12:45:47.682Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6884lb # [ 186.818517] niks3-server[1069]: 2026/09/20 12:45:47 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6885lb # [ 186.822774] niks3-server[1069]: 2026/09/20 12:45:47 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6886lb # [ 186.825900] niks3-server[1069]: 2026/09/20 12:45:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6887lb # [ 186.828168] niks3-server[1069]: 2026/09/20 12:45:47 INFO Signed narinfos id=65 count=16888worker2 # [ 186.776652] nix-grpc-daemon[1563]: time=2026-09-20T12:45:47.709Z level=INFO msg="Uploading 1 narinfos"6889lb # [ 186.833162] niks3-server[1069]: 2026/09/20 12:45:47 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6890lb # [ 186.838069] niks3-server[1069]: 2026/09/20 12:45:47 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6891lb # [ 186.841284] niks3-server[1069]: 2026/09/20 12:45:47 INFO Completed upload id=656892worker2 # [ 186.790117] nix-grpc-daemon[1563]: time=2026-09-20T12:45:47.723Z level=INFO msg="Upload complete. (210ms)"6893lb # [ 186.846100] niks3-server[1069]: 2026/09/20 12:45:47 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6894worker2 # [ 186.797290] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16895worker2 # [ 186.804116] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:47Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06896worker2 # [ 186.806629] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:47Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16897worker2 # [ 186.812255] nix-daemon[924]: reaped child process 1775, status = succeeded6898worker2 # [ 186.847223] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386899worker2 # [ 186.854132] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502386900worker2 # [ 186.860040] nix-grpc-daemon[1563]: time=2026-09-20T12:45:47.793Z level=INFO msg="All 1 paths already cached"6901client # copying 1 paths to 'lb:50051'...6902worker2 # [ 186.865510] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502386903client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6904worker2 # [ 186.870798] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6905worker2 # [ 187.011407] nix-daemon[924]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6906worker2 # [ 187.027105] nix-daemon[924]: reaped child process 1804, status = succeeded6907worker2 # [ 187.189355] nix-daemon[924]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6908worker2 # [ 187.206089] nix-daemon[924]: reaped child process 1809, status = succeeded6909lb # [ 187.273171] niks3-server[1069]: 2026/09/20 12:45:47 INFO Received uploads request method=POST path=/api/pending_closures6910worker2 # [ 187.224510] nix-grpc-daemon[1563]: time=2026-09-20T12:45:48.157Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6911worker2 # [ 187.226176] nix-grpc-daemon[1563]: time=2026-09-20T12:45:48.159Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6912lb # [ 187.290718] niks3-server[1069]: 2026/09/20 12:45:47 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6913lb # [ 187.294487] niks3-server[1069]: 2026/09/20 12:45:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6914lb # [ 187.296565] niks3-server[1069]: 2026/09/20 12:45:47 INFO Signed narinfos id=66 count=16915worker2 # [ 187.243814] nix-grpc-daemon[1563]: time=2026-09-20T12:45:48.176Z level=INFO msg="Uploading 1 narinfos"6916lb # [ 187.302399] niks3-server[1069]: 2026/09/20 12:45:47 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6917lb # [ 187.306984] niks3-server[1069]: 2026/09/20 12:45:47 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6918lb # [ 187.310281] niks3-server[1069]: 2026/09/20 12:45:47 INFO Completed upload id=666919worker2 # [ 187.259054] nix-grpc-daemon[1563]: time=2026-09-20T12:45:48.192Z level=INFO msg="Upload complete. (380ms)"6920lb # [ 187.312336] niks3-server[1069]: 2026/09/20 12:45:47 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6921worker2 # [ 187.262092] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=0 paths=1 nar_bytes_in=6246922worker2 # [ 187.264904] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386923worker2 # [ 187.272982] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6924worker2 # [ 187.280171] nix-daemon[924]: reaped child process 1799, status = succeeded6925client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6926client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6927worker2 # [ 187.284099] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6928lb # [ 188.051591] envoy[921]: conn peer=[2001:db8:1::1]:58076 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=497 ms=300306929client # farm-b> LOG-farm-b-drain6930worker2 # [ 188.707936] nix-daemon[924]: reaped child process 1813, status = succeeded6931worker2 # [ 188.850646] nix-daemon[924]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)6932worker2 # [ 188.867588] nix-daemon[924]: reaped child process 1831, status = succeeded6933lb # [ 188.932258] niks3-server[1069]: 2026/09/20 12:45:49 INFO Received uploads request method=POST path=/api/pending_closures6934worker2 # [ 188.896452] nix-grpc-daemon[1563]: time=2026-09-20T12:45:49.827Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6935worker2 # [ 188.897884] nix-grpc-daemon[1563]: time=2026-09-20T12:45:49.827Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6936lb # [ 188.982206] niks3-server[1069]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6937lb # [ 188.990126] niks3-server[1069]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6938lb # [ 189.004341] niks3-server[1069]: 2026/09/20 12:45:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6939lb # [ 189.007432] niks3-server[1069]: 2026/09/20 12:45:49 INFO Signed narinfos id=67 count=16940worker2 # [ 188.954600] nix-grpc-daemon[1563]: time=2026-09-20T12:45:49.887Z level=INFO msg="Uploading 1 narinfos"6941lb # [ 189.010866] niks3-server[1069]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6942lb # [ 189.016281] niks3-server[1069]: 2026/09/20 12:45:49 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6943lb # [ 189.018730] niks3-server[1069]: 2026/09/20 12:45:49 INFO Completed upload id=676944lb # [ 189.021664] niks3-server[1069]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6945worker2 # [ 188.967516] nix-grpc-daemon[1563]: time=2026-09-20T12:45:49.900Z level=INFO msg="Upload complete. (272ms)"6946worker2 # [ 188.976398] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16947worker2 # [ 188.981563] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:49Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06948worker2 # [ 188.984285] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:49Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16949worker2 # [ 188.989476] nix-daemon[924]: reaped child process 1811, status = succeeded6950worker2 # [ 189.027117] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386951worker2 # [ 189.034909] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502386952worker2 # [ 189.041550] nix-grpc-daemon[1563]: time=2026-09-20T12:45:49.974Z level=INFO msg="All 2 paths already cached"6953client # copying 1 paths to 'lb:50051'...6954client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6955worker2 # [ 189.047527] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502386956worker2 # [ 189.053586] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6957worker2 # [ 189.214665] nix-daemon[924]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6958worker2 # [ 189.230795] nix-daemon[924]: reaped child process 1840, status = succeeded6959worker2 # [ 189.403713] nix-daemon[924]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6960worker2 # [ 189.419424] nix-daemon[924]: reaped child process 1845, status = succeeded6961lb # [ 189.488519] niks3-server[1069]: 2026/09/20 12:45:49 INFO Received uploads request method=POST path=/api/pending_closures6962worker2 # [ 189.443454] nix-grpc-daemon[1563]: time=2026-09-20T12:45:50.376Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6963worker2 # [ 189.445021] nix-grpc-daemon[1563]: time=2026-09-20T12:45:50.376Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6964lb # [ 189.510783] niks3-server[1069]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6965lb # [ 189.519822] niks3-server[1069]: 2026/09/20 12:45:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6966worker2 # [ 189.471211] nix-grpc-daemon[1563]: time=2026-09-20T12:45:50.404Z level=INFO msg="Uploading 1 narinfos"6967lb # [ 189.524098] niks3-server[1069]: 2026/09/20 12:45:49 INFO Signed narinfos id=68 count=16968lb # [ 189.534569] niks3-server[1069]: 2026/09/20 12:45:49 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6969lb # [ 189.628053] niks3-server[1069]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6970lb # [ 189.631894] niks3-server[1069]: 2026/09/20 12:45:49 INFO Completed upload id=686971worker2 # [ 189.579095] nix-grpc-daemon[1563]: time=2026-09-20T12:45:50.512Z level=INFO msg="Upload complete. (517ms)"6972lb # [ 189.635030] niks3-server[1069]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6973worker2 # [ 189.581927] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=0 paths=1 nar_bytes_in=7526974worker2 # [ 189.584443] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502386975worker2 # [ 189.591614] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6976worker2 # [ 189.595918] nix-daemon[924]: reaped child process 1835, status = succeeded6977client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6978worker2 # [ 189.601842] nix-daemon[924]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6979client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6980client # farm-top> LOG-farm-top-drain6981worker2 # [ 190.417932] nix-daemon[924]: reaped child process 1849, status = succeeded6982worker2 # [ 190.585114] nix-daemon[924]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6983worker2 # [ 190.600234] nix-daemon[924]: reaped child process 1867, status = succeeded6984lb # [ 190.669682] niks3-server[1069]: 2026/09/20 12:45:50 INFO Received uploads request method=POST path=/api/pending_closures6985worker2 # [ 190.622587] nix-grpc-daemon[1563]: time=2026-09-20T12:45:51.555Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6986worker2 # [ 190.624349] nix-grpc-daemon[1563]: time=2026-09-20T12:45:51.557Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6987lb # [ 190.690312] niks3-server[1069]: 2026/09/20 12:45:50 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6988lb # [ 190.694440] niks3-server[1069]: 2026/09/20 12:45:50 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6989lb # [ 190.700073] niks3-server[1069]: 2026/09/20 12:45:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6990lb # [ 190.702606] niks3-server[1069]: 2026/09/20 12:45:51 INFO Signed narinfos id=69 count=16991worker2 # [ 190.649769] nix-grpc-daemon[1563]: time=2026-09-20T12:45:51.582Z level=INFO msg="Uploading 1 narinfos"6992lb # [ 190.704895] niks3-server[1069]: 2026/09/20 12:45:51 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6993lb # [ 190.710324] niks3-server[1069]: 2026/09/20 12:45:51 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6994lb # [ 190.713632] niks3-server[1069]: 2026/09/20 12:45:51 INFO Completed upload id=696995worker2 # [ 190.662098] nix-grpc-daemon[1563]: time=2026-09-20T12:45:51.595Z level=INFO msg="Upload complete. (253ms)"6996lb # [ 190.715492] niks3-server[1069]: 2026/09/20 12:45:51 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6997worker2 # [ 190.665584] nix-grpc-daemon[1425]: ts=2026-09-20T12:45:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50238 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16998worker2 # [ 190.674288] nix-daemon[924]: reaped child process 1847, status = succeeded6999client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.87 seconds)7000client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7001client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.17 seconds)7002worker1: must succeed: rm /nix/.rw-store/fill7003worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7004worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7005worker1 # [ 191.480092] nix-grpc-daemon[1648]: ts=2026-09-20T12:45:52Z level=info event=healthy reason=min_free7006worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)7007(finished: subtest: low disk drains a worker and builds go to the other, in 9.37 seconds)7008(finished: run the VM test script, in 193.11 seconds)7009test script finished in 193.76s7010cleanup7011kill QemuMachine (pid 50)7012client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7013kill QemuMachine (pid 49)7014lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7015kill QemuMachine (pid 51)7016worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7017kill QemuMachine (pid 52)7018worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7019(finished: cleanup, in 1.00 seconds)7020additionally exposed symbols:7021 client, lb, worker1, worker2,7022 vlan1,7023 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh