vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #296
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1: QEMU running (pid 51)14worker1 # Formatting '/build/vm-state-worker1/tmp.fDKFSnIN0a', fmt=raw size=107374182415worker1 # mke2fs 1.47.4 (6-Mar-2025)16worker1 # Discarding device blocks: 0/262144 done17lb: QEMU running (pid 50)18worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes19lb # Disk image does not exist, creating the virtualisation disk image...20worker1 # Filesystem UUID: 43a797eb-f19c-4ece-bdd4-758010111bcf21lb # Formatting '/build/vm-state-lb/tmp.1uoXbATNxM', fmt=raw size=107374182422worker1 # Superblock backups stored on blocks:23lb # mke2fs 1.47.4 (6-Mar-2025)24worker1 # 32768, 98304, 163840, 22937625lb # Discarding device blocks: 0/262144 done26lb # Creating filesystem with 262144 4k blocks and 65536 inodes27worker2: QEMU running (pid 52)28worker1 # 29client # Disk image does not exist, creating the virtualisation disk image...30client # Formatting '/build/vm-state-client/tmp.CXuhofQfh9', fmt=raw size=107374182431worker2 # Disk image does not exist, creating the virtualisation disk image...32lb # Filesystem UUID: bebea37f-6c42-4096-a121-7d1c616ffef833worker1 # Allocating group tables: 0/8 done34client: QEMU running (pid 49)35lb # Superblock backups stored on blocks:36worker1 # Writing inode tables: 0/8 done37worker2 # Formatting '/build/vm-state-worker2/tmp.hD4CoMnCRQ', fmt=raw size=107374182438lb # 32768, 98304, 163840, 22937639worker1 # Creating journal (8192 blocks): done40client # mke2fs 1.47.4 (6-Mar-2025)41lb # 42worker1 # Writing superblocks and filesystem accounting information: 0/8 done43(finished: start all VMs, in 0.22 seconds)44lb # Allocating group tables: 0/8 done45worker1 # 46worker1 # Virtualisation disk image created.47client: waiting for unit network-addresses-eth1.service48client # Discarding device blocks: 0/262144 done49lb # Writing inode tables: 0/8 done50client # Creating filesystem with 262144 4k blocks and 65536 inodes51client: waiting for the VM to finish booting52worker2 # mke2fs 1.47.4 (6-Mar-2025)53lb # Creating journal (8192 blocks): done54lb # Writing superblocks and filesystem accounting information: 0/8 done55client # Filesystem UUID: 5407e002-b34b-404a-b04f-f33a3ab3be8956worker2 # Discarding device blocks: 0/262144 done57client # Superblock backups stored on blocks:58worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes59worker2 # Filesystem UUID: 605a9e08-36dd-43c1-836e-3d52f5a3ae3d60client # 32768, 98304, 163840, 22937661lb # 62worker2 # Superblock backups stored on blocks:63lb # Virtualisation disk image created.64worker2 # 32768, 98304, 163840, 22937665client # 66worker2 # 67worker2 # Allocating group tables: 0/8 done68client # Allocating group tables: 0/8 done69worker2 # Writing inode tables: 0/8 done70client # Writing inode tables: 0/8 done71worker2 # Creating journal (8192 blocks): done72client # Creating journal (8192 blocks): done73worker2 # Writing superblocks and filesystem accounting information: 0/8 done74client # Writing superblocks and filesystem accounting information: 0/8 done75worker2 # 76client # 77worker2 # Virtualisation disk image created.78client # Virtualisation disk image created.79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87lb # 88lb # 89lb # 90lb # 91lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0092worker1 # 93worker1 # 94worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0095lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...96lb # 97lb # 98worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...99worker1 # 100worker1 # 101worker1 # 102worker1 # 103worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00104client # 105client # 106client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00107worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...108worker1 # 109worker1 # 110client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...111client # 112client # 113client # 114client # 115client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117client # 118client # 119worker2 # 120worker2 # 121worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122lb # Booting from ROM...123worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...124worker2 # 125worker2 # 126worker2 # 127worker2 # 128worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00129worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...130worker2 # 131worker2 # 132worker1 # Booting from ROM...133client # Booting from ROM...134worker2 # Booting from ROM...135lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xzrwmx591i25vlijx1m7lk76hjmbxra6-nixos-system-lb-test/init regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration console=ttyS0,115200n8 console=tty0137lb # [ 0.000000] BIOS-provided physical RAM map:138lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148lb # [ 0.000000] NX (Execute Disable) protection: active149lb # [ 0.000000] APIC: Static calls initialized150lb # [ 0.000000] SMBIOS 2.8 present.151lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014152lb # [ 0.000000] DMI: Memory slots populated: 1/1153lb # [ 0.000000] Hypervisor detected: KVM154lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000155lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00156lb # [ 0.000000] kvm-clock: using sched offset of 449594091 cycles157lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns158lb # [ 0.000005] tsc: Detected 2400.012 MHz processor159lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000160lb # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs161lb # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT162worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026163lb # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]164lb # [ 0.002748] Using GB pages for direct mapping165lb # [ 0.002880] RAMDISK: [mem 0x5e367000-0x5ffcffff]166lb # [ 0.002888] ACPI: Early table checksum verification disabled167lb # [ 0.002891] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )168worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hvv0rrwgpqzhm46a4280mwzmbapcmqq9-nixos-system-worker1-test/init regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration console=ttyS0,115200n8 console=tty0169lb # [ 0.002895] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)170worker1 # [ 0.000000] BIOS-provided physical RAM map:171lb # [ 0.002898] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)172worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable173worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved174lb # [ 0.002906] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)175lb # [ 0.002908] ACPI: FACS 0x000000005FFE0000 000040176worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved177lb # [ 0.002909] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)178worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable179worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved180lb # [ 0.002911] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)181worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved182lb # [ 0.002912] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)183worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved184lb # [ 0.002914] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)185worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved186lb # [ 0.002915] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]187worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved188client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026189lb # [ 0.002916] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]190worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved191worker1 # [ 0.000000] NX (Execute Disable) protection: active192lb # [ 0.002917] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]193worker1 # [ 0.000000] APIC: Static calls initialized194lb # [ 0.002917] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]195worker1 # [ 0.000000] SMBIOS 2.8 present.196lb # [ 0.002917] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]197worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014198client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test/init regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration console=ttyS0,115200n8 console=tty0199lb # [ 0.002918] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]200worker1 # [ 0.000000] DMI: Memory slots populated: 1/1201client # [ 0.000000] BIOS-provided physical RAM map:202worker1 # [ 0.000000] Hypervisor detected: KVM203lb # [ 0.002918] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]204client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable205worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000206lb # [ 0.003148] No NUMA configuration found207worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00208client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved209lb # [ 0.003149] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]210worker1 # [ 0.000000] kvm-clock: using sched offset of 464749696 cycles211client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved212lb # [ 0.003151] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]213lb # [ 0.005421] Zone ranges:214client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable215worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns216lb # [ 0.005422] DMA [mem 0x0000000000001000-0x0000000000ffffff]217worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor218client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved219lb # [ 0.005423] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]220lb # [ 0.005424] Normal empty221worker1 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000222client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved223lb # [ 0.005425] Device empty224lb # [ 0.005425] Movable zone start for each node225worker1 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs226client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved227lb # [ 0.005426] Early memory node ranges228worker1 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT229client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved230lb # [ 0.005426] node 0: [mem 0x0000000000001000-0x000000000009efff]231worker1 # [ 0.002735] found SMP MP-table at [mem 0x000f5450-0x000f545f]232lb # [ 0.005427] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]233client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved234worker1 # [ 0.002747] Using GB pages for direct mapping235worker1 # [ 0.002891] RAMDISK: [mem 0x5e367000-0x5ffcffff]236lb # [ 0.005428] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]237client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved238client # [ 0.000000] NX (Execute Disable) protection: active239worker1 # [ 0.002899] ACPI: Early table checksum verification disabled240lb # [ 0.005448] On node 0, zone DMA: 1 pages in unavailable ranges241client # [ 0.000000] APIC: Static calls initialized242worker1 # [ 0.002903] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )243lb # [ 0.005713] On node 0, zone DMA: 97 pages in unavailable ranges244client # [ 0.000000] SMBIOS 2.8 present.245lb # [ 0.033073] On node 0, zone DMA32: 40 pages in unavailable ranges246worker1 # [ 0.002907] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)247lb # [ 0.033532] ACPI: PM-Timer IO Port: 0x608248client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014249client # [ 0.000000] DMI: Memory slots populated: 1/1250lb # [ 0.033544] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])251worker1 # [ 0.002911] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)252client # [ 0.000000] Hypervisor detected: KVM253lb # [ 0.033570] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23254client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000255worker1 # [ 0.002918] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)256lb # [ 0.033573] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)257worker1 # [ 0.002920] ACPI: FACS 0x000000005FFE0000 000040258client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00259client # [ 0.000000] kvm-clock: using sched offset of 468767386 cycles260lb # [ 0.033575] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)261worker1 # [ 0.002921] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)262lb # [ 0.033576] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)263client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns264worker1 # [ 0.002923] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)265lb # [ 0.033577] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)266client # [ 0.000005] tsc: Detected 2400.012 MHz processor267worker1 # [ 0.002924] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)268client # [ 0.000817] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000269lb # [ 0.033578] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)270lb # [ 0.033581] ACPI: Using ACPI (MADT) for SMP configuration information271worker1 # [ 0.002926] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)272client # [ 0.000845] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs273lb # [ 0.033582] ACPI: HPET id: 0x8086a201 base: 0xfed00000274worker1 # [ 0.002927] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]275client # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT276lb # [ 0.033585] TSC deadline timer available277lb # [ 0.033589] CPU topo: Max. logical packages: 1278client # [ 0.002773] found SMP MP-table at [mem 0x000f5450-0x000f545f]279worker1 # [ 0.002928] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]280worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026281client # [ 0.002785] Using GB pages for direct mapping282lb # [ 0.033590] CPU topo: Max. logical dies: 1283worker1 # [ 0.002929] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]284lb # [ 0.033590] CPU topo: Max. dies per package: 1285client # [ 0.002886] RAMDISK: [mem 0x5e366000-0x5ffcffff]286lb # [ 0.033594] CPU topo: Max. threads per core: 1287client # [ 0.002894] ACPI: Early table checksum verification disabled288worker1 # [ 0.002929] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]289lb # [ 0.033594] CPU topo: Num. cores per package: 1290client # [ 0.002897] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )291worker1 # [ 0.002930] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]292lb # [ 0.033595] CPU topo: Num. threads per package: 1293client # [ 0.002901] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)294worker1 # [ 0.002930] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]295lb # [ 0.033595] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs296worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5qgax5h37ybfdjp7r7k01yx9abl6wij-nixos-system-worker2-test/init regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration console=ttyS0,115200n8 console=tty0297worker1 # [ 0.002931] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]298lb # [ 0.033612] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()299client # [ 0.002905] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)300worker2 # [ 0.000000] BIOS-provided physical RAM map:301worker1 # [ 0.003156] No NUMA configuration found302lb # [ 0.033643] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]303worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable304client # [ 0.002912] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)305worker1 # [ 0.003158] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]306client # [ 0.002914] ACPI: FACS 0x000000005FFE0000 000040307lb # [ 0.033645] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]308worker1 # [ 0.003160] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]309worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved310worker1 # [ 0.005492] Zone ranges:311lb # [ 0.033646] [mem 0x60000000-0xafffffff] available for PCI devices312worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved313client # [ 0.002915] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)314worker1 # [ 0.005494] DMA [mem 0x0000000000001000-0x0000000000ffffff]315lb # [ 0.033647] Booting paravirtualized kernel on KVM316worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable317worker1 # [ 0.005495] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]318client # [ 0.002917] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)319worker1 # [ 0.005496] Normal empty320lb # [ 0.033650] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns321worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved322worker1 # [ 0.005497] Device empty323client # [ 0.002918] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)324worker1 # [ 0.005498] Movable zone start for each node325worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved326lb # [ 0.038112] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1327worker1 # [ 0.005498] Early memory node ranges328client # [ 0.002920] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)329worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved330worker1 # [ 0.005499] node 0: [mem 0x0000000000001000-0x000000000009efff]331lb # [ 0.040575] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152332client # [ 0.002921] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]333lb # [ 0.040618] kvm-guest: PV spinlocks disabled, single CPU334worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved335worker1 # [ 0.005500] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]336client # [ 0.002922] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]337worker1 # [ 0.005501] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]338worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved339client # [ 0.002923] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]340worker1 # [ 0.005520] On node 0, zone DMA: 1 pages in unavailable ranges341worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved342worker2 # [ 0.000000] NX (Execute Disable) protection: active343worker1 # [ 0.005794] On node 0, zone DMA: 97 pages in unavailable ranges344client # [ 0.002923] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]345worker2 # [ 0.000000] APIC: Static calls initialized346worker1 # [ 0.034196] On node 0, zone DMA32: 40 pages in unavailable ranges347client # [ 0.002924] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]348worker2 # [ 0.000000] SMBIOS 2.8 present.349worker1 # [ 0.034673] ACPI: PM-Timer IO Port: 0x608350lb # [ 0.040620] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xzrwmx591i25vlijx1m7lk76hjmbxra6-nixos-system-lb-test/init regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration console=ttyS0,115200n8 console=tty0351worker1 # [ 0.034687] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])352client # [ 0.002924] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]353worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014354worker2 # [ 0.000000] DMI: Memory slots populated: 1/1355client # [ 0.002925] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]356worker1 # [ 0.034715] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23357worker2 # [ 0.000000] Hypervisor detected: KVM358client # [ 0.003162] No NUMA configuration found359lb # [ 0.040712] Unknown kernel command line parameters "regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration", will be passed to user space.360worker1 # [ 0.034718] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)361lb # [ 0.040725] random: crng init done362worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000363client # [ 0.003163] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]364worker1 # [ 0.034720] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)365lb # [ 0.040726] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes366worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00367client # [ 0.003165] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]368client # [ 0.005545] Zone ranges:369worker1 # [ 0.034721] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)370worker2 # [ 0.000001] kvm-clock: using sched offset of 498199699 cycles371lb # [ 0.042889] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)372client # [ 0.005546] DMA [mem 0x0000000000001000-0x0000000000ffffff]373worker1 # [ 0.034722] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)374lb # [ 0.043963] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)375worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns376client # [ 0.005548] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]377lb # [ 0.043994] Fallback order for Node 0: 0378client # [ 0.005549] Normal empty379worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor380worker1 # [ 0.034723] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)381client # [ 0.005550] Device empty382worker1 # [ 0.034726] ACPI: Using ACPI (MADT) for SMP configuration information383lb # [ 0.043997] Built 1 zonelists, mobility grouping on. Total pages: 393078384worker2 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000385client # [ 0.005550] Movable zone start for each node386lb # [ 0.043998] Policy zone: DMA32387worker1 # [ 0.034727] ACPI: HPET id: 0x8086a201 base: 0xfed00000388client # [ 0.005551] Early memory node ranges389worker1 # [ 0.034732] TSC deadline timer available390worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs391lb # [ 0.046510] mem auto-init: stack:all(zero), heap alloc:on, heap free:off392client # [ 0.005551] node 0: [mem 0x0000000000001000-0x000000000009efff]393worker1 # [ 0.034736] CPU topo: Max. logical packages: 1394worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT395lb # [ 0.049681] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1396worker1 # [ 0.034737] CPU topo: Max. logical dies: 1397client # [ 0.005552] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]398worker2 # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]399lb # [ 0.053334] allocated 3145728 bytes of page_ext400worker1 # [ 0.034737] CPU topo: Max. dies per package: 1401client # [ 0.005553] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]402worker2 # [ 0.002750] Using GB pages for direct mapping403lb # [ 0.063000] ftrace: allocating 48731 entries in 192 pages404worker1 # [ 0.034742] CPU topo: Max. threads per core: 1405worker2 # [ 0.002882] RAMDISK: [mem 0x5e367000-0x5ffcffff]406client # [ 0.005573] On node 0, zone DMA: 1 pages in unavailable ranges407worker1 # [ 0.034743] CPU topo: Num. cores per package: 1408lb # [ 0.063001] ftrace: allocated 192 pages with 2 groups409lb # [ 0.063845] Dynamic Preempt: lazy410worker2 # [ 0.002891] ACPI: Early table checksum verification disabled411worker1 # [ 0.034743] CPU topo: Num. threads per package: 1412client # [ 0.005846] On node 0, zone DMA: 97 pages in unavailable ranges413lb # [ 0.064031] rcu: Preemptible hierarchical RCU implementation.414worker2 # [ 0.002894] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )415client # [ 0.034671] On node 0, zone DMA32: 40 pages in unavailable ranges416worker1 # [ 0.034744] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs417lb # [ 0.064031] rcu: RCU event tracing is enabled.418client # [ 0.035151] ACPI: PM-Timer IO Port: 0x608419worker1 # [ 0.034771] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()420worker2 # [ 0.002898] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)421client # [ 0.035165] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])422lb # [ 0.064032] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.423lb # [ 0.064034] Trampoline variant of Tasks RCU enabled.424worker1 # [ 0.034811] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]425worker2 # [ 0.002903] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)426client # [ 0.035197] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23427lb # [ 0.064034] Rude variant of Tasks RCU enabled.428worker1 # [ 0.034812] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]429client # [ 0.035200] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)430lb # [ 0.064034] Tracing variant of Tasks RCU enabled.431worker2 # [ 0.002910] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)432worker1 # [ 0.034814] [mem 0x60000000-0xafffffff] available for PCI devices433worker2 # [ 0.002912] ACPI: FACS 0x000000005FFE0000 000040434client # [ 0.035202] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)435lb # [ 0.064035] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.436worker1 # [ 0.034816] Booting paravirtualized kernel on KVM437client # [ 0.035203] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)438worker2 # [ 0.002914] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)439lb # [ 0.064036] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1440client # [ 0.035204] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)441worker1 # [ 0.034820] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns442worker2 # [ 0.002915] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)443lb # [ 0.064062] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.444client # [ 0.035205] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)445worker1 # [ 0.039273] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1446worker2 # [ 0.002917] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)447lb # [ 0.064063] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.448client # [ 0.035207] ACPI: Using ACPI (MADT) for SMP configuration information449worker1 # [ 0.042071] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152450client # [ 0.035208] ACPI: HPET id: 0x8086a201 base: 0xfed00000451worker2 # [ 0.002918] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)452worker1 # [ 0.042121] kvm-guest: PV spinlocks disabled, single CPU453client # [ 0.035213] TSC deadline timer available454lb # [ 0.064064] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.455client # [ 0.035218] CPU topo: Max. logical packages: 1456worker2 # [ 0.002920] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]457lb # [ 0.068436] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16458client # [ 0.035218] CPU topo: Max. logical dies: 1459worker2 # [ 0.002921] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]460lb # [ 0.068720] rcu: srcu_init: Setting srcu_struct sizes based on contention.461client # [ 0.035219] CPU topo: Max. dies per package: 1462client # [ 0.035222] CPU topo: Max. threads per core: 1463worker2 # [ 0.002921] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]464lb # [ 0.068726] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns465client # [ 0.035223] CPU topo: Num. cores per package: 1466worker2 # [ 0.002922] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]467client # [ 0.035223] CPU topo: Num. threads per package: 1468worker1 # [ 0.042122] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hvv0rrwgpqzhm46a4280mwzmbapcmqq9-nixos-system-worker1-test/init regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration console=ttyS0,115200n8 console=tty0469lb # [ 0.068828] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)470worker2 # [ 0.002922] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]471client # [ 0.035224] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs472lb # [ 0.072344] Console: colour VGA+ 80x25473worker2 # [ 0.002923] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]474client # [ 0.035246] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()475lb # [ 0.072347] printk: legacy console [tty0] enabled476worker1 # [ 0.042223] Unknown kernel command line parameters "regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration", will be passed to user space.477worker1 # [ 0.042235] random: crng init done478lb # [ 0.101913] printk: legacy console [ttyS0] enabled479worker2 # [ 0.002923] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]480client # [ 0.035282] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]481worker2 # [ 0.003150] No NUMA configuration found482lb # [ 0.200713] ACPI: Core revision 20250807483worker1 # [ 0.042236] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes484client # [ 0.035283] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]485worker2 # [ 0.003151] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]486client # [ 0.035285] [mem 0x60000000-0xafffffff] available for PCI devices487worker1 # [ 0.044660] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)488worker2 # [ 0.003153] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]489worker2 # [ 0.005496] Zone ranges:490lb # [ 0.201564] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns491client # [ 0.035287] Booting paravirtualized kernel on KVM492worker1 # [ 0.045814] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)493worker2 # [ 0.005497] DMA [mem 0x0000000000001000-0x0000000000ffffff]494worker1 # [ 0.045848] Fallback order for Node 0: 0495lb # [ 0.203103] APIC: Switch to symmetric I/O mode setup496worker2 # [ 0.005498] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]497client # [ 0.035290] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns498lb # [ 0.204031] x2apic enabled499worker1 # [ 0.045851] Built 1 zonelists, mobility grouping on. Total pages: 393078500worker2 # [ 0.005499] Normal empty501worker1 # [ 0.045852] Policy zone: DMA32502worker2 # [ 0.005500] Device empty503client # [ 0.039770] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1504worker2 # [ 0.005501] Movable zone start for each node505lb # [ 0.204763] APIC: Switched APIC routing to: physical x2apic506worker1 # [ 0.048641] mem auto-init: stack:all(zero), heap alloc:on, heap free:off507worker2 # [ 0.005502] Early memory node ranges508client # [ 0.042559] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152509worker1 # [ 0.052200] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1510client # [ 0.042610] kvm-guest: PV spinlocks disabled, single CPU511worker2 # [ 0.005502] node 0: [mem 0x0000000000001000-0x000000000009efff]512worker1 # [ 0.056011] allocated 3145728 bytes of page_ext513worker2 # [ 0.005503] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]514lb # [ 0.206537] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1515worker1 # [ 0.066466] ftrace: allocating 48731 entries in 192 pages516worker2 # [ 0.005504] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]517worker1 # [ 0.066468] ftrace: allocated 192 pages with 2 groups518worker1 # [ 0.067333] Dynamic Preempt: lazy519worker2 # [ 0.005524] On node 0, zone DMA: 1 pages in unavailable ranges520lb # [ 0.207470] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns521worker1 # [ 0.067471] rcu: Preemptible hierarchical RCU implementation.522worker2 # [ 0.006198] On node 0, zone DMA: 97 pages in unavailable ranges523worker1 # [ 0.067472] rcu: RCU event tracing is enabled.524worker2 # [ 0.033690] On node 0, zone DMA32: 40 pages in unavailable ranges525client # [ 0.042611] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test/init regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration console=ttyS0,115200n8 console=tty0526lb # [ 0.209028] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)527worker1 # [ 0.067472] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.528worker2 # [ 0.034162] ACPI: PM-Timer IO Port: 0x608529worker1 # [ 0.067474] Trampoline variant of Tasks RCU enabled.530lb # [ 0.211111] x86/cpu: User Mode Instruction Prevention (UMIP) activated531worker2 # [ 0.034176] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])532worker1 # [ 0.067474] Rude variant of Tasks RCU enabled.533client # [ 0.042712] Unknown kernel command line parameters "regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration", will be passed to user space.534worker2 # [ 0.034210] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23535worker1 # [ 0.067475] Tracing variant of Tasks RCU enabled.536client # [ 0.042725] random: crng init done537lb # [ 0.212179] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127538worker2 # [ 0.034213] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)539client # [ 0.042726] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes540worker1 # [ 0.067476] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.541lb # [ 0.213026] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0542worker2 # [ 0.034214] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)543worker1 # [ 0.067476] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1544client # [ 0.045195] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)545worker2 # [ 0.034216] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)546worker1 # [ 0.067499] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.547client # [ 0.046404] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)548lb # [ 0.214029] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto549worker2 # [ 0.034217] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)550client # [ 0.046437] Fallback order for Node 0: 0551worker1 # [ 0.067501] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.552worker2 # [ 0.034218] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)553client # [ 0.046440] Built 1 zonelists, mobility grouping on. Total pages: 393078554lb # [ 0.215026] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl555client # [ 0.046441] Policy zone: DMA32556worker2 # [ 0.034220] ACPI: Using ACPI (MADT) for SMP configuration information557worker1 # [ 0.067502] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.558lb # [ 0.217026] Transient Scheduler Attacks: Vulnerable: No microcode559worker2 # [ 0.034221] ACPI: HPET id: 0x8086a201 base: 0xfed00000560client # [ 0.049267] mem auto-init: stack:all(zero), heap alloc:on, heap free:off561worker1 # [ 0.071940] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16562worker2 # [ 0.034226] TSC deadline timer available563lb # [ 0.218025] Spectre V2 : Mitigation: Enhanced / Automatic IBRS564client # [ 0.052764] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1565worker2 # [ 0.034230] CPU topo: Max. logical packages: 1566worker1 # [ 0.072226] rcu: srcu_init: Setting srcu_struct sizes based on contention.567client # [ 0.056279] allocated 3145728 bytes of page_ext568worker2 # [ 0.034231] CPU topo: Max. logical dies: 1569lb # [ 0.219025] Speculative Return Stack Overflow: Mitigation: Safe RET570client # [ 0.066550] ftrace: allocating 48731 entries in 192 pages571worker2 # [ 0.034231] CPU topo: Max. dies per package: 1572worker1 # [ 0.072234] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns573client # [ 0.066552] ftrace: allocated 192 pages with 2 groups574worker2 # [ 0.034235] CPU topo: Max. threads per core: 1575client # [ 0.067416] Dynamic Preempt: lazy576worker2 # [ 0.034235] CPU topo: Num. cores per package: 1577worker1 # [ 0.072350] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)578client # [ 0.067592] rcu: Preemptible hierarchical RCU implementation.579worker2 # [ 0.034236] CPU topo: Num. threads per package: 1580worker1 # [ 0.075874] Console: colour VGA+ 80x25581client # [ 0.067593] rcu: RCU event tracing is enabled.582worker1 # [ 0.075876] printk: legacy console [tty0] enabled583worker2 # [ 0.034236] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs584client # [ 0.067594] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.585worker1 # [ 0.105376] printk: legacy console [ttyS0] enabled586worker2 # [ 0.034260] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()587lb # [ 0.219953] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization588client # [ 0.067595] Trampoline variant of Tasks RCU enabled.589worker1 # [ 0.201650] ACPI: Core revision 20250807590client # [ 0.067596] Rude variant of Tasks RCU enabled.591worker2 # [ 0.034296] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]592lb # [ 0.221033] Spectre V2 : Enabling IBPB for BPF593client # [ 0.067596] Tracing variant of Tasks RCU enabled.594worker2 # [ 0.034297] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]595worker1 # [ 0.202520] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns596client # [ 0.067597] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.597lb # [ 0.222027] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier598worker2 # [ 0.034299] [mem 0x60000000-0xafffffff] available for PCI devices599worker1 # [ 0.204017] APIC: Switch to symmetric I/O mode setup600client # [ 0.067598] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1601lb # [ 0.223026] active return thunk: srso_alias_return_thunk602worker2 # [ 0.034300] Booting paravirtualized kernel on KVM603worker1 # [ 0.204970] x2apic enabled604client # [ 0.067619] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.605worker2 # [ 0.034304] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns606lb # [ 0.224643] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'607worker1 # [ 0.205707] APIC: Switched APIC routing to: physical x2apic608client # [ 0.067621] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.609worker2 # [ 0.038774] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1610lb # [ 0.225026] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'611worker2 # [ 0.040964] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152612client # [ 0.067622] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.613lb # [ 0.226742] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'614worker2 # [ 0.041011] kvm-guest: PV spinlocks disabled, single CPU615worker1 # [ 0.207496] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1616client # [ 0.072028] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16617lb # [ 0.227025] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'618client # [ 0.072308] rcu: srcu_init: Setting srcu_struct sizes based on contention.619lb # [ 0.228025] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'620worker1 # [ 0.208454] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns621client # [ 0.072316] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns622lb # [ 0.229025] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'623worker1 # [ 0.210068] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)624client # [ 0.072434] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)625worker2 # [ 0.041012] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5qgax5h37ybfdjp7r7k01yx9abl6wij-nixos-system-worker2-test/init regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration console=ttyS0,115200n8 console=tty0626lb # [ 0.230025] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'627client # [ 0.075981] Console: colour VGA+ 80x25628worker1 # [ 0.211155] x86/cpu: User Mode Instruction Prevention (UMIP) activated629client # [ 0.075984] printk: legacy console [tty0] enabled630lb # [ 0.231026] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256631client # [ 0.105512] printk: legacy console [ttyS0] enabled632worker1 # [ 0.212208] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127633worker2 # [ 0.041113] Unknown kernel command line parameters "regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration", will be passed to user space.634lb # [ 0.232725] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64635client # [ 0.201430] ACPI: Core revision 20250807636worker2 # [ 0.041126] random: crng init done637worker1 # [ 0.213066] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0638lb # [ 0.233025] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512639worker2 # [ 0.041127] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes640client # [ 0.202341] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns641lb # [ 0.234025] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024642worker2 # [ 0.043225] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)643worker1 # [ 0.214069] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto644client # [ 0.203837] APIC: Switch to symmetric I/O mode setup645lb # [ 0.235025] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8646worker2 # [ 0.044274] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)647worker2 # [ 0.044311] Fallback order for Node 0: 0648client # [ 0.204815] x2apic enabled649worker1 # [ 0.215066] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl650worker2 # [ 0.044323] Built 1 zonelists, mobility grouping on. Total pages: 393078651lb # [ 0.236025] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.652worker2 # [ 0.044324] Policy zone: DMA32653client # [ 0.205595] APIC: Switched APIC routing to: physical x2apic654worker1 # [ 0.216066] Transient Scheduler Attacks: Vulnerable: No microcode655worker2 # [ 0.046786] mem auto-init: stack:all(zero), heap alloc:on, heap free:off656worker1 # [ 0.217065] Spectre V2 : Mitigation: Enhanced / Automatic IBRS657worker2 # [ 0.050250] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1658client # [ 0.207393] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1659worker1 # [ 0.218066] Speculative Return Stack Overflow: Mitigation: Safe RET660worker2 # [ 0.053511] allocated 3145728 bytes of page_ext661worker2 # [ 0.063766] ftrace: allocating 48731 entries in 192 pages662worker2 # [ 0.063768] ftrace: allocated 192 pages with 2 groups663client # [ 0.208378] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns664worker2 # [ 0.064619] Dynamic Preempt: lazy665worker2 # [ 0.064747] rcu: Preemptible hierarchical RCU implementation.666client # [ 0.210058] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)667worker2 # [ 0.064748] rcu: RCU event tracing is enabled.668worker2 # [ 0.064749] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.669client # [ 0.211144] x86/cpu: User Mode Instruction Prevention (UMIP) activated670worker1 # [ 0.219066] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization671worker2 # [ 0.064751] Trampoline variant of Tasks RCU enabled.672worker1 # [ 0.221073] Spectre V2 : Enabling IBPB for BPF673client # [ 0.212168] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127674worker2 # [ 0.064752] Rude variant of Tasks RCU enabled.675worker2 # [ 0.064752] Tracing variant of Tasks RCU enabled.676client # [ 0.213746] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0677worker1 # [ 0.222066] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier678worker2 # [ 0.064753] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.679worker1 # [ 0.223067] active return thunk: srso_alias_return_thunk680worker2 # [ 0.064753] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1681client # [ 0.214059] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto682worker1 # [ 0.224087] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'683worker2 # [ 0.064768] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.684client # [ 0.216054] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl685worker1 # [ 0.225065] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'686worker2 # [ 0.064769] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.687client # [ 0.217055] Transient Scheduler Attacks: Vulnerable: No microcode688worker1 # [ 0.226828] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'689worker2 # [ 0.064770] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.690client # [ 0.218778] Spectre V2 : Mitigation: Enhanced / Automatic IBRS691worker1 # [ 0.227835] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'692worker2 # [ 0.069153] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16693client # [ 0.219748] Speculative Return Stack Overflow: Mitigation: Safe RET694worker2 # [ 0.069421] rcu: srcu_init: Setting srcu_struct sizes based on contention.695worker1 # [ 0.228842] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'696worker1 # [ 0.229829] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'697worker2 # [ 0.069430] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns698worker1 # [ 0.230931] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'699worker2 # [ 0.069533] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)700worker2 # [ 0.073048] Console: colour VGA+ 80x25701worker1 # [ 0.232066] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256702worker2 # [ 0.073050] printk: legacy console [tty0] enabled703client # [ 0.220054] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization704worker2 # [ 0.102419] printk: legacy console [ttyS0] enabled705client # [ 0.222062] Spectre V2 : Enabling IBPB for BPF706worker1 # [ 0.233065] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64707worker2 # [ 0.199049] ACPI: Core revision 20250807708worker1 # [ 0.234066] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512709client # [ 0.223055] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier710worker1 # [ 0.235065] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024711worker2 # [ 0.199970] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns712client # [ 0.224055] active return thunk: srso_alias_return_thunk713worker1 # [ 0.236065] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8714worker2 # [ 0.201513] APIC: Switch to symmetric I/O mode setup715client # [ 0.225707] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'716worker2 # [ 0.202513] x2apic enabled717client # [ 0.226958] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'718worker1 # [ 0.237065] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.719worker2 # [ 0.203294] APIC: Switched APIC routing to: physical x2apic720client # [ 0.227805] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'721client # [ 0.228812] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'722worker2 # [ 0.205155] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1723client # [ 0.229823] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'724client # [ 0.230808] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'725worker2 # [ 0.206211] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns726client # [ 0.231845] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'727worker2 # [ 0.207910] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)728client # [ 0.233055] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256729worker2 # [ 0.208999] x86/cpu: User Mode Instruction Prevention (UMIP) activated730client # [ 0.234026] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64731worker2 # [ 0.210988] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127732client # [ 0.235054] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512733worker2 # [ 0.211907] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0734client # [ 0.236054] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024735lb # [ 0.270150] Freeing SMP alternatives memory: 44K736client # [ 0.237054] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8737lb # [ 0.270915] pid_max: default: 32768 minimum: 301738worker2 # [ 0.212911] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto739lb # [ 0.271642] LSM: initializing lsm=capability,landlock,yama,bpf,ima740client # [ 0.238054] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.741worker2 # [ 0.213907] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl742lb # [ 0.272134] landlock: Up and running.743lb # [ 0.273026] Yama: becoming mindful.744worker2 # [ 0.215906] Transient Scheduler Attacks: Vulnerable: No microcode745lb # [ 0.274233] LSM support for eBPF active746worker2 # [ 0.216907] Spectre V2 : Mitigation: Enhanced / Automatic IBRS747lb # [ 0.275040] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)748worker2 # [ 0.217907] Speculative Return Stack Overflow: Mitigation: Safe RET749lb # [ 0.276062] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)750lb # [ 0.278604] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)751worker2 # [ 0.218907] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization752worker2 # [ 0.220915] Spectre V2 : Enabling IBPB for BPF753lb # [ 0.279614] Performance Events: Fam17h+ core perfctr, AMD PMU driver.754lb # [ 0.280030] ... version: 2755worker2 # [ 0.221679] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier756lb # [ 0.280761] ... bit width: 48757worker2 # [ 0.222908] active return thunk: srso_alias_return_thunk758lb # [ 0.281028] ... generic counters: 6759worker2 # [ 0.223929] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'760lb # [ 0.281760] ... generic bitmap: 000000000000003f761lb # [ 0.282038] ... fixed-purpose counters: 0762worker2 # [ 0.224908] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'763lb # [ 0.282755] ... fixed-purpose bitmap: 0000000000000000764worker2 # [ 0.225907] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'765lb # [ 0.283028] ... value mask: 0000ffffffffffff766worker2 # [ 0.226907] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'767lb # [ 0.283955] ... max period: 00007fffffffffff768worker2 # [ 0.227907] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'769lb # [ 0.284725] ... global_ctrl mask: 000000000000003f770worker2 # [ 0.228907] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'771lb # [ 0.285136] signal: max sigframe size: 3376772lb # [ 0.285949] rcu: Hierarchical SRCU implementation.773worker2 # [ 0.229907] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'774lb # [ 0.286642] rcu: Max phase no-delay instances is 400.775worker2 # [ 0.231875] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256776worker2 # [ 0.232907] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64777worker2 # [ 0.233906] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512778worker2 # [ 0.234907] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024779worker1 # [ 0.272036] Freeing SMP alternatives memory: 44K780worker1 # [ 0.272644] pid_max: default: 32768 minimum: 301781worker2 # [ 0.235907] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8782lb # [ 0.291501] smp: Bringing up secondary CPUs ...783worker1 # [ 0.273159] LSM: initializing lsm=capability,landlock,yama,bpf,ima784lb # [ 0.292042] smp: Brought up 1 node, 1 CPU785worker2 # [ 0.236907] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.786worker1 # [ 0.274165] landlock: Up and running.787worker1 # [ 0.275066] Yama: becoming mindful.788lb # [ 0.292697] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)789worker1 # [ 0.276282] LSM support for eBPF active790worker1 # [ 0.277124] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)791lb # [ 0.293198] Memory: 1454544K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)792lb # [ 0.294261] devtmpfs: initialized793worker1 # [ 0.278108] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)794lb # [ 0.295070] x86/mm: Memory block size: 128MB795worker1 # [ 0.280718] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)796lb # [ 0.296837] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)797worker1 # [ 0.281600] Performance Events: Fam17h+ core perfctr, AMD PMU driver.798lb # [ 0.297970] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).799client # [ 0.272912] Freeing SMP alternatives memory: 44K800worker1 # [ 0.282070] ... version: 2801lb # [ 0.299119] pinctrl core: initialized pinctrl subsystem802client # [ 0.273614] pid_max: default: 32768 minimum: 301803worker1 # [ 0.282852] ... bit width: 48804lb # [ 0.300263] PM: RTC time: 10:50:26, date: 2026-09-20805worker1 # [ 0.283068] ... generic counters: 6806client # [ 0.274153] LSM: initializing lsm=capability,landlock,yama,bpf,ima807worker1 # [ 0.283776] ... generic bitmap: 000000000000003f808client # [ 0.275176] landlock: Up and running.809client # [ 0.276055] Yama: becoming mindful.810worker1 # [ 0.284080] ... fixed-purpose counters: 0811client # [ 0.277121] LSM support for eBPF active812worker1 # [ 0.284795] ... fixed-purpose bitmap: 0000000000000000813worker1 # [ 0.285068] ... value mask: 0000ffffffffffff814client # [ 0.277880] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)815lb # [ 0.304067] NET: Registered PF_NETLINK/PF_ROUTE protocol family816worker1 # [ 0.285976] ... max period: 00007fffffffffff817client # [ 0.279090] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker1 # [ 0.286781] ... global_ctrl mask: 000000000000003f819lb # [ 0.305495] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations820worker1 # [ 0.287198] signal: max sigframe size: 3376821lb # [ 0.306051] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations822worker1 # [ 0.288046] rcu: Hierarchical SRCU implementation.823client # [ 0.281354] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)824worker1 # [ 0.288652] rcu: Max phase no-delay instances is 400.825lb # [ 0.307305] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations826client # [ 0.282621] Performance Events: Fam17h+ core perfctr, AMD PMU driver.827lb # [ 0.308037] audit: initializing netlink subsys (disabled)828client # [ 0.283064] ... version: 2829lb # [ 0.309187] thermal_sys: Registered thermal governor 'fair_share'830client # [ 0.283809] ... bit width: 48831client # [ 0.284057] ... generic counters: 6832lb # [ 0.309190] thermal_sys: Registered thermal governor 'bang_bang'833client # [ 0.284757] ... generic bitmap: 000000000000003f834lb # [ 0.310029] thermal_sys: Registered thermal governor 'step_wise'835client # [ 0.285064] ... fixed-purpose counters: 0836worker1 # [ 0.293716] smp: Bringing up secondary CPUs ...837lb # [ 0.311002] thermal_sys: Registered thermal governor 'user_space'838client # [ 0.285777] ... fixed-purpose bitmap: 0000000000000000839worker1 # [ 0.294081] smp: Brought up 1 node, 1 CPU840client # [ 0.286056] ... value mask: 0000ffffffffffff841lb # [ 0.311789] audit: type=2000 audit(1789901427.399:1): state=initialized audit_enabled=0 res=1842worker1 # [ 0.294754] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)843client # [ 0.286908] ... max period: 00007fffffffffff844lb # [ 0.313030] thermal_sys: Registered thermal governor 'power_allocator'845client # [ 0.287717] ... global_ctrl mask: 000000000000003f846lb # [ 0.313055] cpuidle: using governor menu847client # [ 0.288169] signal: max sigframe size: 3376848worker1 # [ 0.295294] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)849client # [ 0.289003] rcu: Hierarchical SRCU implementation.850lb # [ 0.315244] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5851worker1 # [ 0.296365] devtmpfs: initialized852client # [ 0.289649] rcu: Max phase no-delay instances is 400.853worker1 # [ 0.297163] x86/mm: Memory block size: 128MB854lb # [ 0.316259] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]855lb # [ 0.317031] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry856worker1 # [ 0.298904] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)857lb # [ 0.318040] PCI: Using configuration type 1 for base access858worker1 # [ 0.300011] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).859worker2 # [ 0.272087] Freeing SMP alternatives memory: 44K860lb # [ 0.319176] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.861worker1 # [ 0.301162] pinctrl core: initialized pinctrl subsystem862worker2 # [ 0.272858] pid_max: default: 32768 minimum: 301863client # [ 0.294708] smp: Bringing up secondary CPUs ...864worker1 # [ 0.302274] PM: RTC time: 10:50:26, date: 2026-09-20865client # [ 0.295070] smp: Brought up 1 node, 1 CPU866worker2 # [ 0.273568] LSM: initializing lsm=capability,landlock,yama,bpf,ima867worker2 # [ 0.274749] landlock: Up and running.868client # [ 0.295765] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)869worker2 # [ 0.275366] Yama: becoming mindful.870worker2 # [ 0.276122] LSM support for eBPF active871client # [ 0.296231] Memory: 1454544K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)872worker2 # [ 0.277015] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)873worker1 # [ 0.305951] NET: Registered PF_NETLINK/PF_ROUTE protocol family874client # [ 0.297334] devtmpfs: initialized875worker2 # [ 0.277949] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)876client # [ 0.298137] x86/mm: Memory block size: 128MB877worker1 # [ 0.307300] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations878lb # [ 0.326273] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages879lb # [ 0.327030] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page880worker1 # [ 0.308092] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations881client # [ 0.299857] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)882worker2 # [ 0.280631] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)883worker1 # [ 0.309338] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations884client # [ 0.300957] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).885worker2 # [ 0.281468] Performance Events: Fam17h+ core perfctr, AMD PMU driver.886worker1 # [ 0.310078] audit: initializing netlink subsys (disabled)887client # [ 0.302147] pinctrl core: initialized pinctrl subsystem888worker2 # [ 0.281912] ... version: 2889worker1 # [ 0.311226] thermal_sys: Registered thermal governor 'fair_share'890worker2 # [ 0.282626] ... bit width: 48891client # [ 0.303282] PM: RTC time: 10:50:26, date: 2026-09-20892worker2 # [ 0.282910] ... generic counters: 6893worker1 # [ 0.311228] thermal_sys: Registered thermal governor 'bang_bang'894lb # [ 0.332028] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages895worker2 # [ 0.283619] ... generic bitmap: 000000000000003f896worker1 # [ 0.312069] thermal_sys: Registered thermal governor 'step_wise'897lb # [ 0.333029] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page898worker2 # [ 0.283956] ... fixed-purpose counters: 0899worker1 # [ 0.313052] thermal_sys: Registered thermal governor 'user_space'900worker2 # [ 0.284636] ... fixed-purpose bitmap: 0000000000000000901client # [ 0.306841] NET: Registered PF_NETLINK/PF_ROUTE protocol family902worker1 # [ 0.313815] audit: type=2000 audit(1789901427.424:1): state=initialized audit_enabled=0 res=1903worker2 # [ 0.284910] ... value mask: 0000ffffffffffff904worker2 # [ 0.285805] ... max period: 00007fffffffffff905worker1 # [ 0.315072] thermal_sys: Registered thermal governor 'power_allocator'906client # [ 0.308298] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations907worker1 # [ 0.315085] cpuidle: using governor menu908worker2 # [ 0.286637] ... global_ctrl mask: 000000000000003f909worker2 # [ 0.287032] signal: max sigframe size: 3376910client # [ 0.309081] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations911worker1 # [ 0.317349] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5912worker2 # [ 0.287868] rcu: Hierarchical SRCU implementation.913worker2 # [ 0.288530] rcu: Max phase no-delay instances is 400.914client # [ 0.310329] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations915worker1 # [ 0.318308] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]916client # [ 0.311067] audit: initializing netlink subsys (disabled)917worker1 # [ 0.319071] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry918lb # [ 0.341610] ACPI: Added _OSI(Module Device)919client # [ 0.312215] thermal_sys: Registered thermal governor 'fair_share'920lb # [ 0.342030] ACPI: Added _OSI(Processor Device)921worker1 # [ 0.320079] PCI: Using configuration type 1 for base access922client # [ 0.312217] thermal_sys: Registered thermal governor 'bang_bang'923lb # [ 0.342789] ACPI: Added _OSI(Processor Aggregator Device)924client # [ 0.313058] thermal_sys: Registered thermal governor 'step_wise'925worker1 # [ 0.321196] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.926client # [ 0.314002] thermal_sys: Registered thermal governor 'user_space'927worker2 # [ 0.293491] smp: Bringing up secondary CPUs ...928worker2 # [ 0.293924] smp: Brought up 1 node, 1 CPU929client # [ 0.314788] audit: type=2000 audit(1789901427.437:1): state=initialized audit_enabled=0 res=1930worker2 # [ 0.294623] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)931client # [ 0.316060] thermal_sys: Registered thermal governor 'power_allocator'932client # [ 0.316084] cpuidle: using governor menu933worker2 # [ 0.295048] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)934client # [ 0.318282] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5935worker2 # [ 0.296166] devtmpfs: initialized936lb # [ 0.349557] ACPI: 1 ACPI AML tables successfully acquired and loaded937client # [ 0.319280] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]938worker2 # [ 0.297043] x86/mm: Memory block size: 128MB939worker1 # [ 0.328306] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages940client # [ 0.320061] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry941worker1 # [ 0.329070] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page942client # [ 0.321069] PCI: Using configuration type 1 for base access943worker2 # [ 0.298868] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)944lb # [ 0.352977] ACPI: Interpreter enabled945client # [ 0.322162] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.946worker2 # [ 0.299898] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).947lb # [ 0.353464] ACPI: PM: (supports S0 S3 S4 S5)948worker2 # [ 0.301000] pinctrl core: initialized pinctrl subsystem949lb # [ 0.354029] ACPI: Using IOAPIC for interrupt routing950worker2 # [ 0.302086] PM: RTC time: 10:50:26, date: 2026-09-20951lb # [ 0.354929] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug952worker1 # [ 0.334069] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages953worker1 # [ 0.335069] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page954client # [ 0.327174] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages955client # [ 0.328059] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page956worker2 # [ 0.305719] NET: Registered PF_NETLINK/PF_ROUTE protocol family957worker2 # [ 0.307159] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations958lb # [ 0.360028] PCI: Using E820 reservations for host bridge windows959lb # [ 0.361140] ACPI: Enabled 2 GPEs in block 00 to 3F960worker2 # [ 0.307935] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations961worker2 # [ 0.309178] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations962client # [ 0.333059] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages963worker2 # [ 0.309919] audit: initializing netlink subsys (disabled)964client # [ 0.334057] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page965worker1 # [ 0.343667] ACPI: Added _OSI(Module Device)966worker1 # [ 0.344070] ACPI: Added _OSI(Processor Device)967worker2 # [ 0.310912] audit: type=2000 audit(1789901427.462:1): state=initialized audit_enabled=0 res=1968worker1 # [ 0.344846] ACPI: Added _OSI(Processor Aggregator Device)969worker2 # [ 0.312067] thermal_sys: Registered thermal governor 'fair_share'970worker2 # [ 0.312069] thermal_sys: Registered thermal governor 'bang_bang'971worker2 # [ 0.312911] thermal_sys: Registered thermal governor 'step_wise'972worker2 # [ 0.313911] thermal_sys: Registered thermal governor 'user_space'973lb # [ 0.369766] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])974worker2 # [ 0.314910] thermal_sys: Registered thermal governor 'power_allocator'975worker2 # [ 0.315932] cpuidle: using governor menu976lb # [ 0.370502] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]977worker2 # [ 0.318138] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5978lb # [ 0.371106] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]979lb # [ 0.372158] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]980client # [ 0.344446] ACPI: Added _OSI(Module Device)981worker2 # [ 0.319157] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]982client # [ 0.345059] ACPI: Added _OSI(Processor Device)983lb # [ 0.373388] PCI host bridge to bus 0000:00984worker2 # [ 0.319913] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry985worker1 # [ 0.354306] ACPI: 1 ACPI AML tables successfully acquired and loaded986client # [ 0.345766] ACPI: Added _OSI(Processor Aggregator Device)987lb # [ 0.374032] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]988worker2 # [ 0.320922] PCI: Using configuration type 1 for base access989lb # [ 0.375028] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]990worker2 # [ 0.322031] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.991lb # [ 0.376028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]992worker1 # [ 0.358320] ACPI: Interpreter enabled993lb # [ 0.377028] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]994worker1 # [ 0.358949] ACPI: PM: (supports S0 S3 S4 S5)995worker1 # [ 0.359660] ACPI: Using IOAPIC for interrupt routing996lb # [ 0.378028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]997lb # [ 0.379028] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]998worker1 # [ 0.360138] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug999lb # [ 0.380029] pci_bus 0000:00: root bus resource [bus 00-ff]1000worker1 # [ 0.361069] PCI: Using E820 reservations for host bridge windows1001lb # [ 0.381057] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1002worker2 # [ 0.328928] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1003client # [ 0.354599] ACPI: 1 ACPI AML tables successfully acquired and loaded1004worker2 # [ 0.329911] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1005worker1 # [ 0.364174] ACPI: Enabled 2 GPEs in block 00 to 3F1006lb # [ 0.382502] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1007client # [ 0.358491] ACPI: Interpreter enabled1008client # [ 0.359074] ACPI: PM: (supports S0 S3 S4 S5)1009client # [ 0.359771] ACPI: Using IOAPIC for interrupt routing1010worker2 # [ 0.334911] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1011worker2 # [ 0.335910] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1012client # [ 0.360104] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1013client # [ 0.361058] PCI: Using E820 reservations for host bridge windows1014lb # [ 0.385169] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1015worker1 # [ 0.372778] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1016lb # [ 0.386041] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1017client # [ 0.364158] ACPI: Enabled 2 GPEs in block 00 to 3F1018worker1 # [ 0.373075] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1019lb # [ 0.387028] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1020worker1 # [ 0.374264] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1021lb # [ 0.388398] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1022worker1 # [ 0.375191] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1023worker1 # [ 0.376505] PCI host bridge to bus 0000:001024lb # [ 0.389789] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1025worker1 # [ 0.377073] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1026worker1 # [ 0.378069] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1027worker2 # [ 0.347158] ACPI: Added _OSI(Module Device)1028worker2 # [ 0.347893] ACPI: Added _OSI(Processor Device)1029lb # [ 0.393036] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1030worker1 # [ 0.379068] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1031worker2 # [ 0.348442] ACPI: Added _OSI(Processor Aggregator Device)1032lb # [ 0.393887] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1033client # [ 0.372838] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1034worker1 # [ 0.380068] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1035lb # [ 0.394739] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1036worker1 # [ 0.381068] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1037client # [ 0.373822] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1038lb # [ 0.395034] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1039worker1 # [ 0.382069] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1040worker1 # [ 0.383068] pci_bus 0000:00: root bus resource [bus 00-ff]1041client # [ 0.375833] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1042lb # [ 0.397087] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1043client # [ 0.377062] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1044worker1 # [ 0.384043] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1045client # [ 0.378459] PCI host bridge to bus 0000:001046worker2 # [ 0.354643] ACPI: 1 ACPI AML tables successfully acquired and loaded1047worker1 # [ 0.385496] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1048client # [ 0.379063] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1049lb # [ 0.399036] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1050lb # [ 0.399870] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1051client # [ 0.380057] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1052lb # [ 0.400766] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1053client # [ 0.381057] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1054client # [ 0.382057] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1055worker2 # [ 0.358316] ACPI: Interpreter enabled1056lb # [ 0.401584] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1057worker2 # [ 0.358928] ACPI: PM: (supports S0 S3 S4 S5)1058client # [ 0.383057] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1059worker2 # [ 0.359661] ACPI: Using IOAPIC for interrupt routing1060client # [ 0.384057] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1061client # [ 0.385060] pci_bus 0000:00: root bus resource [bus 00-ff]1062client # [ 0.386085] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1063worker1 # [ 0.388156] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1064worker2 # [ 0.361953] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1065lb # [ 0.404278] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1066worker1 # [ 0.389080] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1067lb # [ 0.405641] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1068worker2 # [ 0.362911] PCI: Using E820 reservations for host bridge windows1069client # [ 0.387554] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1070worker1 # [ 0.390020] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1071lb # [ 0.406050] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1072worker1 # [ 0.391446] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1073worker2 # [ 0.366031] ACPI: Enabled 2 GPEs in block 00 to 3F1074lb # [ 0.407615] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1075worker1 # [ 0.392795] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1076lb # [ 0.409060] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1077lb # [ 0.409953] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1078lb # [ 0.410834] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1079worker1 # [ 0.396104] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1080worker1 # [ 0.396981] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1081client # [ 0.390248] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1082worker1 # [ 0.397854] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1083lb # [ 0.413071] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1084worker2 # [ 0.373419] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1085client # [ 0.391140] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1086worker1 # [ 0.398876] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1087client # [ 0.392057] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1088worker2 # [ 0.373917] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1089lb # [ 0.415057] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1090worker2 # [ 0.374988] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1091worker1 # [ 0.400551] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1092lb # [ 0.415938] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1093client # [ 0.393393] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1094worker2 # [ 0.376033] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1095lb # [ 0.416752] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1096worker2 # [ 0.377263] PCI host bridge to bus 0000:001097client # [ 0.394828] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1098worker1 # [ 0.402111] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1099worker2 # [ 0.377914] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1100lb # [ 0.417595] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1101worker1 # [ 0.402955] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1102worker2 # [ 0.378910] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1103worker1 # [ 0.403830] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1104client # [ 0.398091] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1105worker2 # [ 0.379910] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1106client # [ 0.398958] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1107worker1 # [ 0.405589] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1108worker2 # [ 0.380910] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1109lb # [ 0.420057] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1110client # [ 0.399816] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1111worker2 # [ 0.381909] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1112lb # [ 0.421037] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1113client # [ 0.400063] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1114lb # [ 0.421992] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1115worker2 # [ 0.382910] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1116worker2 # [ 0.383910] pci_bus 0000:00: root bus resource [bus 00-ff]1117client # [ 0.401633] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1118lb # [ 0.423494] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1119worker1 # [ 0.408337] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1120worker2 # [ 0.384932] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1121worker1 # [ 0.409695] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1122worker1 # [ 0.410090] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1123lb # [ 0.425036] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1124worker2 # [ 0.386353] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1125client # [ 0.404091] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1126lb # [ 0.425879] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1127client # [ 0.404916] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1128worker1 # [ 0.411655] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1129lb # [ 0.426799] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1130client # [ 0.405860] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1131lb # [ 0.427870] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1132worker1 # [ 0.413099] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1133client # [ 0.407452] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1134worker1 # [ 0.413980] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1135lb # [ 0.429880] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1136worker1 # [ 0.414839] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1137lb # [ 0.432062] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1138worker2 # [ 0.389103] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1139client # [ 0.411091] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1140lb # [ 0.433050] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1141worker1 # [ 0.416496] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1142worker2 # [ 0.389923] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1143client # [ 0.411897] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144worker2 # [ 0.390901] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1145client # [ 0.412778] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1146lb # [ 0.434603] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1147worker1 # [ 0.419100] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1148worker2 # [ 0.392202] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1149worker1 # [ 0.420000] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1150client # [ 0.413621] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1151worker1 # [ 0.420827] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1152lb # [ 0.436901] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1153worker2 # [ 0.393628] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1154client # [ 0.415920] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1155lb # [ 0.437700] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1156worker1 # [ 0.421654] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1157client # [ 0.416701] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158lb # [ 0.438051] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1159worker2 # [ 0.396945] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1160client # [ 0.417079] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1161lb # [ 0.439574] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1162worker2 # [ 0.397829] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1163worker1 # [ 0.424100] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1164client # [ 0.418612] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1165worker2 # [ 0.398706] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1166worker1 # [ 0.425076] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1167lb # [ 0.441057] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1168worker2 # [ 0.399754] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1169worker1 # [ 0.426031] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1170lb # [ 0.441907] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1171client # [ 0.420885] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1172lb # [ 0.442806] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1173worker2 # [ 0.401243] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1174worker1 # [ 0.427486] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1175client # [ 0.421668] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1176client # [ 0.422079] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1177lb # [ 0.444436] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1178worker1 # [ 0.429102] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1179worker1 # [ 0.430026] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1180client # [ 0.423654] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1181lb # [ 0.446387] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1182worker2 # [ 0.404041] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1183worker1 # [ 0.430837] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1184worker2 # [ 0.404916] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1185lb # [ 0.447223] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1186worker1 # [ 0.431947] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1187worker2 # [ 0.405901] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1188lb # [ 0.448625] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1189client # [ 0.426077] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1190worker2 # [ 0.407369] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1191worker1 # [ 0.434111] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1192client # [ 0.426928] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1193client # [ 0.427783] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1194worker1 # [ 0.436069] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1195worker2 # [ 0.408942] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1196client # [ 0.428631] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1197worker2 # [ 0.409805] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1198worker1 # [ 0.437090] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1199worker2 # [ 0.410639] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1200worker1 # [ 0.438603] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1201client # [ 0.430066] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1202worker2 # [ 0.411502] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1203client # [ 0.430925] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1204worker1 # [ 0.440085] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1205client # [ 0.431837] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1206worker2 # [ 0.413944] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1207worker1 # [ 0.440959] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1208client # [ 0.433064] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1209worker2 # [ 0.414839] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1210worker1 # [ 0.442090] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1211worker2 # [ 0.415663] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1212client # [ 0.434575] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1213lb # [ 0.449219] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1214worker1 # [ 0.443640] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1215worker2 # [ 0.416476] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1216lb # [ 0.451683] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1217client # [ 0.436080] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1218lb # [ 0.452263] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1219worker1 # [ 0.445099] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1220client # [ 0.437057] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1221worker1 # [ 0.445925] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1222worker2 # [ 0.417944] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1223lb # [ 0.454008] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1224worker1 # [ 0.446854] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1225client # [ 0.438594] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1226worker2 # [ 0.418917] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1227lb # [ 0.455005] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1228worker2 # [ 0.419852] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1229worker1 # [ 0.448451] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1230lb # [ 0.456313] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1231worker2 # [ 0.421307] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1232client # [ 0.440076] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1233worker1 # [ 0.450506] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1234client # [ 0.441679] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1235lb # [ 0.457313] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1236client # [ 0.442079] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1237worker1 # [ 0.451290] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1238worker2 # [ 0.422945] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1239lb # [ 0.458991] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1240worker2 # [ 0.423847] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1241worker1 # [ 0.452758] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1242lb # [ 0.459689] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1243client # [ 0.443624] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1244worker2 # [ 0.424669] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1245lb # [ 0.460338] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1246client # [ 0.445722] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1247worker2 # [ 0.425532] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1248lb # [ 0.461655] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1249client # [ 0.446086] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1250client # [ 0.447057] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1251worker2 # [ 0.426943] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1252worker2 # [ 0.427813] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1253worker2 # [ 0.428707] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1254lb # [ 0.464968] ACPI: PCI: Interrupt link LNKA configured for IRQ 101255client # [ 0.448652] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1256worker2 # [ 0.429799] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1257lb # [ 0.465885] ACPI: PCI: Interrupt link LNKB configured for IRQ 101258client # [ 0.450894] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1259lb # [ 0.466917] ACPI: PCI: Interrupt link LNKC configured for IRQ 111260worker2 # [ 0.431262] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1261lb # [ 0.467825] ACPI: PCI: Interrupt link LNKD configured for IRQ 111262client # [ 0.451900] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1263lb # [ 0.468833] ACPI: PCI: Interrupt link LNKE configured for IRQ 101264worker2 # [ 0.432950] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1265client # [ 0.453772] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1266worker1 # [ 0.453258] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1267lb # [ 0.470084] ACPI: PCI: Interrupt link LNKF configured for IRQ 101268worker2 # [ 0.433934] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1269lb # [ 0.471128] ACPI: PCI: Interrupt link LNKG configured for IRQ 111270worker1 # [ 0.455699] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1271lb # [ 0.472124] ACPI: PCI: Interrupt link LNKH configured for IRQ 111272worker1 # [ 0.456292] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1273worker2 # [ 0.435497] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1274lb # [ 0.473062] ACPI: PCI: Interrupt link GSIA configured for IRQ 161275lb # [ 0.474019] ACPI: PCI: Interrupt link GSIB configured for IRQ 171276worker1 # [ 0.458039] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1277worker2 # [ 0.436918] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1278lb # [ 0.474836] ACPI: PCI: Interrupt link GSIC configured for IRQ 181279worker2 # [ 0.437757] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1280lb # [ 0.475779] ACPI: PCI: Interrupt link GSID configured for IRQ 191281worker1 # [ 0.459110] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1282worker2 # [ 0.438655] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1283lb # [ 0.476044] ACPI: PCI: Interrupt link GSIE configured for IRQ 201284lb # [ 0.477022] ACPI: PCI: Interrupt link GSIF configured for IRQ 211285worker1 # [ 0.460380] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1286worker2 # [ 0.440351] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1287lb # [ 0.478015] ACPI: PCI: Interrupt link GSIG configured for IRQ 221288worker1 # [ 0.461315] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1289lb # [ 0.478780] ACPI: PCI: Interrupt link GSIH configured for IRQ 231290worker1 # [ 0.463104] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1291lb # [ 0.480799] iommu: Default domain type: Translated1292worker2 # [ 0.442612] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1293worker1 # [ 0.463981] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1294client # [ 0.454252] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1295lb # [ 0.481037] iommu: DMA domain TLB invalidation policy: lazy mode1296worker2 # [ 0.442936] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1297lb # [ 0.482239] ACPI: bus type USB registered1298worker2 # [ 0.443926] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1299client # [ 0.456871] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1300worker1 # [ 0.465108] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1301lb # [ 0.482970] usbcore: registered new interface driver usbfs1302lb # [ 0.483710] usbcore: registered new interface driver hub1303client # [ 0.458113] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1304worker1 # [ 0.466725] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1305worker2 # [ 0.445483] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1306lb # [ 0.484038] usbcore: registered new device driver usb1307worker2 # [ 0.446430] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1308client # [ 0.459771] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1309lb # [ 0.485751] NetLabel: Initializing1310lb # [ 0.486035] NetLabel: domain hash size = 1281311worker2 # [ 0.447145] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1312client # [ 0.460318] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1313lb # [ 0.486724] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1314worker1 # [ 0.470057] ACPI: PCI: Interrupt link LNKA configured for IRQ 101315worker2 # [ 0.448432] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1316lb # [ 0.487067] NetLabel: unlabeled traffic allowed by default1317client # [ 0.461371] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1318worker1 # [ 0.471167] ACPI: PCI: Interrupt link LNKB configured for IRQ 101319lb # [ 0.487968] PCI: Using ACPI for IRQ routing1320worker1 # [ 0.472170] ACPI: PCI: Interrupt link LNKC configured for IRQ 111321client # [ 0.462313] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1322worker1 # [ 0.473179] ACPI: PCI: Interrupt link LNKD configured for IRQ 111323client # [ 0.464100] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1324worker1 # [ 0.474165] ACPI: PCI: Interrupt link LNKE configured for IRQ 101325worker1 # [ 0.475196] ACPI: PCI: Interrupt link LNKF configured for IRQ 101326client # [ 0.464937] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1327worker1 # [ 0.476168] ACPI: PCI: Interrupt link LNKG configured for IRQ 111328client # [ 0.466388] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1329worker1 # [ 0.477175] ACPI: PCI: Interrupt link LNKH configured for IRQ 111330worker1 # [ 0.478106] ACPI: PCI: Interrupt link GSIA configured for IRQ 161331client # [ 0.467875] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1332worker1 # [ 0.479084] ACPI: PCI: Interrupt link GSIB configured for IRQ 171333worker1 # [ 0.480083] ACPI: PCI: Interrupt link GSIC configured for IRQ 181334worker1 # [ 0.481080] ACPI: PCI: Interrupt link GSID configured for IRQ 191335client # [ 0.471679] ACPI: PCI: Interrupt link LNKA configured for IRQ 101336worker1 # [ 0.482080] ACPI: PCI: Interrupt link GSIE configured for IRQ 201337worker2 # [ 0.449100] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1338worker1 # [ 0.483068] ACPI: PCI: Interrupt link GSIF configured for IRQ 211339client # [ 0.472163] ACPI: PCI: Interrupt link LNKB configured for IRQ 101340client # [ 0.473157] ACPI: PCI: Interrupt link LNKC configured for IRQ 111341worker1 # [ 0.484042] ACPI: PCI: Interrupt link GSIG configured for IRQ 221342worker1 # [ 0.484965] ACPI: PCI: Interrupt link GSIH configured for IRQ 231343client # [ 0.474155] ACPI: PCI: Interrupt link LNKD configured for IRQ 111344client # [ 0.475166] ACPI: PCI: Interrupt link LNKE configured for IRQ 101345worker2 # [ 0.452461] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1346worker1 # [ 0.486951] iommu: Default domain type: Translated1347client # [ 0.476153] ACPI: PCI: Interrupt link LNKF configured for IRQ 101348worker1 # [ 0.487707] iommu: DMA domain TLB invalidation policy: lazy mode1349worker2 # [ 0.453146] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1350client # [ 0.477160] ACPI: PCI: Interrupt link LNKG configured for IRQ 111351worker1 # [ 0.488321] ACPI: bus type USB registered1352client # [ 0.478153] ACPI: PCI: Interrupt link LNKH configured for IRQ 111353worker1 # [ 0.489113] usbcore: registered new interface driver usbfs1354worker2 # [ 0.454889] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1355client # [ 0.479095] ACPI: PCI: Interrupt link GSIA configured for IRQ 161356worker1 # [ 0.490033] usbcore: registered new interface driver hub1357client # [ 0.480041] ACPI: PCI: Interrupt link GSIB configured for IRQ 171358worker1 # [ 0.490790] usbcore: registered new device driver usb1359worker2 # [ 0.455952] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1360client # [ 0.480815] ACPI: PCI: Interrupt link GSIC configured for IRQ 181361worker1 # [ 0.491972] NetLabel: Initializing1362worker2 # [ 0.457236] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1363client # [ 0.481070] ACPI: PCI: Interrupt link GSID configured for IRQ 191364worker1 # [ 0.492476] NetLabel: domain hash size = 1281365client # [ 0.482062] ACPI: PCI: Interrupt link GSIE configured for IRQ 201366worker2 # [ 0.458167] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1367worker1 # [ 0.493068] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1368client # [ 0.483070] ACPI: PCI: Interrupt link GSIF configured for IRQ 211369worker1 # [ 0.494113] NetLabel: unlabeled traffic allowed by default1370client # [ 0.484073] ACPI: PCI: Interrupt link GSIG configured for IRQ 221371worker1 # [ 0.495132] PCI: Using ACPI for IRQ routing1372client # [ 0.485072] ACPI: PCI: Interrupt link GSIH configured for IRQ 231373worker2 # [ 0.460645] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1374worker2 # [ 0.460917] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1375client # [ 0.487181] iommu: Default domain type: Translated1376worker2 # [ 0.462239] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1377client # [ 0.488036] iommu: DMA domain TLB invalidation policy: lazy mode1378client # [ 0.489114] ACPI: bus type USB registered1379worker2 # [ 0.463647] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1380client # [ 0.489888] usbcore: registered new interface driver usbfs1381client # [ 0.490772] usbcore: registered new interface driver hub1382client # [ 0.491069] usbcore: registered new device driver usb1383client # [ 0.492880] NetLabel: Initializing1384worker2 # [ 0.466920] ACPI: PCI: Interrupt link LNKA configured for IRQ 101385client # [ 0.493509] NetLabel: domain hash size = 1281386worker2 # [ 0.468020] ACPI: PCI: Interrupt link LNKB configured for IRQ 101387client # [ 0.494062] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1388worker2 # [ 0.469012] ACPI: PCI: Interrupt link LNKC configured for IRQ 111389client # [ 0.495141] NetLabel: unlabeled traffic allowed by default1390client # [ 0.496057] PCI: Using ACPI for IRQ routing1391worker2 # [ 0.470006] ACPI: PCI: Interrupt link LNKD configured for IRQ 111392worker2 # [ 0.472053] ACPI: PCI: Interrupt link LNKE configured for IRQ 101393worker2 # [ 0.473029] ACPI: PCI: Interrupt link LNKF configured for IRQ 101394worker2 # [ 0.474014] ACPI: PCI: Interrupt link LNKG configured for IRQ 111395worker2 # [ 0.475008] ACPI: PCI: Interrupt link LNKH configured for IRQ 111396lb # [ 0.531599] pci 0000:00:01.0: vgaarb: setting as boot VGA device1397worker2 # [ 0.475947] ACPI: PCI: Interrupt link GSIA configured for IRQ 161398lb # [ 0.532024] pci 0000:00:01.0: vgaarb: bridge control possible1399worker2 # [ 0.476925] ACPI: PCI: Interrupt link GSIB configured for IRQ 171400lb # [ 0.532024] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1401lb # [ 0.532034] vgaarb: loaded1402worker2 # [ 0.477927] ACPI: PCI: Interrupt link GSIC configured for IRQ 181403lb # [ 0.532734] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01404worker2 # [ 0.478923] ACPI: PCI: Interrupt link GSID configured for IRQ 191405lb # [ 0.533030] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1406worker2 # [ 0.479925] ACPI: PCI: Interrupt link GSIE configured for IRQ 201407worker2 # [ 0.480891] ACPI: PCI: Interrupt link GSIF configured for IRQ 211408worker2 # [ 0.481731] ACPI: PCI: Interrupt link GSIG configured for IRQ 221409worker2 # [ 0.482747] ACPI: PCI: Interrupt link GSIH configured for IRQ 231410lb # [ 0.536112] clocksource: Switched to clocksource kvm-clock1411worker2 # [ 0.484779] iommu: Default domain type: Translated1412worker2 # [ 0.485552] iommu: DMA domain TLB invalidation policy: lazy mode1413lb # [ 0.539585] VFS: Disk quotas dquot_6.6.01414worker2 # [ 0.486166] ACPI: bus type USB registered1415lb # [ 0.540313] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1416worker2 # [ 0.486951] usbcore: registered new interface driver usbfs1417lb # [ 0.541652] pnp: PnP ACPI init1418worker2 # [ 0.487886] usbcore: registered new interface driver hub1419worker2 # [ 0.488586] usbcore: registered new device driver usb1420lb # [ 0.542480] ACPI: IRQ 4 override to edge(!), high(!)1421lb # [ 0.543493] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1422worker2 # [ 0.489851] NetLabel: Initializing1423worker2 # [ 0.490346] NetLabel: domain hash size = 1281424lb # [ 0.545012] pnp: PnP ACPI: found 6 devices1425worker2 # [ 0.490918] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1426worker2 # [ 0.491985] NetLabel: unlabeled traffic allowed by default1427worker2 # [ 0.492924] PCI: Using ACPI for IRQ routing1428lb # [ 0.552664] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1429lb # [ 0.554153] clocksource: Switched to clocksource acpi_pm1430worker1 # [ 0.539564] pci 0000:00:01.0: vgaarb: setting as boot VGA device1431lb # [ 0.555129] NET: Registered PF_INET protocol family1432worker1 # [ 0.540064] pci 0000:00:01.0: vgaarb: bridge control possible1433worker1 # [ 0.540064] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1434lb # [ 0.556269] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1435worker1 # [ 0.540073] vgaarb: loaded1436worker1 # [ 0.540796] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01437worker1 # [ 0.541069] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1438worker1 # [ 0.544173] clocksource: Switched to clocksource kvm-clock1439client # [ 0.540226] pci 0000:00:01.0: vgaarb: setting as boot VGA device1440worker1 # [ 0.547967] VFS: Disk quotas dquot_6.6.01441client # [ 0.541053] pci 0000:00:01.0: vgaarb: bridge control possible1442worker1 # [ 0.548668] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1443worker1 # [ 0.550005] pnp: PnP ACPI init1444client # [ 0.541053] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1445client # [ 0.541062] vgaarb: loaded1446worker1 # [ 0.550772] ACPI: IRQ 4 override to edge(!), high(!)1447client # [ 0.541801] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01448worker1 # [ 0.551804] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1449client # [ 0.542059] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1450worker1 # [ 0.553307] pnp: PnP ACPI: found 6 devices1451lb # [ 0.572601] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1452lb # [ 0.574178] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1453client # [ 0.545143] clocksource: Switched to clocksource kvm-clock1454lb # [ 0.575547] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1455lb # [ 0.576921] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1456lb # [ 0.578215] TCP: Hash tables configured (established 16384 bind 16384)1457client # [ 0.548625] VFS: Disk quotas dquot_6.6.01458lb # [ 0.579371] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1459client # [ 0.549412] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1460worker1 # [ 0.560877] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1461lb # [ 0.580621] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1462worker1 # [ 0.562263] clocksource: Switched to clocksource acpi_pm1463client # [ 0.551069] pnp: PnP ACPI init1464lb # [ 0.581692] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1465worker1 # [ 0.563236] NET: Registered PF_INET protocol family1466client # [ 0.551903] ACPI: IRQ 4 override to edge(!), high(!)1467lb # [ 0.582876] NET: Registered PF_UNIX/PF_LOCAL protocol family1468worker1 # [ 0.564398] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1469client # [ 0.552897] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1470lb # [ 0.583854] NET: Registered PF_XDP protocol family1471client # [ 0.554404] pnp: PnP ACPI: found 6 devices1472lb # [ 0.584688] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1473lb # [ 0.585730] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1474lb # [ 0.586780] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1475lb # [ 0.587922] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1476lb # [ 0.589070] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1477lb # [ 0.590235] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1478worker2 # [ 0.536628] pci 0000:00:01.0: vgaarb: setting as boot VGA device1479lb # [ 0.592019] ACPI: \_SB_.GSIA: Enabled at IRQ 161480worker2 # [ 0.536906] pci 0000:00:01.0: vgaarb: bridge control possible1481client # [ 0.562069] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1482worker2 # [ 0.536906] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1483client # [ 0.563555] clocksource: Switched to clocksource acpi_pm1484lb # [ 0.594215] ACPI: \_SB_.GSIB: Enabled at IRQ 171485worker2 # [ 0.536914] vgaarb: loaded1486client # [ 0.564535] NET: Registered PF_INET protocol family1487worker2 # [ 0.537630] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01488worker2 # [ 0.537910] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1489lb # [ 0.596489] ACPI: \_SB_.GSIC: Enabled at IRQ 181490client # [ 0.565807] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1491lb # [ 0.598543] ACPI: \_SB_.GSID: Enabled at IRQ 191492lb # [ 0.600339] PCI: CLS 0 bytes, default 641493worker2 # [ 0.540998] clocksource: Switched to clocksource kvm-clock1494worker1 # [ 0.581080] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1495worker1 # [ 0.582610] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1496lb # [ 0.601230] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1497lb # [ 0.602993] Trying to unpack rootfs image as initramfs...1498worker1 # [ 0.583895] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1499worker2 # [ 0.544445] VFS: Disk quotas dquot_6.6.01500worker1 # [ 0.585184] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1501worker2 # [ 0.545190] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1502worker1 # [ 0.586400] TCP: Hash tables configured (established 16384 bind 16384)1503worker2 # [ 0.546580] pnp: PnP ACPI init1504worker1 # [ 0.587495] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1505worker2 # [ 0.547376] ACPI: IRQ 4 override to edge(!), high(!)1506worker1 # [ 0.588711] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1507worker2 # [ 0.548407] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1508worker1 # [ 0.589762] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1509worker2 # [ 0.549984] pnp: PnP ACPI: found 6 devices1510worker1 # [ 0.590952] NET: Registered PF_UNIX/PF_LOCAL protocol family1511worker1 # [ 0.591866] NET: Registered PF_XDP protocol family1512worker1 # [ 0.592642] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1513worker1 # [ 0.593626] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1514client # [ 0.582266] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1515worker1 # [ 0.594602] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1516client # [ 0.583843] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1517worker1 # [ 0.595674] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1518worker1 # [ 0.596738] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1519client # [ 0.585172] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1520worker1 # [ 0.597795] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1521client # [ 0.586525] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1522worker2 # [ 0.557744] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1523worker1 # [ 0.599529] ACPI: \_SB_.GSIA: Enabled at IRQ 161524client # [ 0.587817] TCP: Hash tables configured (established 16384 bind 16384)1525worker2 # [ 0.559184] clocksource: Switched to clocksource acpi_pm1526worker2 # [ 0.560171] NET: Registered PF_INET protocol family1527client # [ 0.588946] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1528worker1 # [ 0.601822] ACPI: \_SB_.GSIB: Enabled at IRQ 171529client # [ 0.590232] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1530worker2 # [ 0.561335] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1531client # [ 0.591337] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1532worker1 # [ 0.604026] ACPI: \_SB_.GSIC: Enabled at IRQ 181533client # [ 0.592583] NET: Registered PF_UNIX/PF_LOCAL protocol family1534client # [ 0.593568] NET: Registered PF_XDP protocol family1535worker1 # [ 0.605979] ACPI: \_SB_.GSID: Enabled at IRQ 191536client # [ 0.594464] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1537client # [ 0.595503] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1538worker1 # [ 0.607685] PCI: CLS 0 bytes, default 641539client # [ 0.596555] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1540worker1 # [ 0.608587] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1541client # [ 0.597711] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1542worker1 # [ 0.610289] Trying to unpack rootfs image as initramfs...1543client # [ 0.598876] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1544client # [ 0.600024] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1545client # [ 0.601927] ACPI: \_SB_.GSIA: Enabled at IRQ 161546client # [ 0.604601] ACPI: \_SB_.GSIB: Enabled at IRQ 171547client # [ 0.606584] ACPI: \_SB_.GSIC: Enabled at IRQ 181548worker2 # [ 0.578090] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1549client # [ 0.608567] ACPI: \_SB_.GSID: Enabled at IRQ 191550worker2 # [ 0.579677] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1551client # [ 0.610298] PCI: CLS 0 bytes, default 641552worker2 # [ 0.581013] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1553worker2 # [ 0.582379] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1554client # [ 0.611176] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1555worker2 # [ 0.583642] TCP: Hash tables configured (established 16384 bind 16384)1556client # [ 0.612903] Trying to unpack rootfs image as initramfs...1557worker2 # [ 0.584787] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1558worker2 # [ 0.586094] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1559worker2 # [ 0.587216] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1560worker2 # [ 0.588467] NET: Registered PF_UNIX/PF_LOCAL protocol family1561lb # [ 0.648265] Initialise system trusted keyrings1562worker2 # [ 0.589467] NET: Registered PF_XDP protocol family1563lb # [ 0.649222] workingset: timestamp_bits=40 max_order=19 bucket_order=01564worker2 # [ 0.590323] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1565worker2 # [ 0.591358] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1566worker2 # [ 0.592411] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1567worker2 # [ 0.593578] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1568worker2 # [ 0.594729] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1569worker2 # [ 0.595880] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1570worker2 # [ 0.597716] ACPI: \_SB_.GSIA: Enabled at IRQ 161571worker2 # [ 0.599924] ACPI: \_SB_.GSIB: Enabled at IRQ 171572worker2 # [ 0.602062] ACPI: \_SB_.GSIC: Enabled at IRQ 181573worker2 # [ 0.604022] ACPI: \_SB_.GSID: Enabled at IRQ 191574worker2 # [ 0.605677] PCI: CLS 0 bytes, default 641575worker2 # [ 0.606617] Trying to unpack rootfs image as initramfs...1576worker2 # [ 0.610881] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1577lb # [ 0.672435] Key type asymmetric registered1578lb # [ 0.673158] Asymmetric key parser 'x509' registered1579worker1 # [ 0.654957] Initialise system trusted keyrings1580lb # [ 0.674064] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1581worker1 # [ 0.657900] workingset: timestamp_bits=40 max_order=19 bucket_order=01582lb # [ 0.679483] io scheduler mq-deadline registered1583lb # [ 0.680277] io scheduler kyber registered1584lb # [ 0.682846] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1585lb # [ 0.684116] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1586client # [ 0.656706] Initialise system trusted keyrings1587lb # [ 0.689705] Linux agpgart interface v0.1031588lb # [ 0.690505] ACPI: bus type drm_connector registered1589client # [ 0.659808] workingset: timestamp_bits=40 max_order=19 bucket_order=01590lb # [ 0.693764] usbcore: registered new interface driver usbserial_generic1591lb # [ 0.694838] usbserial: USB Serial support registered for generic1592worker1 # [ 0.679181] Key type asymmetric registered1593worker1 # [ 0.679898] Asymmetric key parser 'x509' registered1594lb # [ 0.698336] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1595lb # [ 0.698336] Please enable it if your BIOS has the CPPC option.1596worker1 # [ 0.680703] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1597lb # [ 0.700515] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1598lb # [ 0.704440] drop_monitor: Initializing network drop monitor service1599worker1 # [ 0.686014] io scheduler mq-deadline registered1600worker1 # [ 0.686743] io scheduler kyber registered1601lb # [ 0.705670] NET: Registered PF_INET6 protocol family1602lb # [ 0.708423] Segment Routing with IPv61603lb # [ 0.709101] In-situ OAM (IOAM) with IPv61604worker1 # [ 0.690415] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1605worker2 # [ 0.650747] Initialise system trusted keyrings1606worker1 # [ 0.691661] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1607client # [ 0.681118] Key type asymmetric registered1608lb # [ 0.711590] IPI shorthand broadcast: enabled1609worker2 # [ 0.653546] workingset: timestamp_bits=40 max_order=19 bucket_order=01610client # [ 0.683796] Asymmetric key parser 'x509' registered1611worker1 # [ 0.695673] Linux agpgart interface v0.1031612worker1 # [ 0.696458] ACPI: bus type drm_connector registered1613client # [ 0.684673] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1614client # [ 0.687904] io scheduler mq-deadline registered1615client # [ 0.688693] io scheduler kyber registered1616lb # [ 0.719835] sched_clock: Marking stable (583016837, 136298418)->(792221999, -72906744)1617worker1 # [ 0.701274] usbcore: registered new interface driver usbserial_generic1618worker1 # [ 0.702339] usbserial: USB Serial support registered for generic1619client # [ 0.691342] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1620worker1 # [ 0.703328] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1621client # [ 0.692635] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1622worker1 # [ 0.703328] Please enable it if your BIOS has the CPPC option.1623lb # [ 0.724414] registered taskstats version 11624worker1 # [ 0.705468] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1625lb # [ 0.725348] Loading compiled-in X.509 certificates1626client # [ 0.698378] Linux agpgart interface v0.1031627client # [ 0.699168] ACPI: bus type drm_connector registered1628worker1 # [ 0.711017] drop_monitor: Initializing network drop monitor service1629worker1 # [ 0.712134] NET: Registered PF_INET6 protocol family1630client # [ 0.702178] usbcore: registered new interface driver usbserial_generic1631client # [ 0.703283] usbserial: USB Serial support registered for generic1632worker1 # [ 0.715305] Segment Routing with IPv61633worker2 # [ 0.676564] Key type asymmetric registered1634worker1 # [ 0.717872] In-situ OAM (IOAM) with IPv61635worker2 # [ 0.677333] Asymmetric key parser 'x509' registered1636client # [ 0.705791] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1637client # [ 0.705791] Please enable it if your BIOS has the CPPC option.1638worker1 # [ 0.718901] IPI shorthand broadcast: enabled1639worker2 # [ 0.678216] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1640client # [ 0.707975] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1641worker2 # [ 0.681665] io scheduler mq-deadline registered1642lb # [ 0.741792] Demotion targets for Node 0: null1643client # [ 0.711906] drop_monitor: Initializing network drop monitor service1644client # [ 0.713097] NET: Registered PF_INET6 protocol family1645worker2 # [ 0.684539] io scheduler kyber registered1646lb # [ 0.745368] Key type .fscrypt registered1647worker2 # [ 0.685901] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1648lb # [ 0.746018] Key type fscrypt-provisioning registered1649worker1 # [ 0.727047] sched_clock: Marking stable (593013208, 133830671)->(797835341, -70991462)1650client # [ 0.716222] Segment Routing with IPv61651lb # [ 0.746925] ima: No TPM chip found, activating TPM-bypass!1652worker2 # [ 0.687180] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1653client # [ 0.718804] In-situ OAM (IOAM) with IPv61654lb # [ 0.749332] ima: Allocated hash algorithm: sha11655worker1 # [ 0.730936] registered taskstats version 11656lb # [ 0.750113] ima: No architecture policies found1657client # [ 0.719887] IPI shorthand broadcast: enabled1658worker2 # [ 0.691303] Linux agpgart interface v0.1031659worker1 # [ 0.731862] Loading compiled-in X.509 certificates1660worker2 # [ 0.692100] ACPI: bus type drm_connector registered1661lb # [ 0.752327] PM: Magic number: 10:369:8311662lb # [ 0.753922] RAS: Correctable Errors collector initialized.1663worker2 # [ 0.696947] usbcore: registered new interface driver usbserial_generic1664worker2 # [ 0.698059] usbserial: USB Serial support registered for generic1665client # [ 0.728137] sched_clock: Marking stable (594014067, 133737772)->(798340688, -70588849)1666worker2 # [ 0.701515] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1667worker2 # [ 0.701515] Please enable it if your BIOS has the CPPC option.1668client # [ 0.733007] registered taskstats version 11669worker2 # [ 0.703798] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1670lb # [ 0.763975] clk: Disabling unused clocks1671client # [ 0.733974] Loading compiled-in X.509 certificates1672lb # [ 0.766332] PM: genpd: Disabling unused power domains1673worker2 # [ 0.707643] drop_monitor: Initializing network drop monitor service1674worker2 # [ 0.708886] NET: Registered PF_INET6 protocol family1675worker1 # [ 0.749860] Demotion targets for Node 0: null1676worker1 # [ 0.750758] Key type .fscrypt registered1677worker2 # [ 0.711954] Segment Routing with IPv61678worker1 # [ 0.752864] Key type fscrypt-provisioning registered1679worker1 # [ 0.753753] ima: No TPM chip found, activating TPM-bypass!1680worker1 # [ 0.754650] ima: Allocated hash algorithm: sha11681worker2 # [ 0.714527] In-situ OAM (IOAM) with IPv61682worker2 # [ 0.715633] IPI shorthand broadcast: enabled1683worker1 # [ 0.756884] ima: No architecture policies found1684worker1 # [ 0.758866] PM: Magic number: 10:369:8311685worker1 # [ 0.760476] RAS: Correctable Errors collector initialized.1686client # [ 0.752768] Demotion targets for Node 0: null1687worker2 # [ 0.723922] sched_clock: Marking stable (590013795, 133477087)->(797053564, -73562682)1688client # [ 0.753655] Key type .fscrypt registered1689client # [ 0.754365] Key type fscrypt-provisioning registered1690client # [ 0.755325] ima: No TPM chip found, activating TPM-bypass!1691worker2 # [ 0.728747] registered taskstats version 11692worker1 # [ 0.769534] clk: Disabling unused clocks1693worker2 # [ 0.729737] Loading compiled-in X.509 certificates1694client # [ 0.759768] ima: Allocated hash algorithm: sha11695client # [ 0.760572] ima: No architecture policies found1696worker1 # [ 0.772869] PM: genpd: Disabling unused power domains1697client # [ 0.762770] PM: Magic number: 10:369:8311698client # [ 0.764454] RAS: Correctable Errors collector initialized.1699client # [ 0.773450] clk: Disabling unused clocks1700client # [ 0.775770] PM: genpd: Disabling unused power domains1701worker2 # [ 0.748506] Demotion targets for Node 0: null1702worker2 # [ 0.749452] Key type .fscrypt registered1703worker2 # [ 0.750207] Key type fscrypt-provisioning registered1704worker2 # [ 0.751204] ima: No TPM chip found, activating TPM-bypass!1705worker2 # [ 0.755516] ima: Allocated hash algorithm: sha11706worker2 # [ 0.756365] ima: No architecture policies found1707worker2 # [ 0.758510] PM: Magic number: 10:369:8311708worker2 # [ 0.760199] RAS: Correctable Errors collector initialized.1709worker2 # [ 0.769193] clk: Disabling unused clocks1710worker2 # [ 0.772525] PM: genpd: Disabling unused power domains1711lb # [ 0.931004] Freeing initrd memory: 29092K1712lb # [ 0.934107] Freeing unused decrypted memory: 2028K1713lb # [ 0.936693] Freeing unused kernel image (initmem) memory: 3644K1714lb # [ 0.937745] Write protecting the kernel read-only data: 32768k1715lb # [ 0.939588] Freeing unused kernel image (text/rodata gap) memory: 1208K1716lb # [ 0.941040] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker1 # [ 0.936918] Freeing initrd memory: 29092K1718worker1 # [ 0.940182] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.942947] Freeing unused kernel image (initmem) memory: 3644K1720worker1 # [ 0.944047] Write protecting the kernel read-only data: 32768k1721worker1 # [ 0.946055] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker1 # [ 0.947632] Freeing unused kernel image (rodata/data gap) memory: 752K1723client # [ 0.949014] Freeing initrd memory: 29096K1724client # [ 0.952310] Freeing unused decrypted memory: 2028K1725client # [ 0.955052] Freeing unused kernel image (initmem) memory: 3644K1726client # [ 0.956170] Write protecting the kernel read-only data: 32768k1727client # [ 0.958114] Freeing unused kernel image (text/rodata gap) memory: 1208K1728client # [ 0.959673] Freeing unused kernel image (rodata/data gap) memory: 752K1729lb # [ 0.992120] x86/mm: Checked W+X mappings: passed, no W+X pages found.1730lb # [ 0.993166] Run /init as init process1731lb # [ 1.003361] systemd[1]: Inserted module 'autofs4'1732worker2 # [ 0.944057] Freeing initrd memory: 29092K1733worker2 # [ 0.947253] Freeing unused decrypted memory: 2028K1734worker2 # [ 0.949965] Freeing unused kernel image (initmem) memory: 3644K1735worker2 # [ 0.951003] Write protecting the kernel read-only data: 32768k1736worker2 # [ 0.952855] Freeing unused kernel image (text/rodata gap) memory: 1208K1737worker2 # [ 0.954320] Freeing unused kernel image (rodata/data gap) memory: 752K1738worker1 # [ 0.998778] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739worker1 # [ 0.999920] Run /init as init process1740lb # [ 1.019501] fuse: init (API version 7.45)1741lb # [ 1.025382] ACPI: \_SB_.GSIG: Enabled at IRQ 221742lb # [ 1.027710] ACPI: \_SB_.GSIH: Enabled at IRQ 231743worker1 # [ 1.010372] systemd[1]: Inserted module 'autofs4'1744lb # [ 1.030513] ACPI: \_SB_.GSIE: Enabled at IRQ 201745lb # [ 1.032673] ACPI: \_SB_.GSIF: Enabled at IRQ 211746client # [ 1.011115] x86/mm: Checked W+X mappings: passed, no W+X pages found.1747client # [ 1.012198] Run /init as init process1748worker1 # [ 1.026655] fuse: init (API version 7.45)1749worker1 # [ 1.032516] ACPI: \_SB_.GSIG: Enabled at IRQ 221750client # [ 1.022517] systemd[1]: Inserted module 'autofs4'1751worker1 # [ 1.034975] ACPI: \_SB_.GSIH: Enabled at IRQ 231752lb # [ 1.055984] systemd[1]: Successfully made /usr/ read-only.1753worker1 # [ 1.037922] ACPI: \_SB_.GSIE: Enabled at IRQ 201754worker1 # [ 1.040116] ACPI: \_SB_.GSIF: Enabled at IRQ 211755worker2 # [ 1.005434] x86/mm: Checked W+X mappings: passed, no W+X pages found.1756worker2 # [ 1.006536] Run /init as init process1757client # [ 1.038954] fuse: init (API version 7.45)1758client # [ 1.044870] ACPI: \_SB_.GSIG: Enabled at IRQ 221759worker2 # [ 1.016677] systemd[1]: Inserted module 'autofs4'1760client # [ 1.047607] ACPI: \_SB_.GSIH: Enabled at IRQ 231761client # [ 1.051435] ACPI: \_SB_.GSIE: Enabled at IRQ 201762client # [ 1.053937] ACPI: \_SB_.GSIF: Enabled at IRQ 211763worker1 # [ 1.069092] systemd[1]: Successfully made /usr/ read-only.1764worker2 # [ 1.033005] fuse: init (API version 7.45)1765worker2 # [ 1.038809] ACPI: \_SB_.GSIG: Enabled at IRQ 221766worker2 # [ 1.041276] ACPI: \_SB_.GSIH: Enabled at IRQ 231767worker2 # [ 1.044156] ACPI: \_SB_.GSIE: Enabled at IRQ 201768worker2 # [ 1.046397] ACPI: \_SB_.GSIF: Enabled at IRQ 211769client # [ 1.079956] systemd[1]: Successfully made /usr/ read-only.1770worker2 # [ 1.071169] systemd[1]: Successfully made /usr/ read-only.1771lb # [ 1.391444] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772lb # [ 1.396781] systemd[1]: Detected virtualization kvm.1773lb # [ 1.397671] systemd[1]: Detected architecture x86-64.1774lb # [ 1.398594] systemd[1]: Running in initrd.1775lb # [ 1.399700] systemd[1]: Initializing machine ID from random generator.1776lb # [ 1.400908] systemd[1]: Hostname set to <lb>.1777worker1 # [ 1.405881] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker1 # [ 1.418066] systemd[1]: Detected virtualization kvm.1779worker1 # [ 1.420229] systemd[1]: Detected architecture x86-64.1780worker1 # [ 1.422389] systemd[1]: Running in initrd.1781worker1 # [ 1.425110] systemd[1]: Initializing machine ID from random generator.1782worker1 # [ 1.428047] systemd[1]: Hostname set to <worker1>.1783client # [ 1.416843] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784client # [ 1.428828] systemd[1]: Detected virtualization kvm.1785client # [ 1.430976] systemd[1]: Detected architecture x86-64.1786client # [ 1.433070] systemd[1]: Running in initrd.1787client # [ 1.435593] systemd[1]: Initializing machine ID from random generator.1788client # [ 1.438444] systemd[1]: Hostname set to <client>.1789worker2 # [ 1.407742] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker2 # [ 1.419768] systemd[1]: Detected virtualization kvm.1791worker2 # [ 1.421839] systemd[1]: Detected architecture x86-64.1792worker2 # [ 1.423921] systemd[1]: Running in initrd.1793worker2 # [ 1.426328] systemd[1]: Initializing machine ID from random generator.1794worker2 # [ 1.429242] systemd[1]: Hostname set to <worker2>.1795lb # [ 1.605092] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796lb # [ 1.657097] systemd[1]: Queued start job for default target Initrd Default Target.1797lb # [ 1.661617] systemd[1]: Created slice Slice /system/modprobe.1798lb # [ 1.662810] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1799lb # [ 1.664351] systemd[1]: Expecting device /dev/disk/by-label/nixos...1800lb # [ 1.665491] systemd[1]: Reached target Path Units.1801lb # [ 1.666391] systemd[1]: Reached target Slice Units.1802lb # [ 1.667288] systemd[1]: Reached target Swaps.1803lb # [ 1.668093] systemd[1]: Reached target Timer Units.1804lb # [ 1.669074] systemd[1]: Listening on D-Bus System Message Bus Socket.1805lb # [ 1.670351] systemd[1]: Listening on Journal Socket (/dev/log).1806lb # [ 1.671555] systemd[1]: Listening on Journal Sockets.1807lb # [ 1.672579] systemd[1]: Listening on udev Control Socket.1808lb # [ 1.673644] systemd[1]: Listening on udev Kernel Socket.1809lb # [ 1.674662] systemd[1]: Reached target Socket Units.1810worker1 # [ 1.655190] systemd[1]: bpf-restrict-fs: LSM BPF program attached1811lb # [ 1.676453] systemd[1]: Starting Create List of Static Device Nodes...1812lb # [ 1.680105] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1813lb # [ 1.686613] systemd[1]: Starting Load Kernel Module configfs...1814client # [ 1.662726] systemd[1]: bpf-restrict-fs: LSM BPF program attached1815worker2 # [ 1.635162] systemd[1]: bpf-restrict-fs: LSM BPF program attached1816lb # [ 1.702385] systemd[1]: Starting Journal Service...1817lb # [ 1.713384] systemd[1]: Starting Load Kernel Modules...1818worker1 # [ 1.710268] systemd[1]: Queued start job for default target Initrd Default Target.1819worker1 # [ 1.714232] systemd[1]: Created slice Slice /system/modprobe.1820lb # [ 1.721385] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1821worker1 # [ 1.715460] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1822worker1 # [ 1.716897] systemd[1]: Expecting device /dev/disk/by-label/nixos...1823worker1 # [ 1.718017] systemd[1]: Reached target Path Units.1824lb # [ 1.737621] netfs: FS-Cache loaded1825worker1 # [ 1.718917] systemd[1]: Reached target Slice Units.1826worker1 # [ 1.719762] systemd[1]: Reached target Swaps.1827worker1 # [ 1.720611] systemd[1]: Reached target Timer Units.1828worker1 # [ 1.721724] systemd[1]: Listening on D-Bus System Message Bus Socket.1829worker1 # [ 1.722989] systemd[1]: Listening on Journal Socket (/dev/log).1830lb # [ 1.742283] 9pnet: Installing 9P2000 support1831worker1 # [ 1.724222] systemd[1]: Listening on Journal Sockets.1832worker1 # [ 1.725275] systemd[1]: Listening on udev Control Socket.1833worker1 # [ 1.726342] systemd[1]: Listening on udev Kernel Socket.1834worker1 # [ 1.727355] systemd[1]: Reached target Socket Units.1835worker2 # [ 1.687913] systemd[1]: Queued start job for default target Initrd Default Target.1836worker1 # [ 1.729248] systemd[1]: Starting Create List of Static Device Nodes...1837client # [ 1.717580] systemd[1]: Queued start job for default target Initrd Default Target.1838lb # [ 1.750387] systemd[1]: Starting Coldplug All udev Devices...1839worker2 # [ 1.691816] systemd[1]: Created slice Slice /system/modprobe.1840worker2 # [ 1.692935] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1841client # [ 1.722356] systemd[1]: Created slice Slice /system/modprobe.1842worker1 # [ 1.733917] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1843worker2 # [ 1.694268] systemd[1]: Expecting device /dev/disk/by-label/nixos...1844client # [ 1.723492] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1845worker2 # [ 1.695345] systemd[1]: Reached target Path Units.1846client # [ 1.724858] systemd[1]: Expecting device /dev/disk/by-label/nixos...1847worker2 # [ 1.696211] systemd[1]: Reached target Slice Units.1848client # [ 1.725917] systemd[1]: Reached target Path Units.1849worker2 # [ 1.697068] systemd[1]: Reached target Swaps.1850client # [ 1.726724] systemd[1]: Reached target Slice Units.1851worker2 # [ 1.697823] systemd[1]: Reached target Timer Units.1852client # [ 1.727582] systemd[1]: Reached target Swaps.1853worker2 # [ 1.698731] systemd[1]: Listening on D-Bus System Message Bus Socket.1854client # [ 1.728359] systemd[1]: Reached target Timer Units.1855worker2 # [ 1.699906] systemd[1]: Listening on Journal Socket (/dev/log).1856client # [ 1.729311] systemd[1]: Listening on D-Bus System Message Bus Socket.1857worker1 # [ 1.741060] systemd[1]: Starting Load Kernel Module configfs...1858worker2 # [ 1.701045] systemd[1]: Listening on Journal Sockets.1859client # [ 1.730465] systemd[1]: Listening on Journal Socket (/dev/log).1860worker2 # [ 1.701987] systemd[1]: Listening on udev Control Socket.1861client # [ 1.731578] systemd[1]: Listening on Journal Sockets.1862worker2 # [ 1.702940] systemd[1]: Listening on udev Kernel Socket.1863client # [ 1.732550] systemd[1]: Listening on udev Control Socket.1864worker2 # [ 1.703836] systemd[1]: Reached target Socket Units.1865client # [ 1.733537] systemd[1]: Listening on udev Kernel Socket.1866lb # [ 1.764412] systemd[1]: Finished Create List of Static Device Nodes.1867client # [ 1.734457] systemd[1]: Reached target Socket Units.1868worker2 # [ 1.705559] systemd[1]: Starting Create List of Static Device Nodes...1869client # [ 1.736275] systemd[1]: Starting Create List of Static Device Nodes...1870worker2 # [ 1.709320] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1871lb # [ 1.770895] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1872client # [ 1.740825] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1873worker1 # [ 1.756032] systemd[1]: Starting Journal Service...1874worker2 # [ 1.716670] systemd[1]: Starting Load Kernel Module configfs...1875lb # [ 1.777698] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1876client # [ 1.748833] systemd[1]: Starting Load Kernel Module configfs...1877lb # [ 1.783727] systemd[1]: modprobe@configfs.service: Deactivated successfully.1878lb # [ 1.785598] systemd-journald[67]: Collecting audit messages is disabled.1879worker1 # [ 1.768934] systemd[1]: Starting Load Kernel Modules...1880worker2 # [ 1.729142] systemd[1]: Starting Journal Service...1881lb # [ 1.790345] systemd[1]: Finished Load Kernel Module configfs.1882client # [ 1.762829] systemd[1]: Starting Journal Service...1883lb # [ 1.796480] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1884worker2 # [ 1.740573] systemd[1]: Starting Load Kernel Modules...1885lb # [ 1.800549] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1886client # [ 1.774524] systemd[1]: Starting Load Kernel Modules...1887worker1 # [ 1.775932] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1888lb # [ 1.808400] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1889lb # [ 1.809771] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1890worker1 # [ 1.791808] netfs: FS-Cache loaded1891worker1 # [ 1.796635] 9pnet: Installing 9P2000 support1892worker2 # [ 1.748545] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1893client # [ 1.781155] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1894worker2 # [ 1.766567] netfs: FS-Cache loaded1895worker1 # [ 1.807926] systemd[1]: Starting Coldplug All udev Devices...1896client # [ 1.799627] netfs: FS-Cache loaded1897worker2 # [ 1.771253] 9pnet: Installing 9P2000 support1898client # [ 1.806948] 9pnet: Installing 9P2000 support1899lb # [ 1.838857] systemd[1]: Finished Load Kernel Modules.1900worker2 # [ 1.780304] systemd[1]: Starting Coldplug All udev Devices...1901worker1 # [ 1.823975] systemd[1]: Finished Create List of Static Device Nodes.1902lb # [ 1.845418] systemd[1]: Starting Apply Kernel Variables...1903worker1 # [ 1.827529] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1904client # [ 1.821992] systemd[1]: Starting Coldplug All udev Devices...1905worker2 # [ 1.794089] systemd[1]: Finished Create List of Static Device Nodes.1906worker1 # [ 1.835277] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1907lb # [ 1.855475] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1908worker2 # [ 1.799070] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1909worker1 # [ 1.840094] systemd-journald[67]: Collecting audit messages is disabled.1910client # [ 1.828558] systemd-journald[67]: Collecting audit messages is disabled.1911worker1 # [ 1.841704] systemd[1]: modprobe@configfs.service: Deactivated successfully.1912lb # [ 1.864956] systemd[1]: Starting Create Static Device Nodes in /dev...1913worker2 # [ 1.806898] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1914client # [ 1.836211] systemd[1]: Finished Create List of Static Device Nodes.1915worker1 # [ 1.849214] systemd[1]: Finished Load Kernel Module configfs.1916worker2 # [ 1.811925] systemd[1]: modprobe@configfs.service: Deactivated successfully.1917client # [ 1.841420] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1918worker1 # [ 1.854312] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1919worker2 # [ 1.817039] systemd-journald[67]: Collecting audit messages is disabled.1920worker1 # [ 1.858976] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1921worker2 # [ 1.819876] systemd[1]: Finished Load Kernel Module configfs.1922client # [ 1.850198] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1923lb # [ 1.883396] systemd[1]: Finished Apply Kernel Variables.1924worker1 # [ 1.866925] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1925worker2 # [ 1.825954] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1926client # [ 1.856175] systemd[1]: modprobe@configfs.service: Deactivated successfully.1927worker1 # [ 1.868380] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1928worker2 # [ 1.830531] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1929client # [ 1.862157] systemd[1]: Finished Load Kernel Module configfs.1930worker2 # [ 1.836588] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1931worker2 # [ 1.838196] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1932client # [ 1.867221] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1933lb # [ 1.898831] systemd[1]: Finished Create Static Device Nodes in /dev.1934client # [ 1.869817] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1935lb # [ 1.902504] systemd[1]: Started Journal Service.1936lb # [ 1.764894] systemd-modules-load[68]: Inserted module 'dm_mod'1937lb # [ 1.768930] systemd-modules-load[68]: Inserted module 'virtio_balloon'1938lb # [ 1.769950] systemd-modules-load[68]: Inserted module 'virtio_gpu'1939client # [ 1.878848] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1940client # [ 1.880209] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1941lb # [ 1.778078] systemd[1]: Reached target Preparation for Local File Systems.1942lb # [ 1.781070] systemd[1]: Reached target Local File Systems.1943worker1 # [ 1.899407] systemd[1]: Finished Load Kernel Modules.1944lb # [ 1.783863] systemd[1]: Starting Create System Files and Directories...1945worker1 # [ 1.907037] systemd[1]: Starting Apply Kernel Variables...1946worker2 # [ 1.866616] systemd[1]: Finished Load Kernel Modules.1947lb # [ 1.791069] systemd[1]: Starting Rule-based Manager for Device Events and Files...1948worker2 # [ 1.874696] systemd[1]: Starting Apply Kernel Variables...1949worker1 # [ 1.916958] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1950client # [ 1.910333] systemd[1]: Finished Load Kernel Modules.1951worker2 # [ 1.884559] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1952worker1 # [ 1.926039] systemd[1]: Starting Create Static Device Nodes in /dev...1953client # [ 1.917828] systemd[1]: Starting Apply Kernel Variables...1954lb # [ 1.814620] systemd[1]: Finished Create System Files and Directories.1955worker2 # [ 1.895042] systemd[1]: Starting Create Static Device Nodes in /dev...1956worker1 # [ 1.944158] systemd[1]: Finished Apply Kernel Variables.1957client # [ 1.932846] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1958client # [ 1.940982] systemd[1]: Starting Create Static Device Nodes in /dev...1959worker2 # [ 1.913109] systemd[1]: Finished Apply Kernel Variables.1960lb # [ 1.836237] systemd-udevd[83]: Using default interface naming scheme 'v261'.1961worker1 # [ 1.957464] systemd[1]: Started Journal Service.1962worker1 # [ 1.824690] systemd-modules-load[68]: Inserted module 'dm_mod'1963worker1 # [ 1.827318] systemd-modules-load[68]: Inserted module 'virtio_balloon'1964worker1 # [ 1.828946] systemd-modules-load[68]: Inserted module 'virtio_gpu'1965client # [ 1.815436] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.952114] systemd[1]: Started Journal Service.1966client # 1967client # [ 1.819283] systemd-modules-load[68]: Inserted module 'virtio_balloon'1968client # [ 1.820427] systemd-modules-load[68]: Inserted module 'virtio_gpu'1969worker1 # [ 1.832885] systemd[1]: Finished Create Static Device Nodes in /dev.1970worker2 # [ 1.928760] systemd[1]: Started Journal Service.1971worker2 # [ 1.794793] systemd-modules-load[68]: Inserted module 'dm_mod'1972worker2 # [ 1.797929] systemd-modules-load[68]: Inserted module 'virtio_balloon'1973worker2 # [ 1.798947] systemd-modules-load[68]: Inserted module 'virtio_gpu'1974worker1 # [ 1.840130] systemd[1]: Reached target Preparation for Local File Systems.1975worker1 # [ 1.841212] systemd[1]: Reached target Local File Systems.1976lb # [ 1.859139] systemd[1]: Started Rule-based Manager for Device Events and Files.1977worker2 # [ 1.802925] systemd[1]: Finished Create Static Device Nodes in /dev.1978client # [ 1.832168] systemd[1]: Finished Apply Kernel Variables.1979worker1 # [ 1.844072] systemd[1]: Starting Create System Files and Directories...1980worker1 # [ 1.849073] systemd[1]: Starting Rule-based Manager for Device Events and Files...1981worker2 # [ 1.811091] systemd[1]: Reached target Preparation for Local File Systems.1982worker2 # [ 1.812107] systemd[1]: Reached target Local File Systems.1983client # [ 1.843075] systemd[1]: Finished Create Static Device Nodes in /dev.1984worker2 # [ 1.815069] systemd[1]: Starting Create System Files and Directories...1985client # [ 1.846082] systemd[1]: Reached target Preparation for Local File Systems.1986client # [ 1.846987] systemd[1]: Reached target Local File Systems.1987client # [ 1.850054] systemd[1]: Starting Create System Files and Directories...1988worker2 # [ 1.822074] systemd[1]: Starting Rule-based Manager for Device Events and Files...1989client # [ 1.857081] systemd[1]: Starting Rule-based Manager for Device Events and Files...1990worker1 # [ 1.874562] systemd[1]: Finished Create System Files and Directories.1991worker2 # [ 1.845155] systemd[1]: Finished Create System Files and Directories.1992lb # [ 1.904071] systemd[1]: Finished Coldplug All udev Devices.1993lb # [ 1.904859] systemd[1]: Reached target System Initialization.1994lb # [ 1.905702] systemd[1]: Reached target Basic System.1995client # [ 1.880962] systemd[1]: Finished Create System Files and Directories.1996worker1 # [ 1.897353] systemd-udevd[83]: Using default interface naming scheme 'v261'.1997worker2 # [ 1.866230] systemd-udevd[83]: Using default interface naming scheme 'v261'.1998client # [ 1.903898] systemd-udevd[83]: Using default interface naming scheme 'v261'.1999worker1 # [ 1.921371] systemd[1]: Started Rule-based Manager for Device Events and Files.2000worker2 # [ 1.890619] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 1,979 hidden lines…
3980lb # [ 16.814976] envoy[908]: [2026-09-20 10:50:43.153][908][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3981lb # [ 16.817460] envoy[908]: [2026-09-20 10:50:43.153][908][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3982lb # [ 16.819805] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3983lb # [ 16.823630] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3984lb # [ 16.827830] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3985lb # [ 16.829507] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3986lb # [ 16.831258] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3987lb # [ 16.833375] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3988lb # [ 16.835112] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3989lb # [ 16.837052] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3990lb # [ 16.839812] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3991lb # [ 16.842328] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3992lb # [ 16.845989] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3993lb # [ 16.847625] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3994lb # [ 16.849710] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3995lb # [ 16.851406] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3996lb # [ 16.853036] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3997lb # [ 16.857093] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3998lb # [ 16.860783] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3999lb # [ 16.862894] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4000lb # [ 16.867701] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4001lb # [ 16.869681] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4002lb # [ 16.872962] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4003worker1 # [ 16.857189] dhcpcd[693]: eth0: leased 10.0.2.15 for 86400 seconds4004lb # [ 16.875516] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4005worker1 # [ 16.859841] dhcpcd[693]: eth0: adding route to 10.0.2.0/244006lb # [ 16.878097] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4007worker1 # [ 16.862098] dhcpcd[693]: eth0: adding default route via 10.0.2.24008lb # [ 16.879586] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4009lb # [ 16.881290] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4010lb # [ 16.886102] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4011lb # [ 16.888115] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4012lb # [ 16.892598] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4013lb # [ 16.895075] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4014lb # [ 16.896955] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4015lb # [ 16.899306] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4016lb # [ 16.904828] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4017lb # [ 16.906412] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4018lb # [ 16.909484] envoy[908]: [2026-09-20 10:50:43.158][908][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4019lb # [ 16.912068] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4020lb # [ 16.914457] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4021lb # [ 16.939961] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4022lb # [ 16.941636] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4023lb # [ 16.943694] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4024lb # [ 16.946310] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4025lb # [ 16.957475] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4026lb # [ 16.959573] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4027lb # [ 16.962637] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4028lb # [ 16.966860] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4029lb # [ 16.968656] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24030lb # [ 16.970891] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4031lb # [ 16.972917] envoy[908]: [2026-09-20 10:50:43.159][908][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4032lb # [ 16.974724] envoy[908]: [2026-09-20 10:50:43.161][908][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4033lb # [ 16.977434] envoy[908]: [2026-09-20 10:50:43.161][908][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4034lb # [ 16.979589] envoy[908]: [2026-09-20 10:50:43.161][908][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4035lb # [ 16.982213] envoy[908]: [2026-09-20 10:50:43.161][908][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4036lb # [ 16.983764] envoy[908]: [2026-09-20 10:50:43.161][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4037lb # [ 16.986694] envoy[908]: [2026-09-20 10:50:43.161][908][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4038lb # [ 16.988502] envoy[908]: [2026-09-20 10:50:43.161][908][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24039lb # [ 16.990159] envoy[908]: [2026-09-20 10:50:43.161][908][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4040lb # [ 16.991832] envoy[908]: [2026-09-20 10:50:43.167][908][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4041lb # [ 16.994094] envoy[908]: [2026-09-20 10:50:43.167][908][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4042worker1 # [ 16.980252] systemd[1]: Started DHCP Client.4043lb # [ 16.996508] envoy[908]: [2026-09-20 10:50:43.167][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24044worker1 # [ 16.982268] systemd[1]: Reached target Multi-User System.4045lb # [ 16.998146] envoy[908]: [2026-09-20 10:50:43.167][908][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4046worker1 # [ 16.983145] systemd[1]: Startup finished in 869ms (kernel) + 4.054s (initrd) + 12.058s (userspace) = 16.982s.4047lb # [ 17.000266] envoy[908]: [2026-09-20 10:50:43.167][908][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84048lb # [ 17.001960] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4049lb # [ 17.004842] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4050lb # [ 17.007154] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4051lb # [ 17.012939] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4052lb # [ 17.015032] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4053lb # [ 17.016516] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4054lb # [ 17.019126] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4055lb # [ 17.025354] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4056lb # [ 17.027633] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4057lb # [ 17.029476] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4058lb # [ 17.031816] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4059lb # [ 17.033638] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4060lb # [ 17.039098] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4061lb # [ 17.045534] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4062lb # [ 17.047422] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4063lb # [ 17.049345] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4064lb # [ 17.051497] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4065lb # [ 17.053407] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4066lb # [ 17.056469] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4067lb # [ 17.065979] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4068lb # [ 17.068410] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4069lb # [ 17.070404] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4070lb # [ 17.072578] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4071lb # [ 17.074752] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4072lb # [ 17.076635] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4073lb # [ 17.079113] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4074lb # [ 17.083502] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4075lb # [ 17.085050] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4076lb # [ 17.087840] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4077lb # [ 17.090802] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4078lb # [ 17.098132] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4079lb # [ 17.101850] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4080lb # [ 17.103843] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4081lb # [ 17.106755] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4082lb # [ 17.109889] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4083lb # [ 17.111907] envoy[908]: [2026-09-20 10:50:43.168][908][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4084lb # [ 17.114575] envoy[908]: [2026-09-20 10:50:43.169][908][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4085lb # [ 17.117082] envoy[908]: [2026-09-20 10:50:43.169][908][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4086lb # [ 17.118986] envoy[908]: [2026-09-20 10:50:43.169][908][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4087lb # [ 17.120823] envoy[908]: [2026-09-20 10:50:43.300][908][info][main] [source/server/server.cc:531] HTTP header map info:4088lb # [ 17.122149] envoy[908]: [2026-09-20 10:50:43.347][908][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4089lb # [ 17.135042] envoy[908]: [2026-09-20 10:50:43.347][908][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4090lb # [ 17.136487] envoy[908]: [2026-09-20 10:50:43.347][908][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4091lb # [ 17.143844] envoy[908]: [2026-09-20 10:50:43.347][908][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4092lb # [ 17.200057] envoy[908]: [2026-09-20 10:50:43.560][908][info][main] [source/server/server.cc:968] runtime: {}4093lb # [ 17.207300] envoy[908]: [2026-09-20 10:50:43.567][908][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4094lb # [ 17.209384] envoy[908]: [2026-09-20 10:50:43.569][908][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014095lb # [ 17.212827] envoy[908]: [2026-09-20 10:50:43.573][908][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014096lb # [ 17.215698] envoy[908]: [2026-09-20 10:50:43.576][908][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4097lb # [ 17.217591] envoy[908]: [2026-09-20 10:50:43.578][908][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4098lb # [ 17.219171] envoy[908]: [2026-09-20 10:50:43.578][908][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4099lb # [ 17.269814] envoy[908]: [2026-09-20 10:50:43.630][908][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4100worker2 # [ 17.217394] systemd[1]: Starting Nix worker-protocol over gRPC...4101worker1 # [ 17.257955] systemd[1]: Starting Nix worker-protocol over gRPC...4102lb # [ 17.289721] envoy[908]: [2026-09-20 10:50:43.650][908][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4103lb # [ 17.291929] envoy[908]: [2026-09-20 10:50:43.652][908][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4104lb # [ 17.294783] envoy[908]: [2026-09-20 10:50:43.652][908][info][main] [source/server/server.cc:1077] starting main dispatch loop4105worker2 # [ 17.337345] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds4106worker2 # [ 17.342312] dhcpcd[692]: eth0: adding route to 10.0.2.0/244107worker2 # [ 17.344407] dhcpcd[692]: eth0: adding default route via 10.0.2.24108worker2 # [ 17.396112] nix-grpc-daemon[729]: ts=2026-09-20T10:50:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4109worker2 # [ 17.410443] nix-grpc-daemon[729]: ts=2026-09-20T10:50:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14110worker1 # [ 17.454362] nix-grpc-daemon[784]: ts=2026-09-20T10:50:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4111worker1 # [ 17.464934] nix-grpc-daemon[784]: ts=2026-09-20T10:50:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14112worker2 # [ 17.448904] nix-grpc-daemon[729]: ts=2026-09-20T10:50:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4113worker2 # [ 17.453103] systemd[1]: Started Nix worker-protocol over gRPC.4114worker1 # [ 17.498243] nix-grpc-daemon[784]: ts=2026-09-20T10:50:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4115worker1 # [ 17.501267] systemd[1]: Started Nix worker-protocol over gRPC.4116lb # [ 17.563150] envoy[908]: [2026-09-20 10:50:43.923][908][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4117lb # [ 17.565152] envoy[908]: [2026-09-20 10:50:43.923][908][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4118lb # [ 17.567087] envoy[908]: [2026-09-20 10:50:43.923][908][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4119lb # [ 17.568603] envoy[908]: [2026-09-20 10:50:43.925][908][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4120worker2 # [ 17.517391] systemd[1]: Started DHCP Client.4121worker2 # [ 17.519410] systemd[1]: Reached target Multi-User System.4122worker2 # [ 17.520696] systemd[1]: Startup finished in 876ms (kernel) + 4.919s (initrd) + 11.723s (userspace) = 17.519s.4123worker1 # [ 17.568658] nix-grpc-daemon[784]: ts=2026-09-20T10:50:44Z level=info event=scheduler_connected addr=lb:500514124worker2 # [ 17.529205] nix-grpc-daemon[729]: ts=2026-09-20T10:50:43Z level=info event=scheduler_connected addr=lb:500514125worker1 # [ 17.578055] nix-grpc-daemon[784]: ts=2026-09-20T10:50:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4126worker2 # [ 17.537855] nix-grpc-daemon[729]: ts=2026-09-20T10:50:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4127worker2 # [ 17.738568] nix-grpc-daemon[729]: ts=2026-09-20T10:50:44Z level=info event=scheduler_connected addr=lb:500514128worker2 # [ 17.740448] nix-grpc-daemon[729]: ts=2026-09-20T10:50:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4129worker1 # [ 17.780448] nix-grpc-daemon[784]: ts=2026-09-20T10:50:44Z level=info event=scheduler_connected addr=lb:500514130worker1 # [ 17.784319] nix-grpc-daemon[784]: ts=2026-09-20T10:50:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4131worker2 # [ 17.941532] nix-grpc-daemon[729]: ts=2026-09-20T10:50:44Z level=info event=scheduler_connected addr=lb:500514132worker1 # [ 17.985102] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514133worker2 # [ 17.946150] nix-grpc-daemon[729]: ts=2026-09-20T10:50:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4134worker1 # [ 17.989315] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker2 # [ 18.146917] nix-grpc-daemon[729]: ts=2026-09-20T10:50:44Z level=info event=scheduler_connected addr=lb:500514136worker1 # [ 18.190702] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 18.151500] nix-grpc-daemon[729]: ts=2026-09-20T10:50:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker1 # [ 18.193796] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker2 # [ 18.351920] nix-grpc-daemon[729]: ts=2026-09-20T10:50:44Z level=info event=scheduler_connected addr=lb:500514140worker1 # [ 18.391920] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 18.353686] nix-grpc-daemon[729]: ts=2026-09-20T10:50:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker1 # [ 18.395787] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker2 # [ 18.555808] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514144worker1 # [ 18.597788] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514145worker2 # [ 18.559631] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4146worker1 # [ 18.601146] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 18.756679] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514148worker1 # [ 18.798792] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514149worker1 # [ 18.801926] nix-grpc-daemon[784]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4150worker2 # [ 18.761503] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 18.999823] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514152worker2 # [ 18.962144] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514153worker1 # [ 19.004209] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4154worker2 # [ 18.966492] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker1 # [ 19.204927] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 19.167604] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514157worker1 # [ 19.208992] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4158worker2 # [ 19.172090] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 19.409905] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 19.372816] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_connected addr=lb:500514161worker2 # [ 19.375282] nix-grpc-daemon[729]: ts=2026-09-20T10:50:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker1 # [ 19.413825] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker2 # [ 19.574995] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 19.615494] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514165worker2 # [ 19.576969] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4166worker1 # [ 19.619261] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker2 # [ 19.777535] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 19.779399] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 19.819945] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514170worker1 # [ 19.823781] nix-grpc-daemon[784]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker2 # [ 19.980038] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514172worker2 # [ 19.981846] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 20.025638] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 20.028849] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 20.182655] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 20.184806] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 20.227339] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 20.230546] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 20.385432] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 20.387150] nix-grpc-daemon[729]: ts=2026-09-20T10:50:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 20.429364] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 20.432542] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 20.588083] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 20.589690] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 20.630443] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 20.631739] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker2 # [ 20.790997] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 20.831625] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514189worker2 # [ 20.795240] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4190worker1 # [ 20.835616] nix-grpc-daemon[784]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 20.996345] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 21.036517] nix-grpc-daemon[784]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514193worker1 # [ 21.040658] nix-grpc-daemon[784]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4194worker2 # [ 21.000436] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 21.241220] nix-grpc-daemon[784]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514196worker2 # [ 21.201372] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514197worker1 # [ 21.243400] nix-grpc-daemon[784]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198worker2 # [ 21.205330] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker1 # [ 21.443925] nix-grpc-daemon[784]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 21.445923] nix-grpc-daemon[784]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker2 # [ 21.406309] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 21.410314] nix-grpc-daemon[729]: ts=2026-09-20T10:50:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker1 # [ 21.647773] nix-grpc-daemon[784]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 21.651638] nix-grpc-daemon[784]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker2 # [ 21.612632] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 21.615776] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 21.849593] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 21.812818] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514209worker2 # [ 21.814861] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker1 # [ 21.853541] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 22.054692] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 22.015994] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514213worker1 # [ 22.058733] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker2 # [ 22.019716] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker1 # [ 22.259224] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514216worker2 # [ 22.217850] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514217worker1 # [ 22.261051] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker2 # [ 22.221880] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker1 # [ 22.461709] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 22.463625] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 22.422789] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 22.426755] nix-grpc-daemon[729]: ts=2026-09-20T10:50:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 22.665840] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514224worker2 # [ 22.629324] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514225worker1 # [ 22.669523] nix-grpc-daemon[784]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker2 # [ 22.632699] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker2 # [ 22.830096] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 22.870682] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514229worker2 # [ 22.834091] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker1 # [ 22.875503] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker2 # [ 23.034794] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 23.075514] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514233worker2 # [ 23.038806] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker1 # [ 23.079437] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker2 # [ 23.239843] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 23.280455] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514237worker2 # [ 23.243803] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker1 # [ 23.284408] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker2 # [ 23.444610] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_connected addr=lb:500514240worker1 # [ 23.485322] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514241worker2 # [ 23.448516] nix-grpc-daemon[729]: ts=2026-09-20T10:50:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker1 # [ 23.489199] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker2 # [ 23.650356] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514244worker1 # [ 23.690876] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 23.653986] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker1 # [ 23.694614] nix-grpc-daemon[784]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker2 # [ 23.855070] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 23.895573] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514249worker2 # [ 23.858983] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker1 # [ 23.899488] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker2 # [ 24.060233] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514252worker1 # [ 24.100631] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514253worker2 # [ 24.063959] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4254worker1 # [ 24.104633] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker2 # [ 24.264907] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514256worker1 # [ 24.305607] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514257worker2 # [ 24.269133] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker1 # [ 24.309656] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker1 # [ 24.510505] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514260worker2 # [ 24.470100] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_connected addr=lb:500514261worker2 # [ 24.473971] nix-grpc-daemon[729]: ts=2026-09-20T10:50:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker1 # [ 24.514421] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker2 # [ 24.675345] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514264worker1 # [ 24.716746] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514265worker2 # [ 24.679451] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4266worker1 # [ 24.720191] nix-grpc-daemon[784]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker2 # [ 24.880356] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514268worker1 # [ 24.920989] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514269worker2 # [ 24.884444] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4270worker1 # [ 24.924885] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker2 # [ 25.084970] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514272worker2 # [ 25.087186] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker1 # [ 25.125818] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514274worker1 # [ 25.130234] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker2 # [ 25.288165] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514276worker1 # [ 25.331250] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514277worker2 # [ 25.292122] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker1 # [ 25.334823] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker1 # [ 25.532845] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 25.492860] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_connected addr=lb:500514281worker1 # [ 25.536883] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker2 # [ 25.496846] nix-grpc-daemon[729]: ts=2026-09-20T10:50:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker1 # [ 25.737820] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 25.740281] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker2 # [ 25.698666] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514286worker2 # [ 25.702420] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker1 # [ 25.940888] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 25.900402] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514289worker1 # [ 25.942943] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290worker2 # [ 25.904644] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker1 # [ 26.143571] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514292worker1 # [ 26.145540] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker2 # [ 26.105561] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 26.109452] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker1 # [ 26.346059] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514296worker1 # [ 26.347788] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker2 # [ 26.310318] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514298worker2 # [ 26.313980] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker1 # [ 26.548353] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514300worker1 # [ 26.550106] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker2 # [ 26.514985] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514302worker2 # [ 26.519907] nix-grpc-daemon[729]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303lb # [ 26.586400] rustfs-bucket-start[1009]: mb s3://farm4304lb # [ 26.592395] systemd[1]: Finished rustfs-bucket.service.4305lb # [ 26.598597] systemd[1]: Starting niks3 server...4306worker1 # [ 26.752493] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514307worker2 # [ 26.717246] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514308worker1 # [ 26.755822] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker2 # [ 26.718606] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310lb # [ 26.826824] postgres[1024]: [1024] ERROR: relation "goose_db_version" does not exist at character 364311lb # [ 26.828113] postgres[1024]: [1024] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4312lb # [ 26.860830] niks3-server[1018]: 2026/09/20 10:50:53 OK 20241026095416_initial_model.sql (15.45ms)4313lb # [ 26.871919] niks3-server[1018]: 2026/09/20 10:50:53 OK 20251210153512_drop_unused_gin_index.sql (11.18ms)4314lb # [ 26.876780] niks3-server[1018]: 2026/09/20 10:50:53 OK 20251218171726_add_pins.sql (3.53ms)4315lb # [ 26.882142] niks3-server[1018]: 2026/09/20 10:50:53 OK 20260628120000_add_object_size_and_stats.sql (4.08ms)4316lb # [ 26.886314] niks3-server[1018]: 2026/09/20 10:50:53 OK 20260905000000_add_claims.sql (4.11ms)4317lb # [ 26.890673] niks3-server[1018]: 2026/09/20 10:50:53 OK 20260920000000_drop_claims.sql (4.26ms)4318lb # [ 26.891848] niks3-server[1018]: 2026/09/20 10:50:53 goose: successfully migrated database to version: 202609200000004319lb # [ 26.897773] niks3-server[1018]: 2026/09/20 10:50:53 OK 1_commit_pending_closure.sql (7.06ms)4320lb # [ 26.900242] niks3-server[1018]: 2026/09/20 10:50:53 OK 2_object_stats_trigger.sql (2.48ms)4321lb # [ 26.901472] niks3-server[1018]: 2026/09/20 10:50:53 goose: up to current file version: 24322lb # [ 26.908242] niks3-server[1018]: 2026/09/20 10:50:53 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4323worker1 # [ 26.953253] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514324worker1 # [ 26.957096] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker2 # [ 26.918970] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514326worker2 # [ 26.922191] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327lb # [ 26.991996] niks3-server[1018]: 2026/09/20 10:50:53 INFO Read proxy enabled — serving cache objects from S34328lb # [ 26.993841] niks3-server[1018]: 2026/09/20 10:50:53 INFO Using socket-activated listener address=0.0.0.0:57514329lb # [ 26.998741] niks3-server[1018]: 2026/09/20 10:50:53 INFO systemd watchdog enabled interval=15s4330lb # [ 27.000415] systemd[1]: Started niks3 server.4331lb # [ 27.001827] systemd[1]: Reached target Multi-User System.4332lb # [ 27.003862] systemd[1]: Startup finished in 860ms (kernel) + 4.913s (initrd) + 21.229s (userspace) = 27.003s.4333lb # [ 27.005556] niks3-server[1018]: 2026/09/20 10:50:53 INFO Starting HTTP server address=0.0.0.0:57514334lb # [ 27.010568] niks3-server[1018]: 2026/09/20 10:50:53 INFO lead: acquired remote=192.168.1.3:516264335worker1 # [ 27.000987] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_take_over4336worker1 # [ 27.157702] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514337worker2 # [ 27.120100] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514338worker1 # [ 27.161926] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4339worker2 # [ 27.124105] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4340worker1 # [ 27.362766] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514341worker2 # [ 27.324571] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514342worker2 # [ 27.326849] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker1 # [ 27.367248] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4344worker2 # [ 27.527197] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514345worker2 # [ 27.529704] nix-grpc-daemon[729]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4346worker1 # [ 27.567963] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514347worker1 # [ 27.572712] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4348worker2 # [ 27.730832] nix-grpc-daemon[729]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514349worker1 # [ 27.774313] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514350worker1 # [ 27.786636] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04351worker1 # [ 27.792851] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04352lb: (finished: waiting for unit niks3.service, in 16.41 seconds)4353lb: waiting for TCP port 5751 on localhost4354lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4355lb: (finished: waiting for TCP port 5751 on localhost, in 0.06 seconds)4356lb: waiting for unit envoy.service4357lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4358lb: waiting for TCP port 50051 on localhost4359lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4360lb # [ 28.176606] envoy[908]: conn peer=[::1]:43682 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24361lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4362lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4363??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4364 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284365lb: must succeed: curl -sf localhost:9901/clusters4366lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4367lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4368subtest: one node schedules and both builders hold a WorkerSession on it4369lb: must succeed: curl -sf localhost:9901/clusters4370lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4371worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4372worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4373(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.12 seconds)4374subtest: balancer auth: client cert, bearer token, nothing, unknown CN4375worker2 # [ 28.760791] nix-grpc-daemon[729]: ts=2026-09-20T10:50:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:533484376worker2 # [ 28.772800] systemd[1]: Started Nix Daemon.4377worker2 # [ 29.000065] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)4378worker2 # [ 29.051221] nix-grpc-daemon[729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)4379worker2 # [ 29.067659] nix-grpc-daemon[729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 173 ms (attempt 2/5)4380worker2 # [ 29.243514] nix-grpc-daemon[729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 320 ms (attempt 3/5)4381worker2 # [ 29.563842] nix-grpc-daemon[729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 740 ms (attempt 4/5)4382worker2 # [ 30.316116] nix-grpc-daemon[729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4383worker2 # [ 30.343453] nix-grpc-daemon[729]: ts=2026-09-20T10:50:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:533484384worker2 # [ 30.360166] nix-grpc-daemon[729]: ts=2026-09-20T10:50:56Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4385worker1 # [ 30.412538] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:477704386worker1 # [ 30.420884] systemd[1]: Started Nix Daemon.4387worker1 # [ 30.631822] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4388worker1 # [ 30.657794] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)4389worker1 # [ 30.689255] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 2/5)4390worker1 # [ 30.744977] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 175 ms (attempt 3/5)4391worker1 # [ 30.926094] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 701 ms (attempt 4/5)4392worker1 # [ 31.638841] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4393worker1 # [ 31.659088] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4394client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4395client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4396client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4397client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4398worker1 # [ 32.039681] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=oidc_jwks provider=mock keys=14399worker1 # [ 32.041700] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:477704400worker1 # [ 32.052849] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:477704401worker1 # [ 32.057038] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4402worker1 # [ 32.060877] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:477704403worker1 # [ 32.064701] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4404client: must succeed: rm -r /var/lib/nix-grpc-store4405client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.02 seconds)4406worker2 # [ 32.274838] nix-grpc-daemon[729]: ts=2026-09-20T10:50:58Z level=info event=denied method=QueryMissing cn=- role=none4407worker1 # [ 32.525951] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=denied method=QueryMissing cn=stranger role=none4408(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.08 seconds)4409subtest: DAG build is scheduled across workers and lands in the cache4410client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24411worker2 # [ 32.741868] nix-grpc-daemon[729]: ts=2026-09-20T10:50:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:533484412client # don't know how to build these paths:4413client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4414worker2 # [ 32.756137] nix-grpc-daemon[729]: ts=2026-09-20T10:50:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533484415worker1 # [ 32.839911] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04416worker1 # [ 32.844561] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14417worker1 # [ 32.887189] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704418worker1 # [ 32.902257] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704419client # copying 1 paths to 'lb:50051'...4420worker1 # [ 32.910803] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477704421client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4422worker1 # [ 32.925962] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4423worker1 # [ 32.939417] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=niks3_push_started pid=8574424worker1 # [ 33.378506] nix-daemon[845]: accepted connection from pid 869, user nix-grpc-daemon (trusted)4425worker1 # [ 33.392169] nix-daemon[845]: reaped child process 872, status = succeeded4426worker1 # [ 33.541261] nix-daemon[845]: accepted connection from pid 875, user nix-grpc-daemon (trusted)4427worker1 # [ 33.553828] nix-daemon[845]: reaped child process 878, status = succeeded4428lb # [ 33.586467] niks3-server[1018]: 2026/09/20 10:50:59 INFO Received uploads request method=POST path=/api/pending_closures4429worker1 # [ 33.596370] nix-grpc-daemon[857]: time=2026-09-20T10:51:00.749Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4430worker1 # [ 33.598704] nix-grpc-daemon[857]: time=2026-09-20T10:51:00.752Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4431lb # [ 33.652551] niks3-server[1018]: 2026/09/20 10:51:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4432lb # [ 33.655978] niks3-server[1018]: 2026/09/20 10:51:00 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4433lb # [ 33.671346] niks3-server[1018]: 2026/09/20 10:51:00 INFO Signed narinfos id=1 count=14434worker1 # [ 33.656787] nix-grpc-daemon[857]: time=2026-09-20T10:51:00.811Z level=INFO msg="Uploading 1 narinfos"4435lb # [ 33.685338] niks3-server[1018]: 2026/09/20 10:51:00 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4436lb # [ 33.688049] niks3-server[1018]: 2026/09/20 10:51:00 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4437lb # [ 33.693076] niks3-server[1018]: 2026/09/20 10:51:00 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4438lb # [ 33.702990] niks3-server[1018]: 2026/09/20 10:51:00 INFO Completed upload id=14439worker1 # [ 33.688325] nix-grpc-daemon[857]: time=2026-09-20T10:51:00.843Z level=INFO msg="Upload complete. (480ms)"4440worker1 # [ 33.692822] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_in=5044441worker1 # [ 33.700143] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704442worker1 # [ 33.709240] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4443worker1 # [ 33.719235] nix-daemon[845]: reaped child process 855, status = succeeded4444worker1 # [ 33.725121] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4445client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4446client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4447client # farm-a> LOG-farm-a-t14448worker1 # [ 34.537693] nix-daemon[845]: reaped child process 882, status = succeeded4449worker1 # [ 34.682060] nix-daemon[845]: accepted connection from pid 897, user nix-grpc-daemon (trusted)4450worker1 # [ 34.692845] nix-daemon[845]: reaped child process 900, status = succeeded4451lb # [ 34.726577] niks3-server[1018]: 2026/09/20 10:51:01 INFO Received uploads request method=POST path=/api/pending_closures4452worker1 # [ 34.721094] nix-grpc-daemon[857]: time=2026-09-20T10:51:01.875Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4453worker1 # [ 34.722799] nix-grpc-daemon[857]: time=2026-09-20T10:51:01.877Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4454lb # [ 34.779397] niks3-server[1018]: 2026/09/20 10:51:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4455lb # [ 34.791228] niks3-server[1018]: 2026/09/20 10:51:01 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4456worker1 # [ 34.781158] nix-grpc-daemon[857]: time=2026-09-20T10:51:01.935Z level=INFO msg="Uploading 1 narinfos"4457lb # [ 34.796942] niks3-server[1018]: 2026/09/20 10:51:01 INFO Signed narinfos id=2 count=14458lb # [ 34.800355] niks3-server[1018]: 2026/09/20 10:51:01 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4459lb # [ 34.803906] niks3-server[1018]: 2026/09/20 10:51:01 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4460lb # [ 34.813511] niks3-server[1018]: 2026/09/20 10:51:01 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4461lb # [ 34.817882] niks3-server[1018]: 2026/09/20 10:51:01 INFO Completed upload id=24462worker1 # [ 34.801697] nix-grpc-daemon[857]: time=2026-09-20T10:51:01.956Z level=INFO msg="Upload complete. (269ms)"4463lb # [ 34.819508] niks3-server[1018]: 2026/09/20 10:51:01 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4464worker1 # [ 34.804578] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14465worker1 # [ 34.814678] nix-daemon[845]: reaped child process 880, status = succeeded4466worker1 # [ 34.819264] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04467worker1 # [ 34.821116] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14468worker1 # [ 34.871275] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704469worker1 # [ 34.880445] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704470worker1 # [ 34.890737] nix-grpc-daemon[857]: time=2026-09-20T10:51:02.045Z level=INFO msg="All 1 paths already cached"4471client # copying 1 paths to 'lb:50051'...4472worker1 # [ 34.895316] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477704473client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4474worker1 # [ 34.904712] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4475worker1 # [ 35.064368] nix-daemon[845]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4476worker1 # [ 35.078682] nix-daemon[845]: reaped child process 909, status = succeeded4477worker1 # [ 35.230702] nix-daemon[845]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4478worker1 # [ 35.242110] nix-daemon[845]: reaped child process 914, status = succeeded4479lb # [ 35.273498] niks3-server[1018]: 2026/09/20 10:51:01 INFO Received uploads request method=POST path=/api/pending_closures4480worker1 # [ 35.267380] nix-grpc-daemon[857]: time=2026-09-20T10:51:02.421Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4481worker1 # [ 35.268729] nix-grpc-daemon[857]: time=2026-09-20T10:51:02.421Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4482lb # [ 35.304697] niks3-server[1018]: 2026/09/20 10:51:01 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4483lb # [ 35.320403] niks3-server[1018]: 2026/09/20 10:51:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4484worker1 # [ 35.309850] nix-grpc-daemon[857]: time=2026-09-20T10:51:02.464Z level=INFO msg="Uploading 1 narinfos"4485lb # [ 35.325991] niks3-server[1018]: 2026/09/20 10:51:01 INFO Signed narinfos id=3 count=14486lb # [ 35.332310] niks3-server[1018]: 2026/09/20 10:51:01 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4487lb # [ 35.344150] niks3-server[1018]: 2026/09/20 10:51:01 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4488lb # [ 35.351506] niks3-server[1018]: 2026/09/20 10:51:01 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4489lb # [ 35.354724] niks3-server[1018]: 2026/09/20 10:51:01 INFO Completed upload id=34490worker1 # [ 35.338561] nix-grpc-daemon[857]: time=2026-09-20T10:51:02.493Z level=INFO msg="Upload complete. (428ms)"4491worker1 # [ 35.341078] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_in=6244492worker1 # [ 35.343418] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704493worker1 # [ 35.348488] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4494worker1 # [ 35.352646] nix-daemon[845]: reaped child process 904, status = succeeded4495client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4496client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4497worker1 # [ 35.357774] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4498client # farm-b> LOG-farm-b-t14499worker1 # [ 36.971913] nix-daemon[845]: reaped child process 918, status = succeeded4500worker1 # [ 37.106797] nix-daemon[845]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4501worker1 # [ 37.119082] nix-daemon[845]: reaped child process 937, status = succeeded4502lb # [ 37.151370] niks3-server[1018]: 2026/09/20 10:51:03 INFO Received uploads request method=POST path=/api/pending_closures4503worker1 # [ 37.150656] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.305Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4504worker1 # [ 37.152985] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.307Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4505lb # [ 37.198697] niks3-server[1018]: 2026/09/20 10:51:03 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4506lb # [ 37.205673] niks3-server[1018]: 2026/09/20 10:51:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4507lb # [ 37.217295] niks3-server[1018]: 2026/09/20 10:51:03 INFO Signed narinfos id=4 count=14508worker1 # [ 37.201120] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.355Z level=INFO msg="Uploading 1 narinfos"4509lb # [ 37.220556] niks3-server[1018]: 2026/09/20 10:51:03 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4510lb # [ 37.227909] niks3-server[1018]: 2026/09/20 10:51:03 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4511lb # [ 37.236059] niks3-server[1018]: 2026/09/20 10:51:03 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4512lb # [ 37.242683] niks3-server[1018]: 2026/09/20 10:51:03 INFO Completed upload id=44513lb # [ 37.244375] niks3-server[1018]: 2026/09/20 10:51:03 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4514worker1 # [ 37.226754] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.381Z level=INFO msg="Upload complete. (260ms)"4515worker1 # [ 37.235621] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14516worker1 # [ 37.245591] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04517worker1 # [ 37.249491] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14518worker1 # [ 37.258739] nix-daemon[845]: reaped child process 916, status = succeeded4519lb # [ 37.278249] envoy[908]: conn peer=[2001:db8:1::1]:36416 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=29144520lb # [ 37.281577] envoy[908]: conn peer=[2001:db8:1::1]:36420 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=362 ms=474521lb # [ 37.284467] envoy[908]: conn peer=[2001:db8:1::1]:36430 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134522lb # [ 37.287403] envoy[908]: conn peer=[2001:db8:1::1]:36442 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=104523worker1 # [ 37.289682] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704524worker1 # [ 37.303147] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704525worker1 # [ 37.341870] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.496Z level=INFO msg="All 2 paths already cached"4526client # copying 1 paths to 'lb:50051'...4527worker1 # [ 37.350686] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477704528client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4529worker1 # [ 37.370133] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4530worker1 # [ 37.529833] nix-daemon[845]: accepted connection from pid 943, user nix-grpc-daemon (trusted)4531worker1 # [ 37.542550] nix-daemon[845]: reaped child process 946, status = succeeded4532worker1 # [ 37.694245] nix-daemon[845]: accepted connection from pid 948, user nix-grpc-daemon (trusted)4533worker1 # [ 37.706214] nix-daemon[845]: reaped child process 951, status = succeeded4534lb # [ 37.737318] niks3-server[1018]: 2026/09/20 10:51:04 INFO Received uploads request method=POST path=/api/pending_closures4535worker1 # [ 37.741596] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.896Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4536worker1 # [ 37.743712] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.898Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4537lb # [ 37.778157] niks3-server[1018]: 2026/09/20 10:51:04 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4538lb # [ 37.789980] niks3-server[1018]: 2026/09/20 10:51:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4539worker1 # [ 37.778968] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.933Z level=INFO msg="Uploading 1 narinfos"4540lb # [ 37.795616] niks3-server[1018]: 2026/09/20 10:51:04 INFO Signed narinfos id=5 count=14541lb # [ 37.802814] niks3-server[1018]: 2026/09/20 10:51:04 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4542lb # [ 37.811515] niks3-server[1018]: 2026/09/20 10:51:04 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4543lb # [ 37.817610] niks3-server[1018]: 2026/09/20 10:51:04 INFO Completed upload id=54544worker1 # [ 37.801512] nix-grpc-daemon[857]: time=2026-09-20T10:51:04.956Z level=INFO msg="Upload complete. (419ms)"4545lb # [ 37.822303] niks3-server[1018]: 2026/09/20 10:51:04 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4546worker1 # [ 37.806085] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_in=7524547worker1 # [ 37.811592] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704548worker1 # [ 37.822247] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4549worker1 # [ 37.829801] nix-daemon[845]: reaped child process 941, status = succeeded4550worker1 # [ 37.834370] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4551client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4552client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4553client # farm-top> LOG-farm-top-t14554worker1 # [ 39.258522] nix-daemon[845]: reaped child process 955, status = succeeded4555worker1 # [ 39.429303] nix-daemon[845]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4556worker1 # [ 39.441677] nix-daemon[845]: reaped child process 974, status = succeeded4557lb # [ 39.476779] niks3-server[1018]: 2026/09/20 10:51:05 INFO Received uploads request method=POST path=/api/pending_closures4558worker1 # [ 39.473066] nix-grpc-daemon[857]: time=2026-09-20T10:51:06.627Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4559worker1 # [ 39.474796] nix-grpc-daemon[857]: time=2026-09-20T10:51:06.629Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4560lb # [ 39.535197] niks3-server[1018]: 2026/09/20 10:51:05 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4561lb # [ 39.540406] niks3-server[1018]: 2026/09/20 10:51:05 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4562lb # [ 39.547408] niks3-server[1018]: 2026/09/20 10:51:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4563lb # [ 39.550780] niks3-server[1018]: 2026/09/20 10:51:05 INFO Signed narinfos id=6 count=14564worker1 # [ 39.534892] nix-grpc-daemon[857]: time=2026-09-20T10:51:06.689Z level=INFO msg="Uploading 1 narinfos"4565lb # [ 39.560864] niks3-server[1018]: 2026/09/20 10:51:05 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4566lb # [ 39.566241] niks3-server[1018]: 2026/09/20 10:51:05 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4567lb # [ 39.570291] niks3-server[1018]: 2026/09/20 10:51:05 INFO Completed upload id=64568worker1 # [ 39.554285] nix-grpc-daemon[857]: time=2026-09-20T10:51:06.709Z level=INFO msg="Upload complete. (311ms)"4569lb # [ 39.573951] niks3-server[1018]: 2026/09/20 10:51:05 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4570worker1 # [ 39.559971] nix-grpc-daemon[784]: ts=2026-09-20T10:51:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14571worker1 # [ 39.575589] nix-daemon[845]: reaped child process 953, status = succeeded4572client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.05 seconds)4573client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4574client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4575client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4576client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4577client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4578client # copying 3 paths...4579client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4580client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4581client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4582client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.28 seconds)4583(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.62 seconds)4584subtest: repeat is answered Cached by the scheduler without building4585worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4586worker1 # finding garbage collector roots...4587worker1 # deleting specified paths...4588worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4589worker1 # deleting unused links...4590worker1 # note: hard linking is currently saving 0.0 KiB4591worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4592worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4593worker2 # finding garbage collector roots...4594worker2 # deleting specified paths...4595worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4596worker2 # deleting unused links...4597worker2 # note: hard linking is currently saving 0.0 KiB4598worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.29 seconds)4599client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24600worker1 # [ 40.907361] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:477704601client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4602client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4603worker1 # [ 40.926600] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704604worker1 # [ 40.938943] nix-grpc-daemon[857]: time=2026-09-20T10:51:08.093Z level=INFO msg="All 2 paths already cached"4605worker1 # [ 40.953429] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14606client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.30 seconds)4607client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4608client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.21 seconds)4609lb: must succeed: curl -sf localhost:9901/clusters4610lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4611lb: must succeed: curl -sf localhost:9901/clusters4612lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4613worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4614worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4615(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.27 seconds)4616subtest: inputs already in the cache are not sent to the scheduler4617lb: must succeed: curl -sf localhost:9901/clusters4618lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4619lb: must succeed: curl -sf localhost:9901/clusters4620lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4621worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4622worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4623client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24624worker1 # [ 41.789778] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:477704625client # don't know how to build these paths:4626client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4627worker2 # [ 41.756530] nix-grpc-daemon[729]: ts=2026-09-20T10:51:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533484628worker1 # [ 41.813654] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04629worker1 # [ 41.815946] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14630worker1 # [ 41.857504] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704631worker1 # [ 41.863958] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704632worker1 # [ 41.881700] nix-grpc-daemon[857]: time=2026-09-20T10:51:09.036Z level=INFO msg="All 2 paths already cached"4633client # copying 1 paths to 'lb:50051'...4634worker1 # [ 41.887218] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477704635client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4636worker1 # [ 41.898813] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4637worker1 # [ 42.067197] nix-daemon[845]: accepted connection from pid 1003, user nix-grpc-daemon (trusted)4638worker1 # [ 42.082186] nix-daemon[845]: reaped child process 1006, status = succeeded4639worker1 # [ 42.242778] nix-daemon[845]: accepted connection from pid 1008, user nix-grpc-daemon (trusted)4640worker1 # [ 42.256313] nix-daemon[845]: reaped child process 1012, status = succeeded4641lb # [ 42.285612] niks3-server[1018]: 2026/09/20 10:51:08 INFO Received uploads request method=POST path=/api/pending_closures4642worker1 # [ 42.275180] nix-grpc-daemon[857]: time=2026-09-20T10:51:09.429Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4643worker1 # [ 42.276659] nix-grpc-daemon[857]: time=2026-09-20T10:51:09.431Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4644lb # [ 42.304796] niks3-server[1018]: 2026/09/20 10:51:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4645lb # [ 42.306633] niks3-server[1018]: 2026/09/20 10:51:08 INFO Signed narinfos id=7 count=14646worker1 # [ 42.291812] nix-grpc-daemon[857]: time=2026-09-20T10:51:09.446Z level=INFO msg="Uploading 1 narinfos"4647lb # [ 42.310315] niks3-server[1018]: 2026/09/20 10:51:08 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4648lb # [ 42.313456] niks3-server[1018]: 2026/09/20 10:51:08 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4649lb # [ 42.317980] niks3-server[1018]: 2026/09/20 10:51:08 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4650lb # [ 42.321411] niks3-server[1018]: 2026/09/20 10:51:08 INFO Completed upload id=74651worker1 # [ 42.305181] nix-grpc-daemon[857]: time=2026-09-20T10:51:09.459Z level=INFO msg="Upload complete. (399ms)"4652lb # [ 42.324055] niks3-server[1018]: 2026/09/20 10:51:08 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4653worker1 # [ 42.307970] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_in=7524654worker1 # [ 42.310446] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704655worker1 # [ 42.316556] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4656worker1 # [ 42.319168] nix-daemon[845]: reaped child process 1001, status = succeeded4657worker1 # [ 42.324840] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4658client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4659client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4660client # farm-top> LOG-farm-top-t1b4661worker1 # [ 43.538488] nix-daemon[845]: reaped child process 1016, status = succeeded4662worker1 # [ 43.680415] nix-daemon[845]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4663worker1 # [ 43.693064] nix-daemon[845]: reaped child process 1034, status = succeeded4664lb # [ 43.727079] niks3-server[1018]: 2026/09/20 10:51:10 INFO Received uploads request method=POST path=/api/pending_closures4665worker1 # [ 43.720098] nix-grpc-daemon[857]: time=2026-09-20T10:51:10.874Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4666worker1 # [ 43.722146] nix-grpc-daemon[857]: time=2026-09-20T10:51:10.876Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4667lb # [ 43.771906] niks3-server[1018]: 2026/09/20 10:51:10 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4668lb # [ 43.778484] niks3-server[1018]: 2026/09/20 10:51:10 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4669lb # [ 43.785711] niks3-server[1018]: 2026/09/20 10:51:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4670lb # [ 43.791861] niks3-server[1018]: 2026/09/20 10:51:10 INFO Signed narinfos id=8 count=14671worker1 # [ 43.778215] nix-grpc-daemon[857]: time=2026-09-20T10:51:10.932Z level=INFO msg="Uploading 1 narinfos"4672lb # [ 43.796782] niks3-server[1018]: 2026/09/20 10:51:10 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4673lb # [ 43.805943] niks3-server[1018]: 2026/09/20 10:51:10 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4674lb # [ 43.809687] niks3-server[1018]: 2026/09/20 10:51:10 INFO Completed upload id=84675worker1 # [ 43.793549] nix-grpc-daemon[857]: time=2026-09-20T10:51:10.948Z level=INFO msg="Upload complete. (259ms)"4676lb # [ 43.811337] niks3-server[1018]: 2026/09/20 10:51:10 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4677worker1 # [ 43.797065] nix-grpc-daemon[784]: ts=2026-09-20T10:51:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14678worker1 # [ 43.803589] nix-daemon[845]: reaped child process 1014, status = succeeded4679client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.25 seconds)4680client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4681client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.20 seconds)4682lb: must succeed: curl -sf localhost:9901/clusters4683lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4684lb: must succeed: curl -sf localhost:9901/clusters4685lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4686worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4687worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4688(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.74 seconds)4689subtest: two clients wanting the same drv build it once4690client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4691client # [ 44.236953] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4692client # Running as unit: dedup1.service; invocation ID: 1f421fb30a8d48599aad9139d14ea8cb4693client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.08 seconds)4694client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4695client # [ 44.460646] nix[893]: don't know how to build these paths:4696client # [ 44.461577] nix[893]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4697worker2 # [ 44.430079] nix-grpc-daemon[729]: ts=2026-09-20T10:51:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:533484698worker1 # [ 44.480230] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04699worker2 # [ 44.443425] nix-grpc-daemon[729]: ts=2026-09-20T10:51:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533484700worker1 # [ 44.485465] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14701worker2 # [ 44.459524] nix-grpc-daemon[729]: ts=2026-09-20T10:51:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533484702client # [ 44.499048] nix[893]: copying 1 paths to 'lb:50051'...4703client # [ 44.499844] nix[893]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4704worker2 # [ 44.471232] nix-grpc-daemon[729]: ts=2026-09-20T10:51:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533484705worker2 # [ 44.483741] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)4706worker2 # [ 44.505124] nix-grpc-daemon[729]: ts=2026-09-20T10:51:10Z level=info event=niks3_push_started pid=8684707worker2 # [ 44.964088] nix-daemon[841]: accepted connection from pid 880, user nix-grpc-daemon (trusted)4708worker2 # [ 44.979565] nix-daemon[841]: reaped child process 883, status = succeeded4709worker2 # [ 45.146761] nix-daemon[841]: accepted connection from pid 885, user nix-grpc-daemon (trusted)4710worker2 # [ 45.158784] nix-daemon[841]: reaped child process 888, status = succeeded4711lb # [ 45.235301] niks3-server[1018]: 2026/09/20 10:51:11 INFO Received uploads request method=POST path=/api/pending_closures4712worker2 # [ 45.189310] nix-grpc-daemon[868]: time=2026-09-20T10:51:11.633Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4713worker2 # [ 45.192530] nix-grpc-daemon[868]: time=2026-09-20T10:51:11.637Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4714lb # [ 45.282126] niks3-server[1018]: 2026/09/20 10:51:11 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4715lb # [ 45.289951] niks3-server[1018]: 2026/09/20 10:51:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4716lb # [ 45.295147] niks3-server[1018]: 2026/09/20 10:51:11 INFO Signed narinfos id=9 count=14717worker2 # [ 45.242026] nix-grpc-daemon[868]: time=2026-09-20T10:51:11.687Z level=INFO msg="Uploading 1 narinfos"4718lb # [ 45.311546] niks3-server[1018]: 2026/09/20 10:51:11 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4719lb # [ 45.317233] niks3-server[1018]: 2026/09/20 10:51:11 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4720lb # [ 45.324641] niks3-server[1018]: 2026/09/20 10:51:11 INFO Completed upload id=94721worker2 # [ 45.268046] nix-grpc-daemon[868]: time=2026-09-20T10:51:11.713Z level=INFO msg="Upload complete. (473ms)"4722lb # [ 45.326498] niks3-server[1018]: 2026/09/20 10:51:11 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4723worker2 # [ 45.272335] nix-grpc-daemon[729]: ts=2026-09-20T10:51:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=0 paths=1 nar_bytes_in=4324724worker2 # [ 45.277602] nix-grpc-daemon[729]: ts=2026-09-20T10:51:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533484725worker2 # [ 45.286044] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)4726worker2 # [ 45.289130] nix-daemon[841]: reaped child process 866, status = succeeded4727worker2 # [ 45.294519] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)4728client # [ 45.321459] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4729client # [ 45.334665] nix[893]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4730client # Running as unit: dedup2.service; invocation ID: d837a26140f7487b986767fbc851b47d4731client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.10 seconds)4732worker1 # [ 45.598425] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:477704733client # [ 45.593134] nix[909]: don't know how to build these paths:4734client # [ 45.594068] nix[909]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4735worker1 # [ 45.610829] nix-grpc-daemon[784]: ts=2026-09-20T10:51:12Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04736worker2 # [ 45.575848] nix-grpc-daemon[729]: ts=2026-09-20T10:51:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533484737client # [ 45.609425] nix[909]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4738client: must succeed: sleep 24739lb # [ 47.279524] envoy[908]: conn peer=[2001:db8:1::1]:36456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9524 tx=3812 ms=68354740lb # [ 47.283505] envoy[908]: conn peer=[2001:db8:1::1]:52952 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=604741lb # [ 47.286990] envoy[908]: conn peer=[2001:db8:1::1]:52960 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4169 tx=1699 ms=20204742client: (finished: must succeed: sleep 2, in 2.05 seconds)4743??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4744 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394745client: waiting for success: ! systemctl is-active dedup1 dedup24746??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4747 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394748worker2 # [ 48.922603] nix-daemon[841]: reaped child process 892, status = succeeded4749worker2 # [ 49.080504] nix-daemon[841]: accepted connection from pid 921, user nix-grpc-daemon (trusted)4750worker2 # [ 49.093269] nix-daemon[841]: reaped child process 924, status = succeeded4751lb # [ 49.165284] niks3-server[1018]: 2026/09/20 10:51:15 INFO Received uploads request method=POST path=/api/pending_closures4752worker2 # [ 49.131031] nix-grpc-daemon[868]: time=2026-09-20T10:51:15.576Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4753worker2 # [ 49.132890] nix-grpc-daemon[868]: time=2026-09-20T10:51:15.578Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4754lb # [ 49.217968] niks3-server[1018]: 2026/09/20 10:51:15 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4755lb # [ 49.232523] niks3-server[1018]: 2026/09/20 10:51:15 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4756lb # [ 49.244283] niks3-server[1018]: 2026/09/20 10:51:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4757lb # [ 49.249856] niks3-server[1018]: 2026/09/20 10:51:15 INFO Signed narinfos id=10 count=14758worker2 # [ 49.197140] nix-grpc-daemon[868]: time=2026-09-20T10:51:15.642Z level=INFO msg="Uploading 1 narinfos"4759lb # [ 49.255063] niks3-server[1018]: 2026/09/20 10:51:15 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4760lb # [ 49.264419] niks3-server[1018]: 2026/09/20 10:51:15 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4761lb # [ 49.270603] niks3-server[1018]: 2026/09/20 10:51:15 INFO Completed upload id=104762worker2 # [ 49.215299] nix-grpc-daemon[868]: time=2026-09-20T10:51:15.661Z level=INFO msg="Upload complete. (306ms)"4763lb # [ 49.272147] niks3-server[1018]: 2026/09/20 10:51:15 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4764worker2 # [ 49.220181] nix-grpc-daemon[729]: ts=2026-09-20T10:51:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14765worker2 # [ 49.223585] nix-grpc-daemon[729]: ts=2026-09-20T10:51:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14766worker2 # [ 49.230949] nix-daemon[841]: reaped child process 890, status = succeeded4767client # [ 49.296666] systemd[1]: dedup2.service: Deactivated successfully.4768client # [ 49.301131] systemd[1]: dedup2.service: Consumed 127ms CPU time over 3.978s wall clock time, 31.2M memory peak, 7.4K incoming IP traffic, 6.4K outgoing IP traffic.4769client # [ 49.308325] systemd[1]: dedup1.service: Deactivated successfully.4770client # [ 49.312468] systemd[1]: dedup1.service: Consumed 111ms CPU time over 5.074s wall clock time, 37.1M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.4771client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds)4772client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4773client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4774worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4775worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4776worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4777worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4778worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4779worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4780worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4781worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4782(finished: subtest: two clients wanting the same drv build it once, in 5.97 seconds)4783subtest: build hook: nix-daemon with builders = grpc://lb4784client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4785client # [ 50.369435] systemd[1]: Started Nix Daemon.4786client # [ 50.570867] nix-daemon[951]: accepted connection from pid 948, user root (trusted)4787worker2 # [ 50.743639] nix-grpc-daemon[729]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:533484788worker1 # [ 50.790991] nix-grpc-daemon[784]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704789worker1 # [ 50.805243] nix-grpc-daemon[784]: ts=2026-09-20T10:51:17Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04790worker1 # [ 50.810757] nix-grpc-daemon[784]: ts=2026-09-20T10:51:17Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14791worker1 # [ 50.850235] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704792worker1 # [ 50.867062] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704793worker1 # [ 50.879218] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477704794worker1 # [ 50.895508] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4795worker1 # [ 51.080966] nix-daemon[845]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4796worker1 # [ 51.094328] nix-daemon[845]: reaped child process 1073, status = succeeded4797worker1 # [ 51.234432] nix-daemon[845]: accepted connection from pid 1075, user nix-grpc-daemon (trusted)4798worker1 # [ 51.247410] nix-daemon[845]: reaped child process 1078, status = succeeded4799lb # [ 51.277744] niks3-server[1018]: 2026/09/20 10:51:17 INFO Received uploads request method=POST path=/api/pending_closures4800worker1 # [ 51.277450] nix-grpc-daemon[857]: time=2026-09-20T10:51:18.432Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4801worker1 # [ 51.279288] nix-grpc-daemon[857]: time=2026-09-20T10:51:18.433Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4802lb # [ 51.318697] niks3-server[1018]: 2026/09/20 10:51:17 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4803lb # [ 51.327891] niks3-server[1018]: 2026/09/20 10:51:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4804lb # [ 51.333514] niks3-server[1018]: 2026/09/20 10:51:17 INFO Signed narinfos id=11 count=14805worker1 # [ 51.320413] nix-grpc-daemon[857]: time=2026-09-20T10:51:18.475Z level=INFO msg="Uploading 1 narinfos"4806lb # [ 51.344743] niks3-server[1018]: 2026/09/20 10:51:17 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4807lb # [ 51.353114] niks3-server[1018]: 2026/09/20 10:51:17 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4808lb # [ 51.359170] niks3-server[1018]: 2026/09/20 10:51:17 INFO Completed upload id=114809worker1 # [ 51.343073] nix-grpc-daemon[857]: time=2026-09-20T10:51:18.497Z level=INFO msg="Upload complete. (434ms)"4810lb # [ 51.361983] niks3-server[1018]: 2026/09/20 10:51:17 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4811worker1 # [ 51.346794] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_in=5124812worker1 # [ 51.352268] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704813worker1 # [ 51.361208] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4814worker1 # [ 51.370627] nix-daemon[845]: reaped child process 1068, status = succeeded4815worker1 # [ 51.375122] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4816worker1 # [ 52.805715] nix-daemon[845]: reaped child process 1082, status = succeeded4817worker1 # [ 52.942249] nix-daemon[845]: accepted connection from pid 1097, user nix-grpc-daemon (trusted)4818worker1 # [ 52.954286] nix-daemon[845]: reaped child process 1100, status = succeeded4819lb # [ 52.987634] niks3-server[1018]: 2026/09/20 10:51:19 INFO Received uploads request method=POST path=/api/pending_closures4820worker1 # [ 52.978031] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.132Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4821worker1 # [ 52.979764] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.134Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4822lb # [ 53.030353] niks3-server[1018]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4823lb # [ 53.043980] niks3-server[1018]: 2026/09/20 10:51:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4824lb # [ 53.050358] niks3-server[1018]: 2026/09/20 10:51:19 INFO Signed narinfos id=12 count=14825worker1 # [ 53.037085] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.191Z level=INFO msg="Uploading 1 narinfos"4826lb # [ 53.056865] niks3-server[1018]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4827lb # [ 53.060733] niks3-server[1018]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4828lb # [ 53.071278] niks3-server[1018]: 2026/09/20 10:51:19 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4829lb # [ 53.074297] niks3-server[1018]: 2026/09/20 10:51:19 INFO Completed upload id=124830worker1 # [ 53.058337] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.212Z level=INFO msg="Upload complete. (275ms)"4831lb # [ 53.076928] niks3-server[1018]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4832worker1 # [ 53.066610] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14833worker1 # [ 53.074333] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:477704834worker1 # [ 53.085584] nix-daemon[845]: reaped child process 1080, status = succeeded4835worker1 # [ 53.093831] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:477704836worker1 # [ 53.098953] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_out=1284837worker1 # [ 53.368216] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:477704838worker1 # [ 53.374501] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704839worker1 # [ 53.379441] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.533Z level=INFO msg="All 1 paths already cached"4840worker1 # [ 53.389231] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04841worker1 # [ 53.393151] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14842worker1 # [ 53.400678] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704843worker1 # [ 53.412385] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704844worker1 # [ 53.422278] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.576Z level=INFO msg="All 1 paths already cached"4845worker1 # [ 53.428072] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477704846worker1 # [ 53.436280] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4847worker1 # [ 53.601182] nix-daemon[845]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)4848worker1 # [ 53.613946] nix-daemon[845]: reaped child process 1109, status = succeeded4849worker1 # [ 53.761187] nix-daemon[845]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)4850worker1 # [ 53.772551] nix-daemon[845]: reaped child process 1114, status = succeeded4851lb # [ 53.801624] niks3-server[1018]: 2026/09/20 10:51:20 INFO Received uploads request method=POST path=/api/pending_closures4852worker1 # [ 53.793303] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.948Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4853worker1 # [ 53.795156] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.949Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4854lb # [ 53.828667] niks3-server[1018]: 2026/09/20 10:51:20 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4855lb # [ 53.839287] niks3-server[1018]: 2026/09/20 10:51:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4856lb # [ 53.843841] niks3-server[1018]: 2026/09/20 10:51:20 INFO Signed narinfos id=13 count=14857worker1 # [ 53.830627] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.985Z level=INFO msg="Uploading 1 narinfos"4858lb # [ 53.855236] niks3-server[1018]: 2026/09/20 10:51:20 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4859lb # [ 53.862649] niks3-server[1018]: 2026/09/20 10:51:20 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4860worker1 # [ 53.852927] nix-grpc-daemon[857]: time=2026-09-20T10:51:21.007Z level=INFO msg="Upload complete. (404ms)"4861lb # [ 53.869733] niks3-server[1018]: 2026/09/20 10:51:20 INFO Completed upload id=134862worker1 # [ 53.855490] nix-grpc-daemon[784]: ts=2026-09-20T10:51:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_in=6244863lb # [ 53.874858] niks3-server[1018]: 2026/09/20 10:51:20 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4864worker1 # [ 53.860844] nix-grpc-daemon[784]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704865worker1 # [ 53.869350] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4866worker1 # [ 53.875817] nix-daemon[845]: reaped child process 1104, status = succeeded4867worker1 # [ 53.878756] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4868worker1 # [ 54.902893] nix-daemon[845]: reaped child process 1118, status = succeeded4869worker1 # [ 55.059598] nix-daemon[845]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)4870worker1 # [ 55.071836] nix-daemon[845]: reaped child process 1136, status = succeeded4871lb # [ 55.102916] niks3-server[1018]: 2026/09/20 10:51:21 INFO Received uploads request method=POST path=/api/pending_closures4872worker1 # [ 55.095081] nix-grpc-daemon[857]: time=2026-09-20T10:51:22.248Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4873worker1 # [ 55.096459] nix-grpc-daemon[857]: time=2026-09-20T10:51:22.249Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4874lb # [ 55.132305] niks3-server[1018]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4875lb # [ 55.146164] niks3-server[1018]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4876lb # [ 55.154837] niks3-server[1018]: 2026/09/20 10:51:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4877lb # [ 55.160608] niks3-server[1018]: 2026/09/20 10:51:21 INFO Signed narinfos id=14 count=14878worker1 # [ 55.146845] nix-grpc-daemon[857]: time=2026-09-20T10:51:22.301Z level=INFO msg="Uploading 1 narinfos"4879lb # [ 55.164504] niks3-server[1018]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4880lb # [ 55.173793] niks3-server[1018]: 2026/09/20 10:51:21 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4881lb # [ 55.179459] niks3-server[1018]: 2026/09/20 10:51:21 INFO Completed upload id=144882worker1 # [ 55.163424] nix-grpc-daemon[857]: time=2026-09-20T10:51:22.318Z level=INFO msg="Upload complete. (276ms)"4883lb # [ 55.182374] niks3-server[1018]: 2026/09/20 10:51:21 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4884worker1 # [ 55.167951] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14885worker1 # [ 55.180713] nix-daemon[845]: reaped child process 1116, status = succeeded4886worker1 # [ 55.218134] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:477704887worker1 # [ 55.234844] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:477704888worker1 # [ 55.239186] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_out=1764889worker1 # [ 55.525796] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:477704890worker2 # [ 55.489241] nix-grpc-daemon[729]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533484891worker1 # [ 55.552234] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04892worker1 # [ 55.556145] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14893worker1 # [ 55.563573] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704894worker1 # [ 55.575784] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704895worker1 # [ 55.588326] nix-grpc-daemon[857]: time=2026-09-20T10:51:22.743Z level=INFO msg="All 2 paths already cached"4896worker1 # [ 55.595517] nix-grpc-daemon[784]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477704897worker1 # [ 55.603426] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4898worker1 # [ 55.752561] nix-daemon[845]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4899worker1 # [ 55.765119] nix-daemon[845]: reaped child process 1145, status = succeeded4900worker1 # [ 55.906568] nix-daemon[845]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)4901worker1 # [ 55.918265] nix-daemon[845]: reaped child process 1150, status = succeeded4902lb # [ 55.947950] niks3-server[1018]: 2026/09/20 10:51:22 INFO Received uploads request method=POST path=/api/pending_closures4903worker1 # [ 55.939486] nix-grpc-daemon[857]: time=2026-09-20T10:51:23.094Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4904worker1 # [ 55.940944] nix-grpc-daemon[857]: time=2026-09-20T10:51:23.095Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4905lb # [ 55.974894] niks3-server[1018]: 2026/09/20 10:51:22 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4906lb # [ 55.985465] niks3-server[1018]: 2026/09/20 10:51:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4907lb # [ 55.989616] niks3-server[1018]: 2026/09/20 10:51:22 INFO Signed narinfos id=15 count=14908worker1 # [ 55.976389] nix-grpc-daemon[857]: time=2026-09-20T10:51:23.131Z level=INFO msg="Uploading 1 narinfos"4909lb # [ 55.999645] niks3-server[1018]: 2026/09/20 10:51:22 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4910lb # [ 56.008200] niks3-server[1018]: 2026/09/20 10:51:22 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4911worker1 # [ 55.999119] nix-grpc-daemon[857]: time=2026-09-20T10:51:23.153Z level=INFO msg="Upload complete. (381ms)"4912lb # [ 56.015335] niks3-server[1018]: 2026/09/20 10:51:22 INFO Completed upload id=154913lb # [ 56.018518] niks3-server[1018]: 2026/09/20 10:51:22 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4914worker1 # [ 56.001580] nix-grpc-daemon[784]: ts=2026-09-20T10:51:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_in=7524915worker1 # [ 56.006679] nix-grpc-daemon[784]: ts=2026-09-20T10:51:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477704916worker1 # [ 56.015084] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4917worker1 # [ 56.020814] nix-daemon[845]: reaped child process 1140, status = succeeded4918worker1 # [ 56.025175] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4919lb # [ 57.280537] envoy[908]: conn peer=[2001:db8:1::1]:36830 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1082 ms=36824920lb # [ 57.284336] envoy[908]: conn peer=[2001:db8:1::1]:36822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2985 tx=1372 ms=48064921lb # [ 57.287903] envoy[908]: conn peer=[2001:db8:1::1]:36844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3411 tx=1660 ms=23694922lb # [ 57.291393] envoy[908]: conn peer=[2001:db8:1::1]:43402 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=574923lb # [ 57.294743] envoy[908]: conn peer=[2001:db8:1::1]:43416 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=1875 ms=19214924lb # [ 57.298298] envoy[908]: conn peer=[2001:db8:1::1]:43426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=594925worker1 # [ 57.648663] nix-daemon[845]: reaped child process 1154, status = succeeded4926worker1 # [ 57.785261] nix-daemon[845]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)4927worker1 # [ 57.796234] nix-daemon[845]: reaped child process 1172, status = succeeded4928lb # [ 57.826771] niks3-server[1018]: 2026/09/20 10:51:24 INFO Received uploads request method=POST path=/api/pending_closures4929worker1 # [ 57.820158] nix-grpc-daemon[857]: time=2026-09-20T10:51:24.974Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4930worker1 # [ 57.821665] nix-grpc-daemon[857]: time=2026-09-20T10:51:24.976Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4931lb # [ 57.871101] niks3-server[1018]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4932lb # [ 57.880954] niks3-server[1018]: 2026/09/20 10:51:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4933worker1 # [ 57.872200] nix-grpc-daemon[857]: time=2026-09-20T10:51:25.026Z level=INFO msg="Uploading 1 narinfos"4934lb # [ 57.888290] niks3-server[1018]: 2026/09/20 10:51:24 INFO Signed narinfos id=16 count=14935lb # [ 57.892947] niks3-server[1018]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4936lb # [ 57.898788] niks3-server[1018]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4937lb # [ 57.908955] niks3-server[1018]: 2026/09/20 10:51:24 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4938lb # [ 57.914598] niks3-server[1018]: 2026/09/20 10:51:24 INFO Completed upload id=164939worker1 # [ 57.898516] nix-grpc-daemon[857]: time=2026-09-20T10:51:25.053Z level=INFO msg="Upload complete. (265ms)"4940lb # [ 57.917438] niks3-server[1018]: 2026/09/20 10:51:24 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4941worker1 # [ 57.901462] nix-grpc-daemon[784]: ts=2026-09-20T10:51:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14942worker1 # [ 57.907968] nix-daemon[845]: reaped child process 1152, status = succeeded4943worker2 # [ 57.867339] nix-grpc-daemon[729]: ts=2026-09-20T10:51:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:533484944worker2 # [ 57.880759] nix-grpc-daemon[729]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4945worker2 # [ 57.895380] nix-grpc-daemon[729]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4946worker2 # [ 57.907268] nix-grpc-daemon[729]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4947worker2 # [ 57.932526] nix-grpc-daemon[729]: ts=2026-09-20T10:51:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:533484948worker2 # [ 57.937982] nix-grpc-daemon[729]: ts=2026-09-20T10:51:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=0 paths=1 nar_bytes_out=2324949client # [ 58.077947] nix-daemon[951]: reaped child process 953, status = succeeded4950client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.94 seconds)4951client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4952client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4953client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4954client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4955client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4956client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4957(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.04 seconds)4958subtest: an input only one worker has still reaches the builder4959client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4960client # this derivation will be built:4961client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4962client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4963client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.41 seconds)4964client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4965client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.19 seconds)4966worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4967worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4968worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4969worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4970client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24971client # [ 59.159071] nix-daemon[951]: accepted connection from pid 1033, user root (trusted)4972client # this derivation will be built:4973client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4974client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...4975worker1 # [ 59.354862] nix-grpc-daemon[784]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:477704976worker1 # [ 59.360891] nix-grpc-daemon[784]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477704977worker1 # [ 59.533398] nix-daemon[845]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)4978worker1 # [ 59.544508] nix-daemon[845]: reaped child process 1182, status = succeeded4979lb # [ 59.575719] niks3-server[1018]: 2026/09/20 10:51:25 INFO Received uploads request method=POST path=/api/pending_closures4980worker1 # [ 59.567497] nix-grpc-daemon[857]: time=2026-09-20T10:51:26.722Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4981worker1 # [ 59.569529] nix-grpc-daemon[857]: time=2026-09-20T10:51:26.724Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4982lb # [ 59.604154] niks3-server[1018]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4983lb # [ 59.614359] niks3-server[1018]: 2026/09/20 10:51:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4984worker1 # [ 59.602500] nix-grpc-daemon[857]: time=2026-09-20T10:51:26.757Z level=INFO msg="Uploading 1 narinfos"4985lb # [ 59.618690] niks3-server[1018]: 2026/09/20 10:51:25 INFO Signed narinfos id=17 count=14986lb # [ 59.626192] niks3-server[1018]: 2026/09/20 10:51:25 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4987lb # [ 59.634957] niks3-server[1018]: 2026/09/20 10:51:25 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4988lb # [ 59.642359] niks3-server[1018]: 2026/09/20 10:51:26 INFO Completed upload id=174989worker1 # [ 59.626352] nix-grpc-daemon[857]: time=2026-09-20T10:51:26.781Z level=INFO msg="Upload complete. (260ms)"4990lb # [ 59.647201] niks3-server[1018]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4991worker1 # [ 59.645556] nix-grpc-daemon[784]: ts=2026-09-20T10:51:26Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04992worker1 # [ 59.649957] nix-grpc-daemon[784]: ts=2026-09-20T10:51:26Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14993worker2 # [ 59.649455] nix-grpc-daemon[729]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533484994worker2 # [ 59.657963] nix-grpc-daemon[729]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533484995client # copying 2 paths to 'lb:50051'...4996worker2 # [ 59.668895] nix-grpc-daemon[729]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533484997client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4998client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4999worker2 # [ 59.677478] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)5000worker2 # [ 59.820127] nix-daemon[841]: accepted connection from pid 944, user nix-grpc-daemon (trusted)5001worker2 # [ 59.833665] nix-daemon[841]: reaped child process 947, status = succeeded5002worker2 # [ 59.981352] nix-daemon[841]: accepted connection from pid 949, user nix-grpc-daemon (trusted)5003worker2 # [ 59.994119] nix-daemon[841]: reaped child process 952, status = succeeded5004lb # [ 60.065785] niks3-server[1018]: 2026/09/20 10:51:26 INFO Received uploads request method=POST path=/api/pending_closures5005lb # [ 60.074930] niks3-server[1018]: 2026/09/20 10:51:26 INFO Received uploads request method=POST path=/api/pending_closures5006worker2 # [ 60.027202] nix-grpc-daemon[868]: time=2026-09-20T10:51:26.472Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5007worker2 # [ 60.028618] nix-grpc-daemon[868]: time=2026-09-20T10:51:26.472Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5008worker2 # [ 60.030250] nix-grpc-daemon[868]: time=2026-09-20T10:51:26.472Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5009lb # [ 60.108500] niks3-server[1018]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5010lb # [ 60.120857] niks3-server[1018]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5011lb # [ 60.126858] niks3-server[1018]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5012lb # [ 60.133967] niks3-server[1018]: 2026/09/20 10:51:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5013lb # [ 60.139687] niks3-server[1018]: 2026/09/20 10:51:26 INFO Signed narinfos id=19 count=25014lb # [ 60.141647] niks3-server[1018]: 2026/09/20 10:51:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5015lb # [ 60.143353] niks3-server[1018]: 2026/09/20 10:51:26 INFO Signed narinfos id=18 count=15016lb # [ 60.146369] niks3-server[1018]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5017worker2 # [ 60.088845] nix-grpc-daemon[868]: time=2026-09-20T10:51:26.534Z level=INFO msg="Uploading 3 narinfos"5018lb # [ 60.154779] niks3-server[1018]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5019lb # [ 60.161070] niks3-server[1018]: 2026/09/20 10:51:26 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5020lb # [ 60.176816] niks3-server[1018]: 2026/09/20 10:51:26 INFO Completed upload id=195021lb # [ 60.178948] niks3-server[1018]: 2026/09/20 10:51:26 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5022lb # [ 60.180931] niks3-server[1018]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5023lb # [ 60.183251] niks3-server[1018]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5024lb # [ 60.185564] niks3-server[1018]: 2026/09/20 10:51:26 INFO Completed upload id=185025worker2 # [ 60.129367] nix-grpc-daemon[868]: time=2026-09-20T10:51:26.575Z level=INFO msg="Upload complete. (445ms)"5026worker2 # [ 60.134146] nix-grpc-daemon[729]: ts=2026-09-20T10:51:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=0 paths=2 nar_bytes_in=9765027worker2 # [ 60.140399] nix-grpc-daemon[729]: ts=2026-09-20T10:51:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533485028worker2 # [ 60.153131] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)5029worker2 # [ 60.164214] nix-daemon[841]: reaped child process 941, status = succeeded5030worker2 # [ 60.173592] nix-grpc-daemon[729]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5031client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5032worker2 # [ 60.187240] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)5033client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5034worker2 # [ 61.205498] nix-daemon[841]: reaped child process 957, status = succeeded5035worker2 # [ 61.355295] nix-daemon[841]: accepted connection from pid 972, user nix-grpc-daemon (trusted)5036worker2 # [ 61.367060] nix-daemon[841]: reaped child process 975, status = succeeded5037lb # [ 61.440277] niks3-server[1018]: 2026/09/20 10:51:27 INFO Received uploads request method=POST path=/api/pending_closures5038worker2 # [ 61.388377] nix-grpc-daemon[868]: time=2026-09-20T10:51:27.834Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5039lb # [ 61.456702] niks3-server[1018]: 2026/09/20 10:51:27 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5040lb # [ 61.459392] niks3-server[1018]: 2026/09/20 10:51:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5041lb # [ 61.462260] niks3-server[1018]: 2026/09/20 10:51:27 INFO Signed narinfos id=20 count=15042worker2 # [ 61.406052] nix-grpc-daemon[868]: time=2026-09-20T10:51:27.851Z level=INFO msg="Uploading 1 narinfos"5043lb # [ 61.467684] niks3-server[1018]: 2026/09/20 10:51:27 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5044lb # [ 61.471487] niks3-server[1018]: 2026/09/20 10:51:27 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5045lb # [ 61.476432] niks3-server[1018]: 2026/09/20 10:51:27 INFO Completed upload id=205046worker2 # [ 61.419936] nix-grpc-daemon[868]: time=2026-09-20T10:51:27.865Z level=INFO msg="Upload complete. (220ms)"5047lb # [ 61.477747] niks3-server[1018]: 2026/09/20 10:51:27 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5048worker2 # [ 61.422251] nix-grpc-daemon[729]: ts=2026-09-20T10:51:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15049worker2 # [ 61.428814] nix-daemon[841]: reaped child process 954, status = succeeded5050client # copying 1 paths...5051worker1 # [ 61.508984] nix-grpc-daemon[784]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:477705052worker1 # [ 61.516313] nix-grpc-daemon[784]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5053client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5054worker1 # [ 61.542582] nix-grpc-daemon[784]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:477705055worker1 # [ 61.544645] nix-grpc-daemon[784]: ts=2026-09-20T10:51:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_out=1205056client # [ 61.623726] nix-daemon[951]: reaped child process 1036, status = succeeded5057client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.64 seconds)5058client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25059client # [ 61.802108] nix-daemon[951]: accepted connection from pid 1055, user root (trusted)5060client # this derivation will be built:5061client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5062worker2 # [ 61.950974] nix-grpc-daemon[729]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:533485063client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5064worker2 # [ 61.957823] nix-grpc-daemon[729]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533485065worker2 # [ 61.962950] nix-grpc-daemon[868]: time=2026-09-20T10:51:28.407Z level=INFO msg="All 1 paths already cached"5066worker1 # [ 62.011787] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05067worker1 # [ 62.018426] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15068worker1 # [ 62.023947] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477705069worker1 # [ 62.033343] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477705070client # copying 1 paths to 'lb:50051'...5071client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5072worker1 # [ 62.046516] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:477705073worker1 # [ 62.058524] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5074worker1 # [ 62.073821] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)5075worker1 # [ 62.115654] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 2/5)5076worker1 # [ 62.261712] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 3/5)5077worker1 # [ 62.326260] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 423 ms (attempt 4/5)5078worker1 # [ 62.756277] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5079worker1 # [ 62.762298] nix-grpc-daemon[784]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5080worker1 # [ 62.962117] nix-daemon[845]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5081worker1 # [ 62.974362] nix-daemon[845]: reaped child process 1198, status = succeeded5082worker1 # [ 63.132821] nix-daemon[845]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)5083worker1 # [ 63.145557] nix-daemon[845]: reaped child process 1203, status = succeeded5084lb # [ 63.171979] niks3-server[1018]: 2026/09/20 10:51:29 INFO Received uploads request method=POST path=/api/pending_closures5085worker1 # [ 63.162571] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.317Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5086worker1 # [ 63.164434] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.318Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5087lb # [ 63.196933] niks3-server[1018]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5088lb # [ 63.208094] niks3-server[1018]: 2026/09/20 10:51:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5089worker1 # [ 63.196927] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.351Z level=INFO msg="Uploading 1 narinfos"5090lb # [ 63.212961] niks3-server[1018]: 2026/09/20 10:51:29 INFO Signed narinfos id=21 count=15091lb # [ 63.218779] niks3-server[1018]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5092lb # [ 63.229175] niks3-server[1018]: 2026/09/20 10:51:29 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5093lb # [ 63.233614] niks3-server[1018]: 2026/09/20 10:51:29 INFO Completed upload id=215094worker1 # [ 63.219978] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.374Z level=INFO msg="Upload complete. (441ms)"5095worker1 # [ 63.222058] nix-grpc-daemon[784]: ts=2026-09-20T10:51:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 paths=1 nar_bytes_in=5605096lb # [ 63.239173] niks3-server[1018]: 2026/09/20 10:51:29 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5097worker1 # [ 63.226770] nix-grpc-daemon[784]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:477705098worker1 # [ 63.233442] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5099worker1 # [ 63.235200] nix-daemon[845]: reaped child process 1189, status = succeeded5100worker1 # [ 63.240390] nix-daemon[845]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5101client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5102client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5103worker1 # [ 64.267317] nix-daemon[845]: reaped child process 1207, status = succeeded5104worker1 # [ 64.389257] nix-daemon[845]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5105worker1 # [ 64.401137] nix-daemon[845]: reaped child process 1225, status = succeeded5106lb # [ 64.427675] niks3-server[1018]: 2026/09/20 10:51:30 INFO Received uploads request method=POST path=/api/pending_closures5107worker1 # [ 64.416157] nix-grpc-daemon[857]: time=2026-09-20T10:51:31.570Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5108lb # [ 64.449368] niks3-server[1018]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5109lb # [ 64.453241] niks3-server[1018]: 2026/09/20 10:51:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5110lb # [ 64.455324] niks3-server[1018]: 2026/09/20 10:51:30 INFO Signed narinfos id=22 count=15111worker1 # [ 64.439130] nix-grpc-daemon[857]: time=2026-09-20T10:51:31.593Z level=INFO msg="Uploading 1 narinfos"5112lb # [ 64.460947] niks3-server[1018]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5113lb # [ 64.465329] niks3-server[1018]: 2026/09/20 10:51:30 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5114lb # [ 64.468977] niks3-server[1018]: 2026/09/20 10:51:30 INFO Completed upload id=225115worker1 # [ 64.452567] nix-grpc-daemon[857]: time=2026-09-20T10:51:31.607Z level=INFO msg="Upload complete. (204ms)"5116lb # [ 64.471071] niks3-server[1018]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5117worker1 # [ 64.456141] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15118worker1 # [ 64.461630] nix-daemon[845]: reaped child process 1205, status = succeeded5119client # copying 1 paths...5120worker1 # [ 64.498696] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:477705121client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5122worker1 # [ 64.511914] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:477705123worker1 # [ 64.515321] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47770 duration_s=0 paths=1 nar_bytes_out=1205124client # [ 64.593057] nix-daemon[951]: reaped child process 1058, status = succeeded5125client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.97 seconds)5126client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25127client # [ 64.788943] nix-daemon[951]: accepted connection from pid 1077, user root (trusted)5128client # this derivation will be built:5129client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5130worker2 # [ 64.940800] nix-grpc-daemon[729]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:533485131client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5132worker1 # [ 64.988076] nix-grpc-daemon[784]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:477705133worker1 # [ 64.996595] nix-grpc-daemon[857]: time=2026-09-20T10:51:32.151Z level=INFO msg="All 1 paths already cached"5134worker1 # [ 65.010175] nix-grpc-daemon[784]: ts=2026-09-20T10:51:32Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05135worker1 # [ 65.014752] nix-grpc-daemon[784]: ts=2026-09-20T10:51:32Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15136worker2 # [ 65.014810] nix-grpc-daemon[729]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533485137worker2 # [ 65.024363] nix-grpc-daemon[729]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533485138worker2 # [ 65.034696] nix-grpc-daemon[868]: time=2026-09-20T10:51:31.480Z level=INFO msg="All 1 paths already cached"5139client # copying 1 paths to 'lb:50051'...5140worker2 # [ 65.041147] nix-grpc-daemon[729]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533485141client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5142worker2 # [ 65.050398] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)5143worker2 # [ 65.187155] nix-daemon[841]: accepted connection from pid 979, user nix-grpc-daemon (trusted)5144worker2 # [ 65.199881] nix-daemon[841]: reaped child process 982, status = succeeded5145worker2 # [ 65.326964] nix-daemon[841]: accepted connection from pid 984, user nix-grpc-daemon (trusted)5146worker2 # [ 65.339479] nix-daemon[841]: reaped child process 987, status = succeeded5147lb # [ 65.405530] niks3-server[1018]: 2026/09/20 10:51:31 INFO Received uploads request method=POST path=/api/pending_closures5148worker2 # [ 65.357335] nix-grpc-daemon[868]: time=2026-09-20T10:51:31.803Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5149worker2 # [ 65.359137] nix-grpc-daemon[868]: time=2026-09-20T10:51:31.804Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5150lb # [ 65.435196] niks3-server[1018]: 2026/09/20 10:51:31 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5151lb # [ 65.445241] niks3-server[1018]: 2026/09/20 10:51:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5152worker2 # [ 65.393692] nix-grpc-daemon[868]: time=2026-09-20T10:51:31.839Z level=INFO msg="Uploading 1 narinfos"5153lb # [ 65.451131] niks3-server[1018]: 2026/09/20 10:51:31 INFO Signed narinfos id=23 count=15154lb # [ 65.455973] niks3-server[1018]: 2026/09/20 10:51:31 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5155lb # [ 65.467444] niks3-server[1018]: 2026/09/20 10:51:31 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5156worker2 # [ 65.418248] nix-grpc-daemon[868]: time=2026-09-20T10:51:31.864Z level=INFO msg="Upload complete. (361ms)"5157lb # [ 65.474790] niks3-server[1018]: 2026/09/20 10:51:31 INFO Completed upload id=235158worker2 # [ 65.420303] nix-grpc-daemon[729]: ts=2026-09-20T10:51:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=0 paths=1 nar_bytes_in=5605159lb # [ 65.477864] niks3-server[1018]: 2026/09/20 10:51:31 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5160worker2 # [ 65.425662] nix-grpc-daemon[729]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533485161worker2 # [ 65.434173] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)5162worker2 # [ 65.436437] nix-daemon[841]: reaped child process 977, status = succeeded5163client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5164worker2 # [ 65.441447] nix-daemon[841]: accepted connection from pid 729, user nix-grpc-daemon (trusted)5165client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5166worker2 # [ 66.266654] nix-daemon[841]: reaped child process 991, status = succeeded5167worker2 # [ 66.414678] nix-daemon[841]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)5168worker2 # [ 66.426799] nix-daemon[841]: reaped child process 1009, status = succeeded5169lb # [ 66.498507] niks3-server[1018]: 2026/09/20 10:51:32 INFO Received uploads request method=POST path=/api/pending_closures5170worker2 # [ 66.451853] nix-grpc-daemon[868]: time=2026-09-20T10:51:32.897Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5171lb # [ 66.531153] niks3-server[1018]: 2026/09/20 10:51:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5172lb # [ 66.537657] niks3-server[1018]: 2026/09/20 10:51:32 INFO Signed narinfos id=24 count=15173worker2 # [ 66.484306] nix-grpc-daemon[868]: time=2026-09-20T10:51:32.930Z level=INFO msg="Uploading 1 narinfos"5174lb # [ 66.542119] niks3-server[1018]: 2026/09/20 10:51:32 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5175lb # [ 66.547941] niks3-server[1018]: 2026/09/20 10:51:32 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5176lb # [ 66.561103] niks3-server[1018]: 2026/09/20 10:51:32 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5177worker2 # [ 66.509845] nix-grpc-daemon[868]: time=2026-09-20T10:51:32.955Z level=INFO msg="Upload complete. (262ms)"5178lb # [ 66.566560] niks3-server[1018]: 2026/09/20 10:51:32 INFO Completed upload id=245179lb # [ 66.569596] niks3-server[1018]: 2026/09/20 10:51:32 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5180worker2 # [ 66.514472] nix-grpc-daemon[729]: ts=2026-09-20T10:51:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15181worker2 # [ 66.524952] nix-daemon[841]: reaped child process 989, status = succeeded5182client # copying 1 paths...5183worker1 # [ 66.599174] nix-grpc-daemon[784]: ts=2026-09-20T10:51:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:477705184worker1 # [ 66.611266] nix-grpc-daemon[784]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5185client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5186worker2 # [ 66.604176] nix-grpc-daemon[729]: ts=2026-09-20T10:51:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:533485187worker2 # [ 66.608695] nix-grpc-daemon[729]: ts=2026-09-20T10:51:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53348 duration_s=0 paths=1 nar_bytes_out=1205188client # [ 66.698770] nix-daemon[951]: reaped child process 1080, status = succeeded5189client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.10 seconds)5190(finished: subtest: an input only one worker has still reaches the builder, in 8.53 seconds)5191subtest: upload whose reference the worker lacks is completed from the cache5192client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5193client # this derivation will be built:5194client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5195client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5196client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.41 seconds)5197client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5198lb # [ 67.281142] envoy[908]: conn peer=[2001:db8:1::1]:43438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=585199lb # [ 67.282575] envoy[908]: conn peer=[2001:db8:1::1]:43436 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4302 tx=2219 ms=25055200lb # [ 67.284022] envoy[908]: conn peer=[2001:db8:1::1]:43444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3875 tx=1877 ms=22425201lb # [ 67.285458] envoy[908]: conn peer=[2001:db8:1::1]:43458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=525202lb # [ 67.286831] envoy[908]: conn peer=[2001:db8:1::1]:43464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3698 tx=1804 ms=25725203lb # [ 67.288270] envoy[908]: conn peer=[2001:db8:1::1]:59802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=495204lb # [ 67.289673] envoy[908]: conn peer=[2001:db8:1::1]:59810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=175205lb # [ 67.291143] envoy[908]: conn peer=[2001:db8:1::1]:59804 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3686 tx=1891 ms=16835206client # this derivation will be built:5207client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5208client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5209client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.39 seconds)5210client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5211client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5212worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5213worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.21 seconds)5214worker2 # [ 67.933508] systemd[1]: Stopping Nix worker-protocol over gRPC...5215worker2 # [ 68.473337] nix-grpc-daemon[729]: ts=2026-09-20T10:51:34Z level=info event=signal_exit5216worker2 # [ 69.735325] nix-grpc-daemon[729]: ts=2026-09-20T10:51:36Z level=info event=scheduler_restarting5217worker1 # [ 69.783614] nix-grpc-daemon[784]: ts=2026-09-20T10:51:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477565218worker1 # [ 69.785370] nix-grpc-daemon[784]: ts=2026-09-20T10:51:36Z level=info event=worker_gone addr=192.168.1.4:500515219worker2 # [ 69.754221] nix-grpc-daemon[729]: ts=2026-09-20T10:51:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5220worker2 # [ 69.771663] nix-daemon[841]: reaped child process 843, status = succeeded5221worker2 # [ 70.160566] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5222worker2 # [ 70.163607] systemd[1]: Stopped Nix worker-protocol over gRPC.5223worker2 # [ 70.167372] systemd[1]: nix-grpc-daemon.service: Consumed 1.267s CPU time over 52.942s wall clock time, 49.5M memory peak, 45.3K incoming IP traffic, 85.7K outgoing IP traffic.5224worker2 # [ 70.178754] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5225worker2 # [ 70.183216] systemd[1]: Closed Nix worker-protocol over gRPC.5226worker2 # [ 70.186380] systemd[1]: nix-grpc-daemon.socket: Consumed 343us CPU time over 1min 189ms wall clock time, 256K memory peak, 52.8K incoming IP traffic, 49.7K outgoing IP traffic.5227lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5228lb: must succeed: curl -sf localhost:9901/clusters5229lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5230lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5231client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25232worker1 # [ 70.467904] nix-grpc-daemon[784]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:477705233client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5234worker1 # [ 70.474464] nix-grpc-daemon[784]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:477705235client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.21 seconds)5236worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5237worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5238worker2 # [ 70.541690] systemd[1]: Listening on Nix worker-protocol over gRPC.5239worker1 # [ 70.641404] systemd[1]: Stopping Nix worker-protocol over gRPC...5240worker2 # [ 70.609333] systemd[1]: Starting Nix worker-protocol over gRPC...5241worker2 # [ 70.770065] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5242worker2 # [ 70.773283] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15243worker2 # [ 70.779072] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5244worker2 # [ 70.783130] systemd[1]: Started Nix worker-protocol over gRPC.5245worker2 # [ 70.807359] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:37Z level=info event=scheduler_connected addr=lb:500515246worker1 # [ 70.849061] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05247worker1 # [ 71.522666] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=signal_exit5248worker2 # [ 71.751813] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:38Z level=info event=scheduler_restarting addr=lb:500515249worker1 # [ 71.790284] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=scheduler_yield5250worker1 # [ 71.794654] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477565251lb # [ 71.809912] niks3-server[1018]: 2026/09/20 10:51:38 INFO lead: released remote=192.168.1.3:516265252worker1 # [ 71.799451] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=worker_gone addr=192.168.1.4:500515253worker1 # [ 71.804090] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=scheduler_restarting addr=lb:500515254worker1 # [ 71.807130] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:477565255worker1 # [ 71.810407] nix-grpc-daemon[784]: ts=2026-09-20T10:51:38Z level=info event=worker_gone addr=192.168.1.3:500515256worker2 # [ 71.953842] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:38Z level=info event=scheduler_connected addr=lb:500515257worker2 # [ 71.957832] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258worker1 # [ 72.000431] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515259worker1 # [ 72.002138] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5260worker2 # [ 72.155751] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:38Z level=info event=scheduler_connected addr=lb:500515261worker2 # [ 72.159739] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5262worker1 # [ 72.202685] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515263worker1 # [ 72.204444] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5264worker2 # [ 72.360532] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:38Z level=info event=scheduler_connected addr=lb:500515265worker1 # [ 72.405061] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515266worker2 # [ 72.364439] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5267worker1 # [ 72.408875] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5268worker2 # [ 72.565415] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515269worker1 # [ 72.609368] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515270worker2 # [ 72.569563] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5271worker1 # [ 72.612109] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5272worker1 # [ 72.794679] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=scheduler_restarting5273worker2 # [ 72.770460] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515274worker2 # [ 72.774753] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5275worker1 # [ 72.828662] nix-grpc-daemon[784]: ts=2026-09-20T10:51:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5276worker1 # [ 72.837568] nix-daemon[845]: reaped child process 847, status = succeeded5277worker2 # [ 72.975539] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515278worker2 # [ 72.977490] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5279worker2 # [ 73.178268] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515280worker2 # [ 73.182181] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5281worker2 # [ 73.383109] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_connected addr=lb:500515282worker2 # [ 73.387272] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker2 # [ 73.587722] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_connected addr=lb:500515284worker2 # [ 73.590052] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5285worker1 # [ 73.668149] nix-grpc-daemon[784]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5286worker1 # [ 73.669844] nix-grpc-daemon[784]: E0000 00:00:1789901500.822053 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5287worker1 # [ 73.672209] nix-grpc-daemon[784]: E0000 00:00:1789901500.822499 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5288worker2 # [ 73.790443] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_connected addr=lb:500515289worker2 # [ 73.792654] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5290worker2 # [ 73.994950] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_connected addr=lb:500515291worker2 # [ 73.998211] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5292worker1 # [ 74.152626] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5293worker1 # [ 74.157506] systemd[1]: Stopped Nix worker-protocol over gRPC.5294worker1 # [ 74.160331] systemd[1]: nix-grpc-daemon.service: Consumed 2.655s CPU time over 56.894s wall clock time, 64M memory peak, 94.2K incoming IP traffic, 185.2K outgoing IP traffic.5295worker1 # [ 74.170657] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5296worker1 # [ 74.174978] systemd[1]: Closed Nix worker-protocol over gRPC.5297worker1 # [ 74.178513] systemd[1]: nix-grpc-daemon.socket: Consumed 284us CPU time over 1min 4.866s wall clock time, 256K memory peak, 93.6K incoming IP traffic, 74.6K outgoing IP traffic.5298lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5299lb: must succeed: curl -sf localhost:9901/clusters5300worker2 # [ 74.196191] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_connected addr=lb:500515301worker2 # [ 74.200309] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5302lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5303lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5304client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25305worker2 # [ 74.401081] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_connected addr=lb:500515306worker2 # [ 74.405175] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307worker2 # [ 74.443158] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455745308worker2 # [ 74.455215] nix-daemon[841]: accepted connection from pid 1030, user nix-grpc-daemon (trusted)5309worker2 # [ 74.472300] nix-grpc-daemon[1030]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)5310worker2 # [ 74.500154] nix-grpc-daemon[1030]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 2/5)5311worker2 # [ 74.506744] nix-grpc-daemon[1030]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 123 ms (attempt 3/5)5312worker2 # [ 74.605897] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500515313worker2 # [ 74.609919] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5314worker2 # [ 74.635702] nix-grpc-daemon[1030]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 665 ms (attempt 4/5)5315worker2 # [ 74.810489] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500515316worker2 # [ 74.814393] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker2 # [ 75.014986] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500515318worker2 # [ 75.016878] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5319worker2 # [ 75.217459] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500515320worker2 # [ 75.219220] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5321worker2 # [ 75.307270] nix-grpc-daemon[1030]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5322client # copying 2 paths...5323client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5324client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5325worker2 # [ 75.327812] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455745326worker2 # [ 75.337569] nix-daemon[841]: accepted connection from pid 1030, user nix-grpc-daemon (trusted)5327worker2 # [ 75.359256] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=niks3_push_started pid=10865328worker2 # [ 75.419676] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_connected addr=lb:500515329worker2 # [ 75.421782] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5330worker2 # [ 75.622649] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:42Z level=info event=scheduler_connected addr=lb:500515331worker2 # [ 75.626491] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5332worker2 # [ 75.722220] nix-daemon[841]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)5333worker2 # [ 75.733203] nix-daemon[841]: reaped child process 1102, status = succeeded5334lb # [ 75.807488] niks3-server[1018]: 2026/09/20 10:51:42 INFO Received uploads request method=POST path=/api/pending_closures5335lb # [ 75.818863] niks3-server[1018]: 2026/09/20 10:51:42 INFO Received uploads request method=POST path=/api/pending_closures5336worker2 # [ 75.770537] nix-grpc-daemon[1086]: time=2026-09-20T10:51:42.216Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5337worker2 # [ 75.774436] nix-grpc-daemon[1086]: time=2026-09-20T10:51:42.218Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5338worker2 # [ 75.776790] nix-grpc-daemon[1086]: time=2026-09-20T10:51:42.218Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5339worker2 # [ 75.792307] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:42Z level=info event=scheduler_take_over5340lb # [ 75.848626] niks3-server[1018]: 2026/09/20 10:51:42 INFO lead: acquired remote=192.168.1.4:394465341lb # [ 75.881383] niks3-server[1018]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5342worker2 # [ 75.825539] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:42Z level=info event=scheduler_connected addr=lb:500515343lb # [ 75.886922] niks3-server[1018]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5344worker2 # [ 75.830179] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05345lb # [ 75.890730] niks3-server[1018]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5346lb # [ 75.893563] niks3-server[1018]: 2026/09/20 10:51:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5347lb # [ 75.895453] niks3-server[1018]: 2026/09/20 10:51:42 INFO Signed narinfos id=25 count=15348lb # [ 75.897767] niks3-server[1018]: 2026/09/20 10:51:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5349lb # [ 75.900146] niks3-server[1018]: 2026/09/20 10:51:42 INFO Signed narinfos id=26 count=25350worker2 # [ 75.845564] nix-grpc-daemon[1086]: time=2026-09-20T10:51:42.290Z level=INFO msg="Uploading 3 narinfos"5351lb # [ 75.903804] niks3-server[1018]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5352lb # [ 75.914818] niks3-server[1018]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5353lb # [ 75.919424] niks3-server[1018]: 2026/09/20 10:51:42 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5354lb # [ 75.922782] niks3-server[1018]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5355lb # [ 75.927179] niks3-server[1018]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5356lb # [ 75.937226] niks3-server[1018]: 2026/09/20 10:51:42 INFO Completed upload id=255357lb # [ 75.938435] niks3-server[1018]: 2026/09/20 10:51:42 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5358lb # [ 75.941578] niks3-server[1018]: 2026/09/20 10:51:42 INFO Completed upload id=265359worker2 # [ 75.885459] nix-grpc-daemon[1086]: time=2026-09-20T10:51:42.331Z level=INFO msg="Upload complete. (327ms)"5360worker2 # [ 75.889980] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45574 duration_s=0 paths=2 nar_bytes_in=3125361worker2 # [ 75.902237] nix-daemon[841]: reaped child process 1084, status = succeeded5362client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.70 seconds)5363worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5364worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5365worker1 # [ 76.041810] systemd[1]: Listening on Nix worker-protocol over gRPC.5366worker1 # [ 76.048793] systemd[1]: Starting Nix worker-protocol over gRPC...5367worker1 # [ 76.216177] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5368worker1 # [ 76.220076] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15369worker1 # [ 76.226498] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5370worker1 # [ 76.231206] systemd[1]: Started Nix worker-protocol over gRPC.5371lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5372lb: must succeed: curl -sf localhost:9901/clusters5373worker1 # [ 76.264630] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500515374worker2 # [ 76.228512] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05375lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5376lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5377(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.60 seconds)5378subtest: scheduler down: the other node takes the lock and keeps it5379lb: must succeed: curl -sf localhost:9901/clusters5380lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5381lb: must succeed: curl -sf localhost:9901/clusters5382lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5383lb: must succeed: curl -sf localhost:9901/clusters5384lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5385lb: must succeed: curl -sf localhost:9901/clusters5386lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5387worker2 # [ 76.548574] systemd[1]: Stopping Nix worker-protocol over gRPC...5388worker2 # [ 76.784658] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=signal_exit5389worker2 # [ 76.996750] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=scheduler_yield5390lb # [ 77.054748] niks3-server[1018]: 2026/09/20 10:51:43 INFO lead: released remote=192.168.1.4:394465391worker1 # [ 77.042973] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:44Z level=info event=scheduler_restarting addr=lb:500515392worker2 # [ 77.005383] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455885393worker2 # [ 77.011304] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=worker_gone addr=192.168.1.3:500515394worker2 # [ 77.014472] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=scheduler_restarting addr=lb:500515395worker2 # [ 77.017426] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455885396worker2 # [ 77.020690] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=worker_gone addr=192.168.1.4:500515397worker1 # [ 77.244589] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500515398worker1 # [ 77.250684] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5399worker2 # [ 77.211171] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500515400worker2 # [ 77.215154] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5401lb # [ 77.281572] envoy[908]: conn peer=[2001:db8:1::4]:43994 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10832 tx=2106 ms=522245402lb # [ 77.285608] envoy[908]: conn peer=[2001:db8:1::1]:59822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=165403lb # [ 77.289585] envoy[908]: conn peer=[2001:db8:1::3]:60802 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12965 tx=2920 ms=552575404lb # [ 77.293689] envoy[908]: conn peer=[2001:db8:1::1]:34208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=305 ms=14595405worker1 # [ 77.450786] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500515406worker1 # [ 77.454750] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker2 # [ 77.417418] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=scheduler_connected addr=lb:500515408worker2 # [ 77.420528] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5409worker1 # [ 77.655598] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500515410worker2 # [ 77.618550] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500515411worker1 # [ 77.659615] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker2 # [ 77.621713] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5413worker1 # [ 77.860071] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500515414worker2 # [ 77.819487] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:44Z level=info event=scheduler_connected addr=lb:500515415worker2 # [ 77.823540] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker1 # [ 77.863748] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5417worker2 # [ 78.000806] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:44Z level=info event=scheduler_restarting5418worker2 # [ 78.013580] nix-daemon[841]: reaped child process 1079, status = succeeded5419worker1 # [ 78.064502] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500515420worker2 # [ 78.029119] nix-grpc-daemon[1030]: ts=2026-09-20T10:51:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5421worker1 # [ 78.068391] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5422worker1 # [ 78.270339] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500515423worker1 # [ 78.273452] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5424worker1 # [ 78.471418] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500515425worker1 # [ 78.475294] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5426worker1 # [ 78.675894] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_connected addr=lb:500515427worker1 # [ 78.679813] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5428worker2 # [ 78.812855] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5429worker2 # [ 78.817847] systemd[1]: Stopped Nix worker-protocol over gRPC.5430worker2 # [ 78.819968] systemd[1]: nix-grpc-daemon.service: Consumed 347ms CPU time over 8.202s wall clock time, 32.4M memory peak, 21.2K incoming IP traffic, 31.1K outgoing IP traffic.5431worker2 # [ 78.830936] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5432worker2 # [ 78.835262] systemd[1]: Closed Nix worker-protocol over gRPC.5433worker2 # [ 78.838940] systemd[1]: nix-grpc-daemon.socket: Consumed 286us CPU time over 8.297s wall clock time, 256K memory peak, 19.5K incoming IP traffic, 15K outgoing IP traffic.5434worker1 # [ 78.880716] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500515435worker1 # [ 78.884684] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5436worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5437worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5438worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5439worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5440worker1 # [ 79.085382] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500515441worker1 # [ 79.089113] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442worker1 # [ 79.289844] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500515443worker1 # [ 79.291936] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5444worker1 # [ 79.492426] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500515445worker1 # [ 79.494324] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5446worker1 # [ 79.695254] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_connected addr=lb:500515447worker1 # [ 79.699315] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5448worker1 # [ 79.900265] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500515449worker1 # [ 79.904548] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5451worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5452worker1 # [ 80.104934] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500515453worker1 # [ 80.106693] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5454worker1 # [ 80.307622] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500515455worker1 # [ 80.309245] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5456worker1 # [ 80.509904] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500515457worker1 # [ 80.512089] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5458worker1 # [ 80.712612] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_connected addr=lb:500515459worker1 # [ 80.714403] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5460worker1 # [ 80.915384] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500515461worker1 # [ 80.919228] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5462worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5463worker1 # [ 81.119889] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500515464worker1 # [ 81.123575] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5465worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5466worker1 # [ 81.240876] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:48Z level=info event=scheduler_take_over5467lb # [ 81.257613] niks3-server[1018]: 2026/09/20 10:51:47 INFO lead: acquired remote=192.168.1.3:409045468worker1 # [ 81.324492] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500515469worker1 # [ 81.331491] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05470worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5471worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5472client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25473worker1 # [ 82.426651] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569945474worker1 # [ 82.437786] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5475client # don't know how to build these paths:5476client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5477worker1 # [ 82.452355] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569945478worker1 # [ 82.463226] nix-grpc-daemon[1251]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)5479worker1 # [ 82.564088] nix-grpc-daemon[1251]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 157 ms (attempt 2/5)5480worker1 # [ 82.726884] nix-grpc-daemon[1251]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 3/5)5481worker1 # [ 82.738738] nix-grpc-daemon[1251]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 622 ms (attempt 4/5)5482worker1 # [ 83.366868] nix-grpc-daemon[1251]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5483worker1 # [ 83.384460] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:50Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05484worker1 # [ 83.386928] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:50Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15485worker1 # [ 83.429041] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569945486worker1 # [ 83.437847] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569945487client # copying 1 paths to 'lb:50051'...5488worker1 # [ 83.446821] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569945489client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5490worker1 # [ 83.453892] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5491worker1 # [ 83.464756] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:50Z level=info event=niks3_push_started pid=13275492worker1 # [ 83.812574] nix-daemon[845]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5493worker1 # [ 83.823481] nix-daemon[845]: reaped child process 1343, status = succeeded5494worker1 # [ 83.993176] nix-daemon[845]: accepted connection from pid 1345, user nix-grpc-daemon (trusted)5495worker1 # [ 84.004885] nix-daemon[845]: reaped child process 1348, status = succeeded5496lb # [ 84.037592] niks3-server[1018]: 2026/09/20 10:51:50 INFO Received uploads request method=POST path=/api/pending_closures5497worker1 # [ 84.029559] nix-grpc-daemon[1327]: time=2026-09-20T10:51:51.184Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5498worker1 # [ 84.032829] nix-grpc-daemon[1327]: time=2026-09-20T10:51:51.185Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5499lb # [ 84.074073] niks3-server[1018]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5500lb # [ 84.084343] niks3-server[1018]: 2026/09/20 10:51:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5501lb # [ 84.089392] niks3-server[1018]: 2026/09/20 10:51:50 INFO Signed narinfos id=27 count=15502worker1 # [ 84.076464] nix-grpc-daemon[1327]: time=2026-09-20T10:51:51.230Z level=INFO msg="Uploading 1 narinfos"5503lb # [ 84.103149] niks3-server[1018]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5504lb # [ 84.111776] niks3-server[1018]: 2026/09/20 10:51:50 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5505worker1 # [ 84.100314] nix-grpc-daemon[1327]: time=2026-09-20T10:51:51.255Z level=INFO msg="Upload complete. (447ms)"5506lb # [ 84.117142] niks3-server[1018]: 2026/09/20 10:51:50 INFO Completed upload id=275507worker1 # [ 84.102601] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56994 duration_s=0 paths=1 nar_bytes_in=5205508lb # [ 84.120083] niks3-server[1018]: 2026/09/20 10:51:50 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5509worker1 # [ 84.106407] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569945510worker1 # [ 84.113094] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5511worker1 # [ 84.120098] nix-daemon[845]: reaped child process 1325, status = succeeded5512worker1 # [ 84.123322] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5513client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5514client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5515client # farm-a> LOG-farm-a-on-standby5516worker1 # [ 85.537439] nix-daemon[845]: reaped child process 1352, status = succeeded5517worker1 # [ 85.670309] nix-daemon[845]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5518worker1 # [ 85.682134] nix-daemon[845]: reaped child process 1370, status = succeeded5519lb # [ 85.713688] niks3-server[1018]: 2026/09/20 10:51:52 INFO Received uploads request method=POST path=/api/pending_closures5520worker1 # [ 85.705155] nix-grpc-daemon[1327]: time=2026-09-20T10:51:52.859Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5521worker1 # [ 85.706841] nix-grpc-daemon[1327]: time=2026-09-20T10:51:52.861Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5522lb # [ 85.746400] niks3-server[1018]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5523lb # [ 85.758454] niks3-server[1018]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5524lb # [ 85.770154] niks3-server[1018]: 2026/09/20 10:51:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5525worker1 # [ 85.759403] nix-grpc-daemon[1327]: time=2026-09-20T10:51:52.914Z level=INFO msg="Uploading 1 narinfos"5526lb # [ 85.775413] niks3-server[1018]: 2026/09/20 10:51:52 INFO Signed narinfos id=28 count=15527lb # [ 85.780386] niks3-server[1018]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5528lb # [ 85.790477] niks3-server[1018]: 2026/09/20 10:51:52 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5529lb # [ 85.795426] niks3-server[1018]: 2026/09/20 10:51:52 INFO Completed upload id=285530worker1 # [ 85.780775] nix-grpc-daemon[1327]: time=2026-09-20T10:51:52.935Z level=INFO msg="Upload complete. (249ms)"5531lb # [ 85.798391] niks3-server[1018]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5532worker1 # [ 85.788681] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56994 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15533worker1 # [ 85.798809] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:52Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05534worker1 # [ 85.802197] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:52Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15535worker1 # [ 85.809975] nix-daemon[845]: reaped child process 1350, status = succeeded5536worker1 # [ 85.842497] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569945537worker1 # [ 85.854641] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569945538worker1 # [ 85.867170] nix-grpc-daemon[1327]: time=2026-09-20T10:51:53.021Z level=INFO msg="All 1 paths already cached"5539client # copying 1 paths to 'lb:50051'...5540client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5541worker1 # [ 85.875062] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569945542worker1 # [ 85.885103] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5543worker1 # [ 86.052650] nix-daemon[845]: accepted connection from pid 1376, user nix-grpc-daemon (trusted)5544worker1 # [ 86.064824] nix-daemon[845]: reaped child process 1379, status = succeeded5545worker1 # [ 86.240905] nix-daemon[845]: accepted connection from pid 1381, user nix-grpc-daemon (trusted)5546worker1 # [ 86.252902] nix-daemon[845]: reaped child process 1384, status = succeeded5547lb # [ 86.285905] niks3-server[1018]: 2026/09/20 10:51:52 INFO Received uploads request method=POST path=/api/pending_closures5548worker1 # [ 86.277313] nix-grpc-daemon[1327]: time=2026-09-20T10:51:53.432Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5549worker1 # [ 86.278791] nix-grpc-daemon[1327]: time=2026-09-20T10:51:53.433Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5550lb # [ 86.313057] niks3-server[1018]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5551lb # [ 86.322295] niks3-server[1018]: 2026/09/20 10:51:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5552lb # [ 86.326482] niks3-server[1018]: 2026/09/20 10:51:52 INFO Signed narinfos id=29 count=15553worker1 # [ 86.313335] nix-grpc-daemon[1327]: time=2026-09-20T10:51:53.467Z level=INFO msg="Uploading 1 narinfos"5554lb # [ 86.337335] niks3-server[1018]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5555lb # [ 86.346152] niks3-server[1018]: 2026/09/20 10:51:52 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5556lb # [ 86.350961] niks3-server[1018]: 2026/09/20 10:51:52 INFO Completed upload id=295557worker1 # [ 86.336812] nix-grpc-daemon[1327]: time=2026-09-20T10:51:53.491Z level=INFO msg="Upload complete. (442ms)"5558lb # [ 86.354807] niks3-server[1018]: 2026/09/20 10:51:52 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5559worker1 # [ 86.339613] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56994 duration_s=0 paths=1 nar_bytes_in=6405560worker1 # [ 86.344830] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569945561worker1 # [ 86.355033] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5562worker1 # [ 86.358401] nix-daemon[845]: reaped child process 1374, status = succeeded5563worker1 # [ 86.363502] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5564client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5565client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5566lb # [ 87.282340] envoy[908]: conn peer=[2001:db8:1::4]:45804 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5259 tx=1045 ms=72205567client # farm-b> LOG-farm-b-on-standby5568worker1 # [ 87.590645] nix-daemon[845]: reaped child process 1388, status = succeeded5569worker1 # [ 87.748367] nix-daemon[845]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)5570worker1 # [ 87.760174] nix-daemon[845]: reaped child process 1406, status = succeeded5571lb # [ 87.791670] niks3-server[1018]: 2026/09/20 10:51:54 INFO Received uploads request method=POST path=/api/pending_closures5572worker1 # [ 87.782840] nix-grpc-daemon[1327]: time=2026-09-20T10:51:54.937Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5573worker1 # [ 87.784299] nix-grpc-daemon[1327]: time=2026-09-20T10:51:54.939Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5574lb # [ 87.820282] niks3-server[1018]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5575lb # [ 87.834634] niks3-server[1018]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5576lb # [ 87.847106] niks3-server[1018]: 2026/09/20 10:51:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5577worker1 # [ 87.834829] nix-grpc-daemon[1327]: time=2026-09-20T10:51:54.989Z level=INFO msg="Uploading 1 narinfos"5578lb # [ 87.851093] niks3-server[1018]: 2026/09/20 10:51:54 INFO Signed narinfos id=30 count=15579lb # [ 87.856222] niks3-server[1018]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5580lb # [ 87.865092] niks3-server[1018]: 2026/09/20 10:51:54 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5581lb # [ 87.869639] niks3-server[1018]: 2026/09/20 10:51:54 INFO Completed upload id=305582worker1 # [ 87.853423] nix-grpc-daemon[1327]: time=2026-09-20T10:51:55.008Z level=INFO msg="Upload complete. (282ms)"5583lb # [ 87.871965] niks3-server[1018]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5584worker1 # [ 87.859524] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56994 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15585worker1 # [ 87.870477] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:55Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05586worker1 # [ 87.875120] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:55Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15587worker1 # [ 87.883293] nix-daemon[845]: reaped child process 1386, status = succeeded5588worker1 # [ 87.914080] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569945589worker1 # [ 87.926256] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569945590worker1 # [ 87.937616] nix-grpc-daemon[1327]: time=2026-09-20T10:51:55.092Z level=INFO msg="All 2 paths already cached"5591client # copying 1 paths to 'lb:50051'...5592client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5593worker1 # [ 87.945617] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569945594worker1 # [ 87.953874] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5595worker1 # [ 88.099655] nix-daemon[845]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5596worker1 # [ 88.111797] nix-daemon[845]: reaped child process 1415, status = succeeded5597worker1 # [ 88.251929] nix-daemon[845]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)5598worker1 # [ 88.263921] nix-daemon[845]: reaped child process 1420, status = succeeded5599lb # [ 88.294437] niks3-server[1018]: 2026/09/20 10:51:54 INFO Received uploads request method=POST path=/api/pending_closures5600worker1 # [ 88.290530] nix-grpc-daemon[1327]: time=2026-09-20T10:51:55.445Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5601worker1 # [ 88.292427] nix-grpc-daemon[1327]: time=2026-09-20T10:51:55.446Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5602lb # [ 88.328477] niks3-server[1018]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5603lb # [ 88.335678] niks3-server[1018]: 2026/09/20 10:51:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5604lb # [ 88.340109] niks3-server[1018]: 2026/09/20 10:51:54 INFO Signed narinfos id=31 count=15605worker1 # [ 88.326958] nix-grpc-daemon[1327]: time=2026-09-20T10:51:55.481Z level=INFO msg="Uploading 1 narinfos"5606lb # [ 88.351538] niks3-server[1018]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5607lb # [ 88.359213] niks3-server[1018]: 2026/09/20 10:51:54 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5608lb # [ 88.364952] niks3-server[1018]: 2026/09/20 10:51:54 INFO Completed upload id=315609worker1 # [ 88.348816] nix-grpc-daemon[1327]: time=2026-09-20T10:51:55.503Z level=INFO msg="Upload complete. (382ms)"5610lb # [ 88.368204] niks3-server[1018]: 2026/09/20 10:51:54 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5611worker1 # [ 88.352615] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56994 duration_s=0 paths=1 nar_bytes_in=7685612worker1 # [ 88.358196] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569945613worker1 # [ 88.368174] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5614worker1 # [ 88.371097] nix-daemon[845]: reaped child process 1410, status = succeeded5615worker1 # [ 88.376137] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5616client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5617client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5618client # farm-top> LOG-farm-top-on-standby5619worker1 # [ 89.604623] nix-daemon[845]: reaped child process 1424, status = succeeded5620worker1 # [ 89.745090] nix-daemon[845]: accepted connection from pid 1439, user nix-grpc-daemon (trusted)5621worker1 # [ 89.756608] nix-daemon[845]: reaped child process 1442, status = succeeded5622lb # [ 89.782922] niks3-server[1018]: 2026/09/20 10:51:56 INFO Received uploads request method=POST path=/api/pending_closures5623worker1 # [ 89.770851] nix-grpc-daemon[1327]: time=2026-09-20T10:51:56.925Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5624worker1 # [ 89.772432] nix-grpc-daemon[1327]: time=2026-09-20T10:51:56.927Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5625lb # [ 89.805382] niks3-server[1018]: 2026/09/20 10:51:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5626lb # [ 89.807555] niks3-server[1018]: 2026/09/20 10:51:56 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5627lb # [ 89.811380] niks3-server[1018]: 2026/09/20 10:51:56 INFO Signed narinfos id=32 count=15628worker1 # [ 89.795230] nix-grpc-daemon[1327]: time=2026-09-20T10:51:56.949Z level=INFO msg="Uploading 1 narinfos"5629lb # [ 89.812431] niks3-server[1018]: 2026/09/20 10:51:56 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5630lb # [ 89.821593] niks3-server[1018]: 2026/09/20 10:51:56 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5631lb # [ 89.825411] niks3-server[1018]: 2026/09/20 10:51:56 INFO Completed upload id=325632worker1 # [ 89.810558] nix-grpc-daemon[1327]: time=2026-09-20T10:51:56.965Z level=INFO msg="Upload complete. (225ms)"5633lb # [ 89.827671] niks3-server[1018]: 2026/09/20 10:51:56 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5634worker1 # [ 89.813132] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56994 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15635lb # [ 89.833783] niks3-server[1018]: 2026/09/20 10:51:56 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5636worker1 # [ 89.820883] nix-daemon[845]: reaped child process 1422, status = succeeded5637client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.67 seconds)5638client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5639client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5640worker2 # [ 90.109987] systemd[1]: Listening on Nix worker-protocol over gRPC.5641worker2 # [ 90.118904] systemd[1]: Starting Nix worker-protocol over gRPC...5642worker2 # [ 90.281446] nix-grpc-daemon[1123]: ts=2026-09-20T10:51:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5643worker2 # [ 90.285394] nix-grpc-daemon[1123]: ts=2026-09-20T10:51:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15644worker2 # [ 90.291512] nix-grpc-daemon[1123]: ts=2026-09-20T10:51:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5645worker2 # [ 90.295680] systemd[1]: Started Nix worker-protocol over gRPC.5646lb: must succeed: curl -sf localhost:9901/clusters5647worker2 # [ 90.324093] nix-grpc-daemon[1123]: ts=2026-09-20T10:51:56Z level=info event=scheduler_connected addr=lb:500515648worker1 # [ 90.367285] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05649lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5650worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5651worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5652lb: must succeed: curl -sf localhost:9901/clusters5653lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5654lb: must succeed: curl -sf localhost:9901/clusters5655lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5656worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5657worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5658(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.30 seconds)5659subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5660client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5661client # [ 90.665251] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5662client # Running as unit: rs.service; invocation ID: bcfee0f660fd48b6bab5958f5a3298825663client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5664worker1 # [ 90.890531] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569945665client # [ 90.885325] nix[1180]: don't know how to build these paths:5666client # [ 90.886307] nix[1180]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5667worker1 # [ 90.904249] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:58Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05668worker1 # [ 90.909502] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:58Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15669worker1 # [ 90.948169] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569945670worker1 # [ 90.960137] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569945671client # [ 90.959134] nix[1180]: copying 1 paths to 'lb:50051'...5672client # [ 90.960712] nix[1180]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5673worker1 # [ 90.971445] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569945674worker1 # [ 90.983261] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5675worker1 # [ 91.146177] nix-daemon[845]: accepted connection from pid 1455, user nix-grpc-daemon (trusted)5676worker1 # [ 91.158422] nix-daemon[845]: reaped child process 1458, status = succeeded5677worker1 # [ 91.313813] nix-daemon[845]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)5678worker1 # [ 91.326614] nix-daemon[845]: reaped child process 1463, status = succeeded5679lb # [ 91.359770] niks3-server[1018]: 2026/09/20 10:51:57 INFO Received uploads request method=POST path=/api/pending_closures5680worker1 # [ 91.353501] nix-grpc-daemon[1327]: time=2026-09-20T10:51:58.508Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5681worker1 # [ 91.354999] nix-grpc-daemon[1327]: time=2026-09-20T10:51:58.509Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5682lb # [ 91.395230] niks3-server[1018]: 2026/09/20 10:51:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5683lb # [ 91.400505] niks3-server[1018]: 2026/09/20 10:51:57 INFO Signed narinfos id=33 count=15684worker1 # [ 91.387420] nix-grpc-daemon[1327]: time=2026-09-20T10:51:58.542Z level=INFO msg="Uploading 1 narinfos"5685lb # [ 91.415058] niks3-server[1018]: 2026/09/20 10:51:57 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5686lb # [ 91.419295] niks3-server[1018]: 2026/09/20 10:51:57 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5687lb # [ 91.428290] niks3-server[1018]: 2026/09/20 10:51:57 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5688worker1 # [ 91.415710] nix-grpc-daemon[1327]: time=2026-09-20T10:51:58.570Z level=INFO msg="Upload complete. (421ms)"5689lb # [ 91.431964] niks3-server[1018]: 2026/09/20 10:51:57 INFO Completed upload id=335690worker1 # [ 91.418681] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56994 duration_s=0 paths=1 nar_bytes_in=4165691lb # [ 91.436118] niks3-server[1018]: 2026/09/20 10:51:57 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5692worker1 # [ 91.424122] nix-grpc-daemon[1251]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569945693worker1 # [ 91.432625] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5694worker1 # [ 91.437283] nix-daemon[845]: reaped child process 1453, status = succeeded5695worker1 # [ 91.442422] nix-daemon[845]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5696client # [ 91.430668] nix[1180]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5697lb: must succeed: curl -sf localhost:9901/clusters5698lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5699lb: must succeed: curl -sf localhost:9901/clusters5700lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5701lb: must succeed: curl -sf localhost:9901/clusters5702lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5703lb: must succeed: curl -sf localhost:9901/clusters5704lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5705worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5706worker2 # [ 93.506686] systemd-journald[287]: Received client request to rotate journal, rotating.5707worker2 # [ 93.524304] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/23c1268bc835417fa8ce6c95d207b324.5708worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5709worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5710worker1 # [ 93.480693] systemd[1]: Stopping Nix worker-protocol over gRPC...5711worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5712??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5713 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735714client: must succeed: sleep 5.05715worker1 # [ 94.233559] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:01Z level=info event=draining5716lb # [ 97.281810] envoy[908]: conn peer=[2001:db8:1::1]:34214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9577 tx=3841 ms=74095717client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5718worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5719worker1 # [ 98.862711] nix-daemon[845]: reaped child process 1467, status = succeeded5720worker1 # [ 98.993625] nix-daemon[845]: accepted connection from pid 1507, user nix-grpc-daemon (trusted)5721worker1 # [ 99.005262] nix-daemon[845]: reaped child process 1510, status = succeeded5722lb # [ 99.033412] niks3-server[1018]: 2026/09/20 10:52:05 INFO Received uploads request method=POST path=/api/pending_closures5723worker1 # [ 99.026324] nix-grpc-daemon[1327]: time=2026-09-20T10:52:06.180Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5724worker1 # [ 99.027813] nix-grpc-daemon[1327]: time=2026-09-20T10:52:06.182Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5725lb # [ 99.073763] niks3-server[1018]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5726lb # [ 99.084230] niks3-server[1018]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5727lb # [ 99.092615] niks3-server[1018]: 2026/09/20 10:52:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5728lb # [ 99.099316] niks3-server[1018]: 2026/09/20 10:52:05 INFO Signed narinfos id=34 count=15729worker1 # [ 99.085747] nix-grpc-daemon[1327]: time=2026-09-20T10:52:06.240Z level=INFO msg="Uploading 1 narinfos"5730lb # [ 99.102163] niks3-server[1018]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5731lb # [ 99.111944] niks3-server[1018]: 2026/09/20 10:52:05 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5732lb # [ 99.118706] niks3-server[1018]: 2026/09/20 10:52:05 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5733lb # [ 99.121488] niks3-server[1018]: 2026/09/20 10:52:05 INFO Completed upload id=345734worker1 # [ 99.105431] nix-grpc-daemon[1327]: time=2026-09-20T10:52:06.260Z level=INFO msg="Upload complete. (250ms)"5735worker1 # [ 99.113700] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56994 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15736worker1 # [ 99.124341] nix-daemon[845]: reaped child process 1465, status = succeeded5737client # [ 99.134474] systemd[1]: rs.service: Deactivated successfully.5738client # [ 99.138202] systemd[1]: rs.service: Consumed 120ms CPU time over 8.472s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 8.9K outgoing IP traffic.5739worker1 # [ 99.236843] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=signal_exit5740worker1 # [ 99.458348] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=scheduler_yield5741lb # [ 99.477546] niks3-server[1018]: 2026/09/20 10:52:05 INFO lead: released remote=192.168.1.3:409045742worker1 # [ 99.462313] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=scheduler_restarting addr=lb:500515743worker2 # [ 99.423561] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:05Z level=info event=scheduler_restarting addr=lb:500515744worker1 # [ 99.466944] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:569805745worker1 # [ 99.470501] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=worker_gone addr=192.168.1.4:500515746worker1 # [ 99.473420] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:569805747worker1 # [ 99.476668] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=worker_gone addr=192.168.1.3:500515748worker2 # [ 99.623506] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515749worker1 # [ 99.663526] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515750worker1 # [ 99.667643] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5751worker2 # [ 99.627573] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5752worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.08 seconds)5753client: waiting for success: ! systemctl is-active rs5754worker2 # [ 99.827884] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515755worker1 # [ 99.868556] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500515756worker2 # [ 99.831499] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5757worker1 # [ 99.872592] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5758client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5759client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5760client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5761client: must fail: journalctl -u rs | grep -q 'reconnecting'5762client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5763lb: must succeed: curl -sf localhost:9901/clusters5764lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5765worker2 # [ 100.032464] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515766worker1 # [ 100.073173] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500515767worker2 # [ 100.036689] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5768worker1 # [ 100.077064] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5769worker2 # [ 100.237346] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515770worker1 # [ 100.278587] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500515771worker2 # [ 100.241336] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5772worker1 # [ 100.282297] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5773worker2 # [ 100.301574] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_take_over5774lb # [ 100.358519] niks3-server[1018]: 2026/09/20 10:52:06 INFO lead: acquired remote=192.168.1.4:578485775worker1 # [ 100.459171] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:07Z level=info event=scheduler_restarting5776worker1 # [ 100.480596] nix-daemon[845]: reaped child process 1318, status = succeeded5777worker2 # [ 100.443518] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_connected addr=lb:500515778worker2 # [ 100.446776] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5779worker1 # [ 100.494975] nix-grpc-daemon[1251]: ts=2026-09-20T10:52:07Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5780worker2 # [ 100.644098] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500515781worker2 # [ 100.645913] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5782worker2 # [ 100.846799] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500515783worker2 # [ 100.850711] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5784lb: must succeed: curl -sf localhost:9901/clusters5785worker2 # [ 101.051667] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:07Z level=info event=scheduler_connected addr=lb:500515786lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5787worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5788worker2 # [ 101.059555] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05789worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5790worker1 # [ 101.288889] nix-grpc-daemon[1251]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5791worker1 # [ 101.292892] nix-grpc-daemon[1251]: E0000 00:00:1789901528.442810 1294 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5792worker1 # [ 101.297568] nix-grpc-daemon[1251]: E0000 00:00:1789901528.447317 1294 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5793lb: must succeed: curl -sf localhost:9901/clusters5794lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5795worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5796worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5797worker1 # [ 102.515091] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5798worker1 # [ 102.517065] systemd[1]: Stopped Nix worker-protocol over gRPC.5799worker1 # [ 102.517958] systemd[1]: nix-grpc-daemon.service: Consumed 1.275s CPU time over 26.468s wall clock time, 35M memory peak, 49K incoming IP traffic, 95K outgoing IP traffic.5800worker1 # [ 102.525631] systemd[1]: Starting Nix worker-protocol over gRPC...5801worker1 # [ 102.683716] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5802worker1 # [ 102.687682] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15803worker1 # [ 102.693623] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5804worker1 # [ 102.697378] systemd[1]: Started Nix worker-protocol over gRPC.5805worker1 # [ 102.719556] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:09Z level=info event=scheduler_connected addr=lb:500515806worker2 # [ 102.682452] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05807lb: must succeed: curl -sf localhost:9901/clusters5808lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5809worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5810worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5811(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.76 seconds)5812subtest: scheduler restart mid-build: build finishes, no double build5813client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5814client # [ 103.423492] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5815client # Running as unit: mid.service; invocation ID: 0e8965c30bed4b12a4e4356fd4f09dbc5816client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5817worker2 # [ 103.612896] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:597265818worker2 # [ 103.618191] nix-daemon[841]: accepted connection from pid 1123, user nix-grpc-daemon (trusted)5819client # [ 103.652733] nix[1216]: don't know how to build these paths:5820client # [ 103.654378] nix[1216]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5821worker2 # [ 103.631458] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:10Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05822worker2 # [ 103.633226] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:10Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15823worker2 # [ 103.678676] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597265824worker2 # [ 103.684938] nix-grpc-daemon[1123]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 1/5)5825worker2 # [ 103.714409] nix-grpc-daemon[1123]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 2/5)5826worker2 # [ 103.757441] nix-grpc-daemon[1123]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 225 ms (attempt 3/5)5827worker2 # [ 103.985203] nix-grpc-daemon[1123]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 103 ms (attempt 4/5)5828worker2 # [ 104.094426] nix-grpc-daemon[1123]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5829worker2 # [ 104.111241] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:597265830client # [ 104.150149] nix[1216]: copying 1 paths to 'lb:50051'...5831client # [ 104.153192] nix[1216]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5832worker2 # [ 104.125191] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:597265833worker2 # [ 104.133505] nix-daemon[841]: accepted connection from pid 1123, user nix-grpc-daemon (trusted)5834worker2 # [ 104.153481] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:10Z level=info event=niks3_push_started pid=12385835worker2 # [ 104.474747] nix-daemon[841]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5836worker2 # [ 104.486826] nix-daemon[841]: reaped child process 1254, status = succeeded5837worker2 # [ 104.653834] nix-daemon[841]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5838worker2 # [ 104.669215] nix-daemon[841]: reaped child process 1263, status = succeeded5839lb # [ 104.741652] niks3-server[1018]: 2026/09/20 10:52:11 INFO Received uploads request method=POST path=/api/pending_closures5840worker2 # [ 104.696109] nix-grpc-daemon[1238]: time=2026-09-20T10:52:11.141Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5841worker2 # [ 104.697512] nix-grpc-daemon[1238]: time=2026-09-20T10:52:11.141Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5842lb # [ 104.769418] niks3-server[1018]: 2026/09/20 10:52:11 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5843lb # [ 104.773848] niks3-server[1018]: 2026/09/20 10:52:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5844lb # [ 104.776054] niks3-server[1018]: 2026/09/20 10:52:11 INFO Signed narinfos id=35 count=15845worker2 # [ 104.721053] nix-grpc-daemon[1238]: time=2026-09-20T10:52:11.166Z level=INFO msg="Uploading 1 narinfos"5846lb # [ 104.783065] niks3-server[1018]: 2026/09/20 10:52:11 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5847lb # [ 104.786313] niks3-server[1018]: 2026/09/20 10:52:11 INFO Completed upload id=355848worker2 # [ 104.729953] nix-grpc-daemon[1238]: time=2026-09-20T10:52:11.175Z level=INFO msg="Upload complete. (402ms)"5849lb # [ 104.788283] niks3-server[1018]: 2026/09/20 10:52:11 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5850worker2 # [ 104.732332] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59726 duration_s=0 paths=1 nar_bytes_in=4245851worker2 # [ 104.737508] nix-grpc-daemon[1123]: ts=2026-09-20T10:52:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597265852worker2 # [ 104.739520] nix-grpc-daemon[1238]: time=2026-09-20T10:52:11.185Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"5853worker2 # [ 104.745101] nix-daemon[841]: accepted connection from pid 1123, user nix-grpc-daemon (trusted)5854worker2 # [ 104.747436] nix-daemon[841]: reaped child process 1236, status = succeeded5855worker2 # [ 104.752477] nix-daemon[841]: accepted connection from pid 1123, user nix-grpc-daemon (trusted)5856client # [ 104.781292] nix[1216]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5857lb: must succeed: curl -sf localhost:9901/clusters5858lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5859lb: must succeed: curl -sf localhost:9901/clusters5860lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5861worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5862worker2 # [ 106.056322] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1123 (nix-grpc-daemon) on client request.5863worker2 # [ 106.058981] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5864lb # [ 106.120397] niks3-server[1018]: 2026/09/20 10:52:12 INFO lead: released remote=192.168.1.4:578485865client # [ 106.099393] nix[1216]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5866worker1 # [ 106.109566] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:13Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5867worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5868client # [ 106.103496] nix[1216]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5869client: must succeed: sleep 5.05870worker2 # [ 106.088912] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5871worker2 # [ 106.091346] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5872worker2 # [ 106.092801] systemd[1]: nix-grpc-daemon.service: Consumed 424ms CPU time over 15.969s wall clock time, 33M memory peak, 15.2K incoming IP traffic, 24K outgoing IP traffic.5873worker2 # [ 106.096440] nix-daemon[841]: reaped child process 1265, status = succeeded5874worker2 # [ 106.097386] nix-daemon[841]: reaped child process 1229, status = succeeded5875worker2 # [ 106.103251] nix-daemon[1267]: unexpected Nix daemon error: error: interrupted by the user5876worker2 # [ 106.107607] nix-daemon[841]: reaped child process 1267, status = failed with exit code 15877worker2 # [ 106.218419] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5878worker2 # [ 106.222360] systemd[1]: Starting Nix worker-protocol over gRPC...5879worker1 # [ 106.310100] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:13Z level=info event=scheduler_connected addr=lb:500515880worker2 # [ 106.384765] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5881worker2 # [ 106.386967] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15882worker2 # [ 106.393061] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5883worker2 # [ 106.397201] systemd[1]: Started Nix worker-protocol over gRPC.5884lb # [ 106.461766] niks3-server[1018]: 2026/09/20 10:52:12 INFO lead: acquired remote=192.168.1.4:547405885worker1 # [ 106.444715] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5886worker2 # [ 106.408535] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:12Z level=info event=scheduler_take_over5887worker2 # [ 106.423479] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:12Z level=info event=scheduler_connected addr=lb:500515888worker2 # [ 106.425247] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05889worker2 # [ 106.579896] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:13Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05890worker2 # [ 106.585288] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:13Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15891worker1 # [ 106.645677] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:13Z level=info event=scheduler_connected addr=lb:500515892worker2 # [ 106.606819] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05893worker2 # [ 106.621986] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597445894worker2 # [ 106.629840] nix-daemon[841]: accepted connection from pid 1299, user nix-grpc-daemon (trusted)5895worker2 # [ 106.645309] nix-daemon[841]: accepted connection from pid 1299, user nix-grpc-daemon (trusted)5896client # [ 106.685656] nix[1216]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5897worker2 # [ 106.658445] nix-daemon[841]: accepted connection from pid 1299, user nix-grpc-daemon (trusted)5898lb # [ 107.282157] envoy[908]: conn peer=[2001:db8:1::1]:35102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2930 tx=1307 ms=82325899lb # [ 107.286089] envoy[908]: conn peer=[2001:db8:1::3]:45714 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=6407 tx=1509 ms=242285900lb # [ 107.289830] envoy[908]: conn peer=[2001:db8:1::4]:33780 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2142 tx=590 ms=157505901client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5902client: waiting for success: ! systemctl is-active mid5903worker2 # [ 111.479240] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:17Z level=info event=niks3_push_started pid=13785904worker2 # [ 111.507784] nix-daemon[841]: reaped child process 1354, status = succeeded5905worker2 # [ 111.830364] nix-daemon[841]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5906worker2 # [ 111.842631] nix-daemon[841]: reaped child process 1394, status = succeeded5907lb # [ 111.917384] niks3-server[1018]: 2026/09/20 10:52:18 INFO Received uploads request method=POST path=/api/pending_closures5908worker2 # [ 111.871258] nix-grpc-daemon[1378]: time=2026-09-20T10:52:18.316Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5909worker2 # [ 111.873794] nix-grpc-daemon[1378]: time=2026-09-20T10:52:18.318Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5910lb # [ 111.947235] niks3-server[1018]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5911lb # [ 111.952310] niks3-server[1018]: 2026/09/20 10:52:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5912lb # [ 111.955087] niks3-server[1018]: 2026/09/20 10:52:18 INFO Signed narinfos id=36 count=15913worker2 # [ 111.898944] nix-grpc-daemon[1378]: time=2026-09-20T10:52:18.344Z level=INFO msg="Uploading 1 narinfos"5914lb # [ 111.959364] niks3-server[1018]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5915lb # [ 111.964070] niks3-server[1018]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5916lb # [ 111.966502] niks3-server[1018]: 2026/09/20 10:52:18 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5917lb # [ 111.969100] niks3-server[1018]: 2026/09/20 10:52:18 INFO Completed upload id=365918worker2 # [ 111.913967] nix-grpc-daemon[1378]: time=2026-09-20T10:52:18.359Z level=INFO msg="Upload complete. (236ms)"5919lb # [ 111.971531] niks3-server[1018]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5920worker2 # [ 111.916308] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59744 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15921worker2 # [ 111.925102] nix-daemon[841]: reaped child process 1352, status = succeeded5922client # [ 111.993074] systemd[1]: mid.service: Deactivated successfully.5923client # [ 111.996601] systemd[1]: mid.service: Consumed 132ms CPU time over 8.572s wall clock time, 39.3M memory peak, 9.6K incoming IP traffic, 10.5K outgoing IP traffic.5924client: (finished: waiting for success: ! systemctl is-active mid, in 1.11 seconds)5925client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5926client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5927(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.17 seconds)5928subtest: interrupting the client stops the build on the worker5929client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5930client # [ 112.567792] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5931client # Running as unit: intr.service; invocation ID: e34718855fc94b46bb4dac5a35d279175932client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5933worker2 # [ 112.735397] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:597445934client # [ 112.770441] nix[1252]: don't know how to build these paths:5935client # [ 112.771874] nix[1252]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5936worker2 # [ 112.751143] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:19Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05937worker2 # [ 112.755285] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:19Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15938worker1 # [ 112.843564] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414385939worker1 # [ 112.854334] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5940worker1 # [ 112.870861] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)5941worker1 # [ 112.904695] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 2/5)5942worker1 # [ 113.094746] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 3/5)5943worker1 # [ 113.144418] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 278 ms (attempt 4/5)5944worker1 # [ 113.428329] nix-grpc-daemon[1517]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5945worker1 # [ 113.440456] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414385946client # [ 113.435855] nix[1252]: copying 1 paths to 'lb:50051'...5947worker1 # [ 113.451001] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414385948client # [ 113.439194] nix[1252]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5949worker1 # [ 113.458474] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5950worker1 # [ 113.469573] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:20Z level=info event=niks3_push_started pid=16015951worker1 # [ 113.806811] nix-daemon[845]: accepted connection from pid 1614, user nix-grpc-daemon (trusted)5952worker1 # [ 113.818507] nix-daemon[845]: reaped child process 1621, status = succeeded5953worker1 # [ 113.978684] nix-daemon[845]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)5954worker1 # [ 113.990596] nix-daemon[845]: reaped child process 1626, status = succeeded5955lb # [ 114.025767] niks3-server[1018]: 2026/09/20 10:52:20 INFO Received uploads request method=POST path=/api/pending_closures5956worker1 # [ 114.018819] nix-grpc-daemon[1601]: time=2026-09-20T10:52:21.172Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5957worker1 # [ 114.020206] nix-grpc-daemon[1601]: time=2026-09-20T10:52:21.172Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5958lb # [ 114.060993] niks3-server[1018]: 2026/09/20 10:52:20 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5959lb # [ 114.066198] niks3-server[1018]: 2026/09/20 10:52:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5960lb # [ 114.070492] niks3-server[1018]: 2026/09/20 10:52:20 INFO Signed narinfos id=37 count=15961worker1 # [ 114.057667] nix-grpc-daemon[1601]: time=2026-09-20T10:52:21.212Z level=INFO msg="Uploading 1 narinfos"5962lb # [ 114.084643] niks3-server[1018]: 2026/09/20 10:52:20 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5963lb # [ 114.088385] niks3-server[1018]: 2026/09/20 10:52:20 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5964lb # [ 114.094993] niks3-server[1018]: 2026/09/20 10:52:20 INFO Completed upload id=375965worker1 # [ 114.081070] nix-grpc-daemon[1601]: time=2026-09-20T10:52:21.235Z level=INFO msg="Upload complete. (435ms)"5966worker1 # [ 114.083591] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=4245967lb # [ 114.100294] niks3-server[1018]: 2026/09/20 10:52:20 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5968worker1 # [ 114.087968] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414385969worker1 # [ 114.094171] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5970worker1 # [ 114.096454] nix-daemon[845]: reaped child process 1599, status = succeeded5971worker1 # [ 114.101838] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)5972client # [ 114.090466] nix[1252]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5973client: must succeed: systemctl kill -s INT intr5974client # [ 116.096223] systemd[1]: intr.service: Sent signal SIGINT to main process 1252 (nix) on client request.5975worker1 # [ 116.113522] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:23Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5976client # [ 116.103170] nix[1252]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5977client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)5978client # [ 116.109740] nix[1252]: error: interrupted by the user5979client # [ 116.135485] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5980client # [ 116.136721] systemd[1]: intr.service: Failed with result 'exit-code'.5981client # [ 116.138413] systemd[1]: intr.service: Consumed 109ms CPU time over 3.567s wall clock time, 39.1M memory peak, 7.9K incoming IP traffic, 8.7K outgoing IP traffic.5982worker1 # [ 116.319426] nix-daemon[845]: reaped child process 1628, status = succeeded5983worker1 # [ 116.323241] nix-daemon[1630]: unexpected Nix daemon error: error: interrupted by the user5984worker1 # [ 116.327439] nix-daemon[845]: reaped child process 1630, status = failed with exit code 15985lb # [ 117.283275] envoy[908]: conn peer=[2001:db8:1::1]:53268 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3835 tx=1638 ms=83335986lb # [ 117.287101] envoy[908]: conn peer=[2001:db8:1::1]:53270 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=753 ms=33575987(finished: subtest: interrupting the client stops the build on the worker, in 4.77 seconds)5988subtest: a deploy mid-build drains: the build finishes, the new generation takes over5989client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw5990client # [ 117.359729] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.5991client # Running as unit: sw.service; invocation ID: fc338957d33d4c608e2e786bdb066cab5992client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)5993worker1 # [ 117.578815] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414385994client # [ 117.573793] nix[1272]: don't know how to build these paths:5995client # [ 117.574766] nix[1272]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv5996worker2 # [ 117.554877] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:23Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=05997worker2 # [ 117.558985] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:23Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=15998worker2 # [ 117.597653] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597445999worker2 # [ 117.609401] nix-grpc-daemon[1299]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 1/5)6000worker2 # [ 117.682633] nix-grpc-daemon[1299]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 114 ms (attempt 2/5)6001worker2 # [ 117.802702] nix-grpc-daemon[1299]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 3/5)6002worker2 # [ 117.819765] nix-grpc-daemon[1299]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 291 ms (attempt 4/5)6003worker2 # [ 118.117072] nix-grpc-daemon[1299]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6004worker2 # [ 118.135675] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:597446005client # [ 118.174740] nix[1272]: copying 1 paths to 'lb:50051'...6006client # [ 118.178146] nix[1272]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6007worker2 # [ 118.149992] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:597446008worker2 # [ 118.158958] nix-daemon[841]: accepted connection from pid 1299, user nix-grpc-daemon (trusted)6009worker2 # [ 118.326560] nix-daemon[841]: accepted connection from pid 1433, user nix-grpc-daemon (trusted)6010worker2 # [ 118.338259] nix-daemon[841]: reaped child process 1436, status = succeeded6011worker2 # [ 118.494270] nix-daemon[841]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)6012worker2 # [ 118.505917] nix-daemon[841]: reaped child process 1441, status = succeeded6013lb # [ 118.579456] niks3-server[1018]: 2026/09/20 10:52:24 INFO Received uploads request method=POST path=/api/pending_closures6014worker2 # [ 118.530670] nix-grpc-daemon[1378]: time=2026-09-20T10:52:24.976Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6015worker2 # [ 118.532161] nix-grpc-daemon[1378]: time=2026-09-20T10:52:24.978Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6016lb # [ 118.617978] niks3-server[1018]: 2026/09/20 10:52:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6017lb # [ 118.624608] niks3-server[1018]: 2026/09/20 10:52:24 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6018lb # [ 118.632535] niks3-server[1018]: 2026/09/20 10:52:24 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6019lb # [ 118.642892] niks3-server[1018]: 2026/09/20 10:52:25 INFO Signed narinfos id=38 count=16020worker2 # [ 118.586488] nix-grpc-daemon[1378]: time=2026-09-20T10:52:25.032Z level=INFO msg="Uploading 1 narinfos"6021lb # [ 118.648480] niks3-server[1018]: 2026/09/20 10:52:25 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6022lb # [ 118.656270] niks3-server[1018]: 2026/09/20 10:52:25 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6023lb # [ 118.659701] niks3-server[1018]: 2026/09/20 10:52:25 INFO Completed upload id=386024worker2 # [ 118.603283] nix-grpc-daemon[1378]: time=2026-09-20T10:52:25.049Z level=INFO msg="Upload complete. (430ms)"6025worker2 # [ 118.605460] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59744 duration_s=0 paths=1 nar_bytes_in=4166026worker2 # [ 118.608266] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:597446027worker2 # [ 118.614617] nix-daemon[841]: accepted connection from pid 1299, user nix-grpc-daemon (trusted)6028worker2 # [ 118.617700] nix-daemon[841]: reaped child process 1431, status = succeeded6029worker2 # [ 118.622531] nix-daemon[841]: accepted connection from pid 1299, user nix-grpc-daemon (trusted)6030client # [ 118.651287] nix[1272]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6031worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26032worker2 # Checking switch inhibitors... done6033worker2 # [ 120.573388] nixos[1476]: switching to system configuration /nix/store/sk820hc3azag8pgag8zqa4r8rvx2cp9w-nixos-system-worker2-test6034worker2 # activating the configuration...6035worker2 # [ 120.576390] systemd[1]: Stopped target Remote File Systems.6036worker2 # [ 120.577731] systemd[1]: Stopped target Local File Systems.6037worker2 # setting up /etc...6038worker2 # [ 121.199539] systemd[1]: Reload requested from client PID 1476 ('.switch-to-conf') (unit backdoor.service)...6039worker2 # [ 121.201588] systemd[1]: Reloading...6040worker2 # [ 121.826936] systemd[1]: Reloading finished in 622 ms.6041worker2 # restarting sysinit-reactivation.target6042worker2 # [ 121.882042] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6043worker2 # [ 122.014583] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6044worker2 # [ 122.018593] systemd[1]: Reached target Reactivate sysinit units.6045worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6046worker2 # [ 122.023421] systemd[1]: Reloading D-Bus System Message Bus...6047worker2 # [ 122.024454] dbus-broker-launch[534]: Caught SIGHUP, trigger reload.6048worker2 # [ 122.028677] systemd[1]: Reloading Nix worker-protocol over gRPC...6049worker2 # [ 122.059927] dbus-broker-launch[534]: Looking up NSS user entry for 'systemd-timesync'...6050worker2 # [ 122.062593] dbus-broker-launch[534]: NSS returned no entry for 'systemd-timesync'6051worker2 # [ 122.063679] dbus-broker-launch[534]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6052worker2 # [ 122.072611] systemd[1]: Reloaded Nix worker-protocol over gRPC.6053worker2 # [ 122.092441] systemd[1]: Reloaded D-Bus System Message Bus.6054worker2 # [ 122.094643] systemd[1]: Reached target Remote File Systems.6055worker2 # [ 122.099419] systemd[1]: Reached target Local File Systems.6056worker2 # [ 122.103854] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6057worker2 # [ 122.109159] systemd[1]: Starting Create SUID/SGID Wrappers...6058worker2 # [ 122.110304] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6059worker2 # [ 122.111533] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676060worker2 # [ 122.114221] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6061worker2 # [ 122.115788] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6062worker2 # [ 122.120844] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6063worker2 # [ 122.122153] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6064worker2 # [ 122.123346] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6065worker2 # [ 122.398643] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:28Z level=info event=draining6066worker2 # [ 122.531100] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6067worker2 # [ 122.534164] systemd[1]: Finished Create SUID/SGID Wrappers.6068worker2 # [ 122.537789] systemd[1]: Starting linger-users.service...6069worker2 # [ 122.577599] systemd[1]: linger-users.service: Deactivated successfully.6070worker2 # [ 122.578725] systemd[1]: Finished linger-users.service.6071worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6072worker2 # [ 122.833068] nixos[1476]: finished switching to system configuration /nix/store/sk820hc3azag8pgag8zqa4r8rvx2cp9w-nixos-system-worker2-test6073worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.02 seconds)6074worker2: must succeed: pgrep -f 'read -t [9]0 x'6075worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6076client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26077worker1 # [ 123.150523] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414386078client # don't know how to build these paths:6079client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6080worker1 # [ 123.161271] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386081worker2 # [ 123.143636] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:29Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06082worker2 # [ 123.145480] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:29Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16083worker1 # [ 123.228770] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386084worker1 # [ 123.241348] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386085client # copying 1 paths to 'lb:50051'...6086worker1 # [ 123.254815] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386087client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6088worker1 # [ 123.282776] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6089worker1 # [ 123.473963] nix-daemon[845]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6090worker1 # [ 123.485745] nix-daemon[845]: reaped child process 1683, status = succeeded6091worker1 # [ 123.653287] nix-daemon[845]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6092worker1 # [ 123.663676] nix-daemon[845]: reaped child process 1688, status = succeeded6093lb # [ 123.691858] niks3-server[1018]: 2026/09/20 10:52:30 INFO Received uploads request method=POST path=/api/pending_closures6094worker1 # [ 123.686826] nix-grpc-daemon[1601]: time=2026-09-20T10:52:30.841Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6095worker1 # [ 123.688302] nix-grpc-daemon[1601]: time=2026-09-20T10:52:30.843Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6096lb # [ 123.725147] niks3-server[1018]: 2026/09/20 10:52:30 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6097lb # [ 123.735354] niks3-server[1018]: 2026/09/20 10:52:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6098worker1 # [ 123.724124] nix-grpc-daemon[1601]: time=2026-09-20T10:52:30.878Z level=INFO msg="Uploading 1 narinfos"6099lb # [ 123.741130] niks3-server[1018]: 2026/09/20 10:52:30 INFO Signed narinfos id=39 count=16100lb # [ 123.747520] niks3-server[1018]: 2026/09/20 10:52:30 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6101lb # [ 123.757363] niks3-server[1018]: 2026/09/20 10:52:30 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6102worker1 # [ 123.746034] nix-grpc-daemon[1601]: time=2026-09-20T10:52:30.900Z level=INFO msg="Upload complete. (451ms)"6103lb # [ 123.762305] niks3-server[1018]: 2026/09/20 10:52:30 INFO Completed upload id=396104worker1 # [ 123.748713] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=5286105lb # [ 123.767118] niks3-server[1018]: 2026/09/20 10:52:30 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6106worker1 # [ 123.754723] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386107worker1 # [ 123.765512] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6108worker1 # [ 123.767506] nix-daemon[845]: reaped child process 1678, status = succeeded6109client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6110worker1 # [ 123.772434] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6111client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6112client # farm-a> LOG-farm-a-during-drain6113worker1 # [ 124.598789] nix-daemon[845]: reaped child process 1692, status = succeeded6114worker1 # [ 124.733206] nix-daemon[845]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6115worker1 # [ 124.744975] nix-daemon[845]: reaped child process 1710, status = succeeded6116lb # [ 124.772998] niks3-server[1018]: 2026/09/20 10:52:31 INFO Received uploads request method=POST path=/api/pending_closures6117worker1 # [ 124.766060] nix-grpc-daemon[1601]: time=2026-09-20T10:52:31.920Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6118worker1 # [ 124.767633] nix-grpc-daemon[1601]: time=2026-09-20T10:52:31.922Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6119lb # [ 124.805630] niks3-server[1018]: 2026/09/20 10:52:31 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6120lb # [ 124.820178] niks3-server[1018]: 2026/09/20 10:52:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6121worker1 # [ 124.808241] nix-grpc-daemon[1601]: time=2026-09-20T10:52:31.962Z level=INFO msg="Uploading 1 narinfos"6122lb # [ 124.824312] niks3-server[1018]: 2026/09/20 10:52:31 INFO Signed narinfos id=40 count=16123lb # [ 124.836263] niks3-server[1018]: 2026/09/20 10:52:31 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6124lb # [ 124.838921] niks3-server[1018]: 2026/09/20 10:52:31 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6125lb # [ 124.844581] niks3-server[1018]: 2026/09/20 10:52:31 INFO Completed upload id=406126lb # [ 124.846759] niks3-server[1018]: 2026/09/20 10:52:31 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6127worker1 # [ 124.829808] nix-grpc-daemon[1601]: time=2026-09-20T10:52:31.984Z level=INFO msg="Upload complete. (250ms)"6128lb # [ 124.848783] niks3-server[1018]: 2026/09/20 10:52:31 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6129worker1 # [ 124.834959] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16130worker2 # [ 124.799765] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:31Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06131worker2 # [ 124.802301] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:31Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16132worker1 # [ 124.845683] nix-daemon[845]: reaped child process 1690, status = succeeded6133worker1 # [ 124.883575] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386134worker1 # [ 124.895291] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386135worker1 # [ 124.906859] nix-grpc-daemon[1601]: time=2026-09-20T10:52:32.061Z level=INFO msg="All 1 paths already cached"6136client # copying 1 paths to 'lb:50051'...6137worker1 # [ 124.915057] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386138client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6139worker1 # [ 124.935160] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6140worker1 # [ 125.077975] nix-daemon[845]: accepted connection from pid 1714, user nix-grpc-daemon (trusted)6141worker1 # [ 125.089936] nix-daemon[845]: reaped child process 1717, status = succeeded6142worker1 # [ 125.217644] nix-daemon[845]: accepted connection from pid 1719, user nix-grpc-daemon (trusted)6143worker1 # [ 125.229315] nix-daemon[845]: reaped child process 1722, status = succeeded6144lb # [ 125.255927] niks3-server[1018]: 2026/09/20 10:52:31 INFO Received uploads request method=POST path=/api/pending_closures6145worker1 # [ 125.249280] nix-grpc-daemon[1601]: time=2026-09-20T10:52:32.404Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6146worker1 # [ 125.251357] nix-grpc-daemon[1601]: time=2026-09-20T10:52:32.405Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6147lb # [ 125.285070] niks3-server[1018]: 2026/09/20 10:52:31 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6148lb # [ 125.291269] niks3-server[1018]: 2026/09/20 10:52:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6149lb # [ 125.293595] niks3-server[1018]: 2026/09/20 10:52:31 INFO Signed narinfos id=41 count=16150worker1 # [ 125.277455] nix-grpc-daemon[1601]: time=2026-09-20T10:52:32.432Z level=INFO msg="Uploading 1 narinfos"6151lb # [ 125.296259] niks3-server[1018]: 2026/09/20 10:52:31 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6152lb # [ 125.300091] niks3-server[1018]: 2026/09/20 10:52:31 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6153lb # [ 125.306050] niks3-server[1018]: 2026/09/20 10:52:31 INFO Completed upload id=416154worker1 # [ 125.291096] nix-grpc-daemon[1601]: time=2026-09-20T10:52:32.445Z level=INFO msg="Upload complete. (344ms)"6155lb # [ 125.307883] niks3-server[1018]: 2026/09/20 10:52:31 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6156worker1 # [ 125.293161] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=6406157worker1 # [ 125.295847] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386158worker1 # [ 125.302438] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6159worker1 # [ 125.305405] nix-daemon[845]: reaped child process 1712, status = succeeded6160worker1 # [ 125.310456] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6161client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6162client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6163client # farm-b> LOG-farm-b-during-drain6164worker1 # [ 126.334644] nix-daemon[845]: reaped child process 1726, status = succeeded6165worker1 # [ 126.492865] nix-daemon[845]: accepted connection from pid 1741, user nix-grpc-daemon (trusted)6166worker1 # [ 126.503514] nix-daemon[845]: reaped child process 1744, status = succeeded6167lb # [ 126.535239] niks3-server[1018]: 2026/09/20 10:52:32 INFO Received uploads request method=POST path=/api/pending_closures6168worker1 # [ 126.523376] nix-grpc-daemon[1601]: time=2026-09-20T10:52:33.677Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6169worker1 # [ 126.524915] nix-grpc-daemon[1601]: time=2026-09-20T10:52:33.679Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6170lb # [ 126.554225] niks3-server[1018]: 2026/09/20 10:52:32 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6171lb # [ 126.556950] niks3-server[1018]: 2026/09/20 10:52:32 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6172lb # [ 126.560581] niks3-server[1018]: 2026/09/20 10:52:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6173lb # [ 126.562197] niks3-server[1018]: 2026/09/20 10:52:32 INFO Signed narinfos id=42 count=16174worker1 # [ 126.546349] nix-grpc-daemon[1601]: time=2026-09-20T10:52:33.701Z level=INFO msg="Uploading 1 narinfos"6175lb # [ 126.568530] niks3-server[1018]: 2026/09/20 10:52:32 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6176lb # [ 126.571110] niks3-server[1018]: 2026/09/20 10:52:32 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6177lb # [ 126.573280] niks3-server[1018]: 2026/09/20 10:52:32 INFO Completed upload id=426178worker1 # [ 126.557080] nix-grpc-daemon[1601]: time=2026-09-20T10:52:33.711Z level=INFO msg="Upload complete. (237ms)"6179lb # [ 126.575140] niks3-server[1018]: 2026/09/20 10:52:32 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6180worker1 # [ 126.559887] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16181worker1 # [ 126.566620] nix-daemon[845]: reaped child process 1724, status = succeeded6182worker2 # [ 126.526946] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:32Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06183worker2 # [ 126.529224] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:32Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16184worker1 # [ 126.611107] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386185worker1 # [ 126.621610] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386186worker1 # [ 126.631999] nix-grpc-daemon[1601]: time=2026-09-20T10:52:33.786Z level=INFO msg="All 2 paths already cached"6187client # copying 1 paths to 'lb:50051'...6188client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6189worker1 # [ 126.639526] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386190worker1 # [ 126.647251] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6191worker1 # [ 126.807125] nix-daemon[845]: accepted connection from pid 1748, user nix-grpc-daemon (trusted)6192worker1 # [ 126.819178] nix-daemon[845]: reaped child process 1751, status = succeeded6193worker1 # [ 126.980238] nix-daemon[845]: accepted connection from pid 1753, user nix-grpc-daemon (trusted)6194worker1 # [ 126.991893] nix-daemon[845]: reaped child process 1756, status = succeeded6195lb # [ 127.024489] niks3-server[1018]: 2026/09/20 10:52:33 INFO Received uploads request method=POST path=/api/pending_closures6196worker1 # [ 127.016527] nix-grpc-daemon[1601]: time=2026-09-20T10:52:34.171Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6197worker1 # [ 127.018447] nix-grpc-daemon[1601]: time=2026-09-20T10:52:34.172Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6198lb # [ 127.050667] niks3-server[1018]: 2026/09/20 10:52:33 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6199lb # [ 127.061760] niks3-server[1018]: 2026/09/20 10:52:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6200lb # [ 127.066986] niks3-server[1018]: 2026/09/20 10:52:33 INFO Signed narinfos id=43 count=16201worker1 # [ 127.053797] nix-grpc-daemon[1601]: time=2026-09-20T10:52:34.208Z level=INFO msg="Uploading 1 narinfos"6202lb # [ 127.074133] niks3-server[1018]: 2026/09/20 10:52:33 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6203lb # [ 127.084400] niks3-server[1018]: 2026/09/20 10:52:33 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6204worker1 # [ 127.074248] nix-grpc-daemon[1601]: time=2026-09-20T10:52:34.229Z level=INFO msg="Upload complete. (415ms)"6205lb # [ 127.090491] niks3-server[1018]: 2026/09/20 10:52:33 INFO Completed upload id=436206lb # [ 127.093870] niks3-server[1018]: 2026/09/20 10:52:33 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6207worker1 # [ 127.077378] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=7686208worker1 # [ 127.081969] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386209worker1 # [ 127.091060] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6210worker1 # [ 127.095688] nix-daemon[845]: reaped child process 1746, status = succeeded6211worker1 # [ 127.100757] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6212client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6213client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6214worker1 # [ 128.313099] nix-daemon[845]: reaped child process 1760, status = succeeded6215client # farm-top> LOG-farm-top-during-drain6216worker1 # [ 128.465850] nix-daemon[845]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6217worker1 # [ 128.477800] nix-daemon[845]: reaped child process 1778, status = succeeded6218lb # [ 128.512336] niks3-server[1018]: 2026/09/20 10:52:34 INFO Received uploads request method=POST path=/api/pending_closures6219worker1 # [ 128.503684] nix-grpc-daemon[1601]: time=2026-09-20T10:52:35.658Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6220worker1 # [ 128.506059] nix-grpc-daemon[1601]: time=2026-09-20T10:52:35.659Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6221lb # [ 128.543514] niks3-server[1018]: 2026/09/20 10:52:34 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6222lb # [ 128.558306] niks3-server[1018]: 2026/09/20 10:52:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6223lb # [ 128.563711] niks3-server[1018]: 2026/09/20 10:52:34 INFO Signed narinfos id=44 count=16224worker1 # [ 128.550638] nix-grpc-daemon[1601]: time=2026-09-20T10:52:35.705Z level=INFO msg="Uploading 1 narinfos"6225lb # [ 128.573314] niks3-server[1018]: 2026/09/20 10:52:34 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6226lb # [ 128.580593] niks3-server[1018]: 2026/09/20 10:52:34 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6227lb # [ 128.587344] niks3-server[1018]: 2026/09/20 10:52:34 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6228lb # [ 128.591655] niks3-server[1018]: 2026/09/20 10:52:34 INFO Completed upload id=446229worker1 # [ 128.576825] nix-grpc-daemon[1601]: time=2026-09-20T10:52:35.731Z level=INFO msg="Upload complete. (268ms)"6230lb # [ 128.594792] niks3-server[1018]: 2026/09/20 10:52:34 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6231worker1 # [ 128.582614] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16232worker1 # [ 128.594867] nix-daemon[845]: reaped child process 1758, status = succeeded6233client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.69 seconds)6234client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6235client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6236worker2: must succeed: pkill -f 'read -t [9]0 x'6237worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6238client: waiting for success: ! systemctl is-active sw6239worker2 # [ 128.846844] nix-daemon[841]: reaped child process 1449, status = succeeded6240worker2 # [ 128.973401] nix-daemon[841]: accepted connection from pid 1622, user nix-grpc-daemon (trusted)6241worker2 # [ 128.983620] nix-daemon[841]: reaped child process 1625, status = succeeded6242lb # [ 129.051297] niks3-server[1018]: 2026/09/20 10:52:35 INFO Received uploads request method=POST path=/api/pending_closures6243worker2 # [ 128.998830] nix-grpc-daemon[1378]: time=2026-09-20T10:52:35.444Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6244worker2 # [ 129.000538] nix-grpc-daemon[1378]: time=2026-09-20T10:52:35.446Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6245lb # [ 129.070896] niks3-server[1018]: 2026/09/20 10:52:35 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6246lb # [ 129.073631] niks3-server[1018]: 2026/09/20 10:52:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6247lb # [ 129.075401] niks3-server[1018]: 2026/09/20 10:52:35 INFO Signed narinfos id=45 count=16248worker2 # [ 129.019488] nix-grpc-daemon[1378]: time=2026-09-20T10:52:35.465Z level=INFO msg="Uploading 1 narinfos"6249lb # [ 129.076618] niks3-server[1018]: 2026/09/20 10:52:35 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6250lb # [ 129.082533] niks3-server[1018]: 2026/09/20 10:52:35 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6251lb # [ 129.085740] niks3-server[1018]: 2026/09/20 10:52:35 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6252lb # [ 129.088351] niks3-server[1018]: 2026/09/20 10:52:35 INFO Completed upload id=456253worker2 # [ 129.031965] nix-grpc-daemon[1378]: time=2026-09-20T10:52:35.477Z level=INFO msg="Upload complete. (191ms)"6254lb # [ 129.091075] niks3-server[1018]: 2026/09/20 10:52:35 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6255worker2 # [ 129.035409] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59744 duration_s=10 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16256worker2 # [ 129.041509] nix-daemon[841]: reaped child process 1447, status = succeeded6257client # [ 129.097313] systemd[1]: sw.service: Deactivated successfully.6258client # [ 129.098963] systemd[1]: sw.service: Consumed 116ms CPU time over 11.739s wall clock time, 37.1M memory peak, 8.7K incoming IP traffic, 9.1K outgoing IP traffic.6259worker2 # [ 129.400953] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:35Z level=info event=signal_exit6260worker2 # [ 129.625868] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_yield6261lb # [ 129.685306] niks3-server[1018]: 2026/09/20 10:52:36 INFO lead: released remote=192.168.1.4:547406262worker1 # [ 129.669289] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:36Z level=info event=scheduler_restarting addr=lb:500516263worker2 # [ 129.632316] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:597426264worker2 # [ 129.634633] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=worker_gone addr=192.168.1.3:500516265worker2 # [ 129.636670] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_restarting addr=lb:500516266worker2 # [ 129.638519] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:597426267worker2 # [ 129.640176] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=worker_gone addr=192.168.1.4:500516268worker1 # [ 129.870265] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500516269worker1 # [ 129.874219] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6270worker2 # [ 129.835490] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500516271worker2 # [ 129.840642] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6272client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6273client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6274client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6275worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6276worker1 # [ 130.074949] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500516277worker1 # [ 130.078902] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6278worker2 # [ 130.042434] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500516279worker2 # [ 130.044878] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6280worker1 # [ 130.279335] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500516281worker1 # [ 130.281176] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6282worker2 # [ 130.243438] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500516283worker2 # [ 130.247344] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6284worker1 # [ 130.482159] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500516285worker2 # [ 130.447656] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_connected addr=lb:500516286worker1 # [ 130.486118] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6287worker2 # [ 130.449413] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6288worker2 # [ 130.627270] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:37Z level=info event=scheduler_restarting6289worker2 # [ 130.647144] nix-daemon[841]: reaped child process 1350, status = succeeded6290worker1 # [ 130.687451] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_connected addr=lb:500516291worker1 # [ 130.691186] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6292worker2 # [ 130.660618] nix-grpc-daemon[1299]: ts=2026-09-20T10:52:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6293worker1 # [ 130.891810] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500516294worker1 # [ 130.893805] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6295worker2 # [ 130.932694] nix-grpc-daemon[1299]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6296worker2 # [ 130.936175] nix-grpc-daemon[1299]: E0000 00:00:1789901557.377909 1342 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6297worker2 # [ 130.940821] nix-grpc-daemon[1299]: E0000 00:00:1789901557.378496 1342 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6298worker1 # [ 131.094328] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500516299worker1 # [ 131.096218] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6300worker1 # [ 131.296672] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500516301worker1 # [ 131.298413] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6302worker1 # [ 131.499143] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500516303worker1 # [ 131.501140] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6304worker1 # [ 131.701990] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_connected addr=lb:500516305worker1 # [ 131.703701] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6306worker1 # [ 131.904261] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516307worker1 # [ 131.906119] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6308worker1 # [ 132.106700] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516309worker1 # [ 132.108541] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6310worker1 # [ 132.309186] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516311worker1 # [ 132.311099] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6312worker2 # [ 132.437529] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6313worker2 # [ 132.441149] systemd[1]: nix-grpc-daemon.service: Consumed 643ms CPU time over 26.348s wall clock time, 33.6M memory peak, 23.6K incoming IP traffic, 32.7K outgoing IP traffic.6314worker2 # [ 132.457473] systemd[1]: Starting Nix worker-protocol over gRPC...6315worker1 # [ 132.512090] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516316worker1 # [ 132.516067] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6317worker2 # [ 132.618592] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6318worker2 # [ 132.622440] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16319worker2 # [ 132.628481] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6320worker2 # [ 132.633130] systemd[1]: Started Nix worker-protocol over gRPC.6321worker2 # [ 132.638100] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_take_over6322lb # [ 132.694897] niks3-server[1018]: 2026/09/20 10:52:39 INFO lead: acquired remote=192.168.1.4:405046323worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.69 seconds)6324lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6325lb: must succeed: curl -sf localhost:9901/clusters6326worker2 # [ 132.658951] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516327worker2 # [ 132.660555] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6328worker1 # [ 132.717410] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516329worker1 # [ 132.721180] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6331worker2 # [ 132.861623] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516332worker2 # [ 132.865568] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6333worker1 # [ 132.918871] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:40Z level=info event=scheduler_connected addr=lb:500516334worker1 # [ 132.922841] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6335worker2 # [ 133.066398] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516336worker2 # [ 133.070250] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6337worker1 # [ 133.123686] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:40Z level=info event=scheduler_connected addr=lb:500516338worker1 # [ 133.127660] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6339worker2 # [ 133.271069] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516340worker2 # [ 133.274912] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6341worker1 # [ 133.328692] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:40Z level=info event=scheduler_connected addr=lb:500516342worker1 # [ 133.332987] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6343worker2 # [ 133.475580] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=scheduler_connected addr=lb:500516344worker2 # [ 133.482071] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06345worker1 # [ 133.533666] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:40Z level=info event=scheduler_connected addr=lb:500516346worker2 # [ 133.494716] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06347lb: must succeed: curl -sf localhost:9901/clusters6348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6349lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)6350lb: must succeed: curl -sf localhost:9901/clusters6351lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6352worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6353worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6354(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.62 seconds)6355subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6356client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16357worker2 # [ 134.101819] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:543266358worker2 # [ 134.113264] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6359worker2 # [ 134.127479] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543266360worker2 # [ 134.139523] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 1/5)6361worker2 # [ 134.204331] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 2/5)6362worker2 # [ 134.379786] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 224 ms (attempt 3/5)6363worker2 # [ 134.609864] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 454 ms (attempt 4/5)6364worker2 # [ 135.070715] nix-grpc-daemon[1639]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6365worker2 # [ 135.113463] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:41Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06366worker2 # [ 135.117449] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:41Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16367worker2 # [ 135.121700] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:41Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06368worker2 # [ 135.125451] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:41Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06369worker1 # [ 135.198775] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386370worker1 # [ 135.211840] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386371worker1 # [ 135.223374] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386372worker1 # [ 135.234867] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6373worker1 # [ 135.398198] nix-daemon[845]: accepted connection from pid 1782, user nix-grpc-daemon (trusted)6374worker1 # [ 135.410157] nix-daemon[845]: reaped child process 1785, status = succeeded6375worker1 # [ 135.530388] nix-daemon[845]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6376worker1 # [ 135.541872] nix-daemon[845]: reaped child process 1790, status = succeeded6377lb # [ 135.573247] niks3-server[1018]: 2026/09/20 10:52:41 INFO Received uploads request method=POST path=/api/pending_closures6378worker1 # [ 135.562411] nix-grpc-daemon[1601]: time=2026-09-20T10:52:42.717Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6379worker1 # [ 135.563882] nix-grpc-daemon[1601]: time=2026-09-20T10:52:42.718Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6380lb # [ 135.590036] niks3-server[1018]: 2026/09/20 10:52:41 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6381lb # [ 135.595800] niks3-server[1018]: 2026/09/20 10:52:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6382lb # [ 135.597975] niks3-server[1018]: 2026/09/20 10:52:41 INFO Signed narinfos id=46 count=16383worker1 # [ 135.583180] nix-grpc-daemon[1601]: time=2026-09-20T10:52:42.737Z level=INFO msg="Uploading 1 narinfos"6384lb # [ 135.602446] niks3-server[1018]: 2026/09/20 10:52:41 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6385lb # [ 135.608355] niks3-server[1018]: 2026/09/20 10:52:41 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6386lb # [ 135.611883] niks3-server[1018]: 2026/09/20 10:52:41 INFO Completed upload id=466387worker1 # [ 135.595646] nix-grpc-daemon[1601]: time=2026-09-20T10:52:42.750Z level=INFO msg="Upload complete. (348ms)"6388worker1 # [ 135.597371] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=5046389lb # [ 135.614680] niks3-server[1018]: 2026/09/20 10:52:41 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6390worker1 # [ 135.600899] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386391worker1 # [ 135.607113] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6392worker1 # [ 135.609341] nix-daemon[845]: reaped child process 1780, status = succeeded6393worker1 # [ 135.614456] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6394worker1 # [ 136.427302] nix-daemon[845]: reaped child process 1794, status = succeeded6395worker1 # [ 136.568974] nix-daemon[845]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6396worker1 # [ 136.582161] nix-daemon[845]: reaped child process 1812, status = succeeded6397lb # [ 136.612442] niks3-server[1018]: 2026/09/20 10:52:42 INFO Received uploads request method=POST path=/api/pending_closures6398worker1 # [ 136.604513] nix-grpc-daemon[1601]: time=2026-09-20T10:52:43.759Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6399worker1 # [ 136.606336] nix-grpc-daemon[1601]: time=2026-09-20T10:52:43.760Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6400lb # [ 136.646480] niks3-server[1018]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6401lb # [ 136.658083] niks3-server[1018]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6402lb # [ 136.668163] niks3-server[1018]: 2026/09/20 10:52:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6403lb # [ 136.672444] niks3-server[1018]: 2026/09/20 10:52:43 INFO Signed narinfos id=47 count=16404worker1 # [ 136.660232] nix-grpc-daemon[1601]: time=2026-09-20T10:52:43.814Z level=INFO msg="Uploading 1 narinfos"6405lb # [ 136.678940] niks3-server[1018]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6406lb # [ 136.687256] niks3-server[1018]: 2026/09/20 10:52:43 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6407lb # [ 136.692680] niks3-server[1018]: 2026/09/20 10:52:43 INFO Completed upload id=476408worker1 # [ 136.676572] nix-grpc-daemon[1601]: time=2026-09-20T10:52:43.831Z level=INFO msg="Upload complete. (256ms)"6409lb # [ 136.696796] niks3-server[1018]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6410worker2 # [ 136.641630] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:43Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16411worker1 # [ 136.682613] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16412worker2 # [ 136.646775] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:43Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06413worker1 # [ 136.695883] nix-daemon[845]: reaped child process 1792, status = succeeded6414worker1 # [ 136.727807] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386415worker1 # [ 136.739792] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386416worker1 # [ 136.750435] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386417worker1 # [ 136.758360] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6418worker1 # [ 136.903702] nix-daemon[845]: accepted connection from pid 1816, user nix-grpc-daemon (trusted)6419worker1 # [ 136.915512] nix-daemon[845]: reaped child process 1819, status = succeeded6420worker1 # [ 137.069346] nix-daemon[845]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6421worker1 # [ 137.080859] nix-daemon[845]: reaped child process 1824, status = succeeded6422lb # [ 137.112356] niks3-server[1018]: 2026/09/20 10:52:43 INFO Received uploads request method=POST path=/api/pending_closures6423worker1 # [ 137.103716] nix-grpc-daemon[1601]: time=2026-09-20T10:52:44.258Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6424worker1 # [ 137.105075] nix-grpc-daemon[1601]: time=2026-09-20T10:52:44.258Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6425lb # [ 137.141835] niks3-server[1018]: 2026/09/20 10:52:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6426lb # [ 137.148600] niks3-server[1018]: 2026/09/20 10:52:43 INFO Signed narinfos id=48 count=16427worker1 # [ 137.135293] nix-grpc-daemon[1601]: time=2026-09-20T10:52:44.289Z level=INFO msg="Uploading 1 narinfos"6428lb # [ 137.155457] niks3-server[1018]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6429lb # [ 137.175626] niks3-server[1018]: 2026/09/20 10:52:43 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6430lb # [ 137.183214] niks3-server[1018]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6431lb # [ 137.187578] niks3-server[1018]: 2026/09/20 10:52:43 INFO Completed upload id=486432worker1 # [ 137.171576] nix-grpc-daemon[1601]: time=2026-09-20T10:52:44.326Z level=INFO msg="Upload complete. (408ms)"6433lb # [ 137.189848] niks3-server[1018]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6434worker1 # [ 137.176163] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=5046435worker1 # [ 137.182317] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386436worker1 # [ 137.194109] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6437worker1 # [ 137.200766] nix-daemon[845]: reaped child process 1814, status = succeeded6438worker1 # [ 137.208429] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6439lb # [ 137.285296] envoy[908]: conn peer=[2001:db8:1::1]:51932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9572 tx=3820 ms=54456440lb # [ 137.288991] envoy[908]: conn peer=[2001:db8:1::1]:47420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2936 tx=1352 ms=115086441lb # [ 137.292568] envoy[908]: conn peer=[2001:db8:1::4]:59660 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1721 tx=545 ms=242326442worker1 # [ 137.821191] nix-daemon[845]: reaped child process 1828, status = succeeded6443worker1 # [ 137.946475] nix-daemon[845]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6444worker1 # [ 137.958645] nix-daemon[845]: reaped child process 1846, status = succeeded6445lb # [ 137.992310] niks3-server[1018]: 2026/09/20 10:52:44 INFO Received uploads request method=POST path=/api/pending_closures6446worker1 # [ 137.983606] nix-grpc-daemon[1601]: time=2026-09-20T10:52:45.138Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6447worker1 # [ 137.985626] nix-grpc-daemon[1601]: time=2026-09-20T10:52:45.140Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6448lb # [ 138.026371] niks3-server[1018]: 2026/09/20 10:52:44 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6449lb # [ 138.040601] niks3-server[1018]: 2026/09/20 10:52:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6450lb # [ 138.045916] niks3-server[1018]: 2026/09/20 10:52:44 INFO Signed narinfos id=49 count=16451worker1 # [ 138.032719] nix-grpc-daemon[1601]: time=2026-09-20T10:52:45.187Z level=INFO msg="Uploading 1 narinfos"6452lb # [ 138.054232] niks3-server[1018]: 2026/09/20 10:52:44 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6453lb # [ 138.061127] niks3-server[1018]: 2026/09/20 10:52:44 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6454lb # [ 138.069552] niks3-server[1018]: 2026/09/20 10:52:44 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6455lb # [ 138.074409] niks3-server[1018]: 2026/09/20 10:52:44 INFO Completed upload id=496456worker1 # [ 138.058284] nix-grpc-daemon[1601]: time=2026-09-20T10:52:45.213Z level=INFO msg="Upload complete. (244ms)"6457lb # [ 138.076807] niks3-server[1018]: 2026/09/20 10:52:44 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6458worker2 # [ 138.023939] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:44Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16459worker1 # [ 138.063450] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16460worker2 # [ 138.033543] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:44Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06461worker1 # [ 138.076920] nix-daemon[845]: reaped child process 1826, status = succeeded6462worker1 # [ 138.113458] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386463worker1 # [ 138.125701] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386464worker1 # [ 138.136971] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386465worker1 # [ 138.145541] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6466worker1 # [ 138.307664] nix-daemon[845]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6467worker1 # [ 138.320128] nix-daemon[845]: reaped child process 1853, status = succeeded6468worker1 # [ 138.462065] nix-daemon[845]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6469worker1 # [ 138.474236] nix-daemon[845]: reaped child process 1858, status = succeeded6470lb # [ 138.501646] niks3-server[1018]: 2026/09/20 10:52:44 INFO Received uploads request method=POST path=/api/pending_closures6471worker1 # [ 138.492983] nix-grpc-daemon[1601]: time=2026-09-20T10:52:45.647Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6472worker1 # [ 138.494879] nix-grpc-daemon[1601]: time=2026-09-20T10:52:45.649Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6473lb # [ 138.530686] niks3-server[1018]: 2026/09/20 10:52:44 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6474lb # [ 138.538218] niks3-server[1018]: 2026/09/20 10:52:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6475lb # [ 138.542727] niks3-server[1018]: 2026/09/20 10:52:44 INFO Signed narinfos id=50 count=16476worker1 # [ 138.529536] nix-grpc-daemon[1601]: time=2026-09-20T10:52:45.684Z level=INFO msg="Uploading 1 narinfos"6477lb # [ 138.556085] niks3-server[1018]: 2026/09/20 10:52:44 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6478lb # [ 138.563109] niks3-server[1018]: 2026/09/20 10:52:44 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6479lb # [ 138.568130] niks3-server[1018]: 2026/09/20 10:52:44 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6480worker1 # [ 138.557850] nix-grpc-daemon[1601]: time=2026-09-20T10:52:45.712Z level=INFO msg="Upload complete. (399ms)"6481lb # [ 138.574163] niks3-server[1018]: 2026/09/20 10:52:44 INFO Completed upload id=506482worker1 # [ 138.560558] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=5046483worker1 # [ 138.566298] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386484worker1 # [ 138.575481] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6485worker1 # [ 138.583622] nix-daemon[845]: reaped child process 1848, status = succeeded6486worker1 # [ 138.587364] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6487worker1 # [ 139.398620] nix-daemon[845]: reaped child process 1862, status = succeeded6488worker1 # [ 139.540181] nix-daemon[845]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)6489worker1 # [ 139.551950] nix-daemon[845]: reaped child process 1880, status = succeeded6490lb # [ 139.584419] niks3-server[1018]: 2026/09/20 10:52:45 INFO Received uploads request method=POST path=/api/pending_closures6491worker1 # [ 139.578521] nix-grpc-daemon[1601]: time=2026-09-20T10:52:46.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6492worker1 # [ 139.580270] nix-grpc-daemon[1601]: time=2026-09-20T10:52:46.734Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6493lb # [ 139.626555] niks3-server[1018]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6494lb # [ 139.633189] niks3-server[1018]: 2026/09/20 10:52:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6495lb # [ 139.642676] niks3-server[1018]: 2026/09/20 10:52:46 INFO Signed narinfos id=51 count=16496worker1 # [ 139.628937] nix-grpc-daemon[1601]: time=2026-09-20T10:52:46.783Z level=INFO msg="Uploading 1 narinfos"6497lb # [ 139.645328] niks3-server[1018]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6498lb # [ 139.650387] niks3-server[1018]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6499lb # [ 139.661158] niks3-server[1018]: 2026/09/20 10:52:46 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6500lb # [ 139.666376] niks3-server[1018]: 2026/09/20 10:52:46 INFO Completed upload id=516501worker1 # [ 139.650228] nix-grpc-daemon[1601]: time=2026-09-20T10:52:46.804Z level=INFO msg="Upload complete. (257ms)"6502lb # [ 139.668298] niks3-server[1018]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6503worker1 # [ 139.652902] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16504worker2 # [ 139.615748] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:46Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16505worker1 # [ 139.659895] nix-daemon[845]: reaped child process 1860, status = succeeded6506worker1 # [ 139.698364] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386507worker2 # [ 139.661877] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:46Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06508worker1 # [ 139.708170] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386509worker1 # [ 139.717436] nix-grpc-daemon[1601]: time=2026-09-20T10:52:46.872Z level=INFO msg="All 1 paths already cached"6510worker1 # [ 139.723727] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386511worker1 # [ 139.728803] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6512worker1 # [ 139.875618] nix-daemon[845]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6513worker1 # [ 139.887564] nix-daemon[845]: reaped child process 1887, status = succeeded6514worker1 # [ 140.039533] nix-daemon[845]: accepted connection from pid 1889, user nix-grpc-daemon (trusted)6515worker1 # [ 140.051599] nix-daemon[845]: reaped child process 1892, status = succeeded6516lb # [ 140.081809] niks3-server[1018]: 2026/09/20 10:52:46 INFO Received uploads request method=POST path=/api/pending_closures6517worker1 # [ 140.073957] nix-grpc-daemon[1601]: time=2026-09-20T10:52:47.228Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6518worker1 # [ 140.075361] nix-grpc-daemon[1601]: time=2026-09-20T10:52:47.228Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6519lb # [ 140.105422] niks3-server[1018]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6520lb # [ 140.118256] niks3-server[1018]: 2026/09/20 10:52:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6521worker1 # [ 140.108204] nix-grpc-daemon[1601]: time=2026-09-20T10:52:47.262Z level=INFO msg="Uploading 1 narinfos"6522lb # [ 140.124170] niks3-server[1018]: 2026/09/20 10:52:46 INFO Signed narinfos id=52 count=16523lb # [ 140.128643] niks3-server[1018]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6524lb # [ 140.140823] niks3-server[1018]: 2026/09/20 10:52:46 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6525worker1 # [ 140.131357] nix-grpc-daemon[1601]: time=2026-09-20T10:52:47.286Z level=INFO msg="Upload complete. (397ms)"6526lb # [ 140.147821] niks3-server[1018]: 2026/09/20 10:52:46 INFO Completed upload id=526527worker1 # [ 140.133354] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=6246528lb # [ 140.150731] niks3-server[1018]: 2026/09/20 10:52:46 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6529worker1 # [ 140.139408] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386530worker1 # [ 140.143551] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6531worker1 # [ 140.145712] nix-daemon[845]: reaped child process 1882, status = succeeded6532worker1 # [ 140.150885] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6533worker1 # [ 140.763600] nix-daemon[845]: reaped child process 1896, status = succeeded6534worker1 # [ 140.893254] nix-daemon[845]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6535worker1 # [ 140.905168] nix-daemon[845]: reaped child process 1914, status = succeeded6536lb # [ 140.937431] niks3-server[1018]: 2026/09/20 10:52:47 INFO Received uploads request method=POST path=/api/pending_closures6537worker1 # [ 140.929392] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.083Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6538worker1 # [ 140.930888] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.085Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6539lb # [ 140.977635] niks3-server[1018]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6540lb # [ 140.987324] niks3-server[1018]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6541lb # [ 140.994436] niks3-server[1018]: 2026/09/20 10:52:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6542lb # [ 140.998640] niks3-server[1018]: 2026/09/20 10:52:47 INFO Signed narinfos id=53 count=16543worker1 # [ 140.985251] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.139Z level=INFO msg="Uploading 1 narinfos"6544lb # [ 141.007132] niks3-server[1018]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6545lb # [ 141.014091] niks3-server[1018]: 2026/09/20 10:52:47 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6546lb # [ 141.021757] niks3-server[1018]: 2026/09/20 10:52:47 INFO Completed upload id=536547worker1 # [ 141.005702] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.160Z level=INFO msg="Upload complete. (249ms)"6548lb # [ 141.023723] niks3-server[1018]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6549worker2 # [ 140.971672] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:47Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16550worker1 # [ 141.010729] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16551worker1 # [ 141.025201] nix-daemon[845]: reaped child process 1894, status = succeeded6552worker1 # [ 141.054909] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386553worker2 # [ 141.017669] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:47Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06554worker1 # [ 141.067221] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386555worker1 # [ 141.079389] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.234Z level=INFO msg="All 1 paths already cached"6556worker1 # [ 141.086096] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386557worker1 # [ 141.096148] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6558worker1 # [ 141.241494] nix-daemon[845]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)6559worker1 # [ 141.253317] nix-daemon[845]: reaped child process 1921, status = succeeded6560worker1 # [ 141.404151] nix-daemon[845]: accepted connection from pid 1923, user nix-grpc-daemon (trusted)6561worker1 # [ 141.415815] nix-daemon[845]: reaped child process 1926, status = succeeded6562lb # [ 141.447392] niks3-server[1018]: 2026/09/20 10:52:47 INFO Received uploads request method=POST path=/api/pending_closures6563worker1 # [ 141.439060] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.593Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6564worker1 # [ 141.440924] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.595Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6565lb # [ 141.480111] niks3-server[1018]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6566lb # [ 141.488818] niks3-server[1018]: 2026/09/20 10:52:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6567worker1 # [ 141.476973] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.631Z level=INFO msg="Uploading 1 narinfos"6568lb # [ 141.493099] niks3-server[1018]: 2026/09/20 10:52:47 INFO Signed narinfos id=54 count=16569lb # [ 141.498130] niks3-server[1018]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6570lb # [ 141.508134] niks3-server[1018]: 2026/09/20 10:52:47 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6571lb # [ 141.513849] niks3-server[1018]: 2026/09/20 10:52:47 INFO Completed upload id=546572worker1 # [ 141.497651] nix-grpc-daemon[1601]: time=2026-09-20T10:52:48.652Z level=INFO msg="Upload complete. (390ms)"6573lb # [ 141.516391] niks3-server[1018]: 2026/09/20 10:52:47 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6574worker1 # [ 141.501385] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=6246575worker1 # [ 141.506404] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386576worker1 # [ 141.513633] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6577worker1 # [ 141.519500] nix-daemon[845]: reaped child process 1916, status = succeeded6578worker1 # [ 141.524657] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6579worker1 # [ 142.748988] nix-daemon[845]: reaped child process 1930, status = succeeded6580worker1 # [ 142.874442] nix-daemon[845]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)6581worker1 # [ 142.886373] nix-daemon[845]: reaped child process 1948, status = succeeded6582lb # [ 142.914750] niks3-server[1018]: 2026/09/20 10:52:49 INFO Received uploads request method=POST path=/api/pending_closures6583worker1 # [ 142.908203] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.062Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6584worker1 # [ 142.910922] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.064Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6585lb # [ 142.944919] niks3-server[1018]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6586lb # [ 142.958122] niks3-server[1018]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6587lb # [ 142.967715] niks3-server[1018]: 2026/09/20 10:52:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6588lb # [ 142.974113] niks3-server[1018]: 2026/09/20 10:52:49 INFO Signed narinfos id=55 count=16589worker1 # [ 142.960673] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.115Z level=INFO msg="Uploading 1 narinfos"6590lb # [ 142.979937] niks3-server[1018]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6591lb # [ 142.990292] niks3-server[1018]: 2026/09/20 10:52:49 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6592lb # [ 142.995046] niks3-server[1018]: 2026/09/20 10:52:49 INFO Completed upload id=556593worker1 # [ 142.980796] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.135Z level=INFO msg="Upload complete. (248ms)"6594lb # [ 142.997838] niks3-server[1018]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6595worker1 # [ 142.983209] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16596worker1 # [ 142.987483] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386597worker2 # [ 142.945587] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:49Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16598worker1 # [ 142.993543] nix-daemon[845]: reaped child process 1928, status = succeeded6599worker1 # [ 142.994816] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386600worker2 # [ 142.955364] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:49Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06601worker1 # [ 143.000487] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.155Z level=INFO msg="All 1 paths already cached"6602worker1 # [ 143.046490] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386603worker1 # [ 143.056076] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6604worker1 # [ 143.205023] nix-daemon[845]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)6605worker1 # [ 143.216820] nix-daemon[845]: reaped child process 1955, status = succeeded6606worker1 # [ 143.359992] nix-daemon[845]: accepted connection from pid 1957, user nix-grpc-daemon (trusted)6607worker1 # [ 143.372079] nix-daemon[845]: reaped child process 1960, status = succeeded6608lb # [ 143.399190] niks3-server[1018]: 2026/09/20 10:52:49 INFO Received uploads request method=POST path=/api/pending_closures6609worker1 # [ 143.391278] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.545Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6610worker1 # [ 143.392649] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.545Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6611lb # [ 143.421670] niks3-server[1018]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6612lb # [ 143.433622] niks3-server[1018]: 2026/09/20 10:52:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6613worker1 # [ 143.422090] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.576Z level=INFO msg="Uploading 1 narinfos"6614lb # [ 143.439500] niks3-server[1018]: 2026/09/20 10:52:49 INFO Signed narinfos id=56 count=16615lb # [ 143.443414] niks3-server[1018]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6616lb # [ 143.454781] niks3-server[1018]: 2026/09/20 10:52:49 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6617lb # [ 143.462629] niks3-server[1018]: 2026/09/20 10:52:49 INFO Completed upload id=566618worker1 # [ 143.448522] nix-grpc-daemon[1601]: time=2026-09-20T10:52:50.603Z level=INFO msg="Upload complete. (387ms)"6619worker1 # [ 143.450672] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=6246620lb # [ 143.466458] niks3-server[1018]: 2026/09/20 10:52:49 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6621worker1 # [ 143.456377] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386622worker1 # [ 143.460371] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6623worker1 # [ 143.462627] nix-daemon[845]: reaped child process 1950, status = succeeded6624worker1 # [ 143.467741] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6625worker1 # [ 144.281192] nix-daemon[845]: reaped child process 1964, status = succeeded6626worker1 # [ 144.423780] nix-daemon[845]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)6627worker1 # [ 144.434567] nix-daemon[845]: reaped child process 1982, status = succeeded6628lb # [ 144.466546] niks3-server[1018]: 2026/09/20 10:52:50 INFO Received uploads request method=POST path=/api/pending_closures6629worker1 # [ 144.458480] nix-grpc-daemon[1601]: time=2026-09-20T10:52:51.613Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6630worker1 # [ 144.459967] nix-grpc-daemon[1601]: time=2026-09-20T10:52:51.614Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6631lb # [ 144.492467] niks3-server[1018]: 2026/09/20 10:52:50 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6632lb # [ 144.506139] niks3-server[1018]: 2026/09/20 10:52:50 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6633lb # [ 144.513320] niks3-server[1018]: 2026/09/20 10:52:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6634lb # [ 144.518574] niks3-server[1018]: 2026/09/20 10:52:50 INFO Signed narinfos id=57 count=16635worker1 # [ 144.504710] nix-grpc-daemon[1601]: time=2026-09-20T10:52:51.659Z level=INFO msg="Uploading 1 narinfos"6636lb # [ 144.522384] niks3-server[1018]: 2026/09/20 10:52:50 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6637lb # [ 144.530916] niks3-server[1018]: 2026/09/20 10:52:50 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6638lb # [ 144.533785] niks3-server[1018]: 2026/09/20 10:52:50 INFO Completed upload id=576639worker1 # [ 144.517612] nix-grpc-daemon[1601]: time=2026-09-20T10:52:51.672Z level=INFO msg="Upload complete. (244ms)"6640lb # [ 144.536402] niks3-server[1018]: 2026/09/20 10:52:50 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6641worker1 # [ 144.521784] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16642worker2 # [ 144.485602] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:50Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16643worker1 # [ 144.528720] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386644worker2 # [ 144.493253] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:50Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06645worker1 # [ 144.538319] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386646worker1 # [ 144.540520] nix-daemon[845]: reaped child process 1962, status = succeeded6647worker1 # [ 144.544051] nix-grpc-daemon[1601]: time=2026-09-20T10:52:51.698Z level=INFO msg="All 2 paths already cached"6648worker1 # [ 144.547672] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386649worker1 # [ 144.551553] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6650worker1 # [ 144.701062] nix-daemon[845]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)6651worker1 # [ 144.713740] nix-daemon[845]: reaped child process 1989, status = succeeded6652worker1 # [ 144.867846] nix-daemon[845]: accepted connection from pid 1991, user nix-grpc-daemon (trusted)6653worker1 # [ 144.879584] nix-daemon[845]: reaped child process 1994, status = succeeded6654lb # [ 144.910886] niks3-server[1018]: 2026/09/20 10:52:51 INFO Received uploads request method=POST path=/api/pending_closures6655worker1 # [ 144.898826] nix-grpc-daemon[1601]: time=2026-09-20T10:52:52.053Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6656worker1 # [ 144.900315] nix-grpc-daemon[1601]: time=2026-09-20T10:52:52.055Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6657lb # [ 144.926738] niks3-server[1018]: 2026/09/20 10:52:51 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6658lb # [ 144.929679] niks3-server[1018]: 2026/09/20 10:52:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6659lb # [ 144.932029] niks3-server[1018]: 2026/09/20 10:52:51 INFO Signed narinfos id=58 count=16660worker1 # [ 144.915732] nix-grpc-daemon[1601]: time=2026-09-20T10:52:52.070Z level=INFO msg="Uploading 1 narinfos"6661lb # [ 144.935059] niks3-server[1018]: 2026/09/20 10:52:51 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6662lb # [ 144.939059] niks3-server[1018]: 2026/09/20 10:52:51 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6663lb # [ 144.941932] niks3-server[1018]: 2026/09/20 10:52:51 INFO Completed upload id=586664worker1 # [ 144.925611] nix-grpc-daemon[1601]: time=2026-09-20T10:52:52.080Z level=INFO msg="Upload complete. (368ms)"6665lb # [ 144.944398] niks3-server[1018]: 2026/09/20 10:52:51 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6666worker1 # [ 144.927532] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=7526667worker1 # [ 144.930391] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386668worker1 # [ 144.936055] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6669worker1 # [ 144.939951] nix-daemon[845]: reaped child process 1984, status = succeeded6670worker1 # [ 144.944821] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6671worker1 # [ 145.758435] nix-daemon[845]: reaped child process 1998, status = succeeded6672worker1 # [ 145.891081] nix-daemon[845]: accepted connection from pid 2013, user nix-grpc-daemon (trusted)6673worker1 # [ 145.902937] nix-daemon[845]: reaped child process 2016, status = succeeded6674lb # [ 145.934582] niks3-server[1018]: 2026/09/20 10:52:52 INFO Received uploads request method=POST path=/api/pending_closures6675worker1 # [ 145.923525] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.077Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6676worker1 # [ 145.924846] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.077Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6677lb # [ 145.950366] niks3-server[1018]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6678lb # [ 145.953121] niks3-server[1018]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6679lb # [ 145.956483] niks3-server[1018]: 2026/09/20 10:52:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6680lb # [ 145.958243] niks3-server[1018]: 2026/09/20 10:52:52 INFO Signed narinfos id=59 count=16681worker1 # [ 145.943301] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.097Z level=INFO msg="Uploading 1 narinfos"6682lb # [ 145.962937] niks3-server[1018]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6683lb # [ 145.965817] niks3-server[1018]: 2026/09/20 10:52:52 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6684lb # [ 145.969457] niks3-server[1018]: 2026/09/20 10:52:52 INFO Completed upload id=596685worker1 # [ 145.954411] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.109Z level=INFO msg="Upload complete. (203ms)"6686lb # [ 145.971214] niks3-server[1018]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6687worker2 # [ 145.917479] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:52Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16688worker1 # [ 145.957731] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16689worker1 # [ 145.963445] nix-daemon[845]: reaped child process 1996, status = succeeded6690worker1 # [ 146.000706] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386691worker1 # [ 146.008541] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386692worker1 # [ 146.017488] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.172Z level=INFO msg="All 2 paths already cached"6693worker1 # [ 146.023121] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386694worker1 # [ 146.031985] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6695worker1 # [ 146.183327] nix-daemon[845]: accepted connection from pid 2020, user nix-grpc-daemon (trusted)6696worker1 # [ 146.195506] nix-daemon[845]: reaped child process 2023, status = succeeded6697worker1 # [ 146.344397] nix-daemon[845]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)6698worker1 # [ 146.356205] nix-daemon[845]: reaped child process 2028, status = succeeded6699lb # [ 146.387694] niks3-server[1018]: 2026/09/20 10:52:52 INFO Received uploads request method=POST path=/api/pending_closures6700worker1 # [ 146.375229] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.530Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6701worker1 # [ 146.377185] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.531Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6702lb # [ 146.412901] niks3-server[1018]: 2026/09/20 10:52:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6703lb # [ 146.417897] niks3-server[1018]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6704worker1 # [ 146.407924] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.562Z level=INFO msg="Uploading 1 narinfos"6705lb # [ 146.424706] niks3-server[1018]: 2026/09/20 10:52:52 INFO Signed narinfos id=60 count=16706lb # [ 146.431132] niks3-server[1018]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6707lb # [ 146.442673] niks3-server[1018]: 2026/09/20 10:52:52 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6708lb # [ 146.447509] niks3-server[1018]: 2026/09/20 10:52:52 INFO Completed upload id=606709worker1 # [ 146.433439] nix-grpc-daemon[1601]: time=2026-09-20T10:52:53.588Z level=INFO msg="Upload complete. (394ms)"6710worker1 # [ 146.435663] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=7526711lb # [ 146.452323] niks3-server[1018]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6712worker1 # [ 146.439890] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386713worker1 # [ 146.445614] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6714worker1 # [ 146.447915] nix-daemon[845]: reaped child process 2018, status = succeeded6715worker1 # [ 146.453225] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6716worker1 # [ 147.265670] nix-daemon[845]: reaped child process 2032, status = succeeded6717worker1 # [ 147.402726] nix-daemon[845]: accepted connection from pid 2047, user nix-grpc-daemon (trusted)6718worker1 # [ 147.414768] nix-daemon[845]: reaped child process 2050, status = succeeded6719lb # [ 147.447139] niks3-server[1018]: 2026/09/20 10:52:53 INFO Received uploads request method=POST path=/api/pending_closures6720worker1 # [ 147.439331] nix-grpc-daemon[1601]: time=2026-09-20T10:52:54.593Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6721worker1 # [ 147.440733] nix-grpc-daemon[1601]: time=2026-09-20T10:52:54.593Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6722lb # [ 147.472279] niks3-server[1018]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6723lb # [ 147.490926] niks3-server[1018]: 2026/09/20 10:52:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6724lb # [ 147.497811] niks3-server[1018]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6725lb # [ 147.503476] niks3-server[1018]: 2026/09/20 10:52:53 INFO Signed narinfos id=61 count=16726worker1 # [ 147.489885] nix-grpc-daemon[1601]: time=2026-09-20T10:52:54.644Z level=INFO msg="Uploading 1 narinfos"6727lb # [ 147.509525] niks3-server[1018]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6728lb # [ 147.516923] niks3-server[1018]: 2026/09/20 10:52:53 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6729lb # [ 147.523820] niks3-server[1018]: 2026/09/20 10:52:53 INFO Completed upload id=616730worker1 # [ 147.507649] nix-grpc-daemon[1601]: time=2026-09-20T10:52:54.662Z level=INFO msg="Upload complete. (248ms)"6731lb # [ 147.526064] niks3-server[1018]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6732worker2 # [ 147.472216] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:53Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16733worker1 # [ 147.515588] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16734worker1 # [ 147.521495] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386735worker1 # [ 147.532529] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386736worker1 # [ 147.538773] nix-daemon[845]: reaped child process 2030, status = succeeded6737worker1 # [ 147.541265] nix-grpc-daemon[1601]: time=2026-09-20T10:52:54.696Z level=INFO msg="All 2 paths already cached"6738worker1 # [ 147.544820] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414386739worker1 # [ 147.548705] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6740worker1 # [ 147.704564] nix-daemon[845]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)6741worker1 # [ 147.716743] nix-daemon[845]: reaped child process 2057, status = succeeded6742worker1 # [ 147.879261] nix-daemon[845]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)6743worker1 # [ 147.890760] nix-daemon[845]: reaped child process 2062, status = succeeded6744lb # [ 147.922800] niks3-server[1018]: 2026/09/20 10:52:54 INFO Received uploads request method=POST path=/api/pending_closures6745worker1 # [ 147.914292] nix-grpc-daemon[1601]: time=2026-09-20T10:52:55.068Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6746worker1 # [ 147.915648] nix-grpc-daemon[1601]: time=2026-09-20T10:52:55.068Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6747lb # [ 147.947667] niks3-server[1018]: 2026/09/20 10:52:54 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6748lb # [ 147.956945] niks3-server[1018]: 2026/09/20 10:52:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6749lb # [ 147.961804] niks3-server[1018]: 2026/09/20 10:52:54 INFO Signed narinfos id=62 count=16750worker1 # [ 147.948553] nix-grpc-daemon[1601]: time=2026-09-20T10:52:55.103Z level=INFO msg="Uploading 1 narinfos"6751lb # [ 147.967407] niks3-server[1018]: 2026/09/20 10:52:54 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6752lb # [ 147.979349] niks3-server[1018]: 2026/09/20 10:52:54 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6753worker1 # [ 147.969160] nix-grpc-daemon[1601]: time=2026-09-20T10:52:55.123Z level=INFO msg="Upload complete. (414ms)"6754lb # [ 147.985424] niks3-server[1018]: 2026/09/20 10:52:54 INFO Completed upload id=626755worker1 # [ 147.971113] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=0 paths=1 nar_bytes_in=7526756lb # [ 147.988480] niks3-server[1018]: 2026/09/20 10:52:54 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6757worker1 # [ 147.976332] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414386758worker1 # [ 147.984614] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6759worker1 # [ 147.985940] nix-daemon[845]: reaped child process 2052, status = succeeded6760worker1 # [ 147.991487] nix-daemon[845]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6761worker1 # [ 149.215817] nix-daemon[845]: reaped child process 2066, status = succeeded6762worker1 # [ 149.348092] nix-daemon[845]: accepted connection from pid 2081, user nix-grpc-daemon (trusted)6763worker1 # [ 149.359312] nix-daemon[845]: reaped child process 2084, status = succeeded6764lb # [ 149.387957] niks3-server[1018]: 2026/09/20 10:52:55 INFO Received uploads request method=POST path=/api/pending_closures6765worker1 # [ 149.380084] nix-grpc-daemon[1601]: time=2026-09-20T10:52:56.534Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6766worker1 # [ 149.381674] nix-grpc-daemon[1601]: time=2026-09-20T10:52:56.536Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6767lb # [ 149.419524] niks3-server[1018]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6768lb # [ 149.433858] niks3-server[1018]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6769lb # [ 149.441797] niks3-server[1018]: 2026/09/20 10:52:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6770worker1 # [ 149.432091] nix-grpc-daemon[1601]: time=2026-09-20T10:52:56.586Z level=INFO msg="Uploading 1 narinfos"6771lb # [ 149.448617] niks3-server[1018]: 2026/09/20 10:52:55 INFO Signed narinfos id=63 count=16772lb # [ 149.452638] niks3-server[1018]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6773lb # [ 149.461442] niks3-server[1018]: 2026/09/20 10:52:55 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6774lb # [ 149.466540] niks3-server[1018]: 2026/09/20 10:52:55 INFO Completed upload id=636775worker1 # [ 149.450426] nix-grpc-daemon[1601]: time=2026-09-20T10:52:56.605Z level=INFO msg="Upload complete. (252ms)"6776lb # [ 149.469520] niks3-server[1018]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6777worker1 # [ 149.456637] nix-grpc-daemon[1517]: ts=2026-09-20T10:52:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41438 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16778worker1 # [ 149.468689] nix-daemon[845]: reaped child process 2064, status = succeeded6779client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.57 seconds)6780worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6781worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6782client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16783worker2 # [ 149.756900] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:543266784worker2 # [ 149.763760] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543266785worker2 # [ 149.783410] nix-grpc-daemon[1639]: ts=2026-09-20T10:52:56Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06786lb # [ 157.285505] envoy[908]: conn peer=[2001:db8:1::1]:43488 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27025 tx=10636 ms=153326787client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.29 seconds)6788(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.92 seconds)6789subtest: low disk drains a worker and builds go to the other6790worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6791worker1 # [ 180.721984] nix-grpc-daemon[1517]: ts=2026-09-20T10:53:27Z level=info event=unhealthy reason=min_free6792worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.97 seconds)6793worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6794worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.05 seconds)6795client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26796worker1 # [ 181.090806] nix-grpc-daemon[1517]: ts=2026-09-20T10:53:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414386797client # don't know how to build these paths:6798client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6799worker1 # [ 181.099676] nix-grpc-daemon[1517]: ts=2026-09-20T10:53:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414386800worker2 # [ 181.081879] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:27Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06801worker2 # [ 181.087669] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:27Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16802worker2 # [ 181.131432] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543266803worker2 # [ 181.144577] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543266804client # copying 1 paths to 'lb:50051'...6805client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6806worker2 # [ 181.156850] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:543266807worker2 # [ 181.165939] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6808worker2 # [ 181.186841] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:27Z level=info event=niks3_push_started pid=17166809worker2 # [ 181.521978] nix-daemon[841]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6810worker2 # [ 181.532853] nix-daemon[841]: reaped child process 1732, status = succeeded6811worker2 # [ 181.686363] nix-daemon[841]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6812worker2 # [ 181.698366] nix-daemon[841]: reaped child process 1737, status = succeeded6813lb # [ 181.769198] niks3-server[1018]: 2026/09/20 10:53:28 INFO Received uploads request method=POST path=/api/pending_closures6814worker2 # [ 181.718765] nix-grpc-daemon[1716]: time=2026-09-20T10:53:28.163Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6815worker2 # [ 181.721561] nix-grpc-daemon[1716]: time=2026-09-20T10:53:28.164Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6816lb # [ 181.790905] niks3-server[1018]: 2026/09/20 10:53:28 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6817lb # [ 181.795268] niks3-server[1018]: 2026/09/20 10:53:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6818lb # [ 181.797595] niks3-server[1018]: 2026/09/20 10:53:28 INFO Signed narinfos id=64 count=16819worker2 # [ 181.740970] nix-grpc-daemon[1716]: time=2026-09-20T10:53:28.186Z level=INFO msg="Uploading 1 narinfos"6820lb # [ 181.801825] niks3-server[1018]: 2026/09/20 10:53:28 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6821lb # [ 181.809422] niks3-server[1018]: 2026/09/20 10:53:28 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6822lb # [ 181.813334] niks3-server[1018]: 2026/09/20 10:53:28 INFO Completed upload id=646823worker2 # [ 181.756969] nix-grpc-daemon[1716]: time=2026-09-20T10:53:28.202Z level=INFO msg="Upload complete. (386ms)"6824lb # [ 181.815316] niks3-server[1018]: 2026/09/20 10:53:28 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6825worker2 # [ 181.759124] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54326 duration_s=0 paths=1 nar_bytes_in=5126826worker2 # [ 181.763432] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543266827worker2 # [ 181.769115] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6828worker2 # [ 181.771406] nix-daemon[841]: reaped child process 1714, status = succeeded6829worker2 # [ 181.776604] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6830client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6831client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6832client # farm-a> LOG-farm-a-drain6833worker2 # [ 182.601858] nix-daemon[841]: reaped child process 1741, status = succeeded6834worker2 # [ 182.757224] nix-daemon[841]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6835worker2 # [ 182.769170] nix-daemon[841]: reaped child process 1759, status = succeeded6836lb # [ 182.841129] niks3-server[1018]: 2026/09/20 10:53:29 INFO Received uploads request method=POST path=/api/pending_closures6837worker2 # [ 182.789987] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.235Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6838worker2 # [ 182.791584] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.237Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6839lb # [ 182.861033] niks3-server[1018]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6840lb # [ 182.867628] niks3-server[1018]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6841lb # [ 182.869449] niks3-server[1018]: 2026/09/20 10:53:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6842lb # [ 182.871770] niks3-server[1018]: 2026/09/20 10:53:29 INFO Signed narinfos id=65 count=16843worker2 # [ 182.815542] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.261Z level=INFO msg="Uploading 1 narinfos"6844lb # [ 182.876635] niks3-server[1018]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6845lb # [ 182.880970] niks3-server[1018]: 2026/09/20 10:53:29 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6846lb # [ 182.883369] niks3-server[1018]: 2026/09/20 10:53:29 INFO Completed upload id=656847worker2 # [ 182.828080] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.274Z level=INFO msg="Upload complete. (244ms)"6848lb # [ 182.885761] niks3-server[1018]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6849worker2 # [ 182.830486] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54326 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16850worker2 # [ 182.837541] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:29Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06851worker2 # [ 182.839172] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:29Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16852worker2 # [ 182.842868] nix-daemon[841]: reaped child process 1739, status = succeeded6853worker2 # [ 182.881328] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543266854worker2 # [ 182.888872] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543266855worker2 # [ 182.897876] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.343Z level=INFO msg="All 1 paths already cached"6856client # copying 1 paths to 'lb:50051'...6857client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6858worker2 # [ 182.904350] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:543266859worker2 # [ 182.911323] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6860worker2 # [ 183.064969] nix-daemon[841]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6861worker2 # [ 183.077323] nix-daemon[841]: reaped child process 1768, status = succeeded6862worker2 # [ 183.228152] nix-daemon[841]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6863worker2 # [ 183.240161] nix-daemon[841]: reaped child process 1773, status = succeeded6864lb # [ 183.312144] niks3-server[1018]: 2026/09/20 10:53:29 INFO Received uploads request method=POST path=/api/pending_closures6865worker2 # [ 183.263098] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.709Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6866worker2 # [ 183.265070] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.710Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6867lb # [ 183.338503] niks3-server[1018]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6868lb # [ 183.349777] niks3-server[1018]: 2026/09/20 10:53:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6869worker2 # [ 183.298314] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.744Z level=INFO msg="Uploading 1 narinfos"6870lb # [ 183.354611] niks3-server[1018]: 2026/09/20 10:53:29 INFO Signed narinfos id=66 count=16871lb # [ 183.361362] niks3-server[1018]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6872lb # [ 183.373720] niks3-server[1018]: 2026/09/20 10:53:29 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6873worker2 # [ 183.322459] nix-grpc-daemon[1716]: time=2026-09-20T10:53:29.768Z level=INFO msg="Upload complete. (402ms)"6874lb # [ 183.378907] niks3-server[1018]: 2026/09/20 10:53:29 INFO Completed upload id=666875worker2 # [ 183.325238] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54326 duration_s=0 paths=1 nar_bytes_in=6246876lb # [ 183.382947] niks3-server[1018]: 2026/09/20 10:53:29 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6877worker2 # [ 183.330861] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543266878worker2 # [ 183.339499] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6879worker2 # [ 183.344844] nix-daemon[841]: reaped child process 1763, status = succeeded6880client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6881worker2 # [ 183.350840] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6882client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6883client # farm-b> LOG-farm-b-drain6884worker2 # [ 184.364328] nix-daemon[841]: reaped child process 1777, status = succeeded6885worker2 # [ 184.493360] nix-daemon[841]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6886worker2 # [ 184.505136] nix-daemon[841]: reaped child process 1795, status = succeeded6887lb # [ 184.577389] niks3-server[1018]: 2026/09/20 10:53:30 INFO Received uploads request method=POST path=/api/pending_closures6888worker2 # [ 184.529644] nix-grpc-daemon[1716]: time=2026-09-20T10:53:30.975Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6889worker2 # [ 184.531172] nix-grpc-daemon[1716]: time=2026-09-20T10:53:30.977Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6890lb # [ 184.616454] niks3-server[1018]: 2026/09/20 10:53:30 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6891lb # [ 184.624106] niks3-server[1018]: 2026/09/20 10:53:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6892lb # [ 184.629481] niks3-server[1018]: 2026/09/20 10:53:30 INFO Signed narinfos id=67 count=16893worker2 # [ 184.576488] nix-grpc-daemon[1716]: time=2026-09-20T10:53:31.022Z level=INFO msg="Uploading 1 narinfos"6894lb # [ 184.635728] niks3-server[1018]: 2026/09/20 10:53:30 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6895lb # [ 184.645910] niks3-server[1018]: 2026/09/20 10:53:31 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6896lb # [ 184.652457] niks3-server[1018]: 2026/09/20 10:53:31 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6897lb # [ 184.657481] niks3-server[1018]: 2026/09/20 10:53:31 INFO Completed upload id=676898worker2 # [ 184.601239] nix-grpc-daemon[1716]: time=2026-09-20T10:53:31.047Z level=INFO msg="Upload complete. (244ms)"6899lb # [ 184.661382] niks3-server[1018]: 2026/09/20 10:53:31 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6900worker2 # [ 184.608180] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54326 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16901worker2 # [ 184.619849] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:31Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06902worker2 # [ 184.623209] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:31Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16903worker2 # [ 184.631498] nix-daemon[841]: reaped child process 1775, status = succeeded6904worker2 # [ 184.662933] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543266905worker2 # [ 184.675400] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543266906worker2 # [ 184.687842] nix-grpc-daemon[1716]: time=2026-09-20T10:53:31.133Z level=INFO msg="All 2 paths already cached"6907client # copying 1 paths to 'lb:50051'...6908client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6909worker2 # [ 184.695454] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:543266910worker2 # [ 184.703649] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6911worker2 # [ 184.843065] nix-daemon[841]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6912worker2 # [ 184.855543] nix-daemon[841]: reaped child process 1804, status = succeeded6913worker2 # [ 184.981969] nix-daemon[841]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6914worker2 # [ 184.994227] nix-daemon[841]: reaped child process 1809, status = succeeded6915lb # [ 185.059647] niks3-server[1018]: 2026/09/20 10:53:31 INFO Received uploads request method=POST path=/api/pending_closures6916worker2 # [ 185.012181] nix-grpc-daemon[1716]: time=2026-09-20T10:53:31.458Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6917worker2 # [ 185.013984] nix-grpc-daemon[1716]: time=2026-09-20T10:53:31.459Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6918lb # [ 185.087422] niks3-server[1018]: 2026/09/20 10:53:31 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6919lb # [ 185.095896] niks3-server[1018]: 2026/09/20 10:53:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6920lb # [ 185.102236] niks3-server[1018]: 2026/09/20 10:53:31 INFO Signed narinfos id=68 count=16921worker2 # [ 185.048783] nix-grpc-daemon[1716]: time=2026-09-20T10:53:31.494Z level=INFO msg="Uploading 1 narinfos"6922lb # [ 185.114131] niks3-server[1018]: 2026/09/20 10:53:31 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6923lb # [ 185.124462] niks3-server[1018]: 2026/09/20 10:53:31 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6924worker2 # [ 185.074074] nix-grpc-daemon[1716]: time=2026-09-20T10:53:31.520Z level=INFO msg="Upload complete. (358ms)"6925lb # [ 185.130421] niks3-server[1018]: 2026/09/20 10:53:31 INFO Completed upload id=686926worker2 # [ 185.076143] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54326 duration_s=0 paths=1 nar_bytes_in=7526927lb # [ 185.133578] niks3-server[1018]: 2026/09/20 10:53:31 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6928worker2 # [ 185.081059] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543266929worker2 # [ 185.086559] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6930worker2 # [ 185.088806] nix-daemon[841]: reaped child process 1799, status = succeeded6931worker2 # [ 185.094095] nix-daemon[841]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6932client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6933client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6934client # farm-top> LOG-farm-top-drain6935worker2 # [ 186.309258] nix-daemon[841]: reaped child process 1813, status = succeeded6936worker2 # [ 186.419327] nix-daemon[841]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)6937worker2 # [ 186.429937] nix-daemon[841]: reaped child process 1831, status = succeeded6938lb # [ 186.501262] niks3-server[1018]: 2026/09/20 10:53:32 INFO Received uploads request method=POST path=/api/pending_closures6939worker2 # [ 186.452368] nix-grpc-daemon[1716]: time=2026-09-20T10:53:32.898Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6940worker2 # [ 186.455038] nix-grpc-daemon[1716]: time=2026-09-20T10:53:32.899Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6941lb # [ 186.532465] niks3-server[1018]: 2026/09/20 10:53:32 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6942lb # [ 186.546128] niks3-server[1018]: 2026/09/20 10:53:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6943lb # [ 186.554710] niks3-server[1018]: 2026/09/20 10:53:32 INFO Signed narinfos id=69 count=16944worker2 # [ 186.501257] nix-grpc-daemon[1716]: time=2026-09-20T10:53:32.947Z level=INFO msg="Uploading 1 narinfos"6945lb # [ 186.558495] niks3-server[1018]: 2026/09/20 10:53:32 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6946lb # [ 186.565474] niks3-server[1018]: 2026/09/20 10:53:32 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6947lb # [ 186.575465] niks3-server[1018]: 2026/09/20 10:53:32 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6948lb # [ 186.581401] niks3-server[1018]: 2026/09/20 10:53:32 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6949lb # [ 186.584467] niks3-server[1018]: 2026/09/20 10:53:32 INFO Completed upload id=696950worker2 # [ 186.529060] nix-grpc-daemon[1716]: time=2026-09-20T10:53:32.974Z level=INFO msg="Upload complete. (227ms)"6951worker2 # [ 186.537233] nix-grpc-daemon[1639]: ts=2026-09-20T10:53:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54326 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16952worker2 # [ 186.548928] nix-daemon[841]: reaped child process 1811, status = succeeded6953client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.73 seconds)6954client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6955client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.18 seconds)6956worker1: must succeed: rm /nix/.rw-store/fill6957worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.10 seconds)6958worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6959lb # [ 187.287783] envoy[908]: conn peer=[2001:db8:1::1]:51566 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300466960lb # [ 187.291575] envoy[908]: conn peer=[2001:db8:1::1]:55516 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9503 tx=3823 ms=54986961worker1 # [ 187.725381] nix-grpc-daemon[1517]: ts=2026-09-20T10:53:34Z level=info event=healthy reason=min_free6962worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)6963(finished: subtest: low disk drains a worker and builds go to the other, in 8.19 seconds)6964(finished: run the VM test script, in 188.94 seconds)6965test script finished in 189.17s6966cleanup6967kill QemuMachine (pid 49)6968client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6969kill QemuMachine (pid 50)6970lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6971kill QemuMachine (pid 51)6972worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6973kill QemuMachine (pid 52)6974worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6975(finished: cleanup, in 0.99 seconds)6976additionally exposed symbols:6977 client, lb, worker1, worker2,6978 vlan1,6979 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh