vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #298
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker2: QEMU running (pid 51)14worker2 # Disk image does not exist, creating the virtualisation disk image...15worker1: QEMU running (pid 52)16worker2 # Formatting '/build/vm-state-worker2/tmp.ZxGVfmDywN', fmt=raw size=107374182417worker1 # Formatting '/build/vm-state-worker1/tmp.EpTVvJNVTO', fmt=raw size=107374182418worker1 # mke2fs 1.47.4 (6-Mar-2025)19lb: QEMU running (pid 50)20worker2 # mke2fs 1.47.4 (6-Mar-2025)21lb # Disk image does not exist, creating the virtualisation disk image...22worker2 # Discarding device blocks: 0/262144 done23worker1 # Discarding device blocks: 0/262144 done24lb # Formatting '/build/vm-state-lb/tmp.l88YMBFsgI', fmt=raw size=107374182425worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes26worker2 # Filesystem UUID: a36464e1-8044-4057-910f-34a949df17e227worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes28worker1 # Filesystem UUID: f29c1835-3ea2-4209-b8f4-03138cda7bc529worker1 # Superblock backups stored on blocks:30client # Disk image does not exist, creating the virtualisation disk image...31client: QEMU running (pid 49)32lb # mke2fs 1.47.4 (6-Mar-2025)33client # Formatting '/build/vm-state-client/tmp.8JUQFB5rNk', fmt=raw size=107374182434lb # Discarding device blocks: 0/262144 done35client # mke2fs 1.47.4 (6-Mar-2025)36lb # Creating filesystem with 262144 4k blocks and 65536 inodes37client # Discarding device blocks: 0/262144 done38(finished: start all VMs, in 0.18 seconds)39lb # Filesystem UUID: 14aefc91-170c-494e-93c5-ab6351bc562e40client # Creating filesystem with 262144 4k blocks and 65536 inodes41client: waiting for unit network-addresses-eth1.service42lb # Superblock backups stored on blocks:43client # Filesystem UUID: 187a62d7-ed5a-415e-9968-c29f958bffd944client: waiting for the VM to finish booting45lb # 32768, 98304, 163840, 22937646client # Superblock backups stored on blocks:47worker1 # 32768, 98304, 163840, 22937648worker2 # Superblock backups stored on blocks:49lb # 50client # 32768, 98304, 163840, 22937651worker1 # 52worker2 # 32768, 98304, 163840, 22937653lb # Allocating group tables: 0/8 done54client # 55worker1 # Allocating group tables: 0/8 done56worker2 # 57lb # Writing inode tables: 0/8 done58client # Allocating group tables: 0/8 done59worker1 # Writing inode tables: 0/8 done60worker2 # Allocating group tables: 0/8 done61lb # Creating journal (8192 blocks): done62client # Writing inode tables: 0/8 done63worker1 # Creating journal (8192 blocks): done64worker2 # Writing inode tables: 0/8 done65lb # Writing superblocks and filesystem accounting information: 0/8 done66client # Creating journal (8192 blocks): done67worker1 # Writing superblocks and filesystem accounting information: 0/8 done68worker2 # Creating journal (8192 blocks): done69lb # 70client # Writing superblocks and filesystem accounting information: 0/8 done71worker1 # 72worker2 # Writing superblocks and filesystem accounting information: 0/8 done73lb # Virtualisation disk image created.74client # 75worker1 # Virtualisation disk image created.76client # Virtualisation disk image created.77worker2 # 78worker2 # Virtualisation disk image created.79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87lb # 88lb # 89lb # 90lb # 91lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0092lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...93lb # 94lb # 95worker2 # 96worker2 # 97client # 98client # 99worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00100client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00101worker1 # 102worker1 # 103worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00104worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...105worker2 # 106worker2 # 107worker2 # 108worker2 # 109client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...110client # 111client # 112client # 113client # 114worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00115client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...117worker1 # 118worker1 # 119worker1 # 120worker1 # 121worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00122worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...123worker2 # 124worker2 # 125client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126client # 127client # 128worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker1 # 130worker1 # 131lb # Booting from ROM...132worker2 # Booting from ROM...133client # Booting from ROM...134worker1 # Booting from ROM...135lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xzrwmx591i25vlijx1m7lk76hjmbxra6-nixos-system-lb-test/init regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration console=ttyS0,115200n8 console=tty0137lb # [ 0.000000] BIOS-provided physical RAM map:138lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148lb # [ 0.000000] NX (Execute Disable) protection: active149lb # [ 0.000000] APIC: Static calls initialized150lb # [ 0.000000] SMBIOS 2.8 present.151lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014152lb # [ 0.000000] DMI: Memory slots populated: 1/1153lb # [ 0.000000] Hypervisor detected: KVM154lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000155lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00156lb # [ 0.000000] kvm-clock: using sched offset of 445612371 cycles157lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns158lb # [ 0.000004] tsc: Detected 2400.012 MHz processor159lb # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000160lb # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs161lb # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT162lb # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]163lb # [ 0.002741] Using GB pages for direct mapping164lb # [ 0.002893] RAMDISK: [mem 0x5e367000-0x5ffcffff]165lb # [ 0.002900] ACPI: Early table checksum verification disabled166lb # [ 0.002903] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )167worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026168lb # [ 0.002907] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)169client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026170lb # [ 0.002911] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)171lb # [ 0.002917] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)172worker1 # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026173lb # [ 0.002920] ACPI: FACS 0x000000005FFE0000 000040174lb # [ 0.002921] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)175worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8406p3fswj6h3s9840p4nnx6zl61n26s-nixos-system-worker2-test/init regInfo=/nix/store/9gsll5rv3mr0i84a6nw4dranvpwj5n3q-closure-info/registration console=ttyS0,115200n8 console=tty0176worker2 # [ 0.000000] BIOS-provided physical RAM map:177client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test/init regInfo=/nix/store/qg7kkzbx4402mlhafr9635k4l8c0jiy9-closure-info/registration console=ttyS0,115200n8 console=tty0178lb # [ 0.002923] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)179worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable180client # [ 0.000000] BIOS-provided physical RAM map:181lb # [ 0.002924] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)182worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved183client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable184worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/41z8s7g0mnn0a1i9plh0bj17rca4hf8c-nixos-system-worker1-test/init regInfo=/nix/store/7n2qfgy34jnxzxm88039rdvpp8lh9x4i-closure-info/registration console=ttyS0,115200n8 console=tty0185worker1 # [ 0.000000] BIOS-provided physical RAM map:186worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved187lb # [ 0.002926] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)188client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved189worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable190worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable191client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved192lb # [ 0.002927] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]193worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved194worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable196lb # [ 0.002928] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]197worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved198client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved199worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved200lb # [ 0.002928] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]201worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable202client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved203lb # [ 0.002929] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]204worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved205worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved206client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved207worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved208lb # [ 0.002929] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]209worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved210client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved211worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved212lb # [ 0.002930] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]213worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved214client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved215worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved216lb # [ 0.002930] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]217worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved218worker2 # [ 0.000000] NX (Execute Disable) protection: active219lb # [ 0.003158] No NUMA configuration found220client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved221worker2 # [ 0.000000] APIC: Static calls initialized222lb # [ 0.003159] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]223client # [ 0.000000] NX (Execute Disable) protection: active224worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved225worker2 # [ 0.000000] SMBIOS 2.8 present.226client # [ 0.000000] APIC: Static calls initialized227lb # [ 0.003161] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]228client # [ 0.000000] SMBIOS 2.8 present.229worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved230lb # [ 0.005334] Zone ranges:231worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014232worker1 # [ 0.000000] NX (Execute Disable) protection: active233lb # [ 0.005335] DMA [mem 0x0000000000001000-0x0000000000ffffff]234worker2 # [ 0.000000] DMI: Memory slots populated: 1/1235worker1 # [ 0.000000] APIC: Static calls initialized236client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014237worker1 # [ 0.000000] SMBIOS 2.8 present.238lb # [ 0.005337] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]239worker2 # [ 0.000000] Hypervisor detected: KVM240client # [ 0.000000] DMI: Memory slots populated: 1/1241lb # [ 0.005338] Normal empty242client # [ 0.000000] Hypervisor detected: KVM243worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000244lb # [ 0.005339] Device empty245worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014246worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00247lb # [ 0.005339] Movable zone start for each node248client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000249worker1 # [ 0.000000] DMI: Memory slots populated: 1/1250lb # [ 0.005340] Early memory node ranges251client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00252worker2 # [ 0.000000] kvm-clock: using sched offset of 467188783 cycles253worker1 # [ 0.000000] Hypervisor detected: KVM254lb # [ 0.005340] node 0: [mem 0x0000000000001000-0x000000000009efff]255client # [ 0.000001] kvm-clock: using sched offset of 470164749 cycles256worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000257worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns258lb # [ 0.005341] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]259worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00260worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor261client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns262worker1 # [ 0.000000] kvm-clock: using sched offset of 469004665 cycles263lb # [ 0.005342] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]264client # [ 0.000005] tsc: Detected 2400.012 MHz processor265worker2 # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000266lb # [ 0.005361] On node 0, zone DMA: 1 pages in unavailable ranges267client # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000268worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns269worker2 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs270lb # [ 0.005618] On node 0, zone DMA: 97 pages in unavailable ranges271worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor272client # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs273worker2 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT274lb # [ 0.031222] On node 0, zone DMA32: 40 pages in unavailable ranges275worker1 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000276lb # [ 0.031671] ACPI: PM-Timer IO Port: 0x608277client # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT278worker2 # [ 0.002743] found SMP MP-table at [mem 0x000f5450-0x000f545f]279worker2 # [ 0.002754] Using GB pages for direct mapping280client # [ 0.002732] found SMP MP-table at [mem 0x000f5450-0x000f545f]281lb # [ 0.031683] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])282worker1 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs283worker2 # [ 0.002874] RAMDISK: [mem 0x5e367000-0x5ffcffff]284client # [ 0.002743] Using GB pages for direct mapping285lb # [ 0.031708] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23286worker1 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT287client # [ 0.002826] RAMDISK: [mem 0x5e366000-0x5ffcffff]288worker2 # [ 0.002881] ACPI: Early table checksum verification disabled289lb # [ 0.031711] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)290worker1 # [ 0.002732] found SMP MP-table at [mem 0x000f5450-0x000f545f]291client # [ 0.002833] ACPI: Early table checksum verification disabled292worker2 # [ 0.002884] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )293worker1 # [ 0.002743] Using GB pages for direct mapping294lb # [ 0.031713] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)295client # [ 0.002836] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )296worker1 # [ 0.002859] RAMDISK: [mem 0x5e367000-0x5ffcffff]297worker2 # [ 0.002888] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)298lb # [ 0.031714] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)299worker1 # [ 0.002867] ACPI: Early table checksum verification disabled300client # [ 0.002840] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)301lb # [ 0.031715] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)302worker2 # [ 0.002892] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)303worker1 # [ 0.002870] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )304client # [ 0.002843] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)305lb # [ 0.031715] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)306worker2 # [ 0.002899] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)307worker1 # [ 0.002874] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)308client # [ 0.002850] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)309lb # [ 0.031718] ACPI: Using ACPI (MADT) for SMP configuration information310worker2 # [ 0.002901] ACPI: FACS 0x000000005FFE0000 000040311worker1 # [ 0.002877] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)312client # [ 0.002852] ACPI: FACS 0x000000005FFE0000 000040313lb # [ 0.031719] ACPI: HPET id: 0x8086a201 base: 0xfed00000314worker2 # [ 0.002904] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)315lb # [ 0.031721] TSC deadline timer available316client # [ 0.002853] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)317worker1 # [ 0.002884] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)318lb # [ 0.031726] CPU topo: Max. logical packages: 1319worker2 # [ 0.002905] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)320worker1 # [ 0.002886] ACPI: FACS 0x000000005FFE0000 000040321lb # [ 0.031726] CPU topo: Max. logical dies: 1322client # [ 0.002855] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)323lb # [ 0.031727] CPU topo: Max. dies per package: 1324worker2 # [ 0.002907] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)325worker1 # [ 0.002888] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)326lb # [ 0.031730] CPU topo: Max. threads per core: 1327client # [ 0.002856] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)328lb # [ 0.031731] CPU topo: Num. cores per package: 1329worker2 # [ 0.002908] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)330worker1 # [ 0.002889] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)331lb # [ 0.031731] CPU topo: Num. threads per package: 1332client # [ 0.002858] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker2 # [ 0.002910] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]334worker1 # [ 0.002891] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)335lb # [ 0.031731] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs336client # [ 0.002859] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]337worker2 # [ 0.002911] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]338lb # [ 0.031748] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()339worker1 # [ 0.002893] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)340client # [ 0.002860] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]341worker2 # [ 0.002911] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]342lb # [ 0.031776] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]343worker1 # [ 0.002894] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]344client # [ 0.002861] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]345worker2 # [ 0.002912] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]346lb # [ 0.031778] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]347worker1 # [ 0.002895] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]348client # [ 0.002861] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]349worker2 # [ 0.002912] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]350lb # [ 0.031779] [mem 0x60000000-0xafffffff] available for PCI devices351client # [ 0.002862] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]352worker1 # [ 0.002895] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]353lb # [ 0.031781] Booting paravirtualized kernel on KVM354worker2 # [ 0.002913] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]355worker1 # [ 0.002896] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]356client # [ 0.002862] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]357worker2 # [ 0.002913] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]358worker1 # [ 0.002896] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]359client # [ 0.002863] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]360lb # [ 0.031783] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns361worker2 # [ 0.003133] No NUMA configuration found362client # [ 0.003081] No NUMA configuration found363worker1 # [ 0.002897] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]364worker2 # [ 0.003134] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]365lb # [ 0.036236] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1366client # [ 0.003082] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]367worker2 # [ 0.003136] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]368worker1 # [ 0.002897] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]369lb # [ 0.038490] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152370client # [ 0.003084] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]371worker2 # [ 0.005332] Zone ranges:372worker1 # [ 0.003114] No NUMA configuration found373client # [ 0.005280] Zone ranges:374lb # [ 0.038531] kvm-guest: PV spinlocks disabled, single CPU375worker2 # [ 0.005333] DMA [mem 0x0000000000001000-0x0000000000ffffff]376worker1 # [ 0.003115] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]377client # [ 0.005281] DMA [mem 0x0000000000001000-0x0000000000ffffff]378worker2 # [ 0.005335] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]379worker1 # [ 0.003118] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]380client # [ 0.005282] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]381worker2 # [ 0.005336] Normal empty382worker1 # [ 0.005285] Zone ranges:383client # [ 0.005283] Normal empty384worker2 # [ 0.005336] Device empty385client # [ 0.005284] Device empty386worker1 # [ 0.005286] DMA [mem 0x0000000000001000-0x0000000000ffffff]387worker2 # [ 0.005337] Movable zone start for each node388client # [ 0.005285] Movable zone start for each node389worker2 # [ 0.005338] Early memory node ranges390client # [ 0.005285] Early memory node ranges391worker1 # [ 0.005287] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]392worker1 # [ 0.005288] Normal empty393worker2 # [ 0.005338] node 0: [mem 0x0000000000001000-0x000000000009efff]394client # [ 0.005286] node 0: [mem 0x0000000000001000-0x000000000009efff]395lb # [ 0.038533] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xzrwmx591i25vlijx1m7lk76hjmbxra6-nixos-system-lb-test/init regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration console=ttyS0,115200n8 console=tty0396worker1 # [ 0.005289] Device empty397worker2 # [ 0.005339] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]398worker1 # [ 0.005289] Movable zone start for each node399client # [ 0.005287] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]400worker1 # [ 0.005290] Early memory node ranges401worker2 # [ 0.005340] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]402client # [ 0.005288] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]403worker1 # [ 0.005290] node 0: [mem 0x0000000000001000-0x000000000009efff]404lb # [ 0.038622] Unknown kernel command line parameters "regInfo=/nix/store/5b1jdrkcd54zycf7j131lh8wai3gzg44-closure-info/registration", will be passed to user space.405worker2 # [ 0.005358] On node 0, zone DMA: 1 pages in unavailable ranges406lb # [ 0.038635] random: crng init done407client # [ 0.005307] On node 0, zone DMA: 1 pages in unavailable ranges408worker1 # [ 0.005291] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]409worker2 # [ 0.005616] On node 0, zone DMA: 97 pages in unavailable ranges410client # [ 0.005561] On node 0, zone DMA: 97 pages in unavailable ranges411lb # [ 0.038635] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes412worker1 # [ 0.005292] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]413worker2 # [ 0.031465] On node 0, zone DMA32: 40 pages in unavailable ranges414client # [ 0.031213] On node 0, zone DMA32: 40 pages in unavailable ranges415lb # [ 0.040694] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)416worker1 # [ 0.005311] On node 0, zone DMA: 1 pages in unavailable ranges417worker2 # [ 0.031919] ACPI: PM-Timer IO Port: 0x608418client # [ 0.031659] ACPI: PM-Timer IO Port: 0x608419worker1 # [ 0.005565] On node 0, zone DMA: 97 pages in unavailable ranges420worker2 # [ 0.031929] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])421client # [ 0.031669] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])422lb # [ 0.041737] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)423lb # [ 0.041772] Fallback order for Node 0: 0424worker1 # [ 0.031202] On node 0, zone DMA32: 40 pages in unavailable ranges425worker2 # [ 0.031955] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23426client # [ 0.031693] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23427worker1 # [ 0.031659] ACPI: PM-Timer IO Port: 0x608428lb # [ 0.041775] Built 1 zonelists, mobility grouping on. Total pages: 393078429worker2 # [ 0.031958] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)430client # [ 0.031695] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)431lb # [ 0.041776] Policy zone: DMA32432worker1 # [ 0.031669] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])433worker2 # [ 0.031959] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)434client # [ 0.031697] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)435lb # [ 0.044200] mem auto-init: stack:all(zero), heap alloc:on, heap free:off436worker1 # [ 0.031695] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23437worker2 # [ 0.031960] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)438client # [ 0.031698] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)439worker1 # [ 0.031698] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)440lb # [ 0.047168] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1441lb # [ 0.050560] allocated 3145728 bytes of page_ext442worker2 # [ 0.031961] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)443client # [ 0.031699] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)444worker1 # [ 0.031700] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)445lb # [ 0.060219] ftrace: allocating 48731 entries in 192 pages446worker2 # [ 0.031962] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)447client # [ 0.031700] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)448worker1 # [ 0.031701] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)449lb # [ 0.060221] ftrace: allocated 192 pages with 2 groups450worker2 # [ 0.031964] ACPI: Using ACPI (MADT) for SMP configuration information451lb # [ 0.061033] Dynamic Preempt: lazy452client # [ 0.031702] ACPI: Using ACPI (MADT) for SMP configuration information453worker1 # [ 0.031702] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)454worker2 # [ 0.031965] ACPI: HPET id: 0x8086a201 base: 0xfed00000455client # [ 0.031703] ACPI: HPET id: 0x8086a201 base: 0xfed00000456lb # [ 0.061195] rcu: Preemptible hierarchical RCU implementation.457worker1 # [ 0.031703] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)458client # [ 0.031706] TSC deadline timer available459worker2 # [ 0.031968] TSC deadline timer available460lb # [ 0.061196] rcu: RCU event tracing is enabled.461worker2 # [ 0.031972] CPU topo: Max. logical packages: 1462client # [ 0.031710] CPU topo: Max. logical packages: 1463worker1 # [ 0.031705] ACPI: Using ACPI (MADT) for SMP configuration information464lb # [ 0.061197] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.465worker2 # [ 0.031973] CPU topo: Max. logical dies: 1466client # [ 0.031711] CPU topo: Max. logical dies: 1467worker1 # [ 0.031706] ACPI: HPET id: 0x8086a201 base: 0xfed00000468worker2 # [ 0.031974] CPU topo: Max. dies per package: 1469lb # [ 0.061198] Trampoline variant of Tasks RCU enabled.470client # [ 0.031712] CPU topo: Max. dies per package: 1471worker1 # [ 0.031709] TSC deadline timer available472lb # [ 0.061198] Rude variant of Tasks RCU enabled.473worker2 # [ 0.031977] CPU topo: Max. threads per core: 1474client # [ 0.031715] CPU topo: Max. threads per core: 1475worker1 # [ 0.031713] CPU topo: Max. logical packages: 1476lb # [ 0.061199] Tracing variant of Tasks RCU enabled.477worker2 # [ 0.031978] CPU topo: Num. cores per package: 1478client # [ 0.031716] CPU topo: Num. cores per package: 1479worker1 # [ 0.031714] CPU topo: Max. logical dies: 1480worker2 # [ 0.031978] CPU topo: Num. threads per package: 1481client # [ 0.031716] CPU topo: Num. threads per package: 1482worker1 # [ 0.031714] CPU topo: Max. dies per package: 1483lb # [ 0.061200] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.484worker1 # [ 0.031717] CPU topo: Max. threads per core: 1485worker2 # [ 0.031978] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs486client # [ 0.031716] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs487lb # [ 0.061200] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1488worker1 # [ 0.031718] CPU topo: Num. cores per package: 1489client # [ 0.031743] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()490worker2 # [ 0.031994] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()491worker1 # [ 0.031718] CPU topo: Num. threads per package: 1492lb # [ 0.061221] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.493client # [ 0.031773] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]494worker2 # [ 0.032024] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]495worker1 # [ 0.031719] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs496lb # [ 0.061222] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.497client # [ 0.031775] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]498worker2 # [ 0.032026] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]499worker1 # [ 0.031736] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()500client # [ 0.031776] [mem 0x60000000-0xafffffff] available for PCI devices501worker2 # [ 0.032027] [mem 0x60000000-0xafffffff] available for PCI devices502lb # [ 0.061223] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.503worker1 # [ 0.031765] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]504client # [ 0.031778] Booting paravirtualized kernel on KVM505worker2 # [ 0.032028] Booting paravirtualized kernel on KVM506lb # [ 0.065538] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16507worker1 # [ 0.031767] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]508lb # [ 0.065808] rcu: srcu_init: Setting srcu_struct sizes based on contention.509client # [ 0.031781] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns510worker2 # [ 0.032031] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns511worker1 # [ 0.031768] [mem 0x60000000-0xafffffff] available for PCI devices512worker1 # [ 0.031769] Booting paravirtualized kernel on KVM513client # [ 0.036218] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1514worker2 # [ 0.036476] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1515lb # [ 0.065814] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns516client # [ 0.038493] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152517worker2 # [ 0.038575] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152518worker1 # [ 0.031771] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns519lb # [ 0.065910] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)520client # [ 0.038532] kvm-guest: PV spinlocks disabled, single CPU521worker2 # [ 0.038617] kvm-guest: PV spinlocks disabled, single CPU522worker1 # [ 0.036208] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1523lb # [ 0.069454] Console: colour VGA+ 80x25524lb # [ 0.069458] printk: legacy console [tty0] enabled525worker1 # [ 0.038300] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152526lb # [ 0.098861] printk: legacy console [ttyS0] enabled527worker1 # [ 0.038342] kvm-guest: PV spinlocks disabled, single CPU528lb # [ 0.197111] ACPI: Core revision 20250807529worker2 # [ 0.038618] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8406p3fswj6h3s9840p4nnx6zl61n26s-nixos-system-worker2-test/init regInfo=/nix/store/9gsll5rv3mr0i84a6nw4dranvpwj5n3q-closure-info/registration console=ttyS0,115200n8 console=tty0530lb # [ 0.198005] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns531lb # [ 0.199471] APIC: Switch to symmetric I/O mode setup532client # [ 0.038534] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test/init regInfo=/nix/store/qg7kkzbx4402mlhafr9635k4l8c0jiy9-closure-info/registration console=ttyS0,115200n8 console=tty0533lb # [ 0.200395] x2apic enabled534worker2 # [ 0.038709] Unknown kernel command line parameters "regInfo=/nix/store/9gsll5rv3mr0i84a6nw4dranvpwj5n3q-closure-info/registration", will be passed to user space.535worker1 # [ 0.038344] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/41z8s7g0mnn0a1i9plh0bj17rca4hf8c-nixos-system-worker1-test/init regInfo=/nix/store/7n2qfgy34jnxzxm88039rdvpp8lh9x4i-closure-info/registration console=ttyS0,115200n8 console=tty0536worker2 # [ 0.038722] random: crng init done537lb # [ 0.201128] APIC: Switched APIC routing to: physical x2apic538worker2 # [ 0.038723] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes539client # [ 0.038624] Unknown kernel command line parameters "regInfo=/nix/store/qg7kkzbx4402mlhafr9635k4l8c0jiy9-closure-info/registration", will be passed to user space.540worker1 # [ 0.038433] Unknown kernel command line parameters "regInfo=/nix/store/7n2qfgy34jnxzxm88039rdvpp8lh9x4i-closure-info/registration", will be passed to user space.541client # [ 0.038637] random: crng init done542worker1 # [ 0.038446] random: crng init done543worker2 # [ 0.040888] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)544client # [ 0.038638] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes545lb # [ 0.202923] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1546worker1 # [ 0.038447] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes547worker2 # [ 0.042100] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)548client # [ 0.040711] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)549worker2 # [ 0.042133] Fallback order for Node 0: 0550worker1 # [ 0.040514] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)551worker2 # [ 0.042136] Built 1 zonelists, mobility grouping on. Total pages: 393078552client # [ 0.041753] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)553lb # [ 0.203887] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns554worker2 # [ 0.042137] Policy zone: DMA32555client # [ 0.041784] Fallback order for Node 0: 0556worker1 # [ 0.041717] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)557worker1 # [ 0.041747] Fallback order for Node 0: 0558worker2 # [ 0.044574] mem auto-init: stack:all(zero), heap alloc:on, heap free:off559client # [ 0.041787] Built 1 zonelists, mobility grouping on. Total pages: 393078560lb # [ 0.205527] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)561client # [ 0.041788] Policy zone: DMA32562worker1 # [ 0.041749] Built 1 zonelists, mobility grouping on. Total pages: 393078563worker2 # [ 0.047569] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1564worker1 # [ 0.041750] Policy zone: DMA32565worker2 # [ 0.050800] allocated 3145728 bytes of page_ext566lb # [ 0.206610] x86/cpu: User Mode Instruction Prevention (UMIP) activated567worker1 # [ 0.044466] mem auto-init: stack:all(zero), heap alloc:on, heap free:off568client # [ 0.044212] mem auto-init: stack:all(zero), heap alloc:on, heap free:off569worker2 # [ 0.060614] ftrace: allocating 48731 entries in 192 pages570client # [ 0.047197] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1571worker1 # [ 0.047468] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1572worker2 # [ 0.060616] ftrace: allocated 192 pages with 2 groups573lb # [ 0.207672] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127574client # [ 0.050576] allocated 3145728 bytes of page_ext575worker2 # [ 0.061434] Dynamic Preempt: lazy576worker1 # [ 0.050690] allocated 3145728 bytes of page_ext577client # [ 0.060312] ftrace: allocating 48731 entries in 192 pages578worker1 # [ 0.060308] ftrace: allocating 48731 entries in 192 pages579worker2 # [ 0.061617] rcu: Preemptible hierarchical RCU implementation.580lb # [ 0.208524] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0581client # [ 0.060314] ftrace: allocated 192 pages with 2 groups582worker1 # [ 0.060309] ftrace: allocated 192 pages with 2 groups583worker2 # [ 0.061618] rcu: RCU event tracing is enabled.584client # [ 0.061139] Dynamic Preempt: lazy585worker1 # [ 0.061131] Dynamic Preempt: lazy586worker2 # [ 0.061618] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.587worker1 # [ 0.061260] rcu: Preemptible hierarchical RCU implementation.588lb # [ 0.209527] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto589client # [ 0.061320] rcu: Preemptible hierarchical RCU implementation.590worker2 # [ 0.061620] Trampoline variant of Tasks RCU enabled.591worker1 # [ 0.061260] rcu: RCU event tracing is enabled.592client # [ 0.061321] rcu: RCU event tracing is enabled.593worker2 # [ 0.061620] Rude variant of Tasks RCU enabled.594worker1 # [ 0.061261] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.595lb # [ 0.210524] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl596worker2 # [ 0.061620] Tracing variant of Tasks RCU enabled.597client # [ 0.061321] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.598worker1 # [ 0.061263] Trampoline variant of Tasks RCU enabled.599worker2 # [ 0.061621] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.600lb # [ 0.211524] Transient Scheduler Attacks: Vulnerable: No microcode601worker1 # [ 0.061263] Rude variant of Tasks RCU enabled.602client # [ 0.061322] Trampoline variant of Tasks RCU enabled.603worker1 # [ 0.061263] Tracing variant of Tasks RCU enabled.604worker2 # [ 0.061622] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1605client # [ 0.061323] Rude variant of Tasks RCU enabled.606lb # [ 0.212523] Spectre V2 : Mitigation: Enhanced / Automatic IBRS607client # [ 0.061323] Tracing variant of Tasks RCU enabled.608worker1 # [ 0.061264] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.609worker2 # [ 0.061631] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.610lb # [ 0.213523] Speculative Return Stack Overflow: Mitigation: Safe RET611worker1 # [ 0.061265] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1612client # [ 0.061324] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.613worker2 # [ 0.061633] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.614client # [ 0.061325] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1615worker1 # [ 0.061282] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.616worker2 # [ 0.061634] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.617client # [ 0.061339] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.618worker1 # [ 0.061283] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.619worker2 # [ 0.065938] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16620client # [ 0.061341] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.621worker2 # [ 0.066397] rcu: srcu_init: Setting srcu_struct sizes based on contention.622worker1 # [ 0.061284] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623lb # [ 0.214523] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization624worker1 # [ 0.065753] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16625client # [ 0.061341] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.626lb # [ 0.216530] Spectre V2 : Enabling IBPB for BPF627worker2 # [ 0.066402] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns628client # [ 0.065684] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16629worker1 # [ 0.066055] rcu: srcu_init: Setting srcu_struct sizes based on contention.630worker2 # [ 0.066505] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)631lb # [ 0.217235] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier632client # [ 0.065962] rcu: srcu_init: Setting srcu_struct sizes based on contention.633worker2 # [ 0.070034] Console: colour VGA+ 80x25634worker1 # [ 0.066061] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns635lb # [ 0.218524] active return thunk: srso_alias_return_thunk636worker2 # [ 0.070037] printk: legacy console [tty0] enabled637client # [ 0.065968] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns638worker2 # [ 0.099419] printk: legacy console [ttyS0] enabled639worker1 # [ 0.066163] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)640worker1 # [ 0.069687] Console: colour VGA+ 80x25641lb # [ 0.219545] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'642worker2 # [ 0.195452] ACPI: Core revision 20250807643client # [ 0.066068] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)644worker1 # [ 0.069690] printk: legacy console [tty0] enabled645client # [ 0.069577] Console: colour VGA+ 80x25646worker1 # [ 0.099135] printk: legacy console [ttyS0] enabled647lb # [ 0.220523] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'648client # [ 0.069579] printk: legacy console [tty0] enabled649worker2 # [ 0.196298] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns650worker1 # [ 0.194476] ACPI: Core revision 20250807651lb # [ 0.221523] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'652client # [ 0.098934] printk: legacy console [ttyS0] enabled653client # [ 0.196419] ACPI: Core revision 20250807654worker2 # [ 0.197776] APIC: Switch to symmetric I/O mode setup655lb # [ 0.222523] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'656worker2 # [ 0.198707] x2apic enabled657worker1 # [ 0.195333] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns658lb # [ 0.223523] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'659client # [ 0.197276] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns660worker1 # [ 0.196872] APIC: Switch to symmetric I/O mode setup661worker2 # [ 0.199444] APIC: Switched APIC routing to: physical x2apic662lb # [ 0.224523] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'663worker1 # [ 0.197808] x2apic enabled664client # [ 0.198831] APIC: Switch to symmetric I/O mode setup665client # [ 0.199772] x2apic enabled666worker1 # [ 0.198538] APIC: Switched APIC routing to: physical x2apic667lb # [ 0.225523] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'668worker2 # [ 0.201243] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1669client # [ 0.200528] APIC: Switched APIC routing to: physical x2apic670lb # [ 0.226524] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256671worker2 # [ 0.202212] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns672worker1 # [ 0.200308] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1673lb # [ 0.227523] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64674client # [ 0.202380] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1675lb # [ 0.228523] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512676worker2 # [ 0.203842] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)677worker1 # [ 0.201278] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns678lb # [ 0.229523] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024679worker2 # [ 0.204932] x86/cpu: User Mode Instruction Prevention (UMIP) activated680client # [ 0.203335] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns681worker1 # [ 0.202910] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)682lb # [ 0.230524] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8683worker2 # [ 0.205966] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127684client # [ 0.204939] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)685worker1 # [ 0.203993] x86/cpu: User Mode Instruction Prevention (UMIP) activated686worker2 # [ 0.206839] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0687lb # [ 0.231523] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.688client # [ 0.206021] x86/cpu: User Mode Instruction Prevention (UMIP) activated689worker1 # [ 0.205035] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127690client # [ 0.207054] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127691worker2 # [ 0.207843] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto692worker1 # [ 0.205908] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0693client # [ 0.207937] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0694worker2 # [ 0.208840] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl695worker1 # [ 0.206911] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto696worker2 # [ 0.209839] Transient Scheduler Attacks: Vulnerable: No microcode697client # [ 0.208940] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto698worker1 # [ 0.207908] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl699worker2 # [ 0.210839] Spectre V2 : Mitigation: Enhanced / Automatic IBRS700client # [ 0.209936] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl701worker1 # [ 0.208908] Transient Scheduler Attacks: Vulnerable: No microcode702worker2 # [ 0.211839] Speculative Return Stack Overflow: Mitigation: Safe RET703client # [ 0.210937] Transient Scheduler Attacks: Vulnerable: No microcode704worker1 # [ 0.209907] Spectre V2 : Mitigation: Enhanced / Automatic IBRS705client # [ 0.211936] Spectre V2 : Mitigation: Enhanced / Automatic IBRS706worker1 # [ 0.210908] Speculative Return Stack Overflow: Mitigation: Safe RET707client # [ 0.212936] Speculative Return Stack Overflow: Mitigation: Safe RET708worker2 # [ 0.212839] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization709worker2 # [ 0.214847] Spectre V2 : Enabling IBPB for BPF710worker1 # [ 0.211907] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization711worker2 # [ 0.215840] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier712client # [ 0.213936] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization713worker1 # [ 0.213915] Spectre V2 : Enabling IBPB for BPF714worker2 # [ 0.216840] active return thunk: srso_alias_return_thunk715client # [ 0.215944] Spectre V2 : Enabling IBPB for BPF716worker1 # [ 0.214908] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier717worker2 # [ 0.217861] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'718client # [ 0.216937] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier719worker1 # [ 0.215908] active return thunk: srso_alias_return_thunk720worker2 # [ 0.218839] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'721client # [ 0.217937] active return thunk: srso_alias_return_thunk722worker1 # [ 0.216928] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'723worker2 # [ 0.219839] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'724client # [ 0.218957] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'725worker1 # [ 0.217907] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'726worker2 # [ 0.220839] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'727client # [ 0.219936] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'728worker1 # [ 0.218907] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'729worker2 # [ 0.221839] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'730client # [ 0.220936] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'731worker1 # [ 0.219907] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'732worker2 # [ 0.222839] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'733client # [ 0.221937] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'734worker1 # [ 0.220908] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'735worker2 # [ 0.223839] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'736client # [ 0.222936] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'737worker1 # [ 0.221907] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'738worker2 # [ 0.224840] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256739client # [ 0.223936] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'740worker1 # [ 0.222907] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'741worker2 # [ 0.226552] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64742client # [ 0.224936] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'743worker1 # [ 0.223908] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256744worker2 # [ 0.226839] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512745client # [ 0.225937] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256746worker1 # [ 0.224907] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64747worker2 # [ 0.227839] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024748client # [ 0.226936] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64749worker1 # [ 0.225907] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512750worker2 # [ 0.228839] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8751client # [ 0.227936] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512752worker1 # [ 0.226908] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024753client # [ 0.228936] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024754worker2 # [ 0.229839] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.755worker1 # [ 0.227907] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8756client # [ 0.229936] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8757worker1 # [ 0.228907] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.758client # [ 0.230936] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759lb # [ 0.265273] Freeing SMP alternatives memory: 44K760lb # [ 0.265526] pid_max: default: 32768 minimum: 301761lb # [ 0.266606] LSM: initializing lsm=capability,landlock,yama,bpf,ima762lb # [ 0.267636] landlock: Up and running.763lb # [ 0.268525] Yama: becoming mindful.764lb # [ 0.269616] LSM support for eBPF active765lb # [ 0.270390] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766lb # [ 0.271566] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767lb # [ 0.273968] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)768lb # [ 0.275065] Performance Events: Fam17h+ core perfctr, AMD PMU driver.769lb # [ 0.275533] ... version: 2770lb # [ 0.276268] ... bit width: 48771lb # [ 0.276526] ... generic counters: 6772lb # [ 0.277265] ... generic bitmap: 000000000000003f773lb # [ 0.277525] ... fixed-purpose counters: 0774lb # [ 0.278264] ... fixed-purpose bitmap: 0000000000000000775lb # [ 0.278526] ... value mask: 0000ffffffffffff776lb # [ 0.279460] ... max period: 00007fffffffffff777lb # [ 0.280241] ... global_ctrl mask: 000000000000003f778lb # [ 0.280621] signal: max sigframe size: 3376779lb # [ 0.281433] rcu: Hierarchical SRCU implementation.780lb # [ 0.282161] rcu: Max phase no-delay instances is 400.781lb # [ 0.287027] smp: Bringing up secondary CPUs ...782worker2 # [ 0.264763] Freeing SMP alternatives memory: 44K783lb # [ 0.287540] smp: Brought up 1 node, 1 CPU784worker2 # [ 0.265392] pid_max: default: 32768 minimum: 301785lb # [ 0.288210] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)786worker1 # [ 0.262271] Freeing SMP alternatives memory: 44K787worker1 # [ 0.262910] pid_max: default: 32768 minimum: 301788worker2 # [ 0.265942] LSM: initializing lsm=capability,landlock,yama,bpf,ima789client # [ 0.265673] Freeing SMP alternatives memory: 44K790worker2 # [ 0.266938] landlock: Up and running.791worker1 # [ 0.263736] LSM: initializing lsm=capability,landlock,yama,bpf,ima792client # [ 0.265939] pid_max: default: 32768 minimum: 301793worker2 # [ 0.267840] Yama: becoming mindful.794lb # [ 0.288624] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)795worker1 # [ 0.264967] landlock: Up and running.796worker2 # [ 0.268932] LSM support for eBPF active797lb # [ 0.289753] devtmpfs: initialized798client # [ 0.267017] LSM: initializing lsm=capability,landlock,yama,bpf,ima799worker1 # [ 0.265571] Yama: becoming mindful.800client # [ 0.268037] landlock: Up and running.801lb # [ 0.290555] x86/mm: Memory block size: 128MB802worker1 # [ 0.266120] LSM support for eBPF active803worker2 # [ 0.269651] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)804client # [ 0.268937] Yama: becoming mindful.805client # [ 0.269754] LSM support for eBPF active806worker1 # [ 0.267025] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)807worker2 # [ 0.270840] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)808lb # [ 0.292314] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)809client # [ 0.270497] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)810worker1 # [ 0.267946] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)811lb # [ 0.293428] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).812worker2 # [ 0.273002] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)813client # [ 0.270978] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)814lb # [ 0.294614] pinctrl core: initialized pinctrl subsystem815worker1 # [ 0.270512] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)816worker2 # [ 0.274397] Performance Events: Fam17h+ core perfctr, AMD PMU driver.817lb # [ 0.295749] PM: RTC time: 10:50:33, date: 2026-09-20818worker2 # [ 0.274848] ... version: 2819client # [ 0.274027] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820worker1 # [ 0.271434] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821worker2 # [ 0.275521] ... bit width: 48822worker1 # [ 0.271912] ... version: 2823worker2 # [ 0.275842] ... generic counters: 6824worker1 # [ 0.272615] ... bit width: 48825client # [ 0.275544] Performance Events: Fam17h+ core perfctr, AMD PMU driver.826worker2 # [ 0.276525] ... generic bitmap: 000000000000003f827worker1 # [ 0.272910] ... generic counters: 6828client # [ 0.275946] ... version: 2829worker2 # [ 0.276842] ... fixed-purpose counters: 0830client # [ 0.276634] ... bit width: 48831worker1 # [ 0.273584] ... generic bitmap: 000000000000003f832lb # [ 0.299414] NET: Registered PF_NETLINK/PF_ROUTE protocol family833worker2 # [ 0.277518] ... fixed-purpose bitmap: 0000000000000000834client # [ 0.276939] ... generic counters: 6835worker1 # [ 0.273921] ... fixed-purpose counters: 0836worker2 # [ 0.277842] ... value mask: 0000ffffffffffff837worker1 # [ 0.274610] ... fixed-purpose bitmap: 0000000000000000838client # [ 0.277685] ... generic bitmap: 000000000000003f839lb # [ 0.300798] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations840worker2 # [ 0.278732] ... max period: 00007fffffffffff841client # [ 0.277939] ... fixed-purpose counters: 0842worker1 # [ 0.274910] ... value mask: 0000ffffffffffff843lb # [ 0.301545] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations844worker2 # [ 0.279482] ... global_ctrl mask: 000000000000003f845client # [ 0.278601] ... fixed-purpose bitmap: 0000000000000000846worker1 # [ 0.275753] ... max period: 00007fffffffffff847worker2 # [ 0.279936] signal: max sigframe size: 3376848client # [ 0.278938] ... value mask: 0000ffffffffffff849worker1 # [ 0.276539] ... global_ctrl mask: 000000000000003f850lb # [ 0.302794] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations851worker2 # [ 0.280737] rcu: Hierarchical SRCU implementation.852client # [ 0.279811] ... max period: 00007fffffffffff853worker1 # [ 0.277036] signal: max sigframe size: 3376854lb # [ 0.303535] audit: initializing netlink subsys (disabled)855worker2 # [ 0.281396] rcu: Max phase no-delay instances is 400.856worker1 # [ 0.277795] rcu: Hierarchical SRCU implementation.857client # [ 0.280645] ... global_ctrl mask: 000000000000003f858lb # [ 0.304661] thermal_sys: Registered thermal governor 'fair_share'859worker1 # [ 0.278485] rcu: Max phase no-delay instances is 400.860client # [ 0.281062] signal: max sigframe size: 3376861lb # [ 0.304664] thermal_sys: Registered thermal governor 'bang_bang'862client # [ 0.281870] rcu: Hierarchical SRCU implementation.863lb # [ 0.305527] thermal_sys: Registered thermal governor 'step_wise'864client # [ 0.282543] rcu: Max phase no-delay instances is 400.865lb # [ 0.306510] thermal_sys: Registered thermal governor 'user_space'866worker2 # [ 0.286377] smp: Bringing up secondary CPUs ...867lb # [ 0.307268] audit: type=2000 audit(1789901434.539:1): state=initialized audit_enabled=0 res=1868worker2 # [ 0.286856] smp: Brought up 1 node, 1 CPU869worker1 # [ 0.283389] smp: Bringing up secondary CPUs ...870lb # [ 0.308529] thermal_sys: Registered thermal governor 'power_allocator'871worker2 # [ 0.287516] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)872worker1 # [ 0.283925] smp: Brought up 1 node, 1 CPU873lb # [ 0.308555] cpuidle: using governor menu874worker1 # [ 0.284587] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)875client # [ 0.287405] smp: Bringing up secondary CPUs ...876lb # [ 0.310744] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5877client # [ 0.287952] smp: Brought up 1 node, 1 CPU878worker2 # [ 0.287958] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)879worker2 # [ 0.289080] devtmpfs: initialized880client # [ 0.288618] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)881lb # [ 0.311780] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]882worker1 # [ 0.285156] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)883worker2 # [ 0.289876] x86/mm: Memory block size: 128MB884worker1 # [ 0.286128] devtmpfs: initialized885lb # [ 0.312529] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry886worker1 # [ 0.286869] x86/mm: Memory block size: 128MB887client # [ 0.289054] Memory: 1454528K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)888lb # [ 0.313538] PCI: Using configuration type 1 for base access889client # [ 0.290202] devtmpfs: initialized890worker2 # [ 0.291594] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)891lb # [ 0.314589] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.892client # [ 0.291083] x86/mm: Memory block size: 128MB893worker1 # [ 0.288466] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)894worker2 # [ 0.292712] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).895worker2 # [ 0.293924] pinctrl core: initialized pinctrl subsystem896worker1 # [ 0.288934] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).897client # [ 0.292821] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)898worker2 # [ 0.295007] PM: RTC time: 10:50:33, date: 2026-09-20899worker1 # [ 0.289992] pinctrl core: initialized pinctrl subsystem900worker1 # [ 0.291074] PM: RTC time: 10:50:33, date: 2026-09-20901client # [ 0.293964] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).902client # [ 0.295018] pinctrl core: initialized pinctrl subsystem903lb # [ 0.319745] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages904client # [ 0.296090] PM: RTC time: 10:50:33, date: 2026-09-20905worker2 # [ 0.298542] NET: Registered PF_NETLINK/PF_ROUTE protocol family906lb # [ 0.320527] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page907worker1 # [ 0.294520] NET: Registered PF_NETLINK/PF_ROUTE protocol family908worker2 # [ 0.299347] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations909worker1 # [ 0.295387] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations910worker2 # [ 0.299862] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations911client # [ 0.299689] NET: Registered PF_NETLINK/PF_ROUTE protocol family912worker1 # [ 0.295929] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations913worker2 # [ 0.301114] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations914client # [ 0.300434] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations915lb # [ 0.325527] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages916worker2 # [ 0.301852] audit: initializing netlink subsys (disabled)917worker1 # [ 0.297183] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations918lb # [ 0.326526] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page919client # [ 0.300960] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations920worker1 # [ 0.297919] audit: initializing netlink subsys (disabled)921worker2 # [ 0.303043] thermal_sys: Registered thermal governor 'fair_share'922worker2 # [ 0.303045] thermal_sys: Registered thermal governor 'bang_bang'923worker1 # [ 0.299024] thermal_sys: Registered thermal governor 'fair_share'924client # [ 0.302204] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations925worker2 # [ 0.303843] thermal_sys: Registered thermal governor 'step_wise'926worker1 # [ 0.299026] thermal_sys: Registered thermal governor 'bang_bang'927client # [ 0.302947] audit: initializing netlink subsys (disabled)928worker2 # [ 0.304762] thermal_sys: Registered thermal governor 'user_space'929worker1 # [ 0.299911] thermal_sys: Registered thermal governor 'step_wise'930client # [ 0.304053] thermal_sys: Registered thermal governor 'fair_share'931worker1 # [ 0.300834] thermal_sys: Registered thermal governor 'user_space'932worker2 # [ 0.305553] audit: type=2000 audit(1789901434.567:1): state=initialized audit_enabled=0 res=1933client # [ 0.304055] thermal_sys: Registered thermal governor 'bang_bang'934worker2 # [ 0.306844] thermal_sys: Registered thermal governor 'power_allocator'935worker1 # [ 0.301636] audit: type=2000 audit(1789901434.566:1): state=initialized audit_enabled=0 res=1936client # [ 0.304939] thermal_sys: Registered thermal governor 'step_wise'937worker2 # [ 0.306866] cpuidle: using governor menu938worker1 # [ 0.302913] thermal_sys: Registered thermal governor 'power_allocator'939client # [ 0.305907] thermal_sys: Registered thermal governor 'user_space'940worker1 # [ 0.302926] cpuidle: using governor menu941worker2 # [ 0.309040] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5942client # [ 0.306677] audit: type=2000 audit(1789901434.568:1): state=initialized audit_enabled=0 res=1943worker1 # [ 0.305111] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5944client # [ 0.307942] thermal_sys: Registered thermal governor 'power_allocator'945worker2 # [ 0.310075] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]946client # [ 0.307957] cpuidle: using governor menu947lb # [ 0.336918] ACPI: Added _OSI(Module Device)948worker2 # [ 0.310845] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry949worker1 # [ 0.306140] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]950lb # [ 0.337528] ACPI: Added _OSI(Processor Device)951worker2 # [ 0.311854] PCI: Using configuration type 1 for base access952client # [ 0.310163] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5953worker1 # [ 0.306913] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry954lb # [ 0.338250] ACPI: Added _OSI(Processor Aggregator Device)955worker1 # [ 0.307921] PCI: Using configuration type 1 for base access956worker2 # [ 0.312904] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.957client # [ 0.311166] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]958worker1 # [ 0.309001] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.959client # [ 0.311941] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry960client # [ 0.312951] PCI: Using configuration type 1 for base access961client # [ 0.314104] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.962worker2 # [ 0.317975] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages963worker2 # [ 0.318843] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page964lb # [ 0.347065] ACPI: 1 ACPI AML tables successfully acquired and loaded965worker1 # [ 0.316189] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages966client # [ 0.319161] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages967worker1 # [ 0.316911] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page968client # [ 0.319940] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page969worker2 # [ 0.323843] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages970worker2 # [ 0.324841] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page971lb # [ 0.350937] ACPI: Interpreter enabled972lb # [ 0.351541] ACPI: PM: (supports S0 S3 S4 S5)973lb # [ 0.352274] ACPI: Using IOAPIC for interrupt routing974worker1 # [ 0.321911] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages975client # [ 0.324940] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages976worker1 # [ 0.322911] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page977lb # [ 0.352566] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug978client # [ 0.325938] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page979lb # [ 0.353526] PCI: Using E820 reservations for host bridge windows980lb # [ 0.356625] ACPI: Enabled 2 GPEs in block 00 to 3F981worker2 # [ 0.335211] ACPI: Added _OSI(Module Device)982worker2 # [ 0.335844] ACPI: Added _OSI(Processor Device)983worker2 # [ 0.336548] ACPI: Added _OSI(Processor Aggregator Device)984worker1 # [ 0.331559] ACPI: Added _OSI(Module Device)985worker1 # [ 0.331912] ACPI: Added _OSI(Processor Device)986worker1 # [ 0.332638] ACPI: Added _OSI(Processor Aggregator Device)987client # [ 0.336329] ACPI: Added _OSI(Module Device)988client # [ 0.336941] ACPI: Added _OSI(Processor Device)989client # [ 0.337690] ACPI: Added _OSI(Processor Aggregator Device)990lb # [ 0.365255] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])991lb # [ 0.365533] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]992lb # [ 0.366981] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]993lb # [ 0.367649] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]994worker2 # [ 0.345153] ACPI: 1 ACPI AML tables successfully acquired and loaded995worker1 # [ 0.339455] ACPI: 1 ACPI AML tables successfully acquired and loaded996lb # [ 0.368879] PCI host bridge to bus 0000:00997lb # [ 0.369530] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]998lb # [ 0.370526] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]999worker1 # [ 0.342831] ACPI: Interpreter enabled1000worker2 # [ 0.349242] ACPI: Interpreter enabled1001lb # [ 0.371527] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1002worker1 # [ 0.343370] ACPI: PM: (supports S0 S3 S4 S5)1003worker2 # [ 0.349857] ACPI: PM: (supports S0 S3 S4 S5)1004client # [ 0.346490] ACPI: 1 ACPI AML tables successfully acquired and loaded1005worker1 # [ 0.343911] ACPI: Using IOAPIC for interrupt routing1006lb # [ 0.372526] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1007worker2 # [ 0.350547] ACPI: Using IOAPIC for interrupt routing1008worker1 # [ 0.344758] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1009lb # [ 0.373526] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1010lb # [ 0.374526] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1011lb # [ 0.375527] pci_bus 0000:00: root bus resource [bus 00-ff]1012client # [ 0.350318] ACPI: Interpreter enabled1013worker2 # [ 0.352888] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1014client # [ 0.350955] ACPI: PM: (supports S0 S3 S4 S5)1015worker2 # [ 0.353843] PCI: Using E820 reservations for host bridge windows1016lb # [ 0.376481] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1017client # [ 0.351643] ACPI: Using IOAPIC for interrupt routing1018worker1 # [ 0.349910] PCI: Using E820 reservations for host bridge windows1019client # [ 0.352007] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1020lb # [ 0.377980] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1021worker1 # [ 0.350998] ACPI: Enabled 2 GPEs in block 00 to 3F1022worker2 # [ 0.356901] ACPI: Enabled 2 GPEs in block 00 to 3F1023client # [ 0.352939] PCI: Using E820 reservations for host bridge windows1024client # [ 0.356033] ACPI: Enabled 2 GPEs in block 00 to 3F1025lb # [ 0.380584] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1026worker2 # [ 0.364279] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1027lb # [ 0.381538] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1028worker1 # [ 0.359542] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1029lb # [ 0.382518] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1030worker2 # [ 0.364848] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1031worker2 # [ 0.365921] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1032worker1 # [ 0.360416] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1033lb # [ 0.383873] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1034worker2 # [ 0.366964] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1035worker1 # [ 0.360988] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1036client # [ 0.364714] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1037worker2 # [ 0.368203] PCI host bridge to bus 0000:001038worker1 # [ 0.362034] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1039lb # [ 0.385233] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1040client # [ 0.365695] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1041worker2 # [ 0.368846] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1042worker1 # [ 0.363277] PCI host bridge to bus 0000:001043worker2 # [ 0.369842] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1044worker1 # [ 0.363915] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1045client # [ 0.367505] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1046lb # [ 0.387534] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1047worker2 # [ 0.370842] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1048worker1 # [ 0.364910] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1049client # [ 0.368063] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1050lb # [ 0.388377] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1051worker2 # [ 0.371842] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1052worker1 # [ 0.365911] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1053client # [ 0.369309] PCI host bridge to bus 0000:001054lb # [ 0.389261] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1055worker2 # [ 0.372842] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1056worker1 # [ 0.366911] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1057lb # [ 0.389532] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1058client # [ 0.369943] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1059worker2 # [ 0.373842] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1060worker1 # [ 0.367910] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1061client # [ 0.370939] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1062worker2 # [ 0.374842] pci_bus 0000:00: root bus resource [bus 00-ff]1063worker1 # [ 0.368910] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1064lb # [ 0.391041] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1065client # [ 0.371939] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1066worker2 # [ 0.375750] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1067worker1 # [ 0.369910] pci_bus 0000:00: root bus resource [bus 00-ff]1068client # [ 0.372939] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1069worker1 # [ 0.370883] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1070worker2 # [ 0.377236] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1071client # [ 0.373939] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1072lb # [ 0.392534] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1073lb # [ 0.393358] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1074client # [ 0.374939] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1075worker1 # [ 0.372347] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1076client # [ 0.375939] pci_bus 0000:00: root bus resource [bus 00-ff]1077lb # [ 0.394286] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1078client # [ 0.376903] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1079lb # [ 0.395070] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1080client # [ 0.378435] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1081worker2 # [ 0.379905] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1082lb # [ 0.397541] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1083worker2 # [ 0.380855] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1084lb # [ 0.398534] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1085worker1 # [ 0.375089] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1086worker2 # [ 0.381795] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1087lb # [ 0.399469] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1088worker1 # [ 0.375963] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1089worker1 # [ 0.376915] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1090worker2 # [ 0.382937] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1091lb # [ 0.400921] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1092worker1 # [ 0.378201] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1093worker2 # [ 0.384554] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1094client # [ 0.381003] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1095client # [ 0.381951] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1096lb # [ 0.402534] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1097worker1 # [ 0.379619] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1098client # [ 0.382939] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1099lb # [ 0.403380] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1100worker2 # [ 0.386859] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1101lb # [ 0.404251] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1102client # [ 0.384134] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1103worker2 # [ 0.387685] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1104worker1 # [ 0.381941] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1105worker2 # [ 0.388560] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1106worker1 # [ 0.382784] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1107lb # [ 0.405072] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1108worker2 # [ 0.388848] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1109client # [ 0.385665] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1110worker1 # [ 0.383657] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1111worker1 # [ 0.383916] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1112lb # [ 0.406534] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1113worker2 # [ 0.390343] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1114client # [ 0.387965] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1115lb # [ 0.407383] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1116client # [ 0.388804] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1117worker1 # [ 0.385473] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1118lb # [ 0.408269] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1119worker2 # [ 0.391851] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1120client # [ 0.389648] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1121worker2 # [ 0.392673] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1122client # [ 0.389946] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1123lb # [ 0.409135] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1124worker1 # [ 0.386937] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1125worker2 # [ 0.393548] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1126worker1 # [ 0.387838] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1127client # [ 0.391461] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1128worker1 # [ 0.388658] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1129worker2 # [ 0.394396] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1130lb # [ 0.411345] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1131worker1 # [ 0.389452] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1132lb # [ 0.412148] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1133client # [ 0.392948] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1134lb # [ 0.412549] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1135client # [ 0.393812] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1136worker2 # [ 0.396850] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1137client # [ 0.394656] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1138lb # [ 0.414102] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1139worker2 # [ 0.397849] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1140worker1 # [ 0.391936] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1141worker2 # [ 0.398779] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1142client # [ 0.395506] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1143worker1 # [ 0.392917] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144lb # [ 0.415534] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1145worker1 # [ 0.393838] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1146worker2 # [ 0.400203] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1147lb # [ 0.416369] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1148lb # [ 0.417292] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1149worker1 # [ 0.395279] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150lb # [ 0.418377] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1151worker2 # [ 0.401849] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1152client # [ 0.397962] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1153client # [ 0.398946] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1154worker2 # [ 0.402671] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1155worker1 # [ 0.396935] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1156worker2 # [ 0.403545] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1157worker1 # [ 0.397750] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158client # [ 0.399882] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1159lb # [ 0.419830] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1160worker1 # [ 0.398609] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1161client # [ 0.401288] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1162lb # [ 0.421512] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1163worker2 # [ 0.404392] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1164worker1 # [ 0.399466] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1165lb # [ 0.422275] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1166client # [ 0.402965] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1167worker2 # [ 0.405859] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1168client # [ 0.403859] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1169lb # [ 0.423063] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1170worker1 # [ 0.400941] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1171worker2 # [ 0.406684] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1172client # [ 0.404660] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1173worker1 # [ 0.401780] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1174worker2 # [ 0.407543] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1175worker1 # [ 0.402644] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1176lb # [ 0.424545] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1177worker2 # [ 0.408408] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1178lb # [ 0.425385] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1179client # [ 0.405478] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1180worker1 # [ 0.403479] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1181lb # [ 0.426237] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1182client # [ 0.406963] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1183worker2 # [ 0.410859] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1184worker1 # [ 0.404932] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1185client # [ 0.407836] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1186lb # [ 0.427527] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1187worker1 # [ 0.405750] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1188worker2 # [ 0.411689] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1189client # [ 0.408666] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1190worker2 # [ 0.412864] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1191worker1 # [ 0.406619] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1192lb # [ 0.429541] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1193client # [ 0.409529] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1194lb # [ 0.430499] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1195worker2 # [ 0.414478] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1196worker1 # [ 0.407512] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1197lb # [ 0.431275] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1198client # [ 0.411963] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1199worker1 # [ 0.408942] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1200lb # [ 0.432079] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1201worker2 # [ 0.415850] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1202worker1 # [ 0.409783] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1203worker2 # [ 0.416675] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1204client # [ 0.412821] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1205lb # [ 0.433197] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1206worker1 # [ 0.410656] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1207worker2 # [ 0.417561] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1208client # [ 0.413673] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1209worker1 # [ 0.410916] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1210worker2 # [ 0.417848] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1211lb # [ 0.433721] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1212client # [ 0.414518] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1213worker1 # [ 0.412432] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1214lb # [ 0.435216] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1215worker2 # [ 0.419321] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1216client # [ 0.415961] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1217worker1 # [ 0.414233] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1218worker2 # [ 0.420854] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1219client # [ 0.416849] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1220worker1 # [ 0.414933] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1221worker2 # [ 0.421780] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1222client # [ 0.417680] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1223client # [ 0.417945] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1224worker2 # [ 0.423179] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1225worker1 # [ 0.416468] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1226worker1 # [ 0.417918] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1227client # [ 0.419527] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1228worker2 # [ 0.424849] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1229worker1 # [ 0.418749] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1230worker2 # [ 0.425666] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1231worker1 # [ 0.419649] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1232client # [ 0.421963] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1233worker2 # [ 0.426563] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1234client # [ 0.422924] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1235worker1 # [ 0.420468] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1236worker2 # [ 0.427387] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1237client # [ 0.424346] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1238lb # [ 0.435726] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1239worker1 # [ 0.422451] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1240lb # [ 0.438131] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1241worker2 # [ 0.429849] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1242worker1 # [ 0.422917] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1243client # [ 0.425968] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1244worker2 # [ 0.430675] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1245lb # [ 0.438798] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1246worker1 # [ 0.423862] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1247client # [ 0.426813] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1248worker2 # [ 0.431542] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1249lb # [ 0.440113] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1250client # [ 0.427657] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1251worker1 # [ 0.425285] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1252worker2 # [ 0.432407] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1253lb # [ 0.440764] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1254worker1 # [ 0.426417] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1255client # [ 0.428478] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1256worker2 # [ 0.433460] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1257worker1 # [ 0.427104] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1258lb # [ 0.442235] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1259worker2 # [ 0.434039] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1260client # [ 0.429962] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1261worker1 # [ 0.428426] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1262lb # [ 0.443722] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1263client # [ 0.430833] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1264worker2 # [ 0.436236] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1265client # [ 0.431752] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1266lb # [ 0.445548] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1267lb # [ 0.446415] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1268client # [ 0.433385] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1269lb # [ 0.447584] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1270client # [ 0.434586] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1271lb # [ 0.449105] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1272client # [ 0.435177] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1273client # [ 0.436680] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1274lb # [ 0.452751] ACPI: PCI: Interrupt link LNKA configured for IRQ 101275lb # [ 0.453630] ACPI: PCI: Interrupt link LNKB configured for IRQ 101276lb # [ 0.454622] ACPI: PCI: Interrupt link LNKC configured for IRQ 111277worker1 # [ 0.429106] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1278worker2 # [ 0.437036] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1279lb # [ 0.455623] ACPI: PCI: Interrupt link LNKD configured for IRQ 111280lb # [ 0.456621] ACPI: PCI: Interrupt link LNKE configured for IRQ 101281worker2 # [ 0.439470] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1282lb # [ 0.457622] ACPI: PCI: Interrupt link LNKF configured for IRQ 101283worker1 # [ 0.432514] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1284worker2 # [ 0.440078] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1285lb # [ 0.458636] ACPI: PCI: Interrupt link LNKG configured for IRQ 111286worker1 # [ 0.433150] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1287worker2 # [ 0.441438] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1288lb # [ 0.459622] ACPI: PCI: Interrupt link LNKH configured for IRQ 111289lb # [ 0.460564] ACPI: PCI: Interrupt link GSIA configured for IRQ 161290worker2 # [ 0.442286] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1291worker1 # [ 0.434768] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1292lb # [ 0.461543] ACPI: PCI: Interrupt link GSIB configured for IRQ 171293lb # [ 0.462533] ACPI: PCI: Interrupt link GSIC configured for IRQ 181294worker1 # [ 0.435891] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1295worker2 # [ 0.443138] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1296client # [ 0.437158] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1297lb # [ 0.463474] ACPI: PCI: Interrupt link GSID configured for IRQ 191298worker1 # [ 0.437168] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1299lb # [ 0.464264] ACPI: PCI: Interrupt link GSIE configured for IRQ 201300worker2 # [ 0.444088] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1301client # [ 0.439597] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1302lb # [ 0.464538] ACPI: PCI: Interrupt link GSIF configured for IRQ 211303worker1 # [ 0.438189] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1304client # [ 0.440170] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1305worker2 # [ 0.445835] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1306lb # [ 0.465507] ACPI: PCI: Interrupt link GSIG configured for IRQ 221307worker2 # [ 0.446662] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1308lb # [ 0.466247] ACPI: PCI: Interrupt link GSIH configured for IRQ 231309client # [ 0.441513] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1310worker1 # [ 0.440479] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1311worker2 # [ 0.447849] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1312lb # [ 0.467584] iommu: Default domain type: Translated1313worker1 # [ 0.440918] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1314client # [ 0.442214] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1315lb # [ 0.468347] iommu: DMA domain TLB invalidation policy: lazy mode1316worker2 # [ 0.449456] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1317worker1 # [ 0.442200] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1318lb # [ 0.469536] ACPI: bus type USB registered1319client # [ 0.443224] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1320lb # [ 0.470255] usbcore: registered new interface driver usbfs1321worker1 # [ 0.443544] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1322lb # [ 0.470552] usbcore: registered new interface driver hub1323client # [ 0.444214] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1324lb # [ 0.471407] usbcore: registered new device driver usb1325worker2 # [ 0.453021] ACPI: PCI: Interrupt link LNKA configured for IRQ 101326client # [ 0.445967] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1327lb # [ 0.473057] NetLabel: Initializing1328worker2 # [ 0.453948] ACPI: PCI: Interrupt link LNKB configured for IRQ 101329client # [ 0.446844] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1330lb # [ 0.473528] NetLabel: domain hash size = 1281331worker1 # [ 0.446829] ACPI: PCI: Interrupt link LNKA configured for IRQ 101332worker2 # [ 0.454937] ACPI: PCI: Interrupt link LNKC configured for IRQ 111333lb # [ 0.474245] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1334client # [ 0.447977] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1335worker2 # [ 0.455940] ACPI: PCI: Interrupt link LNKD configured for IRQ 111336worker1 # [ 0.447722] ACPI: PCI: Interrupt link LNKB configured for IRQ 101337lb # [ 0.474587] NetLabel: unlabeled traffic allowed by default1338lb # [ 0.475450] PCI: Using ACPI for IRQ routing1339worker2 # [ 0.456934] ACPI: PCI: Interrupt link LNKE configured for IRQ 101340worker1 # [ 0.448988] ACPI: PCI: Interrupt link LNKC configured for IRQ 111341client # [ 0.449955] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1342worker2 # [ 0.457933] ACPI: PCI: Interrupt link LNKF configured for IRQ 101343worker1 # [ 0.450009] ACPI: PCI: Interrupt link LNKD configured for IRQ 111344worker2 # [ 0.458939] ACPI: PCI: Interrupt link LNKG configured for IRQ 111345worker1 # [ 0.451009] ACPI: PCI: Interrupt link LNKE configured for IRQ 101346worker2 # [ 0.459937] ACPI: PCI: Interrupt link LNKH configured for IRQ 111347worker1 # [ 0.452010] ACPI: PCI: Interrupt link LNKF configured for IRQ 101348worker2 # [ 0.460878] ACPI: PCI: Interrupt link GSIA configured for IRQ 161349worker1 # [ 0.453008] ACPI: PCI: Interrupt link LNKG configured for IRQ 111350client # [ 0.453752] ACPI: PCI: Interrupt link LNKA configured for IRQ 101351worker2 # [ 0.461858] ACPI: PCI: Interrupt link GSIB configured for IRQ 171352worker1 # [ 0.454006] ACPI: PCI: Interrupt link LNKH configured for IRQ 111353client # [ 0.454815] ACPI: PCI: Interrupt link LNKB configured for IRQ 101354worker2 # [ 0.462857] ACPI: PCI: Interrupt link GSIC configured for IRQ 181355worker1 # [ 0.454949] ACPI: PCI: Interrupt link GSIA configured for IRQ 161356client # [ 0.455768] ACPI: PCI: Interrupt link LNKC configured for IRQ 111357worker2 # [ 0.463825] ACPI: PCI: Interrupt link GSID configured for IRQ 191358worker1 # [ 0.455896] ACPI: PCI: Interrupt link GSIB configured for IRQ 171359client # [ 0.456782] ACPI: PCI: Interrupt link LNKD configured for IRQ 111360worker2 # [ 0.464579] ACPI: PCI: Interrupt link GSIE configured for IRQ 201361worker1 # [ 0.456666] ACPI: PCI: Interrupt link GSIC configured for IRQ 181362client # [ 0.457790] ACPI: PCI: Interrupt link LNKE configured for IRQ 101363worker2 # [ 0.464857] ACPI: PCI: Interrupt link GSIF configured for IRQ 211364worker1 # [ 0.456926] ACPI: PCI: Interrupt link GSID configured for IRQ 191365worker2 # [ 0.465787] ACPI: PCI: Interrupt link GSIG configured for IRQ 221366client # [ 0.458784] ACPI: PCI: Interrupt link LNKF configured for IRQ 101367worker1 # [ 0.457925] ACPI: PCI: Interrupt link GSIE configured for IRQ 201368worker2 # [ 0.466543] ACPI: PCI: Interrupt link GSIH configured for IRQ 231369worker1 # [ 0.458922] ACPI: PCI: Interrupt link GSIF configured for IRQ 211370client # [ 0.459998] ACPI: PCI: Interrupt link LNKG configured for IRQ 111371worker1 # [ 0.459917] ACPI: PCI: Interrupt link GSIG configured for IRQ 221372client # [ 0.461050] ACPI: PCI: Interrupt link LNKH configured for IRQ 111373worker2 # [ 0.467904] iommu: Default domain type: Translated1374worker1 # [ 0.460909] ACPI: PCI: Interrupt link GSIH configured for IRQ 231375client # [ 0.461982] ACPI: PCI: Interrupt link GSIA configured for IRQ 161376worker2 # [ 0.468682] iommu: DMA domain TLB invalidation policy: lazy mode1377client # [ 0.462955] ACPI: PCI: Interrupt link GSIB configured for IRQ 171378worker2 # [ 0.469800] ACPI: bus type USB registered1379worker1 # [ 0.462851] iommu: Default domain type: Translated1380client # [ 0.463945] ACPI: PCI: Interrupt link GSIC configured for IRQ 181381worker2 # [ 0.470451] usbcore: registered new interface driver usbfs1382worker1 # [ 0.463610] iommu: DMA domain TLB invalidation policy: lazy mode1383worker2 # [ 0.470857] usbcore: registered new interface driver hub1384client # [ 0.464926] ACPI: PCI: Interrupt link GSID configured for IRQ 191385worker1 # [ 0.464157] ACPI: bus type USB registered1386worker2 # [ 0.471698] usbcore: registered new device driver usb1387client # [ 0.465660] ACPI: PCI: Interrupt link GSIE configured for IRQ 201388worker1 # [ 0.464927] usbcore: registered new interface driver usbfs1389client # [ 0.465952] ACPI: PCI: Interrupt link GSIF configured for IRQ 211390worker2 # [ 0.473330] NetLabel: Initializing1391worker1 # [ 0.465857] usbcore: registered new interface driver hub1392client # [ 0.466950] ACPI: PCI: Interrupt link GSIG configured for IRQ 221393worker2 # [ 0.473843] NetLabel: domain hash size = 1281394worker1 # [ 0.466589] usbcore: registered new device driver usb1395worker2 # [ 0.474571] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1396client # [ 0.467913] ACPI: PCI: Interrupt link GSIH configured for IRQ 231397worker2 # [ 0.474880] NetLabel: unlabeled traffic allowed by default1398worker1 # [ 0.467754] NetLabel: Initializing1399worker2 # [ 0.475786] PCI: Using ACPI for IRQ routing1400worker1 # [ 0.468347] NetLabel: domain hash size = 1281401client # [ 0.469800] iommu: Default domain type: Translated1402worker1 # [ 0.468910] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1403client # [ 0.470524] iommu: DMA domain TLB invalidation policy: lazy mode1404worker1 # [ 0.469914] NetLabel: unlabeled traffic allowed by default1405client # [ 0.471189] ACPI: bus type USB registered1406worker1 # [ 0.470863] PCI: Using ACPI for IRQ routing1407client # [ 0.471959] usbcore: registered new interface driver usbfs1408client # [ 0.472908] usbcore: registered new interface driver hub1409client # [ 0.473658] usbcore: registered new device driver usb1410client # [ 0.474746] NetLabel: Initializing1411client # [ 0.474945] NetLabel: domain hash size = 1281412client # [ 0.475687] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413client # [ 0.476007] NetLabel: unlabeled traffic allowed by default1414client # [ 0.476951] PCI: Using ACPI for IRQ routing1415lb # [ 0.519317] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416lb # [ 0.519522] pci 0000:00:01.0: vgaarb: bridge control possible1417lb # [ 0.519522] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418lb # [ 0.519530] vgaarb: loaded1419lb # [ 0.520267] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420lb # [ 0.520528] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421lb # [ 0.524609] clocksource: Switched to clocksource kvm-clock1422lb # [ 0.528300] VFS: Disk quotas dquot_6.6.01423lb # [ 0.529039] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1424lb # [ 0.530415] pnp: PnP ACPI init1425lb # [ 0.531207] ACPI: IRQ 4 override to edge(!), high(!)1426lb # [ 0.532257] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1427lb # [ 0.533832] pnp: PnP ACPI: found 6 devices1428worker2 # [ 0.519420] pci 0000:00:01.0: vgaarb: setting as boot VGA device1429worker2 # [ 0.519838] pci 0000:00:01.0: vgaarb: bridge control possible1430worker1 # [ 0.514896] pci 0000:00:01.0: vgaarb: setting as boot VGA device1431worker2 # [ 0.519838] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1432worker2 # [ 0.519847] vgaarb: loaded1433worker1 # [ 0.514906] pci 0000:00:01.0: vgaarb: bridge control possible1434lb # [ 0.541509] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1435worker2 # [ 0.520560] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01436lb # [ 0.542932] clocksource: Switched to clocksource acpi_pm1437worker1 # [ 0.514906] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1438worker2 # [ 0.520844] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1439worker1 # [ 0.514915] vgaarb: loaded1440lb # [ 0.543867] NET: Registered PF_INET protocol family1441worker1 # [ 0.515626] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01442lb # [ 0.545049] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1443worker1 # [ 0.515911] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1444worker2 # [ 0.523925] clocksource: Switched to clocksource kvm-clock1445client # [ 0.521250] pci 0000:00:01.0: vgaarb: setting as boot VGA device1446client # [ 0.521935] pci 0000:00:01.0: vgaarb: bridge control possible1447worker1 # [ 0.519997] clocksource: Switched to clocksource kvm-clock1448worker2 # [ 0.527402] VFS: Disk quotas dquot_6.6.01449client # [ 0.521935] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1450client # [ 0.521944] vgaarb: loaded1451worker2 # [ 0.528105] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1452worker1 # [ 0.521618] VFS: Disk quotas dquot_6.6.01453client # [ 0.522671] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01454worker2 # [ 0.529398] pnp: PnP ACPI init1455client # [ 0.522941] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1456worker2 # [ 0.530191] ACPI: IRQ 4 override to edge(!), high(!)1457worker2 # [ 0.531148] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1458worker1 # [ 0.524122] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1459worker1 # [ 0.525529] pnp: PnP ACPI init1460worker2 # [ 0.532649] pnp: PnP ACPI: found 6 devices1461worker1 # [ 0.526344] ACPI: IRQ 4 override to edge(!), high(!)1462client # [ 0.526025] clocksource: Switched to clocksource kvm-clock1463worker1 # [ 0.527387] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1464client # [ 0.527738] VFS: Disk quotas dquot_6.6.01465worker1 # [ 0.528992] pnp: PnP ACPI: found 6 devices1466lb # [ 0.561215] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1467client # [ 0.530261] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1468client # [ 0.531625] pnp: PnP ACPI init1469lb # [ 0.562759] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1470client # [ 0.532419] ACPI: IRQ 4 override to edge(!), high(!)1471worker2 # [ 0.540193] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1472lb # [ 0.564080] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1473client # [ 0.533398] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1474worker2 # [ 0.541603] clocksource: Switched to clocksource acpi_pm1475lb # [ 0.565392] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1476worker2 # [ 0.542543] NET: Registered PF_INET protocol family1477client # [ 0.534916] pnp: PnP ACPI: found 6 devices1478lb # [ 0.566633] TCP: Hash tables configured (established 16384 bind 16384)1479worker2 # [ 0.543669] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1480worker1 # [ 0.536573] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1481lb # [ 0.567774] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1482worker1 # [ 0.538040] clocksource: Switched to clocksource acpi_pm1483lb # [ 0.569005] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1484worker1 # [ 0.538994] NET: Registered PF_INET protocol family1485lb # [ 0.570102] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1486worker1 # [ 0.540148] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1487lb # [ 0.571308] NET: Registered PF_UNIX/PF_LOCAL protocol family1488lb # [ 0.572285] NET: Registered PF_XDP protocol family1489lb # [ 0.573163] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1490lb # [ 0.574214] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1491client # [ 0.542570] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1492client # [ 0.544053] clocksource: Switched to clocksource acpi_pm1493lb # [ 0.575240] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1494client # [ 0.545024] NET: Registered PF_INET protocol family1495lb # [ 0.576333] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1496lb # [ 0.577439] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1497client # [ 0.546236] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1498lb # [ 0.578561] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1499lb # [ 0.580271] ACPI: \_SB_.GSIA: Enabled at IRQ 161500lb # [ 0.582455] ACPI: \_SB_.GSIB: Enabled at IRQ 171501worker2 # [ 0.559944] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1502lb # [ 0.584695] ACPI: \_SB_.GSIC: Enabled at IRQ 181503worker2 # [ 0.561487] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1504lb # [ 0.586694] ACPI: \_SB_.GSID: Enabled at IRQ 191505worker2 # [ 0.562752] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1506worker1 # [ 0.556434] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1507worker2 # [ 0.564074] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1508lb # [ 0.588355] PCI: CLS 0 bytes, default 641509worker2 # [ 0.565300] TCP: Hash tables configured (established 16384 bind 16384)1510worker1 # [ 0.557979] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1511lb # [ 0.589236] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1512worker2 # [ 0.566422] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1513worker1 # [ 0.559296] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1514lb # [ 0.590910] Trying to unpack rootfs image as initramfs...1515worker2 # [ 0.567628] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1516worker1 # [ 0.560606] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1517worker2 # [ 0.568664] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1518worker1 # [ 0.561888] TCP: Hash tables configured (established 16384 bind 16384)1519worker2 # [ 0.569821] NET: Registered PF_UNIX/PF_LOCAL protocol family1520worker1 # [ 0.563028] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1521worker2 # [ 0.570766] NET: Registered PF_XDP protocol family1522client # [ 0.562523] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1523worker2 # [ 0.571556] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1524worker1 # [ 0.564279] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1525client # [ 0.564015] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1526worker2 # [ 0.572511] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1527worker1 # [ 0.565349] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1528client # [ 0.565316] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1529worker2 # [ 0.573459] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1530worker1 # [ 0.566552] NET: Registered PF_UNIX/PF_LOCAL protocol family1531worker2 # [ 0.574519] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1532worker1 # [ 0.567522] NET: Registered PF_XDP protocol family1533client # [ 0.566649] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1534worker2 # [ 0.575552] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1535worker1 # [ 0.568367] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1536client # [ 0.567868] TCP: Hash tables configured (established 16384 bind 16384)1537worker1 # [ 0.569382] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1538worker2 # [ 0.576588] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1539client # [ 0.569010] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1540worker1 # [ 0.570398] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1541worker2 # [ 0.578236] ACPI: \_SB_.GSIA: Enabled at IRQ 161542client # [ 0.570253] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1543worker1 # [ 0.571508] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1544client # [ 0.571349] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1545worker1 # [ 0.572654] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1546worker2 # [ 0.580514] ACPI: \_SB_.GSIB: Enabled at IRQ 171547client # [ 0.572568] NET: Registered PF_UNIX/PF_LOCAL protocol family1548worker1 # [ 0.573810] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1549client # [ 0.573577] NET: Registered PF_XDP protocol family1550client # [ 0.574412] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1551worker2 # [ 0.582705] ACPI: \_SB_.GSIC: Enabled at IRQ 181552worker1 # [ 0.575551] ACPI: \_SB_.GSIA: Enabled at IRQ 161553client # [ 0.575440] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1554client # [ 0.576475] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1555worker2 # [ 0.584694] ACPI: \_SB_.GSID: Enabled at IRQ 191556worker1 # [ 0.577724] ACPI: \_SB_.GSIB: Enabled at IRQ 171557client # [ 0.577578] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1558worker2 # [ 0.586377] PCI: CLS 0 bytes, default 641559worker1 # [ 0.579536] ACPI: \_SB_.GSIC: Enabled at IRQ 181560client # [ 0.578701] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1561client # [ 0.579788] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1562worker2 # [ 0.587310] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1563worker1 # [ 0.581458] ACPI: \_SB_.GSID: Enabled at IRQ 191564worker2 # [ 0.589013] Trying to unpack rootfs image as initramfs...1565client # [ 0.581571] ACPI: \_SB_.GSIA: Enabled at IRQ 161566worker1 # [ 0.583116] PCI: CLS 0 bytes, default 641567client # [ 0.584013] ACPI: \_SB_.GSIB: Enabled at IRQ 171568worker1 # [ 0.584004] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1569worker1 # [ 0.585771] Trying to unpack rootfs image as initramfs...1570client # [ 0.586418] ACPI: \_SB_.GSIC: Enabled at IRQ 181571client # [ 0.588443] ACPI: \_SB_.GSID: Enabled at IRQ 191572client # [ 0.590176] PCI: CLS 0 bytes, default 641573client # [ 0.591141] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574client # [ 0.592926] Trying to unpack rootfs image as initramfs...1575lb # [ 0.637117] Initialise system trusted keyrings1576lb # [ 0.638060] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.631495] Initialise system trusted keyrings1578worker2 # [ 0.634502] workingset: timestamp_bits=40 max_order=19 bucket_order=01579worker1 # [ 0.628689] Initialise system trusted keyrings1580lb # [ 0.660744] Key type asymmetric registered1581lb # [ 0.661469] Asymmetric key parser 'x509' registered1582lb # [ 0.662387] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1583worker1 # [ 0.631911] workingset: timestamp_bits=40 max_order=19 bucket_order=01584lb # [ 0.665857] io scheduler mq-deadline registered1585lb # [ 0.666641] io scheduler kyber registered1586client # [ 0.637247] Initialise system trusted keyrings1587lb # [ 0.670252] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1588client # [ 0.640030] workingset: timestamp_bits=40 max_order=19 bucket_order=01589lb # [ 0.671543] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1590lb # [ 0.675508] Linux agpgart interface v0.1031591lb # [ 0.676293] ACPI: bus type drm_connector registered1592worker2 # [ 0.655631] Key type asymmetric registered1593worker2 # [ 0.656316] Asymmetric key parser 'x509' registered1594worker2 # [ 0.657176] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1595lb # [ 0.681281] usbcore: registered new interface driver usbserial_generic1596lb # [ 0.682367] usbserial: USB Serial support registered for generic1597worker1 # [ 0.653061] Key type asymmetric registered1598worker1 # [ 0.653822] Asymmetric key parser 'x509' registered1599worker1 # [ 0.654735] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1600worker2 # [ 0.662651] io scheduler mq-deadline registered1601lb # [ 0.685740] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1602worker2 # [ 0.663373] io scheduler kyber registered1603lb # [ 0.685740] Please enable it if your BIOS has the CPPC option.1604lb # [ 0.687940] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1605worker2 # [ 0.665995] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1606worker1 # [ 0.659984] io scheduler mq-deadline registered1607worker2 # [ 0.667257] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1608worker1 # [ 0.660743] io scheduler kyber registered1609client # [ 0.660935] Key type asymmetric registered1610lb # [ 0.691959] drop_monitor: Initializing network drop monitor service1611lb # [ 0.693258] NET: Registered PF_INET6 protocol family1612client # [ 0.663019] Asymmetric key parser 'x509' registered1613worker1 # [ 0.664386] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1614client # [ 0.663915] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1615worker2 # [ 0.672864] Linux agpgart interface v0.1031616worker1 # [ 0.665696] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1617lb # [ 0.697191] Segment Routing with IPv61618worker2 # [ 0.673613] ACPI: bus type drm_connector registered1619lb # [ 0.697848] In-situ OAM (IOAM) with IPv61620client # [ 0.667148] io scheduler mq-deadline registered1621client # [ 0.667923] io scheduler kyber registered1622worker1 # [ 0.669631] Linux agpgart interface v0.1031623worker2 # [ 0.676844] usbcore: registered new interface driver usbserial_generic1624lb # [ 0.701012] IPI shorthand broadcast: enabled1625worker1 # [ 0.670445] ACPI: bus type drm_connector registered1626worker2 # [ 0.677898] usbserial: USB Serial support registered for generic1627client # [ 0.672138] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1628worker1 # [ 0.673281] usbcore: registered new interface driver usbserial_generic1629client # [ 0.673401] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630worker1 # [ 0.674665] usbserial: USB Serial support registered for generic1631worker2 # [ 0.681466] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1632worker2 # [ 0.681466] Please enable it if your BIOS has the CPPC option.1633worker2 # [ 0.683748] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1634worker1 # [ 0.676883] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1635worker1 # [ 0.676883] Please enable it if your BIOS has the CPPC option.1636lb # [ 0.709299] sched_clock: Marking stable (573014608, 135709552)->(785142484, -76418324)1637client # [ 0.678792] Linux agpgart interface v0.1031638worker2 # [ 0.686578] drop_monitor: Initializing network drop monitor service1639worker1 # [ 0.679176] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1640client # [ 0.679558] ACPI: bus type drm_connector registered1641worker2 # [ 0.687707] NET: Registered PF_INET6 protocol family1642lb # [ 0.713925] registered taskstats version 11643worker1 # [ 0.682979] drop_monitor: Initializing network drop monitor service1644client # [ 0.682474] usbcore: registered new interface driver usbserial_generic1645worker2 # [ 0.691118] Segment Routing with IPv61646lb # [ 0.714861] Loading compiled-in X.509 certificates1647worker1 # [ 0.684117] NET: Registered PF_INET6 protocol family1648client # [ 0.683519] usbserial: USB Serial support registered for generic1649worker2 # [ 0.693489] In-situ OAM (IOAM) with IPv61650worker2 # [ 0.695616] IPI shorthand broadcast: enabled1651client # [ 0.687007] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1652worker1 # [ 0.689002] Segment Routing with IPv61653client # [ 0.687007] Please enable it if your BIOS has the CPPC option.1654worker1 # [ 0.689665] In-situ OAM (IOAM) with IPv61655client # [ 0.689293] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1656worker1 # [ 0.692141] IPI shorthand broadcast: enabled1657client # [ 0.693138] drop_monitor: Initializing network drop monitor service1658client # [ 0.694356] NET: Registered PF_INET6 protocol family1659worker2 # [ 0.703773] sched_clock: Marking stable (570015878, 133436233)->(775609362, -72157251)1660client # [ 0.697468] Segment Routing with IPv61661lb # [ 0.731234] Demotion targets for Node 0: null1662client # [ 0.700018] In-situ OAM (IOAM) with IPv61663worker1 # [ 0.700431] sched_clock: Marking stable (567014325, 132845716)->(776645687, -76785646)1664worker2 # [ 0.708646] registered taskstats version 11665client # [ 0.701076] IPI shorthand broadcast: enabled1666worker2 # [ 0.709592] Loading compiled-in X.509 certificates1667worker1 # [ 0.703947] registered taskstats version 11668lb # [ 0.734784] Key type .fscrypt registered1669lb # [ 0.735465] Key type fscrypt-provisioning registered1670worker1 # [ 0.704914] Loading compiled-in X.509 certificates1671lb # [ 0.736453] ima: No TPM chip found, activating TPM-bypass!1672lb # [ 0.738738] ima: Allocated hash algorithm: sha11673lb # [ 0.739563] ima: No architecture policies found1674client # [ 0.709293] sched_clock: Marking stable (574015116, 134967485)->(787509542, -78526941)1675lb # [ 0.741737] PM: Magic number: 10:369:8311676lb # [ 0.743333] RAS: Correctable Errors collector initialized.1677client # [ 0.714250] registered taskstats version 11678client # [ 0.715249] Loading compiled-in X.509 certificates1679worker2 # [ 0.726091] Demotion targets for Node 0: null1680lb # [ 0.752390] clk: Disabling unused clocks1681worker2 # [ 0.729508] Key type .fscrypt registered1682worker1 # [ 0.722881] Demotion targets for Node 0: null1683worker2 # [ 0.730197] Key type fscrypt-provisioning registered1684worker2 # [ 0.731159] ima: No TPM chip found, activating TPM-bypass!1685worker1 # [ 0.724908] Key type .fscrypt registered1686lb # [ 0.755743] PM: genpd: Disabling unused power domains1687worker1 # [ 0.725595] Key type fscrypt-provisioning registered1688worker2 # [ 0.733477] ima: Allocated hash algorithm: sha11689worker1 # [ 0.726597] ima: No TPM chip found, activating TPM-bypass!1690worker2 # [ 0.734305] ima: No architecture policies found1691worker1 # [ 0.730875] ima: Allocated hash algorithm: sha11692worker2 # [ 0.738470] PM: Magic number: 10:369:8311693worker1 # [ 0.731692] ima: No architecture policies found1694worker2 # [ 0.740134] RAS: Correctable Errors collector initialized.1695worker1 # [ 0.733881] PM: Magic number: 10:369:8311696client # [ 0.733999] Demotion targets for Node 0: null1697client # [ 0.734904] Key type .fscrypt registered1698worker1 # [ 0.735519] RAS: Correctable Errors collector initialized.1699client # [ 0.735677] Key type fscrypt-provisioning registered1700client # [ 0.738101] ima: No TPM chip found, activating TPM-bypass!1701client # [ 0.739051] ima: Allocated hash algorithm: sha11702client # [ 0.739912] ima: No architecture policies found1703worker2 # [ 0.749148] clk: Disabling unused clocks1704worker2 # [ 0.751467] PM: genpd: Disabling unused power domains1705worker1 # [ 0.744566] clk: Disabling unused clocks1706client # [ 0.745157] PM: Magic number: 10:369:8311707worker1 # [ 0.746879] PM: genpd: Disabling unused power domains1708client # [ 0.746762] RAS: Correctable Errors collector initialized.1709client # [ 0.754618] clk: Disabling unused clocks1710client # [ 0.758993] PM: genpd: Disabling unused power domains1711lb # [ 0.917608] Freeing initrd memory: 29092K1712lb # [ 0.920765] Freeing unused decrypted memory: 2028K1713lb # [ 0.923346] Freeing unused kernel image (initmem) memory: 3644K1714lb # [ 0.924488] Write protecting the kernel read-only data: 32768k1715lb # [ 0.926395] Freeing unused kernel image (text/rodata gap) memory: 1208K1716lb # [ 0.927958] Freeing unused kernel image (rodata/data gap) memory: 752K1717worker2 # [ 0.911243] Freeing initrd memory: 29092K1718worker2 # [ 0.914531] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.907583] Freeing initrd memory: 29092K1720worker2 # [ 0.917068] Freeing unused kernel image (initmem) memory: 3644K1721worker1 # [ 0.910693] Freeing unused decrypted memory: 2028K1722worker2 # [ 0.918196] Write protecting the kernel read-only data: 32768k1723worker2 # [ 0.920070] Freeing unused kernel image (text/rodata gap) memory: 1208K1724worker1 # [ 0.913340] Freeing unused kernel image (initmem) memory: 3644K1725worker1 # [ 0.914456] Write protecting the kernel read-only data: 32768k1726worker2 # [ 0.921644] Freeing unused kernel image (rodata/data gap) memory: 752K1727client # [ 0.915739] Freeing initrd memory: 29096K1728worker1 # [ 0.916341] Freeing unused kernel image (text/rodata gap) memory: 1208K1729worker1 # [ 0.917822] Freeing unused kernel image (rodata/data gap) memory: 752K1730client # [ 0.918825] Freeing unused decrypted memory: 2028K1731client # [ 0.921443] Freeing unused kernel image (initmem) memory: 3644K1732client # [ 0.922560] Write protecting the kernel read-only data: 32768k1733client # [ 0.924442] Freeing unused kernel image (text/rodata gap) memory: 1208K1734client # [ 0.925994] Freeing unused kernel image (rodata/data gap) memory: 752K1735lb # [ 0.979147] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736lb # [ 0.980253] Run /init as init process1737lb # [ 0.990164] systemd[1]: Inserted module 'autofs4'1738worker2 # [ 0.972888] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739worker2 # [ 0.973999] Run /init as init process1740worker1 # [ 0.968869] x86/mm: Checked W+X mappings: passed, no W+X pages found.1741worker1 # [ 0.969992] Run /init as init process1742lb # [ 1.006389] fuse: init (API version 7.45)1743worker2 # [ 0.983935] systemd[1]: Inserted module 'autofs4'1744client # [ 0.976917] x86/mm: Checked W+X mappings: passed, no W+X pages found.1745client # [ 0.978051] Run /init as init process1746worker1 # [ 0.980096] systemd[1]: Inserted module 'autofs4'1747lb # [ 1.012197] ACPI: \_SB_.GSIG: Enabled at IRQ 221748lb # [ 1.014770] ACPI: \_SB_.GSIH: Enabled at IRQ 231749lb # [ 1.018389] ACPI: \_SB_.GSIE: Enabled at IRQ 201750client # [ 0.988162] systemd[1]: Inserted module 'autofs4'1751lb # [ 1.021221] ACPI: \_SB_.GSIF: Enabled at IRQ 211752worker2 # [ 1.000315] fuse: init (API version 7.45)1753worker1 # [ 0.996157] fuse: init (API version 7.45)1754worker2 # [ 1.006168] ACPI: \_SB_.GSIG: Enabled at IRQ 221755worker2 # [ 1.008627] ACPI: \_SB_.GSIH: Enabled at IRQ 231756worker1 # [ 1.001951] ACPI: \_SB_.GSIG: Enabled at IRQ 221757worker1 # [ 1.004297] ACPI: \_SB_.GSIH: Enabled at IRQ 231758worker2 # [ 1.011614] ACPI: \_SB_.GSIE: Enabled at IRQ 201759client # [ 1.004287] fuse: init (API version 7.45)1760worker2 # [ 1.013813] ACPI: \_SB_.GSIF: Enabled at IRQ 211761worker1 # [ 1.007125] ACPI: \_SB_.GSIE: Enabled at IRQ 201762worker1 # [ 1.009285] ACPI: \_SB_.GSIF: Enabled at IRQ 211763client # [ 1.010076] ACPI: \_SB_.GSIG: Enabled at IRQ 221764client # [ 1.012386] ACPI: \_SB_.GSIH: Enabled at IRQ 231765client # [ 1.015352] ACPI: \_SB_.GSIE: Enabled at IRQ 201766client # [ 1.017610] ACPI: \_SB_.GSIF: Enabled at IRQ 211767lb # [ 1.054482] systemd[1]: Successfully made /usr/ read-only.1768worker2 # [ 1.047585] systemd[1]: Successfully made /usr/ read-only.1769worker1 # [ 1.040718] systemd[1]: Successfully made /usr/ read-only.1770client # [ 1.050707] systemd[1]: Successfully made /usr/ read-only.1771lb # [ 1.391034] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772lb # [ 1.403306] systemd[1]: Detected virtualization kvm.1773lb # [ 1.405476] systemd[1]: Detected architecture x86-64.1774lb # [ 1.407682] systemd[1]: Running in initrd.1775lb # [ 1.410296] systemd[1]: Initializing machine ID from random generator.1776worker1 # [ 1.376333] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1777lb # [ 1.413310] systemd[1]: Hostname set to <lb>.1778worker1 # [ 1.382952] systemd[1]: Detected virtualization kvm.1779worker1 # [ 1.385027] systemd[1]: Detected architecture x86-64.1780worker2 # [ 1.384043] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1781worker1 # [ 1.387111] systemd[1]: Running in initrd.1782worker2 # [ 1.394670] systemd[1]: Detected virtualization kvm.1783worker2 # [ 1.396752] systemd[1]: Detected architecture x86-64.1784worker1 # [ 1.389493] systemd[1]: Initializing machine ID from random generator.1785worker2 # [ 1.398868] systemd[1]: Running in initrd.1786worker1 # [ 1.392356] systemd[1]: Hostname set to <worker1>.1787worker2 # [ 1.401336] systemd[1]: Initializing machine ID from random generator.1788worker2 # [ 1.404245] systemd[1]: Hostname set to <worker2>.1789client # [ 1.387208] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790client # [ 1.399287] systemd[1]: Detected virtualization kvm.1791client # [ 1.401389] systemd[1]: Detected architecture x86-64.1792client # [ 1.403480] systemd[1]: Running in initrd.1793client # [ 1.405923] systemd[1]: Initializing machine ID from random generator.1794client # [ 1.408774] systemd[1]: Hostname set to <client>.1795lb # [ 1.633961] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker1 # [ 1.615931] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797worker2 # [ 1.624566] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798client # [ 1.632525] systemd[1]: bpf-restrict-fs: LSM BPF program attached1799lb # [ 1.687165] systemd[1]: Queued start job for default target Initrd Default Target.1800lb # [ 1.691004] systemd[1]: Created slice Slice /system/modprobe.1801lb # [ 1.692180] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1802lb # [ 1.693626] systemd[1]: Expecting device /dev/disk/by-label/nixos...1803lb # [ 1.694797] systemd[1]: Reached target Path Units.1804lb # [ 1.695665] systemd[1]: Reached target Slice Units.1805lb # [ 1.696586] systemd[1]: Reached target Swaps.1806lb # [ 1.697456] systemd[1]: Reached target Timer Units.1807lb # [ 1.698471] systemd[1]: Listening on D-Bus System Message Bus Socket.1808lb # [ 1.699755] systemd[1]: Listening on Journal Socket (/dev/log).1809lb # [ 1.700946] systemd[1]: Listening on Journal Sockets.1810worker1 # [ 1.670403] systemd[1]: Queued start job for default target Initrd Default Target.1811lb # [ 1.701938] systemd[1]: Listening on udev Control Socket.1812worker2 # [ 1.678985] systemd[1]: Queued start job for default target Initrd Default Target.1813lb # [ 1.702937] systemd[1]: Listening on udev Kernel Socket.1814lb # [ 1.703904] systemd[1]: Reached target Socket Units.1815worker1 # [ 1.674233] systemd[1]: Created slice Slice /system/modprobe.1816lb # [ 1.705663] systemd[1]: Starting Create List of Static Device Nodes...1817worker2 # [ 1.682799] systemd[1]: Created slice Slice /system/modprobe.1818worker1 # [ 1.675427] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1819worker1 # [ 1.676784] systemd[1]: Expecting device /dev/disk/by-label/nixos...1820worker2 # [ 1.683974] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1821worker1 # [ 1.677923] systemd[1]: Reached target Path Units.1822worker2 # [ 1.685390] systemd[1]: Expecting device /dev/disk/by-label/nixos...1823worker1 # [ 1.678776] systemd[1]: Reached target Slice Units.1824lb # [ 1.709513] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1825worker2 # [ 1.686583] systemd[1]: Reached target Path Units.1826worker1 # [ 1.679642] systemd[1]: Reached target Swaps.1827worker2 # [ 1.687467] systemd[1]: Reached target Slice Units.1828worker1 # [ 1.680446] systemd[1]: Reached target Timer Units.1829worker2 # [ 1.688359] systemd[1]: Reached target Swaps.1830worker1 # [ 1.681424] systemd[1]: Listening on D-Bus System Message Bus Socket.1831worker2 # [ 1.689181] systemd[1]: Reached target Timer Units.1832worker1 # [ 1.682645] systemd[1]: Listening on Journal Socket (/dev/log).1833worker2 # [ 1.690182] systemd[1]: Listening on D-Bus System Message Bus Socket.1834worker1 # [ 1.683794] systemd[1]: Listening on Journal Sockets.1835worker2 # [ 1.691460] systemd[1]: Listening on Journal Socket (/dev/log).1836worker1 # [ 1.684825] systemd[1]: Listening on udev Control Socket.1837worker2 # [ 1.692610] systemd[1]: Listening on Journal Sockets.1838worker1 # [ 1.685918] systemd[1]: Listening on udev Kernel Socket.1839lb # [ 1.716910] systemd[1]: Starting Load Kernel Module configfs...1840worker2 # [ 1.693602] systemd[1]: Listening on udev Control Socket.1841worker1 # [ 1.686792] systemd[1]: Reached target Socket Units.1842client # [ 1.686222] systemd[1]: Queued start job for default target Initrd Default Target.1843worker2 # [ 1.694571] systemd[1]: Listening on udev Kernel Socket.1844worker2 # [ 1.695548] systemd[1]: Reached target Socket Units.1845worker1 # [ 1.688531] systemd[1]: Starting Create List of Static Device Nodes...1846client # [ 1.689276] systemd[1]: Created slice Slice /system/modprobe.1847worker2 # [ 1.697331] systemd[1]: Starting Create List of Static Device Nodes...1848client # [ 1.690493] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1849client # [ 1.691936] systemd[1]: Expecting device /dev/disk/by-label/nixos...1850worker1 # [ 1.692935] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1851client # [ 1.693100] systemd[1]: Reached target Path Units.1852worker2 # [ 1.701269] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1853client # [ 1.693982] systemd[1]: Reached target Slice Units.1854client # [ 1.694866] systemd[1]: Reached target Swaps.1855client # [ 1.695709] systemd[1]: Reached target Timer Units.1856client # [ 1.696709] systemd[1]: Listening on D-Bus System Message Bus Socket.1857client # [ 1.697950] systemd[1]: Listening on Journal Socket (/dev/log).1858lb # [ 1.730069] systemd[1]: Starting Journal Service...1859client # [ 1.699155] systemd[1]: Listening on Journal Sockets.1860client # [ 1.700146] systemd[1]: Listening on udev Control Socket.1861worker2 # [ 1.708035] systemd[1]: Starting Load Kernel Module configfs...1862worker1 # [ 1.700929] systemd[1]: Starting Load Kernel Module configfs...1863client # [ 1.701198] systemd[1]: Listening on udev Kernel Socket.1864client # [ 1.702144] systemd[1]: Reached target Socket Units.1865client # [ 1.703900] systemd[1]: Starting Create List of Static Device Nodes...1866client # [ 1.707821] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1867lb # [ 1.741794] systemd[1]: Starting Load Kernel Modules...1868worker1 # [ 1.713927] systemd[1]: Starting Journal Service...1869worker2 # [ 1.722531] systemd[1]: Starting Journal Service...1870client # [ 1.715204] systemd[1]: Starting Load Kernel Module configfs...1871worker1 # [ 1.725934] systemd[1]: Starting Load Kernel Modules...1872lb # [ 1.747842] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1873worker2 # [ 1.736523] systemd[1]: Starting Load Kernel Modules...1874client # [ 1.729082] systemd[1]: Starting Journal Service...1875lb # [ 1.765916] netfs: FS-Cache loaded1876lb # [ 1.770521] 9pnet: Installing 9P2000 support1877client # [ 1.741793] systemd[1]: Starting Load Kernel Modules...1878worker1 # [ 1.732912] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1879worker2 # [ 1.741529] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1880worker1 # [ 1.750332] netfs: FS-Cache loaded1881worker2 # [ 1.758979] netfs: FS-Cache loaded1882lb # [ 1.785813] systemd[1]: Starting Coldplug All udev Devices...1883worker1 # [ 1.755552] 9pnet: Installing 9P2000 support1884worker2 # [ 1.763792] 9pnet: Installing 9P2000 support1885client # [ 1.747142] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1886client # [ 1.764604] netfs: FS-Cache loaded1887worker2 # [ 1.773278] systemd[1]: Starting Coldplug All udev Devices...1888client # [ 1.769763] 9pnet: Installing 9P2000 support1889lb # [ 1.800886] systemd[1]: Finished Create List of Static Device Nodes.1890lb # [ 1.802677] systemd-journald[67]: Collecting audit messages is disabled.1891worker1 # [ 1.773139] systemd[1]: Starting Coldplug All udev Devices...1892lb # [ 1.807192] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1893worker2 # [ 1.788544] systemd[1]: Finished Create List of Static Device Nodes.1894lb # [ 1.815142] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1895worker1 # [ 1.787458] systemd-journald[67]: Collecting audit messages is disabled.1896worker2 # [ 1.795046] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897client # [ 1.788091] systemd[1]: Starting Coldplug All udev Devices...1898worker1 # [ 1.789150] systemd[1]: Finished Create List of Static Device Nodes.1899lb # [ 1.821161] systemd[1]: modprobe@configfs.service: Deactivated successfully.1900worker1 # [ 1.794538] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1901worker2 # [ 1.802503] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1902lb # [ 1.828136] systemd[1]: Finished Load Kernel Module configfs.1903worker2 # [ 1.808895] systemd[1]: modprobe@configfs.service: Deactivated successfully.1904client # [ 1.802333] systemd-journald[67]: Collecting audit messages is disabled.1905worker1 # [ 1.804314] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1906worker2 # [ 1.811559] systemd-journald[67]: Collecting audit messages is disabled.1907lb # [ 1.835249] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1908client # [ 1.805112] systemd[1]: Finished Create List of Static Device Nodes.1909worker2 # [ 1.813509] systemd[1]: Finished Load Kernel Module configfs.1910lb # [ 1.840132] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1911worker1 # [ 1.810381] systemd[1]: modprobe@configfs.service: Deactivated successfully.1912client # [ 1.811794] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1913worker2 # [ 1.819937] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1914lb # [ 1.847980] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1915worker1 # [ 1.818286] systemd[1]: Finished Load Kernel Module configfs.1916worker2 # [ 1.824993] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1917lb # [ 1.849358] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1918client # [ 1.819462] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1919worker1 # [ 1.822397] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1920worker2 # [ 1.832570] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1921worker1 # [ 1.825527] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1922worker2 # [ 1.833908] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1923client # [ 1.827473] systemd[1]: modprobe@configfs.service: Deactivated successfully.1924client # [ 1.833586] systemd[1]: Finished Load Kernel Module configfs.1925worker1 # [ 1.835210] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1926worker1 # [ 1.836706] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1927client # [ 1.840328] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1928client # [ 1.843164] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1929lb # [ 1.880288] systemd[1]: Finished Load Kernel Modules.1930client # [ 1.851015] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1931client # [ 1.852952] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1932worker2 # [ 1.863965] systemd[1]: Finished Load Kernel Modules.1933lb # [ 1.887923] systemd[1]: Starting Apply Kernel Variables...1934worker2 # [ 1.871645] systemd[1]: Starting Apply Kernel Variables...1935lb # [ 1.900796] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1936worker1 # [ 1.871577] systemd[1]: Finished Load Kernel Modules.1937worker2 # [ 1.882571] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1938lb # [ 1.909969] systemd[1]: Starting Create Static Device Nodes in /dev...1939worker1 # [ 1.881229] systemd[1]: Starting Apply Kernel Variables...1940worker2 # [ 1.891683] systemd[1]: Starting Create Static Device Nodes in /dev...1941client # [ 1.886706] systemd[1]: Finished Load Kernel Modules.1942worker1 # [ 1.895974] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1943client # [ 1.896295] systemd[1]: Starting Apply Kernel Variables...1944lb # [ 1.792252] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.929181] systemd[1]: Started Journal Service.1945lb # 1946lb # [ 1.794386] systemd-modules-load[68]: Inserted module 'virtio_balloon'1947lb # [ 1.797349] systemd-modules-load[68]: Inserted module 'virtio_gpu'1948worker2 # [ 1.910072] systemd[1]: Finished Apply Kernel Variables.1949worker1 # [ 1.905251] systemd[1]: Starting Create Static Device Nodes in /dev...1950lb # [ 1.802963] systemd[1]: Finished Apply Kernel Variables.1951client # [ 1.912780] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1952worker1 # [ 1.781280] systemd-modules-load[68]: Inserted module 'dm_mod'1953worker2 # [ 1.924232] systemd[1]: Started Journal Service.1954worker1 # [ 1.784302] systemd-modules-load[68]: Inserted module 'virtio_balloon'1955lb # [ 1.812601] systemd[1]: Finished Create Static Device Nodes in /dev.1956worker1 # [ 1.918292] systemd[1]: Started Journal Service.1957worker1 # [ 1.787359] systemd-modules-load[68]: Inserted module 'virtio_gpu'1958lb # [ 1.816077] systemd[1]: Reached target Preparation for Local File Systems.1959lb # [ 1.817040] systemd[1]: Reached target Local File Systems.1960worker2 # [ 1.793296] systemd-modules-load[68]: Inserted module 'dm_mod'1961worker2 # [ 1.797436] systemd-modules-load[68]: Inserted module 'virtio_balloon'1962lb # [ 1.819253] systemd[1]: Starting Create System Files and Directories...1963worker2 # [ 1.798640] systemd-modules-load[68]: Inserted module 'virtio_gpu'1964client # [ 1.924335] systemd[1]: Starting Create Static Device Nodes in /dev...1965worker2 # [ 1.800269] systemd[1]: Finished Create Static Device Nodes in /dev.1966worker2 # [ 1.801357] systemd[1]: Reached target Preparation for Local File Systems.1967worker2 # [ 1.802449] systemd[1]: Reached target Local File Systems.1968worker1 # [ 1.798201] systemd[1]: Finished Apply Kernel Variables.1969worker2 # [ 1.805254] systemd[1]: Starting Create System Files and Directories...1970lb # [ 1.826362] systemd[1]: Starting Rule-based Manager for Device Events and Files...1971client # [ 1.797674] systemd-modules-load[68]: Inserted module 'dm_mod'1972client # [ 1.801989] systemd-modules-load[68]: Inserted module 'virtio_balloon'1973worker2 # [ 1.811943] systemd[1]: Starting Rule-based Manager for Device Events and Files...1974client # [ 1.803310] systemd-modules-load[68]: Inserted module 'virtio_gpu'1975client # [ 1.939246] systemd[1]: Started Journal Service.1976client # [ 1.813183] systemd[1]: Finished Apply Kernel Variables.1977worker1 # [ 1.817142] systemd[1]: Finished Create Static Device Nodes in /dev.1978worker1 # [ 1.818133] systemd[1]: Reached target Preparation for Local File Systems.1979worker1 # [ 1.819083] systemd[1]: Reached target Local File Systems.1980worker1 # [ 1.822067] systemd[1]: Starting Create System Files and Directories...1981lb # [ 1.853189] systemd[1]: Finished Create System Files and Directories.1982worker1 # [ 1.827088] systemd[1]: Starting Rule-based Manager for Device Events and Files...1983client # [ 1.831172] systemd[1]: Finished Create Static Device Nodes in /dev.1984worker2 # [ 1.841278] systemd[1]: Finished Create System Files and Directories.1985client # [ 1.833148] systemd[1]: Reached target Preparation for Local File Systems.1986client # [ 1.836089] systemd[1]: Reached target Local File Systems.1987client # [ 1.839069] systemd[1]: Starting Create System Files and Directories...1988client # [ 1.843606] systemd[1]: Starting Rule-based Manager for Device Events and Files...1989lb # [ 1.875984] systemd-udevd[83]: Using default interface naming scheme 'v261'.1990worker1 # [ 1.859519] systemd[1]: Finished Create System Files and Directories.1991worker2 # [ 1.866352] systemd-udevd[76]: Using default interface naming scheme 'v261'.1992lb # [ 1.899774] systemd[1]: Started Rule-based Manager for Device Events and Files.1993client # [ 1.874217] systemd[1]: Finished Create System Files and Directories.1994worker1 # [ 1.881277] systemd-udevd[83]: Using default interface naming scheme 'v261'.1995worker2 # [ 1.890075] systemd[1]: Started Rule-based Manager for Device Events and Files.1996client # [ 1.900507] systemd-udevd[83]: Using default interface naming scheme 'v261'.1997worker1 # [ 1.904793] systemd[1]: Started Rule-based Manager for Device Events and Files.1998worker2 # [ 1.935890] systemd[1]: Finished Coldplug All udev Devices.1999worker2 # [ 1.936785] systemd[1]: Reached target System Initialization.2000client # [ 1.927451] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 1,994 hidden lines…
3995lb # [ 18.630044] envoy[931]: [2026-09-20 10:50:52.707][931][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3996lb # [ 18.635995] envoy[931]: [2026-09-20 10:50:52.707][931][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83997lb # [ 18.637784] envoy[931]: [2026-09-20 10:50:52.707][931][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3998lb # [ 18.640560] envoy[931]: [2026-09-20 10:50:52.707][931][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3999lb # [ 18.642166] envoy[931]: [2026-09-20 10:50:52.707][931][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts4000lb # [ 18.644960] envoy[931]: [2026-09-20 10:50:52.707][931][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4001lb # [ 18.647455] envoy[931]: [2026-09-20 10:50:52.707][931][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4002lb # [ 18.649862] envoy[931]: [2026-09-20 10:50:52.707][931][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4003lb # [ 18.652225] envoy[931]: [2026-09-20 10:50:52.708][931][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4004lb # [ 18.653916] envoy[931]: [2026-09-20 10:50:52.708][931][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4005lb # [ 18.661715] envoy[931]: [2026-09-20 10:50:52.708][931][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4006lb # [ 18.663834] envoy[931]: [2026-09-20 10:50:52.708][931][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4007lb # [ 18.667903] envoy[931]: [2026-09-20 10:50:52.708][931][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4008lb # [ 18.670513] envoy[931]: [2026-09-20 10:50:52.708][931][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4009lb # [ 18.673680] envoy[931]: [2026-09-20 10:50:52.708][931][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4010lb # [ 18.675939] envoy[931]: [2026-09-20 10:50:52.708][931][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4011lb # [ 18.677590] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4012lb # [ 18.679856] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14013lb # [ 18.682379] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4014lb # [ 18.685264] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4015lb # [ 18.687630] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4016lb # [ 18.689220] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4017lb # [ 18.692085] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4018lb # [ 18.694571] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4019lb # [ 18.696292] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4020lb # [ 18.701048] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4021lb # [ 18.703064] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4022lb # [ 18.705290] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4023lb # [ 18.706976] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4024lb # [ 18.709854] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4025lb # [ 18.711753] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4026lb # [ 18.713893] envoy[931]: [2026-09-20 10:50:52.713][931][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4027lb # [ 18.716274] envoy[931]: [2026-09-20 10:50:52.716][931][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4028lb # [ 18.717917] envoy[931]: [2026-09-20 10:50:52.716][931][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4029lb # [ 18.719422] envoy[931]: [2026-09-20 10:50:52.716][931][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4030lb # [ 18.720932] envoy[931]: [2026-09-20 10:50:52.716][931][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4031lb # [ 18.724149] envoy[931]: [2026-09-20 10:50:52.716][931][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4032lb # [ 18.726212] envoy[931]: [2026-09-20 10:50:52.716][931][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4033lb # [ 18.728281] envoy[931]: [2026-09-20 10:50:52.716][931][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4034lb # [ 18.732817] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4035lb # [ 18.736238] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4036lb # [ 18.738421] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4037lb # [ 18.742266] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4038lb # [ 18.744076] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4039lb # [ 18.746745] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4040lb # [ 18.749271] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4041lb # [ 18.751630] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4042lb # [ 18.754252] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4043lb # [ 18.756061] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24044lb # [ 18.757838] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4045lb # [ 18.759464] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4046lb # [ 18.764463] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24047lb # [ 18.766388] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4048lb # [ 18.776035] envoy[931]: [2026-09-20 10:50:52.722][931][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4049lb # [ 18.781781] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4050lb # [ 18.784742] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4051lb # [ 18.786907] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4052lb # [ 18.791141] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4053lb # [ 18.793525] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4054lb # [ 18.795583] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4055lb # [ 18.797928] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4056lb # [ 18.799724] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4057lb # [ 18.802321] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4058lb # [ 18.805176] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4059lb # [ 18.810141] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4060lb # [ 18.812577] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4061lb # [ 18.814260] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4062lb # [ 18.816120] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4063lb # [ 18.817640] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4064lb # [ 18.820452] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4065lb # [ 18.822981] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4066lb # [ 18.825492] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4067lb # [ 18.827855] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4068lb # [ 18.829546] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4069lb # [ 18.831304] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4070lb # [ 18.833393] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4071lb # [ 18.835899] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4072lb # [ 18.837775] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4073lb # [ 18.840746] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4074lb # [ 18.842743] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4075lb # [ 18.847326] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4076lb # [ 18.859050] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4077lb # [ 18.860795] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4078lb # [ 18.862655] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4079lb # [ 18.864659] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4080lb # [ 18.866737] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4081lb # [ 18.868668] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4082lb # [ 18.871188] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4083lb # [ 18.876588] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24084lb # [ 18.878763] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4085lb # [ 18.881838] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4086lb # [ 18.883514] envoy[931]: [2026-09-20 10:50:52.723][931][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4087lb # [ 18.887517] envoy[931]: [2026-09-20 10:50:52.724][931][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4088lb # [ 18.893400] envoy[931]: [2026-09-20 10:50:52.724][931][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4089lb # [ 18.895648] envoy[931]: [2026-09-20 10:50:52.724][931][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4090lb # [ 18.921104] envoy[931]: [2026-09-20 10:50:52.854][931][info][main] [source/server/server.cc:531] HTTP header map info:4091lb # [ 18.922949] envoy[931]: [2026-09-20 10:50:52.905][931][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4092lb # [ 18.935804] envoy[931]: [2026-09-20 10:50:52.905][931][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4093lb # [ 18.937324] envoy[931]: [2026-09-20 10:50:52.905][931][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4094lb # [ 18.944826] envoy[931]: [2026-09-20 10:50:52.905][931][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4095lb # [ 18.987088] envoy[931]: [2026-09-20 10:50:53.105][931][info][main] [source/server/server.cc:968] runtime: {}4096lb # [ 18.994827] envoy[931]: [2026-09-20 10:50:53.113][931][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4097lb # [ 18.996927] envoy[931]: [2026-09-20 10:50:53.115][931][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014098lb # [ 19.000550] envoy[931]: [2026-09-20 10:50:53.119][931][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014099lb # [ 19.003645] envoy[931]: [2026-09-20 10:50:53.122][931][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4100lb # [ 19.005289] envoy[931]: [2026-09-20 10:50:53.124][931][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4101lb # [ 19.006906] envoy[931]: [2026-09-20 10:50:53.124][931][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4102lb # [ 19.055451] envoy[931]: [2026-09-20 10:50:53.174][931][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4103worker1 # [ 19.031297] systemd[1]: Starting Nix worker-protocol over gRPC...4104worker2 # [ 19.042851] systemd[1]: Starting Nix worker-protocol over gRPC...4105lb # [ 19.076163] envoy[931]: [2026-09-20 10:50:53.194][931][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4106lb # [ 19.078442] envoy[931]: [2026-09-20 10:50:53.197][931][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4107lb # [ 19.081331] envoy[931]: [2026-09-20 10:50:53.197][931][info][main] [source/server/server.cc:1077] starting main dispatch loop4108worker1 # [ 19.242139] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4109worker2 # [ 19.249506] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4110worker2 # [ 19.260347] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14111worker1 # [ 19.255315] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14112worker2 # [ 19.292883] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4113worker2 # [ 19.295741] systemd[1]: Started Nix worker-protocol over gRPC.4114worker1 # [ 19.293320] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4115worker1 # [ 19.297456] systemd[1]: Started Nix worker-protocol over gRPC.4116lb # [ 19.366297] envoy[931]: [2026-09-20 10:50:53.484][931][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4117lb # [ 19.370589] envoy[931]: [2026-09-20 10:50:53.488][931][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4118lb # [ 19.374986] envoy[931]: [2026-09-20 10:50:53.488][931][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4119lb # [ 19.380244] envoy[931]: [2026-09-20 10:50:53.493][931][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4120worker2 # [ 19.381324] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514121worker1 # [ 19.375546] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_connected addr=lb:500514122worker2 # [ 19.391050] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4123worker1 # [ 19.384604] nix-grpc-daemon[784]: ts=2026-09-20T10:50:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4124worker2 # [ 19.592475] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514125worker1 # [ 19.585893] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514126worker2 # [ 19.596422] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4127worker1 # [ 19.590376] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4128worker1 # [ 19.790737] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514129worker2 # [ 19.797251] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514130worker1 # [ 19.794753] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4131worker2 # [ 19.801739] nix-grpc-daemon[785]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4132worker1 # [ 19.995829] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514133worker2 # [ 20.002496] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514134worker1 # [ 20.000084] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker2 # [ 20.006853] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4136worker1 # [ 20.200823] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514137worker2 # [ 20.207733] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514138worker1 # [ 20.204757] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker2 # [ 20.212246] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker1 # [ 20.406891] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 20.414301] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514142worker1 # [ 20.409612] nix-grpc-daemon[784]: ts=2026-09-20T10:50:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker2 # [ 20.417797] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 20.607816] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514145worker2 # [ 20.614768] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514146worker1 # [ 20.611831] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 20.618610] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4148worker2 # [ 20.819481] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514149worker1 # [ 20.812872] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 20.823456] nix-grpc-daemon[785]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 20.817293] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker1 # [ 21.017830] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514153worker2 # [ 21.024376] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514154worker1 # [ 21.021694] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker2 # [ 21.028526] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4156worker2 # [ 21.228968] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514157worker1 # [ 21.222741] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 21.230939] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 21.226652] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4160worker2 # [ 21.431836] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514161worker2 # [ 21.433479] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker1 # [ 21.428601] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_connected addr=lb:500514163worker1 # [ 21.431625] nix-grpc-daemon[784]: ts=2026-09-20T10:50:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164worker2 # [ 21.634777] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514165worker1 # [ 21.629261] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 21.638900] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker1 # [ 21.632896] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4168worker2 # [ 21.839735] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514169worker1 # [ 21.833936] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 21.843946] nix-grpc-daemon[785]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker1 # [ 21.837872] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker2 # [ 22.044786] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514173worker1 # [ 22.038897] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 22.048848] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker1 # [ 22.043136] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker2 # [ 22.249850] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514177worker1 # [ 22.243696] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 22.247692] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 22.254322] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker2 # [ 22.455271] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514181worker2 # [ 22.456844] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker1 # [ 22.449910] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_connected addr=lb:500514183worker1 # [ 22.453241] nix-grpc-daemon[784]: ts=2026-09-20T10:50:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker2 # [ 22.656785] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514185worker1 # [ 22.650646] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 22.660615] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker1 # [ 22.654636] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker2 # [ 22.861355] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514189worker1 # [ 22.855599] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 22.859583] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 22.866831] nix-grpc-daemon[785]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker1 # [ 23.060535] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514193worker2 # [ 23.067629] nix-grpc-daemon[785]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 23.064645] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker2 # [ 23.071649] nix-grpc-daemon[785]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 23.265386] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514197worker2 # [ 23.272337] nix-grpc-daemon[785]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514198worker1 # [ 23.269425] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker2 # [ 23.276237] nix-grpc-daemon[785]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker2 # [ 23.476971] nix-grpc-daemon[785]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514201worker2 # [ 23.478700] nix-grpc-daemon[785]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker1 # [ 23.470920] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_connected addr=lb:500514203worker1 # [ 23.474818] nix-grpc-daemon[784]: ts=2026-09-20T10:50:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker2 # [ 23.679779] nix-grpc-daemon[785]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514205worker1 # [ 23.676215] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 23.683560] nix-grpc-daemon[785]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 23.679867] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker2 # [ 23.884411] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514209worker1 # [ 23.880711] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 23.888279] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 23.884528] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker2 # [ 24.089144] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514213worker1 # [ 24.085206] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514214worker1 # [ 24.087058] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker2 # [ 24.093309] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker2 # [ 24.294338] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514217worker1 # [ 24.287905] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 24.291700] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker2 # [ 24.298754] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker1 # [ 24.494138] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_connected addr=lb:500514221worker2 # [ 24.500867] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 24.497283] nix-grpc-daemon[784]: ts=2026-09-20T10:50:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker2 # [ 24.504102] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker1 # [ 24.694537] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514225worker1 # [ 24.696320] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker2 # [ 24.701566] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 24.705437] nix-grpc-daemon[785]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker1 # [ 24.897418] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514229worker2 # [ 24.905889] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 24.907635] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker1 # [ 24.901311] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker2 # [ 25.108508] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514233worker1 # [ 25.101951] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 25.112438] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker1 # [ 25.105904] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker2 # [ 25.312840] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514237worker2 # [ 25.314865] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker1 # [ 25.306763] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 25.310722] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker2 # [ 25.517346] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514241worker1 # [ 25.512648] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 25.520555] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker1 # [ 25.515798] nix-grpc-daemon[784]: ts=2026-09-20T10:50:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker1 # [ 25.713059] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 25.718203] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514246worker1 # [ 25.714969] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker2 # [ 25.722229] nix-grpc-daemon[785]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker1 # [ 25.915532] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514249worker1 # [ 25.917395] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker2 # [ 25.922950] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 25.926883] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker1 # [ 26.118362] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514253worker2 # [ 26.127524] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514254worker1 # [ 26.122268] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker2 # [ 26.130702] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker1 # [ 26.322499] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514257worker1 # [ 26.324159] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker2 # [ 26.331467] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514259worker2 # [ 26.335209] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker1 # [ 26.526070] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_connected addr=lb:500514261worker2 # [ 26.536152] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514262worker2 # [ 26.537426] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker1 # [ 26.529904] nix-grpc-daemon[784]: ts=2026-09-20T10:50:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker1 # [ 26.727838] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514265worker2 # [ 26.736957] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514266worker1 # [ 26.732424] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker2 # [ 26.740823] nix-grpc-daemon[785]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4268worker1 # [ 26.933290] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514269worker2 # [ 26.942313] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514270worker1 # [ 26.937439] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker2 # [ 26.945455] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker2 # [ 27.143219] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514273worker1 # [ 27.138440] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514274worker2 # [ 27.146911] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker1 # [ 27.142648] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4276worker2 # [ 27.347180] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514277worker2 # [ 27.348867] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker1 # [ 27.343472] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514279worker1 # [ 27.347479] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4280worker2 # [ 27.550329] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514281worker2 # [ 27.553274] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker1 # [ 27.549766] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_connected addr=lb:500514283worker1 # [ 27.552853] nix-grpc-daemon[784]: ts=2026-09-20T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284worker2 # [ 27.754119] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514285worker1 # [ 27.750092] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514286worker1 # [ 27.751680] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker2 # [ 27.757945] nix-grpc-daemon[785]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4288worker1 # [ 27.952287] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514289worker2 # [ 27.958560] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514290worker1 # [ 27.954269] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker2 # [ 27.962362] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4292worker1 # [ 28.155430] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514293worker2 # [ 28.164209] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514294worker1 # [ 28.159328] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker2 # [ 28.167418] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4296worker2 # [ 28.364924] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514297worker1 # [ 28.360376] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514298worker2 # [ 28.368780] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker1 # [ 28.364163] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4300worker1 # [ 28.565124] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_connected addr=lb:500514301worker2 # [ 28.570415] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514302worker1 # [ 28.567118] nix-grpc-daemon[784]: ts=2026-09-20T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker2 # [ 28.574415] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4304worker1 # [ 28.768323] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514305worker2 # [ 28.775559] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 28.772245] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 28.779214] nix-grpc-daemon[785]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4308worker1 # [ 28.973055] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514309worker2 # [ 28.980286] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 28.976990] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 28.984121] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4312worker1 # [ 29.177226] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514313worker1 # [ 29.179152] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4314worker2 # [ 29.185138] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514315worker2 # [ 29.188894] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4316worker1 # [ 29.380192] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514317worker2 # [ 29.390485] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514318worker1 # [ 29.385440] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker2 # [ 29.393535] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4320worker1 # [ 29.583366] nix-grpc-daemon[784]: ts=2026-09-20T10:51:02Z level=info event=scheduler_connected addr=lb:500514321worker2 # [ 29.592729] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514322worker1 # [ 29.587190] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker2 # [ 29.595763] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4324worker1 # [ 29.787644] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514325worker2 # [ 29.793347] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514326worker1 # [ 29.789482] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker2 # [ 29.797167] nix-grpc-daemon[785]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4328worker1 # [ 29.990332] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514329worker2 # [ 29.997973] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514330worker1 # [ 29.993766] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker2 # [ 30.002258] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4332worker1 # [ 30.194634] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514333worker2 # [ 30.203205] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514334worker1 # [ 30.198873] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker2 # [ 30.207091] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4336lb # [ 30.256381] rustfs-bucket-start[1050]: mb s3://farm4337lb # [ 30.264511] systemd[1]: Finished rustfs-bucket.service.4338lb # [ 30.270038] systemd[1]: Starting niks3 server...4339worker1 # [ 30.399530] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_connected addr=lb:500514340worker2 # [ 30.407819] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514341worker1 # [ 30.403465] nix-grpc-daemon[784]: ts=2026-09-20T10:51:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4342worker2 # [ 30.411642] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343lb # [ 30.479424] postgres[1066]: [1066] ERROR: relation "goose_db_version" does not exist at character 364344lb # [ 30.480629] postgres[1066]: [1066] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4345lb # [ 30.515550] niks3-server[1059]: 2026/09/20 10:51:04 OK 20241026095416_initial_model.sql (18.99ms)4346lb # [ 30.523862] niks3-server[1059]: 2026/09/20 10:51:04 OK 20251210153512_drop_unused_gin_index.sql (8.31ms)4347lb # [ 30.532755] niks3-server[1059]: 2026/09/20 10:51:04 OK 20251218171726_add_pins.sql (7.44ms)4348lb # [ 30.537477] niks3-server[1059]: 2026/09/20 10:51:04 OK 20260628120000_add_object_size_and_stats.sql (4.8ms)4349lb # [ 30.546096] niks3-server[1059]: 2026/09/20 10:51:04 OK 20260905000000_add_claims.sql (7.15ms)4350lb # [ 30.577187] niks3-server[1059]: 2026/09/20 10:51:04 OK 20260920000000_drop_claims.sql (31.15ms)4351lb # [ 30.578632] niks3-server[1059]: 2026/09/20 10:51:04 goose: successfully migrated database to version: 202609200000004352lb # [ 30.588322] niks3-server[1059]: 2026/09/20 10:51:04 OK 1_commit_pending_closure.sql (8.22ms)4353lb # [ 30.609673] niks3-server[1059]: 2026/09/20 10:51:04 OK 2_object_stats_trigger.sql (21.41ms)4354lb # [ 30.610944] niks3-server[1059]: 2026/09/20 10:51:04 goose: up to current file version: 24355lb # [ 30.618361] niks3-server[1059]: 2026/09/20 10:51:04 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4356worker1 # [ 30.605991] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514357worker2 # [ 30.613775] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514358worker1 # [ 30.609185] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4359worker2 # [ 30.616838] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4360lb # [ 30.684689] niks3-server[1059]: 2026/09/20 10:51:04 INFO Read proxy enabled — serving cache objects from S34361lb # [ 30.686544] niks3-server[1059]: 2026/09/20 10:51:04 INFO Using socket-activated listener address=0.0.0.0:57514362lb # [ 30.692206] systemd[1]: Started niks3 server.4363lb # [ 30.692847] niks3-server[1059]: 2026/09/20 10:51:04 INFO systemd watchdog enabled interval=15s4364lb # [ 30.694023] systemd[1]: Reached target Multi-User System.4365lb # [ 30.696457] systemd[1]: Startup finished in 847ms (kernel) + 4.873s (initrd) + 24.974s (userspace) = 30.696s.4366lb # [ 30.698167] niks3-server[1059]: 2026/09/20 10:51:04 INFO Starting HTTP server address=0.0.0.0:57514367lb # [ 30.703406] niks3-server[1059]: 2026/09/20 10:51:04 INFO lead: acquired remote=192.168.1.3:436824368worker1 # [ 30.681861] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_take_over4369worker1 # [ 30.806312] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514370worker2 # [ 30.813868] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514371worker2 # [ 30.815579] nix-grpc-daemon[785]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4372worker1 # [ 30.808720] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4373lb: (finished: waiting for unit niks3.service, in 19.89 seconds)4374lb: waiting for TCP port 5751 on localhost4375lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4376lb: (finished: waiting for TCP port 5751 on localhost, in 0.05 seconds)4377lb: waiting for unit envoy.service4378lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4379lb: waiting for TCP port 50051 on localhost4380lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4381lb # [ 31.026676] envoy[931]: conn peer=[::1]:33986 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24382lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4383worker2 # [ 31.016558] nix-grpc-daemon[785]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500514384worker1 # [ 31.010635] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514385worker2 # [ 31.020438] nix-grpc-daemon[785]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4386worker1 # [ 31.014170] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4387lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4388lb: must succeed: curl -sf localhost:9901/clusters4389worker2 # [ 31.218387] nix-grpc-daemon[785]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500514390worker1 # [ 31.211676] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514391lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4392lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4393subtest: one node schedules and both builders hold a WorkerSession on it4394lb: must succeed: curl -sf localhost:9901/clusters4395worker2 # [ 31.222213] nix-grpc-daemon[785]: ts=2026-09-20T10:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4396worker1 # [ 31.216261] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4397lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4398worker2 # [ 31.423163] nix-grpc-daemon[785]: ts=2026-09-20T10:51:05Z level=info event=scheduler_connected addr=lb:500514399worker1 # [ 31.416602] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=scheduler_connected addr=lb:500514400worker1 # [ 31.427221] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04401worker1 # [ 31.434087] nix-grpc-daemon[784]: ts=2026-09-20T10:51:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04402lb: must succeed: curl -sf localhost:9901/clusters4403lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4404worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4405worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4406lb: must succeed: curl -sf localhost:9901/clusters4407lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4408lb: must succeed: curl -sf localhost:9901/clusters4409lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4410worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4411worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4412lb: must succeed: curl -sf localhost:9901/clusters4413lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4414lb: must succeed: curl -sf localhost:9901/clusters4415lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4416worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4417worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4418worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4419worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4420worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4421worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4422worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4423worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4424(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.59 seconds)4425subtest: balancer auth: client cert, bearer token, nothing, unknown CN4426worker1 # [ 33.080447] nix-grpc-daemon[784]: ts=2026-09-20T10:51:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426904427worker1 # [ 33.091820] systemd[1]: Started Nix Daemon.4428worker1 # [ 33.313207] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4429worker1 # [ 33.365651] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)4430worker1 # [ 33.424555] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5)4431worker1 # [ 33.524665] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 3/5)4432worker1 # [ 33.574915] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 502 ms (attempt 4/5)4433worker1 # [ 34.088282] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4434worker2 # [ 34.133871] nix-grpc-daemon[785]: ts=2026-09-20T10:51:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:545144435worker2 # [ 34.140681] systemd[1]: Started Nix Daemon.4436worker2 # [ 34.343506] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4437worker2 # [ 34.376240] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)4438worker2 # [ 34.433115] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 168 ms (attempt 2/5)4439worker2 # [ 34.604226] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 3/5)4440worker2 # [ 34.773438] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 4/5)4441worker2 # [ 34.824436] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4442worker2 # [ 34.835607] nix-grpc-daemon[785]: ts=2026-09-20T10:51:08Z level=debug event=path_info_miss path=rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test error="error: path '/nix/store/rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test' is required, but there is no substituter that can build it"4443worker2 # [ 34.846208] nix-grpc-daemon[785]: ts=2026-09-20T10:51:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:545144444worker2 # [ 34.849943] nix-grpc-daemon[785]: ts=2026-09-20T10:51:08Z level=debug event=path_info_miss path=rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test error="error: path '/nix/store/rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test' is required, but there is no substituter that can build it"4445client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4446client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4447client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4448client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4449worker1 # [ 35.214533] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=info event=oidc_jwks provider=mock keys=14450worker1 # [ 35.217824] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:426904451worker2 # [ 35.246747] nix-grpc-daemon[785]: ts=2026-09-20T10:51:09Z level=info event=oidc_jwks provider=mock keys=14452worker2 # [ 35.250723] nix-grpc-daemon[785]: ts=2026-09-20T10:51:09Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:545144453worker2 # [ 35.258888] nix-grpc-daemon[785]: ts=2026-09-20T10:51:09Z level=debug event=path_info_miss path=rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test error="error: path '/nix/store/rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test' is required, but there is no substituter that can build it"4454worker1 # [ 35.256759] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:426904455worker1 # [ 35.274523] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=debug event=path_info_miss path=rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test error="error: path '/nix/store/rd92mric7clp2ajnph200byp8g4917j4-nixos-system-client-test' is required, but there is no substituter that can build it"4456client: must succeed: rm -r /var/lib/nix-grpc-store4457client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4458worker1 # [ 35.533168] nix-grpc-daemon[784]: ts=2026-09-20T10:51:08Z level=info event=denied method=QueryMissing cn=- role=none4459worker2 # [ 35.753312] nix-grpc-daemon[785]: ts=2026-09-20T10:51:09Z level=info event=denied method=QueryMissing cn=stranger role=none4460(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.96 seconds)4461subtest: DAG build is scheduled across workers and lands in the cache4462client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24463worker1 # [ 35.965179] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426904464client # don't know how to build these paths:4465client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4466worker2 # [ 35.984827] nix-grpc-daemon[785]: ts=2026-09-20T10:51:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144467worker1 # [ 36.013090] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04468worker1 # [ 36.017516] nix-grpc-daemon[784]: ts=2026-09-20T10:51:09Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14469worker2 # [ 36.074545] nix-grpc-daemon[785]: ts=2026-09-20T10:51:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144470worker2 # [ 36.094505] nix-grpc-daemon[785]: ts=2026-09-20T10:51:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144471client # copying 1 paths to 'lb:50051'...4472worker2 # [ 36.107303] nix-grpc-daemon[785]: ts=2026-09-20T10:51:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545144473client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4474worker2 # [ 36.134328] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4475worker2 # [ 36.161373] nix-grpc-daemon[785]: ts=2026-09-20T10:51:10Z level=info event=niks3_push_started pid=8574476worker2 # [ 36.571185] nix-daemon[847]: accepted connection from pid 869, user nix-grpc-daemon (trusted)4477worker2 # [ 36.588501] nix-daemon[847]: reaped child process 872, status = succeeded4478worker2 # [ 36.727206] nix-daemon[847]: accepted connection from pid 874, user nix-grpc-daemon (trusted)4479worker2 # [ 36.737908] nix-daemon[847]: reaped child process 877, status = succeeded4480lb # [ 36.771273] niks3-server[1059]: 2026/09/20 10:51:10 INFO Received uploads request method=POST path=/api/pending_closures4481worker2 # [ 36.779665] nix-grpc-daemon[857]: time=2026-09-20T10:51:10.898Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4482worker2 # [ 36.782863] nix-grpc-daemon[857]: time=2026-09-20T10:51:10.902Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4483lb # [ 36.849651] niks3-server[1059]: 2026/09/20 10:51:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4484lb # [ 36.861650] niks3-server[1059]: 2026/09/20 10:51:10 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4485lb # [ 36.874822] niks3-server[1059]: 2026/09/20 10:51:10 INFO Signed narinfos id=1 count=14486worker2 # [ 36.855354] nix-grpc-daemon[857]: time=2026-09-20T10:51:10.975Z level=INFO msg="Uploading 1 narinfos"4487lb # [ 36.884884] niks3-server[1059]: 2026/09/20 10:51:11 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4488lb # [ 36.887148] niks3-server[1059]: 2026/09/20 10:51:11 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4489lb # [ 36.896305] niks3-server[1059]: 2026/09/20 10:51:11 INFO Completed upload id=14490worker2 # [ 36.875303] nix-grpc-daemon[857]: time=2026-09-20T10:51:10.995Z level=INFO msg="Upload complete. (457ms)"4491worker2 # [ 36.877856] nix-grpc-daemon[785]: ts=2026-09-20T10:51:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=5044492lb # [ 36.901516] niks3-server[1059]: 2026/09/20 10:51:11 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4493worker2 # [ 36.882783] nix-grpc-daemon[785]: ts=2026-09-20T10:51:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144494worker2 # [ 36.889658] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4495worker2 # [ 36.892152] nix-daemon[847]: reaped child process 855, status = succeeded4496worker2 # [ 36.898150] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4497client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4498client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4499client # farm-a> LOG-farm-a-t14500worker2 # [ 38.118558] nix-daemon[847]: reaped child process 881, status = succeeded4501worker2 # [ 38.258258] nix-daemon[847]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4502worker2 # [ 38.268502] nix-daemon[847]: reaped child process 899, status = succeeded4503lb # [ 38.303767] niks3-server[1059]: 2026/09/20 10:51:12 INFO Received uploads request method=POST path=/api/pending_closures4504worker2 # [ 38.294660] nix-grpc-daemon[857]: time=2026-09-20T10:51:12.414Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4505worker2 # [ 38.296254] nix-grpc-daemon[857]: time=2026-09-20T10:51:12.416Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4506lb # [ 38.348635] niks3-server[1059]: 2026/09/20 10:51:12 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4507lb # [ 38.363134] niks3-server[1059]: 2026/09/20 10:51:12 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4508lb # [ 38.372817] niks3-server[1059]: 2026/09/20 10:51:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4509lb # [ 38.376899] niks3-server[1059]: 2026/09/20 10:51:12 INFO Signed narinfos id=2 count=14510worker2 # [ 38.358171] nix-grpc-daemon[857]: time=2026-09-20T10:51:12.478Z level=INFO msg="Uploading 1 narinfos"4511lb # [ 38.383112] niks3-server[1059]: 2026/09/20 10:51:12 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4512lb # [ 38.392795] niks3-server[1059]: 2026/09/20 10:51:12 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4513lb # [ 38.398595] niks3-server[1059]: 2026/09/20 10:51:12 INFO Completed upload id=24514worker2 # [ 38.377783] nix-grpc-daemon[857]: time=2026-09-20T10:51:12.497Z level=INFO msg="Upload complete. (270ms)"4515lb # [ 38.400874] niks3-server[1059]: 2026/09/20 10:51:12 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4516worker2 # [ 38.384651] nix-grpc-daemon[785]: ts=2026-09-20T10:51:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14517worker2 # [ 38.397114] nix-daemon[847]: reaped child process 879, status = succeeded4518worker1 # [ 38.396547] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04519worker1 # [ 38.401315] nix-grpc-daemon[784]: ts=2026-09-20T10:51:11Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14520worker2 # [ 38.449518] nix-grpc-daemon[785]: ts=2026-09-20T10:51:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144521worker2 # [ 38.461721] nix-grpc-daemon[785]: ts=2026-09-20T10:51:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144522worker2 # [ 38.474253] nix-grpc-daemon[857]: time=2026-09-20T10:51:12.594Z level=INFO msg="All 1 paths already cached"4523client # copying 1 paths to 'lb:50051'...4524worker2 # [ 38.481437] nix-grpc-daemon[785]: ts=2026-09-20T10:51:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545144525client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4526worker2 # [ 38.496170] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4527worker2 # [ 38.651387] nix-daemon[847]: accepted connection from pid 903, user nix-grpc-daemon (trusted)4528worker2 # [ 38.664136] nix-daemon[847]: reaped child process 906, status = succeeded4529worker2 # [ 38.799609] nix-daemon[847]: accepted connection from pid 909, user nix-grpc-daemon (trusted)4530worker2 # [ 38.811670] nix-daemon[847]: reaped child process 912, status = succeeded4531lb # [ 38.848781] niks3-server[1059]: 2026/09/20 10:51:12 INFO Received uploads request method=POST path=/api/pending_closures4532worker2 # [ 38.837169] nix-grpc-daemon[857]: time=2026-09-20T10:51:12.957Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4533worker2 # [ 38.839225] nix-grpc-daemon[857]: time=2026-09-20T10:51:12.958Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4534lb # [ 38.876652] niks3-server[1059]: 2026/09/20 10:51:12 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4535lb # [ 38.888541] niks3-server[1059]: 2026/09/20 10:51:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4536lb # [ 38.893112] niks3-server[1059]: 2026/09/20 10:51:13 INFO Signed narinfos id=3 count=14537worker2 # [ 38.875225] nix-grpc-daemon[857]: time=2026-09-20T10:51:12.995Z level=INFO msg="Uploading 1 narinfos"4538lb # [ 38.899546] niks3-server[1059]: 2026/09/20 10:51:13 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4539lb # [ 38.910209] niks3-server[1059]: 2026/09/20 10:51:13 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4540worker2 # [ 38.897167] nix-grpc-daemon[857]: time=2026-09-20T10:51:13.017Z level=INFO msg="Upload complete. (388ms)"4541lb # [ 38.918468] niks3-server[1059]: 2026/09/20 10:51:13 INFO Completed upload id=34542worker2 # [ 38.898995] nix-grpc-daemon[785]: ts=2026-09-20T10:51:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=6244543lb # [ 38.921798] niks3-server[1059]: 2026/09/20 10:51:13 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4544worker2 # [ 38.905223] nix-grpc-daemon[785]: ts=2026-09-20T10:51:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144545worker2 # [ 38.906861] nix-daemon[847]: reaped child process 901, status = succeeded4546worker2 # [ 38.909799] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4547worker2 # [ 38.915994] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4548client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4549client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4550lb # [ 39.062219] envoy[931]: conn peer=[2001:db8:1::1]:48538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=381 ms=18094551lb # [ 39.065899] envoy[931]: conn peer=[2001:db8:1::1]:48550 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=914552lb # [ 39.069225] envoy[931]: conn peer=[2001:db8:1::1]:48566 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=74553lb # [ 39.072576] envoy[931]: conn peer=[2001:db8:1::1]:48568 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84554client # farm-b> LOG-farm-b-t14555worker2 # [ 40.144463] nix-daemon[847]: reaped child process 917, status = succeeded4556worker2 # [ 40.295709] nix-daemon[847]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4557worker2 # [ 40.306923] nix-daemon[847]: reaped child process 935, status = succeeded4558lb # [ 40.338557] niks3-server[1059]: 2026/09/20 10:51:14 INFO Received uploads request method=POST path=/api/pending_closures4559worker2 # [ 40.324354] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.444Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4560worker2 # [ 40.326046] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.446Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4561lb # [ 40.359660] niks3-server[1059]: 2026/09/20 10:51:14 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4562lb # [ 40.361801] niks3-server[1059]: 2026/09/20 10:51:14 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4563lb # [ 40.366410] niks3-server[1059]: 2026/09/20 10:51:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4564lb # [ 40.368042] niks3-server[1059]: 2026/09/20 10:51:14 INFO Signed narinfos id=4 count=14565worker2 # [ 40.348289] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.468Z level=INFO msg="Uploading 1 narinfos"4566lb # [ 40.371740] niks3-server[1059]: 2026/09/20 10:51:14 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4567lb # [ 40.377932] niks3-server[1059]: 2026/09/20 10:51:14 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4568lb # [ 40.380635] niks3-server[1059]: 2026/09/20 10:51:14 INFO Completed upload id=44569worker2 # [ 40.359563] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.479Z level=INFO msg="Upload complete. (234ms)"4570lb # [ 40.382116] niks3-server[1059]: 2026/09/20 10:51:14 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4571worker2 # [ 40.361842] nix-grpc-daemon[785]: ts=2026-09-20T10:51:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14572worker2 # [ 40.368119] nix-daemon[847]: reaped child process 915, status = succeeded4573worker1 # [ 40.364943] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04574worker1 # [ 40.369388] nix-grpc-daemon[784]: ts=2026-09-20T10:51:13Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14575worker2 # [ 40.413930] nix-grpc-daemon[785]: ts=2026-09-20T10:51:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144576worker2 # [ 40.423253] nix-grpc-daemon[785]: ts=2026-09-20T10:51:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144577worker2 # [ 40.432094] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.552Z level=INFO msg="All 2 paths already cached"4578client # copying 1 paths to 'lb:50051'...4579client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4580worker2 # [ 40.438453] nix-grpc-daemon[785]: ts=2026-09-20T10:51:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545144581worker2 # [ 40.449958] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4582worker2 # [ 40.599684] nix-daemon[847]: accepted connection from pid 939, user nix-grpc-daemon (trusted)4583worker2 # [ 40.611703] nix-daemon[847]: reaped child process 942, status = succeeded4584worker2 # [ 40.756598] nix-daemon[847]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4585worker2 # [ 40.766752] nix-daemon[847]: reaped child process 947, status = succeeded4586lb # [ 40.801630] niks3-server[1059]: 2026/09/20 10:51:14 INFO Received uploads request method=POST path=/api/pending_closures4587worker2 # [ 40.789306] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.909Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4588worker2 # [ 40.790837] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.911Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4589lb # [ 40.829175] niks3-server[1059]: 2026/09/20 10:51:14 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4590lb # [ 40.840987] niks3-server[1059]: 2026/09/20 10:51:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4591worker2 # [ 40.824436] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.944Z level=INFO msg="Uploading 1 narinfos"4592lb # [ 40.846076] niks3-server[1059]: 2026/09/20 10:51:14 INFO Signed narinfos id=5 count=14593lb # [ 40.852213] niks3-server[1059]: 2026/09/20 10:51:14 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4594lb # [ 40.862225] niks3-server[1059]: 2026/09/20 10:51:14 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4595lb # [ 40.867596] niks3-server[1059]: 2026/09/20 10:51:14 INFO Completed upload id=54596worker2 # [ 40.848662] nix-grpc-daemon[857]: time=2026-09-20T10:51:14.968Z level=INFO msg="Upload complete. (387ms)"4597lb # [ 40.871763] niks3-server[1059]: 2026/09/20 10:51:14 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4598worker2 # [ 40.851704] nix-grpc-daemon[785]: ts=2026-09-20T10:51:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=7524599worker2 # [ 40.857113] nix-grpc-daemon[785]: ts=2026-09-20T10:51:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144600worker2 # [ 40.866899] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4601worker2 # [ 40.871401] nix-daemon[847]: reaped child process 937, status = succeeded4602client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4603worker2 # [ 40.875760] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4604client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4605client # farm-top> LOG-farm-top-t14606worker2 # [ 42.304907] nix-daemon[847]: reaped child process 951, status = succeeded4607worker2 # [ 42.436630] nix-daemon[847]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4608worker2 # [ 42.449280] nix-daemon[847]: reaped child process 970, status = succeeded4609lb # [ 42.479683] niks3-server[1059]: 2026/09/20 10:51:16 INFO Received uploads request method=POST path=/api/pending_closures4610worker2 # [ 42.466606] nix-grpc-daemon[857]: time=2026-09-20T10:51:16.586Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4611worker2 # [ 42.468176] nix-grpc-daemon[857]: time=2026-09-20T10:51:16.588Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4612lb # [ 42.512277] niks3-server[1059]: 2026/09/20 10:51:16 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4613lb # [ 42.528328] niks3-server[1059]: 2026/09/20 10:51:16 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4614lb # [ 42.536977] niks3-server[1059]: 2026/09/20 10:51:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4615worker2 # [ 42.520425] nix-grpc-daemon[857]: time=2026-09-20T10:51:16.640Z level=INFO msg="Uploading 1 narinfos"4616lb # [ 42.541647] niks3-server[1059]: 2026/09/20 10:51:16 INFO Signed narinfos id=6 count=14617lb # [ 42.545646] niks3-server[1059]: 2026/09/20 10:51:16 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4618lb # [ 42.554289] niks3-server[1059]: 2026/09/20 10:51:16 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4619lb # [ 42.557236] niks3-server[1059]: 2026/09/20 10:51:16 INFO Completed upload id=64620worker2 # [ 42.537691] nix-grpc-daemon[857]: time=2026-09-20T10:51:16.657Z level=INFO msg="Upload complete. (253ms)"4621lb # [ 42.559642] niks3-server[1059]: 2026/09/20 10:51:16 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4622worker2 # [ 42.541106] nix-grpc-daemon[785]: ts=2026-09-20T10:51:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14623worker2 # [ 42.547158] nix-daemon[847]: reaped child process 949, status = succeeded4624client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.80 seconds)4625client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4626client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4627client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4628client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4629client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4630client # copying 3 paths...4631client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4632client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4633client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4634client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.20 seconds)4635(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.30 seconds)4636subtest: repeat is answered Cached by the scheduler without building4637worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4638worker1 # finding garbage collector roots...4639worker1 # deleting specified paths...4640worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4641worker1 # deleting unused links...4642worker1 # note: hard linking is currently saving 0.0 KiB4643worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4644worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4645worker2 # finding garbage collector roots...4646worker2 # deleting specified paths...4647worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4648worker2 # deleting unused links...4649worker2 # note: hard linking is currently saving 0.0 KiB4650worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4651client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24652worker1 # [ 43.724833] nix-grpc-daemon[784]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426904653client # don't know how to build these paths:4654client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4655worker1 # [ 43.732990] nix-grpc-daemon[784]: ts=2026-09-20T10:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426904656worker1 # [ 43.759867] nix-grpc-daemon[784]: ts=2026-09-20T10:51:17Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14657client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.22 seconds)4658client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4659client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4660lb: must succeed: curl -sf localhost:9901/clusters4661lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4662lb: must succeed: curl -sf localhost:9901/clusters4663lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4664worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4665worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4666(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.15 seconds)4667subtest: inputs already in the cache are not sent to the scheduler4668lb: must succeed: curl -sf localhost:9901/clusters4669lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4670lb: must succeed: curl -sf localhost:9901/clusters4671lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4672worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4673worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4674client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24675worker2 # [ 44.575236] nix-grpc-daemon[785]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545144676client # don't know how to build these paths:4677client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4678worker2 # [ 44.583686] nix-grpc-daemon[785]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144679worker2 # [ 44.592530] nix-grpc-daemon[857]: time=2026-09-20T10:51:18.712Z level=INFO msg="All 2 paths already cached"4680worker1 # [ 44.598180] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04681worker1 # [ 44.602606] nix-grpc-daemon[784]: ts=2026-09-20T10:51:18Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14682worker2 # [ 44.648854] nix-grpc-daemon[785]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144683worker2 # [ 44.661158] nix-grpc-daemon[785]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144684worker2 # [ 44.672671] nix-grpc-daemon[857]: time=2026-09-20T10:51:18.792Z level=INFO msg="All 2 paths already cached"4685client # copying 1 paths to 'lb:50051'...4686client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4687worker2 # [ 44.680795] nix-grpc-daemon[785]: ts=2026-09-20T10:51:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545144688worker2 # [ 44.696193] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4689worker2 # [ 44.864164] nix-daemon[847]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4690worker2 # [ 44.880189] nix-daemon[847]: reaped child process 990, status = succeeded4691worker2 # [ 45.032478] nix-daemon[847]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4692worker2 # [ 45.044280] nix-daemon[847]: reaped child process 995, status = succeeded4693lb # [ 45.082537] niks3-server[1059]: 2026/09/20 10:51:19 INFO Received uploads request method=POST path=/api/pending_closures4694worker2 # [ 45.067528] nix-grpc-daemon[857]: time=2026-09-20T10:51:19.187Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4695worker2 # [ 45.069034] nix-grpc-daemon[857]: time=2026-09-20T10:51:19.189Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4696lb # [ 45.106632] niks3-server[1059]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4697lb # [ 45.119769] niks3-server[1059]: 2026/09/20 10:51:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4698worker2 # [ 45.103999] nix-grpc-daemon[857]: time=2026-09-20T10:51:19.224Z level=INFO msg="Uploading 1 narinfos"4699lb # [ 45.125252] niks3-server[1059]: 2026/09/20 10:51:19 INFO Signed narinfos id=7 count=14700lb # [ 45.131419] niks3-server[1059]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4701lb # [ 45.143100] niks3-server[1059]: 2026/09/20 10:51:19 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4702lb # [ 45.147799] niks3-server[1059]: 2026/09/20 10:51:19 INFO Completed upload id=74703worker2 # [ 45.129120] nix-grpc-daemon[857]: time=2026-09-20T10:51:19.249Z level=INFO msg="Upload complete. (418ms)"4704worker2 # [ 45.130745] nix-grpc-daemon[785]: ts=2026-09-20T10:51:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=7524705lb # [ 45.152297] niks3-server[1059]: 2026/09/20 10:51:19 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4706worker2 # [ 45.137687] nix-grpc-daemon[785]: ts=2026-09-20T10:51:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144707worker2 # [ 45.146342] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4708worker2 # [ 45.148663] nix-daemon[847]: reaped child process 985, status = succeeded4709worker2 # [ 45.153729] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4710client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4711client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4712client # farm-top> LOG-farm-top-t1b4713worker2 # [ 46.167135] nix-daemon[847]: reaped child process 999, status = succeeded4714worker2 # [ 46.310820] nix-daemon[847]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4715worker2 # [ 46.321951] nix-daemon[847]: reaped child process 1017, status = succeeded4716lb # [ 46.357774] niks3-server[1059]: 2026/09/20 10:51:20 INFO Received uploads request method=POST path=/api/pending_closures4717worker2 # [ 46.345236] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.465Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4718worker2 # [ 46.346867] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.466Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4719lb # [ 46.396139] niks3-server[1059]: 2026/09/20 10:51:20 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4720lb # [ 46.406189] niks3-server[1059]: 2026/09/20 10:51:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4721lb # [ 46.414259] niks3-server[1059]: 2026/09/20 10:51:20 INFO Signed narinfos id=8 count=14722worker2 # [ 46.396831] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.516Z level=INFO msg="Uploading 1 narinfos"4723lb # [ 46.420998] niks3-server[1059]: 2026/09/20 10:51:20 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4724lb # [ 46.426111] niks3-server[1059]: 2026/09/20 10:51:20 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4725lb # [ 46.435853] niks3-server[1059]: 2026/09/20 10:51:20 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4726lb # [ 46.438446] niks3-server[1059]: 2026/09/20 10:51:20 INFO Completed upload id=84727worker2 # [ 46.417627] nix-grpc-daemon[857]: time=2026-09-20T10:51:20.537Z level=INFO msg="Upload complete. (257ms)"4728lb # [ 46.441078] niks3-server[1059]: 2026/09/20 10:51:20 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4729worker2 # [ 46.422672] nix-grpc-daemon[785]: ts=2026-09-20T10:51:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14730worker2 # [ 46.435488] nix-daemon[847]: reaped child process 997, status = succeeded4731client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.10 seconds)4732client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4733client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds)4734lb: must succeed: curl -sf localhost:9901/clusters4735lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4736lb: must succeed: curl -sf localhost:9901/clusters4737lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4738worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4739worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4740(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.57 seconds)4741subtest: two clients wanting the same drv build it once4742client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4743client # [ 46.837708] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4744client # Running as unit: dedup1.service; invocation ID: 801e3afc5b09485b86082800a56e26874745client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)4746client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4747worker2 # [ 47.066834] nix-grpc-daemon[785]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545144748client # [ 47.064495] nix[894]: don't know how to build these paths:4749client # [ 47.065603] nix[894]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4750worker1 # [ 47.075886] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04751worker1 # [ 47.082891] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14752worker1 # [ 47.087706] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426904753worker1 # [ 47.103115] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426904754client # [ 47.111148] nix[894]: copying 1 paths to 'lb:50051'...4755client # [ 47.112035] nix[894]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4756worker1 # [ 47.114751] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426904757worker1 # [ 47.126545] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4758worker1 # [ 47.210455] nix-grpc-daemon[784]: ts=2026-09-20T10:51:20Z level=info event=niks3_push_started pid=9084759worker1 # [ 47.658636] nix-daemon[857]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4760worker1 # [ 47.673582] nix-daemon[857]: reaped child process 923, status = succeeded4761worker1 # [ 47.842451] nix-daemon[857]: accepted connection from pid 925, user nix-grpc-daemon (trusted)4762worker1 # [ 47.855035] nix-daemon[857]: reaped child process 928, status = succeeded4763lb # [ 47.897219] niks3-server[1059]: 2026/09/20 10:51:22 INFO Received uploads request method=POST path=/api/pending_closures4764worker1 # [ 47.875425] nix-grpc-daemon[908]: time=2026-09-20T10:51:21.291Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4765worker1 # [ 47.878111] nix-grpc-daemon[908]: time=2026-09-20T10:51:21.294Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4766lb # [ 47.921857] niks3-server[1059]: 2026/09/20 10:51:22 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4767lb # [ 47.927290] niks3-server[1059]: 2026/09/20 10:51:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4768lb # [ 47.934646] niks3-server[1059]: 2026/09/20 10:51:22 INFO Signed narinfos id=9 count=14769worker1 # [ 47.909263] nix-grpc-daemon[908]: time=2026-09-20T10:51:21.326Z level=INFO msg="Uploading 1 narinfos"4770lb # [ 47.942124] niks3-server[1059]: 2026/09/20 10:51:22 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4771lb # [ 47.950449] niks3-server[1059]: 2026/09/20 10:51:22 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4772lb # [ 47.955851] niks3-server[1059]: 2026/09/20 10:51:22 INFO Completed upload id=94773worker1 # [ 47.928192] nix-grpc-daemon[908]: time=2026-09-20T10:51:21.345Z level=INFO msg="Upload complete. (449ms)"4774lb # [ 47.957539] niks3-server[1059]: 2026/09/20 10:51:22 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4775worker1 # [ 47.930316] nix-grpc-daemon[784]: ts=2026-09-20T10:51:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=0 paths=1 nar_bytes_in=4324776worker1 # [ 47.936064] nix-daemon[857]: reaped child process 906, status = succeeded4777client # [ 47.922401] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4778worker1 # [ 47.939810] nix-grpc-daemon[784]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426904779client # Running as unit: dedup2.service; invocation ID: 7297a76944a14aeabafceaef1a741b094780worker1 # [ 47.943325] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4781worker1 # [ 47.949979] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4782client # [ 47.946528] nix[894]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4783client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4784worker2 # [ 48.182960] nix-grpc-daemon[785]: ts=2026-09-20T10:51:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545144785client # [ 48.174781] nix[910]: don't know how to build these paths:4786client # [ 48.175709] nix[910]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4787worker1 # [ 48.186422] nix-grpc-daemon[784]: ts=2026-09-20T10:51:21Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04788client # [ 48.189527] nix[910]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4789worker1 # [ 48.192083] nix-grpc-daemon[784]: ts=2026-09-20T10:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426904790lb # [ 49.063688] envoy[931]: conn peer=[2001:db8:1::1]:48582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9524 tx=3822 ms=66114791lb # [ 49.065597] envoy[931]: conn peer=[2001:db8:1::1]:45464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=504792lb # [ 49.067307] envoy[931]: conn peer=[2001:db8:1::1]:45474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4126 tx=1713 ms=18584793client: must succeed: sleep 24794client: (finished: must succeed: sleep 2, in 2.05 seconds)4795client: waiting for success: ! systemctl is-active dedup1 dedup24796worker1 # [ 52.567125] nix-daemon[857]: reaped child process 932, status = succeeded4797worker1 # [ 52.718299] nix-daemon[857]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4798worker1 # [ 52.729712] nix-daemon[857]: reaped child process 971, status = succeeded4799lb # [ 52.773096] niks3-server[1059]: 2026/09/20 10:51:26 INFO Received uploads request method=POST path=/api/pending_closures4800worker1 # [ 52.750416] nix-grpc-daemon[908]: time=2026-09-20T10:51:26.167Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4801worker1 # [ 52.751860] nix-grpc-daemon[908]: time=2026-09-20T10:51:26.169Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4802lb # [ 52.799811] niks3-server[1059]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4803lb # [ 52.804995] niks3-server[1059]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4804lb # [ 52.808605] niks3-server[1059]: 2026/09/20 10:51:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4805lb # [ 52.810709] niks3-server[1059]: 2026/09/20 10:51:26 INFO Signed narinfos id=10 count=14806worker1 # [ 52.785066] nix-grpc-daemon[908]: time=2026-09-20T10:51:26.201Z level=INFO msg="Uploading 1 narinfos"4807lb # [ 52.815425] niks3-server[1059]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4808lb # [ 52.821057] niks3-server[1059]: 2026/09/20 10:51:26 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4809worker1 # [ 52.797614] nix-grpc-daemon[908]: time=2026-09-20T10:51:26.215Z level=INFO msg="Upload complete. (234ms)"4810lb # [ 52.825608] niks3-server[1059]: 2026/09/20 10:51:26 INFO Completed upload id=104811lb # [ 52.827603] niks3-server[1059]: 2026/09/20 10:51:26 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4812worker1 # [ 52.800696] nix-grpc-daemon[784]: ts=2026-09-20T10:51:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14813worker1 # [ 52.804230] nix-grpc-daemon[784]: ts=2026-09-20T10:51:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14814worker1 # [ 52.809928] nix-daemon[857]: reaped child process 930, status = succeeded4815client # [ 52.852334] systemd[1]: dedup2.service: Deactivated successfully.4816client # [ 52.856059] systemd[1]: dedup2.service: Consumed 129ms CPU time over 4.933s wall clock time, 31.2M memory peak, 7.4K incoming IP traffic, 6.3K outgoing IP traffic.4817client # [ 52.860802] systemd[1]: dedup1.service: Deactivated successfully.4818client # [ 52.863986] systemd[1]: dedup1.service: Consumed 134ms CPU time over 6.025s wall clock time, 39.3M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.4819client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.11 seconds)4820client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4821client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4822worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4823worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4824worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4825worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4826worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4827worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4828worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4829worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4830(finished: subtest: two clients wanting the same drv build it once, in 6.97 seconds)4831subtest: build hook: nix-daemon with builders = grpc://lb4832client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4833client # [ 53.939309] systemd[1]: Started Nix Daemon.4834client # [ 54.113369] nix-daemon[952]: accepted connection from pid 949, user root (trusted)4835worker1 # [ 54.292439] nix-grpc-daemon[784]: ts=2026-09-20T10:51:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:426904836worker1 # [ 54.297622] nix-grpc-daemon[784]: ts=2026-09-20T10:51:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426904837worker1 # [ 54.308074] nix-grpc-daemon[784]: ts=2026-09-20T10:51:27Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04838worker1 # [ 54.312561] nix-grpc-daemon[784]: ts=2026-09-20T10:51:27Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14839worker2 # [ 54.359263] nix-grpc-daemon[785]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144840worker2 # [ 54.371506] nix-grpc-daemon[785]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144841worker2 # [ 54.382441] nix-grpc-daemon[785]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545144842worker2 # [ 54.393100] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4843worker2 # [ 54.558187] nix-daemon[847]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4844worker2 # [ 54.570698] nix-daemon[847]: reaped child process 1049, status = succeeded4845worker2 # [ 54.723129] nix-daemon[847]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4846worker2 # [ 54.734761] nix-daemon[847]: reaped child process 1054, status = succeeded4847lb # [ 54.772571] niks3-server[1059]: 2026/09/20 10:51:28 INFO Received uploads request method=POST path=/api/pending_closures4848worker2 # [ 54.758101] nix-grpc-daemon[857]: time=2026-09-20T10:51:28.877Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4849worker2 # [ 54.759551] nix-grpc-daemon[857]: time=2026-09-20T10:51:28.877Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4850lb # [ 54.793947] niks3-server[1059]: 2026/09/20 10:51:28 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4851lb # [ 54.810237] niks3-server[1059]: 2026/09/20 10:51:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4852worker2 # [ 54.795703] nix-grpc-daemon[857]: time=2026-09-20T10:51:28.915Z level=INFO msg="Uploading 1 narinfos"4853lb # [ 54.817292] niks3-server[1059]: 2026/09/20 10:51:28 INFO Signed narinfos id=11 count=14854lb # [ 54.821981] niks3-server[1059]: 2026/09/20 10:51:28 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4855lb # [ 54.835401] niks3-server[1059]: 2026/09/20 10:51:28 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4856lb # [ 54.841787] niks3-server[1059]: 2026/09/20 10:51:28 INFO Completed upload id=114857worker2 # [ 54.820857] nix-grpc-daemon[857]: time=2026-09-20T10:51:28.940Z level=INFO msg="Upload complete. (414ms)"4858lb # [ 54.846809] niks3-server[1059]: 2026/09/20 10:51:28 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4859worker2 # [ 54.824853] nix-grpc-daemon[785]: ts=2026-09-20T10:51:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=5124860worker2 # [ 54.831123] nix-grpc-daemon[785]: ts=2026-09-20T10:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144861worker2 # [ 54.839462] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4862worker2 # [ 54.846388] nix-daemon[847]: reaped child process 1044, status = succeeded4863worker2 # [ 54.851357] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4864worker2 # [ 55.676863] nix-daemon[847]: reaped child process 1058, status = succeeded4865worker2 # [ 55.821775] nix-daemon[847]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)4866worker2 # [ 55.832487] nix-daemon[847]: reaped child process 1076, status = succeeded4867lb # [ 55.868764] niks3-server[1059]: 2026/09/20 10:51:29 INFO Received uploads request method=POST path=/api/pending_closures4868worker2 # [ 55.855511] nix-grpc-daemon[857]: time=2026-09-20T10:51:29.975Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4869worker2 # [ 55.858042] nix-grpc-daemon[857]: time=2026-09-20T10:51:29.977Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4870lb # [ 55.909576] niks3-server[1059]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4871lb # [ 55.921793] niks3-server[1059]: 2026/09/20 10:51:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4872worker2 # [ 55.908755] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.028Z level=INFO msg="Uploading 1 narinfos"4873lb # [ 55.929917] niks3-server[1059]: 2026/09/20 10:51:30 INFO Signed narinfos id=12 count=14874lb # [ 55.934349] niks3-server[1059]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4875lb # [ 55.938581] niks3-server[1059]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4876lb # [ 55.949429] niks3-server[1059]: 2026/09/20 10:51:30 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4877lb # [ 55.953960] niks3-server[1059]: 2026/09/20 10:51:30 INFO Completed upload id=124878lb # [ 55.956306] niks3-server[1059]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4879worker2 # [ 55.934444] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.054Z level=INFO msg="Upload complete. (276ms)"4880worker2 # [ 55.941140] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14881worker1 # [ 55.937446] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:426904882worker2 # [ 55.953146] nix-daemon[847]: reaped child process 1056, status = succeeded4883worker1 # [ 55.950752] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4884worker2 # [ 55.986228] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:545144885worker2 # [ 55.991701] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_out=1284886worker2 # [ 56.219120] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:545144887worker1 # [ 56.215728] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426904888worker1 # [ 56.224837] nix-grpc-daemon[908]: time=2026-09-20T10:51:29.641Z level=INFO msg="All 1 paths already cached"4889worker1 # [ 56.238530] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04890worker2 # [ 56.247758] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144891worker1 # [ 56.242867] nix-grpc-daemon[784]: ts=2026-09-20T10:51:29Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14892worker2 # [ 56.258860] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144893worker2 # [ 56.270536] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.390Z level=INFO msg="All 1 paths already cached"4894worker2 # [ 56.277098] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545144895worker2 # [ 56.286330] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4896worker2 # [ 56.461043] nix-daemon[847]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)4897worker2 # [ 56.473717] nix-daemon[847]: reaped child process 1083, status = succeeded4898worker2 # [ 56.614050] nix-daemon[847]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)4899worker2 # [ 56.625600] nix-daemon[847]: reaped child process 1088, status = succeeded4900lb # [ 56.658833] niks3-server[1059]: 2026/09/20 10:51:30 INFO Received uploads request method=POST path=/api/pending_closures4901worker2 # [ 56.645786] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.765Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4902worker2 # [ 56.647655] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.767Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4903lb # [ 56.684327] niks3-server[1059]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4904lb # [ 56.696162] niks3-server[1059]: 2026/09/20 10:51:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4905worker2 # [ 56.680554] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.800Z level=INFO msg="Uploading 1 narinfos"4906lb # [ 56.701697] niks3-server[1059]: 2026/09/20 10:51:30 INFO Signed narinfos id=13 count=14907lb # [ 56.708584] niks3-server[1059]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4908lb # [ 56.721466] niks3-server[1059]: 2026/09/20 10:51:30 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4909worker2 # [ 56.718458] nix-grpc-daemon[857]: time=2026-09-20T10:51:30.838Z level=INFO msg="Upload complete. (420ms)"4910lb # [ 56.739772] niks3-server[1059]: 2026/09/20 10:51:30 INFO Completed upload id=134911worker2 # [ 56.720587] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=6244912lb # [ 56.743411] niks3-server[1059]: 2026/09/20 10:51:30 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4913worker2 # [ 56.727452] nix-grpc-daemon[785]: ts=2026-09-20T10:51:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144914worker2 # [ 56.738660] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4915worker2 # [ 56.743223] nix-daemon[847]: reaped child process 1078, status = succeeded4916worker2 # [ 56.748915] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4917worker2 # [ 57.961584] nix-daemon[847]: reaped child process 1092, status = succeeded4918worker2 # [ 58.088099] nix-daemon[847]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4919worker2 # [ 58.099674] nix-daemon[847]: reaped child process 1110, status = succeeded4920lb # [ 58.131823] niks3-server[1059]: 2026/09/20 10:51:32 INFO Received uploads request method=POST path=/api/pending_closures4921worker2 # [ 58.114881] nix-grpc-daemon[857]: time=2026-09-20T10:51:32.234Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4922worker2 # [ 58.116453] nix-grpc-daemon[857]: time=2026-09-20T10:51:32.236Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4923lb # [ 58.158993] niks3-server[1059]: 2026/09/20 10:51:32 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4924lb # [ 58.171138] niks3-server[1059]: 2026/09/20 10:51:32 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4925lb # [ 58.183808] niks3-server[1059]: 2026/09/20 10:51:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4926lb # [ 58.189927] niks3-server[1059]: 2026/09/20 10:51:32 INFO Signed narinfos id=14 count=14927worker2 # [ 58.171380] nix-grpc-daemon[857]: time=2026-09-20T10:51:32.291Z level=INFO msg="Uploading 1 narinfos"4928lb # [ 58.192460] niks3-server[1059]: 2026/09/20 10:51:32 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4929lb # [ 58.203112] niks3-server[1059]: 2026/09/20 10:51:32 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4930lb # [ 58.209069] niks3-server[1059]: 2026/09/20 10:51:32 INFO Completed upload id=144931worker2 # [ 58.187834] nix-grpc-daemon[857]: time=2026-09-20T10:51:32.308Z level=INFO msg="Upload complete. (232ms)"4932lb # [ 58.210609] niks3-server[1059]: 2026/09/20 10:51:32 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4933worker2 # [ 58.192278] nix-grpc-daemon[785]: ts=2026-09-20T10:51:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14934worker2 # [ 58.197949] nix-grpc-daemon[785]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:545144935worker2 # [ 58.206684] nix-daemon[847]: reaped child process 1090, status = succeeded4936worker1 # [ 58.204360] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:426904937worker1 # [ 58.208951] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4938worker1 # [ 58.214934] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=0 paths=1 nar_bytes_out=1764939worker2 # [ 58.477858] nix-grpc-daemon[785]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:545144940worker1 # [ 58.478689] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426904941worker1 # [ 58.488837] nix-grpc-daemon[908]: time=2026-09-20T10:51:31.905Z level=INFO msg="All 2 paths already cached"4942worker1 # [ 58.502858] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04943worker1 # [ 58.506982] nix-grpc-daemon[784]: ts=2026-09-20T10:51:31Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14944worker2 # [ 58.553484] nix-grpc-daemon[785]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144945worker2 # [ 58.565676] nix-grpc-daemon[785]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545144946worker2 # [ 58.577655] nix-grpc-daemon[857]: time=2026-09-20T10:51:32.697Z level=INFO msg="All 2 paths already cached"4947worker2 # [ 58.584582] nix-grpc-daemon[785]: ts=2026-09-20T10:51:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545144948worker2 # [ 58.594332] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4949worker2 # [ 58.749822] nix-daemon[847]: accepted connection from pid 1114, user nix-grpc-daemon (trusted)4950worker2 # [ 58.761670] nix-daemon[847]: reaped child process 1117, status = succeeded4951worker2 # [ 58.898143] nix-daemon[847]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4952worker2 # [ 58.909797] nix-daemon[847]: reaped child process 1122, status = succeeded4953lb # [ 58.944570] niks3-server[1059]: 2026/09/20 10:51:33 INFO Received uploads request method=POST path=/api/pending_closures4954worker2 # [ 58.928096] nix-grpc-daemon[857]: time=2026-09-20T10:51:33.047Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4955worker2 # [ 58.929581] nix-grpc-daemon[857]: time=2026-09-20T10:51:33.048Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4956lb # [ 58.957628] niks3-server[1059]: 2026/09/20 10:51:33 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4957lb # [ 58.960595] niks3-server[1059]: 2026/09/20 10:51:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4958lb # [ 58.962430] niks3-server[1059]: 2026/09/20 10:51:33 INFO Signed narinfos id=15 count=14959worker2 # [ 58.942799] nix-grpc-daemon[857]: time=2026-09-20T10:51:33.062Z level=INFO msg="Uploading 1 narinfos"4960lb # [ 58.967988] niks3-server[1059]: 2026/09/20 10:51:33 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4961lb # [ 58.970631] niks3-server[1059]: 2026/09/20 10:51:33 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4962worker2 # [ 58.952914] nix-grpc-daemon[857]: time=2026-09-20T10:51:33.073Z level=INFO msg="Upload complete. (346ms)"4963lb # [ 58.974249] niks3-server[1059]: 2026/09/20 10:51:33 INFO Completed upload id=154964worker2 # [ 58.954719] nix-grpc-daemon[785]: ts=2026-09-20T10:51:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=7524965lb # [ 58.977365] niks3-server[1059]: 2026/09/20 10:51:33 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4966worker2 # [ 58.958522] nix-grpc-daemon[785]: ts=2026-09-20T10:51:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545144967worker2 # [ 58.964410] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4968worker2 # [ 58.966541] nix-daemon[847]: reaped child process 1112, status = succeeded4969worker2 # [ 58.971697] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4970lb # [ 59.064306] envoy[931]: conn peer=[2001:db8:1::1]:45486 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=46484971lb # [ 59.068604] envoy[931]: conn peer=[2001:db8:1::1]:45482 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2985 tx=1343 ms=57664972lb # [ 59.072735] envoy[931]: conn peer=[2001:db8:1::1]:52756 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=264973lb # [ 59.076663] envoy[931]: conn peer=[2001:db8:1::1]:52740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3423 tx=1747 ms=17154974lb # [ 59.080723] envoy[931]: conn peer=[2001:db8:1::1]:52782 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=614975lb # [ 59.084752] envoy[931]: conn peer=[2001:db8:1::1]:52772 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3837 tx=1839 ms=20534976worker2 # [ 59.996821] nix-daemon[847]: reaped child process 1126, status = succeeded4977worker2 # [ 60.123485] nix-daemon[847]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)4978worker2 # [ 60.135363] nix-daemon[847]: reaped child process 1144, status = succeeded4979lb # [ 60.169444] niks3-server[1059]: 2026/09/20 10:51:34 INFO Received uploads request method=POST path=/api/pending_closures4980worker2 # [ 60.158036] nix-grpc-daemon[857]: time=2026-09-20T10:51:34.277Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4981worker2 # [ 60.159533] nix-grpc-daemon[857]: time=2026-09-20T10:51:34.279Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4982lb # [ 60.205161] niks3-server[1059]: 2026/09/20 10:51:34 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4983lb # [ 60.220569] niks3-server[1059]: 2026/09/20 10:51:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4984worker2 # [ 60.205330] nix-grpc-daemon[857]: time=2026-09-20T10:51:34.325Z level=INFO msg="Uploading 1 narinfos"4985lb # [ 60.225915] niks3-server[1059]: 2026/09/20 10:51:34 INFO Signed narinfos id=16 count=14986lb # [ 60.233905] niks3-server[1059]: 2026/09/20 10:51:34 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4987lb # [ 60.239251] niks3-server[1059]: 2026/09/20 10:51:34 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4988lb # [ 60.246158] niks3-server[1059]: 2026/09/20 10:51:34 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4989lb # [ 60.249659] niks3-server[1059]: 2026/09/20 10:51:34 INFO Completed upload id=164990worker2 # [ 60.228539] nix-grpc-daemon[857]: time=2026-09-20T10:51:34.348Z level=INFO msg="Upload complete. (249ms)"4991lb # [ 60.251574] niks3-server[1059]: 2026/09/20 10:51:34 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4992worker2 # [ 60.235131] nix-grpc-daemon[785]: ts=2026-09-20T10:51:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14993worker2 # [ 60.241603] nix-grpc-daemon[785]: ts=2026-09-20T10:51:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:545144994worker2 # [ 60.252172] nix-daemon[847]: reaped child process 1124, status = succeeded4995worker2 # [ 60.254921] nix-grpc-daemon[785]: ts=2026-09-20T10:51:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:545144996worker2 # [ 60.256673] nix-grpc-daemon[785]: ts=2026-09-20T10:51:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_out=2324997client # [ 60.356648] nix-daemon[952]: reaped child process 954, status = succeeded4998client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.62 seconds)4999client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5000client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5001client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5002client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)5003client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5004client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5005(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.71 seconds)5006subtest: an input only one worker has still reaches the builder5007client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5008client # this derivation will be built:5009client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5010client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5011client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.40 seconds)5012client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5013client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds)5014worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5015worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)5016worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5017worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.01 seconds)5018client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25019client # [ 61.434498] nix-daemon[952]: accepted connection from pid 1035, user root (trusted)5020client # this derivation will be built:5021client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5022worker1 # [ 61.646808] nix-grpc-daemon[784]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:426905023worker2 # [ 61.660674] nix-grpc-daemon[785]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545145024client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5025client # copying 1 paths...5026client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5027worker2 # [ 61.668298] nix-grpc-daemon[785]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545145028worker2 # [ 61.676362] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5029worker2 # [ 61.867360] nix-daemon[847]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)5030worker2 # [ 61.883096] nix-daemon[847]: reaped child process 1154, status = succeeded5031lb # [ 61.919628] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received uploads request method=POST path=/api/pending_closures5032worker2 # [ 61.902801] nix-grpc-daemon[857]: time=2026-09-20T10:51:36.022Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5033worker2 # [ 61.904574] nix-grpc-daemon[857]: time=2026-09-20T10:51:36.024Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5034lb # [ 61.934853] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5035lb # [ 61.937149] niks3-server[1059]: 2026/09/20 10:51:36 INFO Signed narinfos id=17 count=15036worker2 # [ 61.917441] nix-grpc-daemon[857]: time=2026-09-20T10:51:36.037Z level=INFO msg="Uploading 1 narinfos"5037lb # [ 61.940785] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5038lb # [ 61.945182] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5039lb # [ 61.948634] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5040lb # [ 61.952286] niks3-server[1059]: 2026/09/20 10:51:36 INFO Completed upload id=175041worker2 # [ 61.931214] nix-grpc-daemon[857]: time=2026-09-20T10:51:36.051Z level=INFO msg="Upload complete. (243ms)"5042worker2 # [ 61.933150] nix-grpc-daemon[785]: ts=2026-09-20T10:51:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_in=1205043lb # [ 61.954751] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5044worker2 # [ 61.938905] nix-daemon[847]: reaped child process 1149, status = succeeded5045worker1 # [ 61.939151] nix-grpc-daemon[784]: ts=2026-09-20T10:51:35Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05046worker1 # [ 61.943389] nix-grpc-daemon[784]: ts=2026-09-20T10:51:35Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15047worker1 # [ 61.993239] nix-grpc-daemon[784]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426905048worker1 # [ 62.005355] nix-grpc-daemon[784]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426905049client # copying 2 paths to 'lb:50051'...5050client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5051worker1 # [ 62.019153] nix-grpc-daemon[784]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426905052client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5053worker1 # [ 62.031843] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5054worker1 # [ 62.201025] nix-daemon[857]: accepted connection from pid 999, user nix-grpc-daemon (trusted)5055worker1 # [ 62.214127] nix-daemon[857]: reaped child process 1002, status = succeeded5056worker1 # [ 62.360591] nix-daemon[857]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)5057worker1 # [ 62.370915] nix-daemon[857]: reaped child process 1007, status = succeeded5058lb # [ 62.413763] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received uploads request method=POST path=/api/pending_closures5059lb # [ 62.422417] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received uploads request method=POST path=/api/pending_closures5060worker1 # [ 62.403210] nix-grpc-daemon[908]: time=2026-09-20T10:51:35.820Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5061worker1 # [ 62.405896] nix-grpc-daemon[908]: time=2026-09-20T10:51:35.822Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5062worker1 # [ 62.407815] nix-grpc-daemon[908]: time=2026-09-20T10:51:35.822Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5063lb # [ 62.454163] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5064lb # [ 62.473122] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5065lb # [ 62.483163] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5066lb # [ 62.487658] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5067lb # [ 62.490731] niks3-server[1059]: 2026/09/20 10:51:36 INFO Signed narinfos id=18 count=15068lb # [ 62.494143] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5069lb # [ 62.496020] niks3-server[1059]: 2026/09/20 10:51:36 INFO Signed narinfos id=19 count=25070worker1 # [ 62.469730] nix-grpc-daemon[908]: time=2026-09-20T10:51:35.886Z level=INFO msg="Uploading 3 narinfos"5071lb # [ 62.499830] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5072lb # [ 62.511872] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5073lb # [ 62.514558] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5074lb # [ 62.517699] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5075lb # [ 62.520942] niks3-server[1059]: 2026/09/20 10:51:36 INFO Completed upload id=185076lb # [ 62.522353] niks3-server[1059]: 2026/09/20 10:51:36 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5077lb # [ 62.524433] niks3-server[1059]: 2026/09/20 10:51:36 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5078lb # [ 62.526489] niks3-server[1059]: 2026/09/20 10:51:36 INFO Completed upload id=195079worker1 # [ 62.498757] nix-grpc-daemon[908]: time=2026-09-20T10:51:35.916Z level=INFO msg="Upload complete. (452ms)"5080worker1 # [ 62.501046] nix-grpc-daemon[784]: ts=2026-09-20T10:51:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=0 paths=2 nar_bytes_in=9765081worker1 # [ 62.504172] nix-grpc-daemon[784]: ts=2026-09-20T10:51:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426905082worker1 # [ 62.508571] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5083worker1 # [ 62.513639] nix-daemon[857]: reaped child process 997, status = succeeded5084worker1 # [ 62.518309] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5085client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5086client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5087worker1 # [ 63.333511] nix-daemon[857]: reaped child process 1011, status = succeeded5088worker1 # [ 63.472562] nix-daemon[857]: accepted connection from pid 1026, user nix-grpc-daemon (trusted)5089worker1 # [ 63.484112] nix-daemon[857]: reaped child process 1029, status = succeeded5090lb # [ 63.524623] niks3-server[1059]: 2026/09/20 10:51:37 INFO Received uploads request method=POST path=/api/pending_closures5091worker1 # [ 63.504935] nix-grpc-daemon[908]: time=2026-09-20T10:51:36.921Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5092lb # [ 63.552070] niks3-server[1059]: 2026/09/20 10:51:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5093worker1 # [ 63.529540] nix-grpc-daemon[908]: time=2026-09-20T10:51:36.946Z level=INFO msg="Uploading 1 narinfos"5094lb # [ 63.557521] niks3-server[1059]: 2026/09/20 10:51:37 INFO Signed narinfos id=20 count=15095lb # [ 63.564408] niks3-server[1059]: 2026/09/20 10:51:37 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5096lb # [ 63.576639] niks3-server[1059]: 2026/09/20 10:51:37 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5097lb # [ 63.582436] niks3-server[1059]: 2026/09/20 10:51:37 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5098worker1 # [ 63.560383] nix-grpc-daemon[908]: time=2026-09-20T10:51:36.977Z level=INFO msg="Upload complete. (236ms)"5099lb # [ 63.588638] niks3-server[1059]: 2026/09/20 10:51:37 INFO Completed upload id=205100worker1 # [ 63.562640] nix-grpc-daemon[784]: ts=2026-09-20T10:51:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15101lb # [ 63.593475] niks3-server[1059]: 2026/09/20 10:51:37 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5102worker1 # [ 63.569413] nix-daemon[857]: reaped child process 1009, status = succeeded5103client # copying 1 paths...5104worker1 # [ 63.609292] nix-grpc-daemon[784]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:426905105client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5106worker1 # [ 63.631217] nix-grpc-daemon[784]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:426905107worker1 # [ 63.632821] nix-grpc-daemon[784]: ts=2026-09-20T10:51:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=0 paths=1 nar_bytes_out=1205108client # [ 63.733930] nix-daemon[952]: reaped child process 1038, status = succeeded5109client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.50 seconds)5110client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25111client # [ 63.907569] nix-daemon[952]: accepted connection from pid 1057, user root (trusted)5112client # this derivation will be built:5113client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5114worker2 # [ 64.140618] nix-grpc-daemon[785]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:545145115client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5116worker1 # [ 64.140133] nix-grpc-daemon[784]: ts=2026-09-20T10:51:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426905117worker1 # [ 64.145555] nix-grpc-daemon[908]: time=2026-09-20T10:51:37.562Z level=INFO msg="All 1 paths already cached"5118worker1 # [ 64.155247] nix-grpc-daemon[784]: ts=2026-09-20T10:51:37Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05119worker1 # [ 64.157565] nix-grpc-daemon[784]: ts=2026-09-20T10:51:37Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15120worker2 # [ 64.166534] nix-grpc-daemon[785]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545145121worker2 # [ 64.178445] nix-grpc-daemon[785]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545145122client # copying 1 paths to 'lb:50051'...5123client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5124worker2 # [ 64.195153] nix-grpc-daemon[785]: ts=2026-09-20T10:51:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545145125worker2 # [ 64.206277] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5126worker2 # [ 64.222389] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 1/5)5127worker2 # [ 64.279246] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 2/5)5128worker2 # [ 64.346087] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 340 ms (attempt 3/5)5129worker2 # [ 64.692136] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 481 ms (attempt 4/5)5130worker2 # [ 65.180202] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5131worker2 # [ 65.185887] nix-grpc-daemon[785]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5132worker2 # [ 65.379767] nix-daemon[847]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5133worker2 # [ 65.393200] nix-daemon[847]: reaped child process 1165, status = succeeded5134worker2 # [ 65.550806] nix-daemon[847]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)5135worker2 # [ 65.563171] nix-daemon[847]: reaped child process 1170, status = succeeded5136lb # [ 65.598224] niks3-server[1059]: 2026/09/20 10:51:39 INFO Received uploads request method=POST path=/api/pending_closures5137worker2 # [ 65.581692] nix-grpc-daemon[857]: time=2026-09-20T10:51:39.701Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5138worker2 # [ 65.583141] nix-grpc-daemon[857]: time=2026-09-20T10:51:39.701Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5139lb # [ 65.611305] niks3-server[1059]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5140lb # [ 65.614628] niks3-server[1059]: 2026/09/20 10:51:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5141lb # [ 65.616355] niks3-server[1059]: 2026/09/20 10:51:39 INFO Signed narinfos id=21 count=15142worker2 # [ 65.595464] nix-grpc-daemon[857]: time=2026-09-20T10:51:39.715Z level=INFO msg="Uploading 1 narinfos"5143lb # [ 65.620136] niks3-server[1059]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5144lb # [ 65.624056] niks3-server[1059]: 2026/09/20 10:51:39 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5145lb # [ 65.627249] niks3-server[1059]: 2026/09/20 10:51:39 INFO Completed upload id=215146worker2 # [ 65.606172] nix-grpc-daemon[857]: time=2026-09-20T10:51:39.726Z level=INFO msg="Upload complete. (397ms)"5147lb # [ 65.629779] niks3-server[1059]: 2026/09/20 10:51:39 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5148worker2 # [ 65.608134] nix-grpc-daemon[785]: ts=2026-09-20T10:51:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 paths=1 nar_bytes_in=5605149worker2 # [ 65.610842] nix-grpc-daemon[785]: ts=2026-09-20T10:51:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545145150worker2 # [ 65.616288] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5151worker2 # [ 65.620815] nix-daemon[847]: reaped child process 1156, status = succeeded5152worker2 # [ 65.625739] nix-daemon[847]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5153client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5154client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5155worker2 # [ 66.652454] nix-daemon[847]: reaped child process 1174, status = succeeded5156worker2 # [ 66.796605] nix-daemon[847]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)5157worker2 # [ 66.808607] nix-daemon[847]: reaped child process 1192, status = succeeded5158lb # [ 66.844536] niks3-server[1059]: 2026/09/20 10:51:40 INFO Received uploads request method=POST path=/api/pending_closures5159worker2 # [ 66.831326] nix-grpc-daemon[857]: time=2026-09-20T10:51:40.951Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5160lb # [ 66.875259] niks3-server[1059]: 2026/09/20 10:51:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5161lb # [ 66.881958] niks3-server[1059]: 2026/09/20 10:51:41 INFO Signed narinfos id=22 count=15162worker2 # [ 66.864780] nix-grpc-daemon[857]: time=2026-09-20T10:51:40.984Z level=INFO msg="Uploading 1 narinfos"5163lb # [ 66.887140] niks3-server[1059]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5164lb # [ 66.893317] niks3-server[1059]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5165lb # [ 66.907095] niks3-server[1059]: 2026/09/20 10:51:41 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5166lb # [ 66.911693] niks3-server[1059]: 2026/09/20 10:51:41 INFO Completed upload id=225167worker2 # [ 66.890744] nix-grpc-daemon[857]: time=2026-09-20T10:51:41.010Z level=INFO msg="Upload complete. (257ms)"5168lb # [ 66.913929] niks3-server[1059]: 2026/09/20 10:51:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5169worker2 # [ 66.896628] nix-grpc-daemon[785]: ts=2026-09-20T10:51:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15170worker2 # [ 66.906985] nix-daemon[847]: reaped child process 1172, status = succeeded5171client # copying 1 paths...5172worker2 # [ 66.926206] nix-grpc-daemon[785]: ts=2026-09-20T10:51:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:545145173client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5174worker1 # [ 66.979734] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:426905175worker1 # [ 66.991371] nix-grpc-daemon[784]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5176worker1 # [ 67.004597] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=0 paths=1 nar_bytes_out=1205177client # [ 67.109696] nix-daemon[952]: reaped child process 1060, status = succeeded5178client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.37 seconds)5179client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25180client # [ 67.256179] nix-daemon[952]: accepted connection from pid 1079, user root (trusted)5181client # this derivation will be built:5182client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5183worker1 # [ 67.431436] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:426905184client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5185worker1 # [ 67.438055] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426905186worker1 # [ 67.445545] nix-grpc-daemon[908]: time=2026-09-20T10:51:40.860Z level=INFO msg="All 1 paths already cached"5187worker1 # [ 67.455714] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05188worker1 # [ 67.462863] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15189worker1 # [ 67.468898] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426905190worker1 # [ 67.480331] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426905191worker1 # [ 67.492978] nix-grpc-daemon[908]: time=2026-09-20T10:51:40.910Z level=INFO msg="All 1 paths already cached"5192client # copying 1 paths to 'lb:50051'...5193client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5194worker1 # [ 67.531213] nix-grpc-daemon[784]: ts=2026-09-20T10:51:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426905195worker1 # [ 67.542332] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5196worker1 # [ 67.700129] nix-daemon[857]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5197worker1 # [ 67.712559] nix-daemon[857]: reaped child process 1041, status = succeeded5198worker1 # [ 67.861996] nix-daemon[857]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5199worker1 # [ 67.873671] nix-daemon[857]: reaped child process 1046, status = succeeded5200lb # [ 67.915582] niks3-server[1059]: 2026/09/20 10:51:42 INFO Received uploads request method=POST path=/api/pending_closures5201worker1 # [ 67.895777] nix-grpc-daemon[908]: time=2026-09-20T10:51:41.312Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5202worker1 # [ 67.897141] nix-grpc-daemon[908]: time=2026-09-20T10:51:41.312Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5203lb # [ 67.942153] niks3-server[1059]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5204lb # [ 67.951078] niks3-server[1059]: 2026/09/20 10:51:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5205lb # [ 67.956415] niks3-server[1059]: 2026/09/20 10:51:42 INFO Signed narinfos id=23 count=15206worker1 # [ 67.931751] nix-grpc-daemon[908]: time=2026-09-20T10:51:41.349Z level=INFO msg="Uploading 1 narinfos"5207lb # [ 67.963463] niks3-server[1059]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5208lb # [ 67.973813] niks3-server[1059]: 2026/09/20 10:51:42 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5209worker1 # [ 67.951920] nix-grpc-daemon[908]: time=2026-09-20T10:51:41.369Z level=INFO msg="Upload complete. (397ms)"5210lb # [ 67.979822] niks3-server[1059]: 2026/09/20 10:51:42 INFO Completed upload id=235211lb # [ 67.982771] niks3-server[1059]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5212worker1 # [ 67.954532] nix-grpc-daemon[784]: ts=2026-09-20T10:51:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=0 paths=1 nar_bytes_in=5605213worker1 # [ 67.959604] nix-grpc-daemon[784]: ts=2026-09-20T10:51:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426905214worker1 # [ 67.968468] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5215worker1 # [ 67.971859] nix-daemon[857]: reaped child process 1036, status = succeeded5216worker1 # [ 67.976947] nix-daemon[857]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5217client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5218client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5219worker1 # [ 68.598709] nix-daemon[857]: reaped child process 1050, status = succeeded5220worker1 # [ 68.744279] nix-daemon[857]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)5221worker1 # [ 68.755769] nix-daemon[857]: reaped child process 1068, status = succeeded5222lb # [ 68.792878] niks3-server[1059]: 2026/09/20 10:51:42 INFO Received uploads request method=POST path=/api/pending_closures5223worker1 # [ 68.768977] nix-grpc-daemon[908]: time=2026-09-20T10:51:42.186Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5224lb # [ 68.817825] niks3-server[1059]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5225lb # [ 68.825367] niks3-server[1059]: 2026/09/20 10:51:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5226lb # [ 68.830794] niks3-server[1059]: 2026/09/20 10:51:42 INFO Signed narinfos id=24 count=15227worker1 # [ 68.805725] nix-grpc-daemon[908]: time=2026-09-20T10:51:42.223Z level=INFO msg="Uploading 1 narinfos"5228lb # [ 68.833939] niks3-server[1059]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5229lb # [ 68.846576] niks3-server[1059]: 2026/09/20 10:51:42 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5230lb # [ 68.851083] niks3-server[1059]: 2026/09/20 10:51:42 INFO Completed upload id=245231worker1 # [ 68.823403] nix-grpc-daemon[908]: time=2026-09-20T10:51:42.240Z level=INFO msg="Upload complete. (238ms)"5232lb # [ 68.855228] niks3-server[1059]: 2026/09/20 10:51:42 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5233worker1 # [ 68.828110] nix-grpc-daemon[784]: ts=2026-09-20T10:51:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42690 duration_s=0 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15234worker1 # [ 68.833218] nix-daemon[857]: reaped child process 1048, status = succeeded5235client # copying 1 paths...5236worker1 # [ 68.872227] nix-grpc-daemon[784]: ts=2026-09-20T10:51:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:426905237client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5238worker2 # [ 68.903937] nix-grpc-daemon[785]: ts=2026-09-20T10:51:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:545145239worker2 # [ 68.918901] nix-grpc-daemon[785]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5240worker2 # [ 68.930099] nix-grpc-daemon[785]: ts=2026-09-20T10:51:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54514 duration_s=0 paths=1 nar_bytes_out=1205241lb # [ 69.063602] envoy[931]: conn peer=[2001:db8:1::1]:52794 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4328 tx=2085 ms=18285242client # [ 69.033286] nix-daemon[952]: reaped child process 1082, status = succeeded5243lb # [ 69.065105] envoy[931]: conn peer=[2001:db8:1::1]:52800 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=525244lb # [ 69.066498] envoy[931]: conn peer=[2001:db8:1::1]:35184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545245lb # [ 69.067903] envoy[931]: conn peer=[2001:db8:1::1]:52802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4217 tx=1750 ms=20355246lb # [ 69.069318] envoy[931]: conn peer=[2001:db8:1::1]:35198 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3711 tx=1806 ms=29255247client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.93 seconds)5248lb # [ 69.070711] envoy[931]: conn peer=[2001:db8:1::1]:35212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=805249(finished: subtest: an input only one worker has still reaches the builder, in 8.58 seconds)5250subtest: upload whose reference the worker lacks is completed from the cache5251client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5252lb # [ 69.072107] envoy[931]: conn peer=[2001:db8:1::1]:35216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3699 tx=1780 ms=15435253lb # [ 69.073442] envoy[931]: conn peer=[2001:db8:1::1]:35224 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=815254client # this derivation will be built:5255client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5256client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5257client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.43 seconds)5258client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5259client # this derivation will be built:5260client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5261client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5262client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.43 seconds)5263client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5264client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.16 seconds)5265worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5266worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.21 seconds)5267worker2 # [ 70.335463] systemd[1]: Stopping Nix worker-protocol over gRPC...5268worker2 # [ 71.315542] nix-grpc-daemon[785]: ts=2026-09-20T10:51:45Z level=info event=signal_exit5269worker2 # [ 72.581959] nix-grpc-daemon[785]: ts=2026-09-20T10:51:46Z level=info event=scheduler_restarting5270worker1 # [ 72.583746] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:390065271worker1 # [ 72.585772] nix-grpc-daemon[784]: ts=2026-09-20T10:51:46Z level=info event=worker_gone addr=192.168.1.4:500515272worker2 # [ 72.603635] nix-grpc-daemon[785]: ts=2026-09-20T10:51:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5273worker2 # [ 72.619426] nix-daemon[847]: reaped child process 849, status = succeeded5274worker2 # [ 73.426959] nix-grpc-daemon[785]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5275worker2 # [ 73.428495] nix-grpc-daemon[785]: E0000 00:00:1789901507.546765 828 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5276worker2 # [ 73.430417] nix-grpc-daemon[785]: E0000 00:00:1789901507.547113 828 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5277worker2 # [ 73.789744] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5278worker2 # [ 73.791605] systemd[1]: Stopped Nix worker-protocol over gRPC.5279worker2 # [ 73.793278] systemd[1]: nix-grpc-daemon.service: Consumed 2.560s CPU time over 54.748s wall clock time, 61.3M memory peak, 85.5K incoming IP traffic, 160.9K outgoing IP traffic.5280worker2 # [ 73.797116] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5281worker2 # [ 73.798779] systemd[1]: Closed Nix worker-protocol over gRPC.5282worker2 # [ 73.800319] systemd[1]: nix-grpc-daemon.socket: Consumed 290us CPU time over 1min 4.475s wall clock time, 256K memory peak, 74.6K incoming IP traffic, 60.5K outgoing IP traffic.5283lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5284lb: must succeed: curl -sf localhost:9901/clusters5285lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5286lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5287client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25288worker1 # [ 74.055276] nix-grpc-daemon[784]: ts=2026-09-20T10:51:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426905289client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5290worker1 # [ 74.060954] nix-grpc-daemon[784]: ts=2026-09-20T10:51:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:426905291client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.22 seconds)5292worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5293worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5294worker2 # [ 74.146154] systemd[1]: Listening on Nix worker-protocol over gRPC.5295worker1 # [ 74.198066] systemd[1]: Stopping Nix worker-protocol over gRPC...5296worker1 # [ 74.319816] nix-grpc-daemon[784]: ts=2026-09-20T10:51:47Z level=info event=signal_exit5297worker1 # [ 74.588253] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_yield5298lb # [ 74.618727] niks3-server[1059]: 2026/09/20 10:51:48 INFO lead: released remote=192.168.1.3:436825299worker1 # [ 74.592247] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_restarting addr=lb:500515300worker1 # [ 74.597620] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:390065301worker1 # [ 74.601463] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=worker_gone addr=192.168.1.3:500515302worker2 # [ 74.786361] systemd[1]: Starting Nix worker-protocol over gRPC...5303worker1 # [ 74.792803] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500515304worker1 # [ 74.796711] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker2 # [ 74.915696] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5306worker2 # [ 74.919495] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15307worker2 # [ 74.925055] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5308worker2 # [ 74.928404] systemd[1]: Started Nix worker-protocol over gRPC.5309worker2 # [ 74.935849] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_take_over5310lb # [ 74.957227] niks3-server[1059]: 2026/09/20 10:51:49 INFO lead: acquired remote=192.168.1.4:361025311worker2 # [ 74.955836] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500515312worker2 # [ 74.957315] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker1 # [ 74.997717] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500515314worker1 # [ 75.002222] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5315worker2 # [ 75.157986] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500515316worker2 # [ 75.159837] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker1 # [ 75.204265] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500515318worker1 # [ 75.207294] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5319worker2 # [ 75.360379] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500515320worker2 # [ 75.362162] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5321worker1 # [ 75.405354] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_connected addr=lb:500515322worker1 # [ 75.409143] nix-grpc-daemon[784]: ts=2026-09-20T10:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker2 # [ 75.562664] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500515324worker2 # [ 75.564506] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5325worker1 # [ 75.588893] nix-grpc-daemon[784]: ts=2026-09-20T10:51:49Z level=info event=scheduler_restarting5326worker1 # [ 75.621631] nix-grpc-daemon[784]: ts=2026-09-20T10:51:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5327worker1 # [ 75.627527] nix-daemon[857]: reaped child process 860, status = succeeded5328worker1 # [ 75.740698] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5329worker1 # [ 75.744462] systemd[1]: Stopped Nix worker-protocol over gRPC.5330worker1 # [ 75.746437] systemd[1]: nix-grpc-daemon.service: Consumed 1.404s CPU time over 56.707s wall clock time, 52.7M memory peak, 58.6K incoming IP traffic, 139.9K outgoing IP traffic.5331worker1 # [ 75.756987] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5332worker2 # [ 75.765418] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_connected addr=lb:500515333worker1 # [ 75.761393] systemd[1]: Closed Nix worker-protocol over gRPC.5334worker2 # [ 75.769293] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5335worker1 # [ 75.764424] systemd[1]: nix-grpc-daemon.socket: Consumed 290us CPU time over 1min 5.706s wall clock time, 256K memory peak, 71.9K incoming IP traffic, 61.3K outgoing IP traffic.5336lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5337lb: must succeed: curl -sf localhost:9901/clusters5338lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5339lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.04 seconds)5340client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25341worker2 # [ 75.970482] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500515342worker2 # [ 75.978616] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05343worker2 # [ 75.988137] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561905344worker2 # [ 75.999271] nix-daemon[847]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)5345worker2 # [ 76.015202] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)5346worker2 # [ 76.104258] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 2/5)5347worker2 # [ 76.185087] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 3/5)5348worker2 # [ 76.260299] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 4/5)5349worker2 # [ 76.575654] nix-grpc-daemon[1214]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5350client # copying 2 paths...5351client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5352worker2 # [ 76.597117] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561905353client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5354worker2 # [ 76.606872] nix-daemon[847]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)5355worker2 # [ 76.628342] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:50Z level=info event=niks3_push_started pid=12705356worker2 # [ 76.910104] nix-daemon[847]: accepted connection from pid 1283, user nix-grpc-daemon (trusted)5357worker2 # [ 76.921884] nix-daemon[847]: reaped child process 1286, status = succeeded5358lb # [ 76.960320] niks3-server[1059]: 2026/09/20 10:51:51 INFO Received uploads request method=POST path=/api/pending_closures5359lb # [ 76.968516] niks3-server[1059]: 2026/09/20 10:51:51 INFO Received uploads request method=POST path=/api/pending_closures5360worker2 # [ 76.957296] nix-grpc-daemon[1270]: time=2026-09-20T10:51:51.075Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5361worker2 # [ 76.958696] nix-grpc-daemon[1270]: time=2026-09-20T10:51:51.075Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5362worker2 # [ 76.961391] nix-grpc-daemon[1270]: time=2026-09-20T10:51:51.075Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5363lb # [ 77.014606] niks3-server[1059]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5364lb # [ 77.022166] niks3-server[1059]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5365lb # [ 77.024153] niks3-server[1059]: 2026/09/20 10:51:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5366lb # [ 77.026054] niks3-server[1059]: 2026/09/20 10:51:51 INFO Signed narinfos id=25 count=15367lb # [ 77.027674] niks3-server[1059]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5368lb # [ 77.029691] niks3-server[1059]: 2026/09/20 10:51:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5369lb # [ 77.031555] niks3-server[1059]: 2026/09/20 10:51:51 INFO Signed narinfos id=26 count=25370worker2 # [ 77.012333] nix-grpc-daemon[1270]: time=2026-09-20T10:51:51.131Z level=INFO msg="Uploading 3 narinfos"5371lb # [ 77.035549] niks3-server[1059]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5372lb # [ 77.046944] niks3-server[1059]: 2026/09/20 10:51:51 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5373lb # [ 77.050798] niks3-server[1059]: 2026/09/20 10:51:51 INFO Completed upload id=255374lb # [ 77.054345] niks3-server[1059]: 2026/09/20 10:51:51 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5375lb # [ 77.056571] niks3-server[1059]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5376lb # [ 77.058617] niks3-server[1059]: 2026/09/20 10:51:51 INFO Completed upload id=265377worker2 # [ 77.037664] nix-grpc-daemon[1270]: time=2026-09-20T10:51:51.157Z level=INFO msg="Upload complete. (256ms)"5378lb # [ 77.061393] niks3-server[1059]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5379worker2 # [ 77.042055] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56190 duration_s=0 paths=2 nar_bytes_in=3125380lb # [ 77.068074] niks3-server[1059]: 2026/09/20 10:51:51 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5381worker2 # [ 77.052957] nix-daemon[847]: reaped child process 1268, status = succeeded5382client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.24 seconds)5383worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5384worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.01 seconds)5385worker1 # [ 77.102943] systemd[1]: Listening on Nix worker-protocol over gRPC.5386worker1 # [ 77.106779] systemd[1]: Starting Nix worker-protocol over gRPC...5387worker1 # [ 77.254924] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5388worker1 # [ 77.258546] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15389worker1 # [ 77.265063] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5390worker1 # [ 77.268135] systemd[1]: Started Nix worker-protocol over gRPC.5391lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5392lb: must succeed: curl -sf localhost:9901/clusters5393worker1 # [ 77.296679] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:50Z level=info event=scheduler_connected addr=lb:500515394worker2 # [ 77.306118] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05395lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5396lb: must succeed: curl -sf localhost:9901/clusters5397lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5398lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.13 seconds)5399(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.37 seconds)5400subtest: scheduler down: the other node takes the lock and keeps it5401lb: must succeed: curl -sf localhost:9901/clusters5402lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5403lb: must succeed: curl -sf localhost:9901/clusters5404lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5405lb: must succeed: curl -sf localhost:9901/clusters5406lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5407lb: must succeed: curl -sf localhost:9901/clusters5408lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5409worker2 # [ 78.616573] systemd[1]: Stopping Nix worker-protocol over gRPC...5410worker2 # [ 78.927163] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=signal_exit5411lb # [ 79.065720] envoy[931]: conn peer=192.168.1.4:43012 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13514 tx=2804 ms=532235412lb # [ 79.069666] envoy[931]: conn peer=[2001:db8:1::1]:54810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=175413lb # [ 79.073225] envoy[931]: conn peer=[2001:db8:1::3]:52560 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13717 tx=2597 ms=562545414lb # [ 79.076881] envoy[931]: conn peer=[2001:db8:1::1]:54826 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=282 ms=10655415worker2 # [ 79.138976] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=scheduler_yield5416lb # [ 79.163613] niks3-server[1059]: 2026/09/20 10:51:53 INFO lead: released remote=192.168.1.4:361025417worker1 # [ 79.136650] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:52Z level=info event=scheduler_restarting addr=lb:500515418worker2 # [ 79.148561] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=scheduler_restarting addr=lb:500515419worker2 # [ 79.154331] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:561805420worker2 # [ 79.157749] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=worker_gone addr=192.168.1.4:500515421worker2 # [ 79.160710] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:561805422worker2 # [ 79.163990] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=worker_gone addr=192.168.1.3:500515423worker1 # [ 79.336923] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:52Z level=info event=scheduler_connected addr=lb:500515424worker1 # [ 79.340903] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425worker2 # [ 79.349638] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=scheduler_connected addr=lb:500515426worker2 # [ 79.354083] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5427worker1 # [ 79.541653] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:52Z level=info event=scheduler_connected addr=lb:500515428worker1 # [ 79.545570] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5429worker2 # [ 79.554587] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=scheduler_connected addr=lb:500515430worker2 # [ 79.558725] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker1 # [ 79.746387] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_connected addr=lb:500515432worker1 # [ 79.750222] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker2 # [ 79.759116] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=scheduler_connected addr=lb:500515434worker2 # [ 79.761078] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker1 # [ 79.950892] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_connected addr=lb:500515436worker2 # [ 79.961910] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:54Z level=info event=scheduler_connected addr=lb:500515437worker2 # [ 79.963234] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5438worker1 # [ 79.954738] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5439worker2 # [ 80.143965] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:54Z level=info event=scheduler_restarting5440worker1 # [ 80.157104] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_connected addr=lb:500515441worker1 # [ 80.160345] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442worker2 # [ 80.170464] nix-daemon[847]: reaped child process 1263, status = succeeded5443worker2 # [ 80.176186] nix-grpc-daemon[1214]: ts=2026-09-20T10:51:54Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5444worker1 # [ 80.358070] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_connected addr=lb:500515445worker1 # [ 80.362085] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5446worker1 # [ 80.562878] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_connected addr=lb:500515447worker1 # [ 80.566817] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5448worker1 # [ 80.767675] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:54Z level=info event=scheduler_connected addr=lb:500515449worker1 # [ 80.771630] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450worker2 # [ 80.946627] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5451worker2 # [ 80.949620] systemd[1]: Stopped Nix worker-protocol over gRPC.5452worker2 # [ 80.954297] systemd[1]: nix-grpc-daemon.service: Consumed 308ms CPU time over 6.159s wall clock time, 32.8M memory peak, 18.2K incoming IP traffic, 24.6K outgoing IP traffic.5453worker2 # [ 80.965712] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5454worker2 # [ 80.970117] systemd[1]: Closed Nix worker-protocol over gRPC.5455worker2 # [ 80.973518] systemd[1]: nix-grpc-daemon.socket: Consumed 284us CPU time over 6.827s wall clock time, 256K memory peak, 18.2K incoming IP traffic, 13.8K outgoing IP traffic.5456worker1 # [ 80.972320] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:54Z level=info event=scheduler_connected addr=lb:500515457worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5458worker1 # [ 80.976266] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5459worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5460worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5461worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5462worker1 # [ 81.176953] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:54Z level=info event=scheduler_connected addr=lb:500515463worker1 # [ 81.178697] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5464worker1 # [ 81.379708] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:54Z level=info event=scheduler_connected addr=lb:500515465worker1 # [ 81.383621] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5466worker1 # [ 81.584522] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_connected addr=lb:500515467worker1 # [ 81.588381] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5468worker1 # [ 81.789223] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_connected addr=lb:500515469worker1 # [ 81.793123] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5470worker1 # [ 81.993428] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_connected addr=lb:500515471worker1 # [ 81.995329] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5472worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5473worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5474worker1 # [ 82.196913] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_connected addr=lb:500515475worker1 # [ 82.200679] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5476worker1 # [ 82.275952] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_take_over5477lb # [ 82.303959] niks3-server[1059]: 2026/09/20 10:51:56 INFO lead: acquired remote=192.168.1.3:358045478worker1 # [ 82.400892] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_connected addr=lb:500515479worker1 # [ 82.402791] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5480worker1 # [ 82.603855] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:56Z level=info event=scheduler_connected addr=lb:500515481worker1 # [ 82.610918] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05482worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5483worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5484client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25485worker1 # [ 83.405135] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409905486worker1 # [ 83.416286] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5487client # don't know how to build these paths:5488client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5489worker1 # [ 83.430224] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409905490worker1 # [ 83.440262] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)5491worker1 # [ 83.521917] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 2/5)5492worker1 # [ 83.595055] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 285 ms (attempt 3/5)5493worker1 # [ 83.885749] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 330 ms (attempt 4/5)5494worker1 # [ 84.218582] nix-grpc-daemon[1091]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5495worker1 # [ 84.238869] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:57Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05496worker1 # [ 84.240984] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:57Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15497worker1 # [ 84.282386] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905498worker1 # [ 84.295130] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409905499client # copying 1 paths to 'lb:50051'...5500client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5501worker1 # [ 84.307374] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409905502worker1 # [ 84.315560] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5503worker1 # [ 84.335184] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:57Z level=info event=niks3_push_started pid=11645504worker1 # [ 84.656320] nix-daemon[857]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5505worker1 # [ 84.667795] nix-daemon[857]: reaped child process 1180, status = succeeded5506worker1 # [ 84.798259] nix-daemon[857]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5507worker1 # [ 84.809937] nix-daemon[857]: reaped child process 1185, status = succeeded5508lb # [ 84.847377] niks3-server[1059]: 2026/09/20 10:51:58 INFO Received uploads request method=POST path=/api/pending_closures5509worker1 # [ 84.828605] nix-grpc-daemon[1164]: time=2026-09-20T10:51:58.245Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5510worker1 # [ 84.830228] nix-grpc-daemon[1164]: time=2026-09-20T10:51:58.247Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5511lb # [ 84.887135] niks3-server[1059]: 2026/09/20 10:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5512lb # [ 84.894678] niks3-server[1059]: 2026/09/20 10:51:59 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5513worker1 # [ 84.872160] nix-grpc-daemon[1164]: time=2026-09-20T10:51:58.289Z level=INFO msg="Uploading 1 narinfos"5514lb # [ 84.899781] niks3-server[1059]: 2026/09/20 10:51:59 INFO Signed narinfos id=27 count=15515lb # [ 84.918108] niks3-server[1059]: 2026/09/20 10:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5516lb # [ 84.923042] niks3-server[1059]: 2026/09/20 10:51:59 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5517lb # [ 84.926668] niks3-server[1059]: 2026/09/20 10:51:59 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5518lb # [ 84.930488] niks3-server[1059]: 2026/09/20 10:51:59 INFO Completed upload id=275519worker1 # [ 84.903084] nix-grpc-daemon[1164]: time=2026-09-20T10:51:58.320Z level=INFO msg="Upload complete. (392ms)"5520worker1 # [ 84.908356] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=0 paths=1 nar_bytes_in=5205521worker1 # [ 84.913257] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905522worker1 # [ 84.922106] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5523worker1 # [ 84.927356] nix-daemon[857]: reaped child process 1162, status = succeeded5524worker1 # [ 84.931857] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5525client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5526client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5527client # farm-a> LOG-farm-a-on-standby5528worker1 # [ 85.756859] nix-daemon[857]: reaped child process 1189, status = succeeded5529worker1 # [ 85.905442] nix-daemon[857]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5530worker1 # [ 85.915975] nix-daemon[857]: reaped child process 1207, status = succeeded5531lb # [ 85.958985] niks3-server[1059]: 2026/09/20 10:52:00 INFO Received uploads request method=POST path=/api/pending_closures5532worker1 # [ 85.941381] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.358Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5533worker1 # [ 85.942970] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.360Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5534lb # [ 86.003404] niks3-server[1059]: 2026/09/20 10:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5535worker1 # [ 85.982607] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.399Z level=INFO msg="Uploading 1 narinfos"5536lb # [ 86.010231] niks3-server[1059]: 2026/09/20 10:52:00 INFO Signed narinfos id=28 count=15537lb # [ 86.015660] niks3-server[1059]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5538lb # [ 86.027764] niks3-server[1059]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5539lb # [ 86.032385] niks3-server[1059]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5540lb # [ 86.039273] niks3-server[1059]: 2026/09/20 10:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5541lb # [ 86.041997] niks3-server[1059]: 2026/09/20 10:52:00 INFO Completed upload id=285542worker1 # [ 86.014569] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.431Z level=INFO msg="Upload complete. (276ms)"5543lb # [ 86.044337] niks3-server[1059]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5544worker1 # [ 86.024545] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15545worker1 # [ 86.032582] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:59Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05546worker1 # [ 86.037202] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:59Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15547worker1 # [ 86.042654] nix-daemon[857]: reaped child process 1187, status = succeeded5548worker1 # [ 86.076108] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905549worker1 # [ 86.088383] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409905550worker1 # [ 86.100820] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.518Z level=INFO msg="All 1 paths already cached"5551client # copying 1 paths to 'lb:50051'...5552client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5553worker1 # [ 86.108765] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409905554worker1 # [ 86.118341] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5555worker1 # [ 86.285347] nix-daemon[857]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5556worker1 # [ 86.297331] nix-daemon[857]: reaped child process 1216, status = succeeded5557worker1 # [ 86.447573] nix-daemon[857]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5558worker1 # [ 86.459197] nix-daemon[857]: reaped child process 1221, status = succeeded5559lb # [ 86.500621] niks3-server[1059]: 2026/09/20 10:52:00 INFO Received uploads request method=POST path=/api/pending_closures5560worker1 # [ 86.480285] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.897Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5561worker1 # [ 86.482276] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.899Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5562lb # [ 86.527066] niks3-server[1059]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5563lb # [ 86.538716] niks3-server[1059]: 2026/09/20 10:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5564lb # [ 86.544591] niks3-server[1059]: 2026/09/20 10:52:00 INFO Signed narinfos id=29 count=15565worker1 # [ 86.519889] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.937Z level=INFO msg="Uploading 1 narinfos"5566lb # [ 86.549356] niks3-server[1059]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5567lb # [ 86.559115] niks3-server[1059]: 2026/09/20 10:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5568worker1 # [ 86.540138] nix-grpc-daemon[1164]: time=2026-09-20T10:51:59.957Z level=INFO msg="Upload complete. (411ms)"5569lb # [ 86.567868] niks3-server[1059]: 2026/09/20 10:52:00 INFO Completed upload id=295570worker1 # [ 86.542119] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=0 paths=1 nar_bytes_in=6405571lb # [ 86.570755] niks3-server[1059]: 2026/09/20 10:52:00 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5572worker1 # [ 86.547351] nix-grpc-daemon[1091]: ts=2026-09-20T10:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905573worker1 # [ 86.552135] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5574worker1 # [ 86.554374] nix-daemon[857]: reaped child process 1211, status = succeeded5575client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5576worker1 # [ 86.560490] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5577client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5578client # farm-b> LOG-farm-b-on-standby5579worker1 # [ 87.589743] nix-daemon[857]: reaped child process 1225, status = succeeded5580worker1 # [ 87.729355] nix-daemon[857]: accepted connection from pid 1240, user nix-grpc-daemon (trusted)5581worker1 # [ 87.741375] nix-daemon[857]: reaped child process 1243, status = succeeded5582lb # [ 87.780704] niks3-server[1059]: 2026/09/20 10:52:01 INFO Received uploads request method=POST path=/api/pending_closures5583worker1 # [ 87.760419] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.177Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5584worker1 # [ 87.761889] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.179Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5585lb # [ 87.818575] niks3-server[1059]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5586lb # [ 87.825552] niks3-server[1059]: 2026/09/20 10:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5587worker1 # [ 87.802426] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.219Z level=INFO msg="Uploading 1 narinfos"5588lb # [ 87.830298] niks3-server[1059]: 2026/09/20 10:52:01 INFO Signed narinfos id=30 count=15589lb # [ 87.840229] niks3-server[1059]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5590lb # [ 87.847837] niks3-server[1059]: 2026/09/20 10:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5591lb # [ 87.855320] niks3-server[1059]: 2026/09/20 10:52:01 INFO Completed upload id=305592worker1 # [ 87.827389] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.244Z level=INFO msg="Upload complete. (260ms)"5593lb # [ 87.860139] niks3-server[1059]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5594lb # [ 87.863993] niks3-server[1059]: 2026/09/20 10:52:01 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5595worker1 # [ 87.833590] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15596worker1 # [ 87.843905] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:01Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05597worker1 # [ 87.848310] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:01Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15598worker1 # [ 87.857163] nix-daemon[857]: reaped child process 1223, status = succeeded5599worker1 # [ 87.891583] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905600worker1 # [ 87.903404] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409905601worker1 # [ 87.916551] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.333Z level=INFO msg="All 2 paths already cached"5602client # copying 1 paths to 'lb:50051'...5603client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5604worker1 # [ 87.924831] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409905605worker1 # [ 87.935156] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5606worker1 # [ 88.084139] nix-daemon[857]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5607worker1 # [ 88.096428] nix-daemon[857]: reaped child process 1252, status = succeeded5608worker1 # [ 88.248802] nix-daemon[857]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)5609worker1 # [ 88.261260] nix-daemon[857]: reaped child process 1257, status = succeeded5610lb # [ 88.303157] niks3-server[1059]: 2026/09/20 10:52:02 INFO Received uploads request method=POST path=/api/pending_closures5611worker1 # [ 88.283446] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.700Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5612worker1 # [ 88.285045] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.702Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5613lb # [ 88.332613] niks3-server[1059]: 2026/09/20 10:52:02 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5614lb # [ 88.344874] niks3-server[1059]: 2026/09/20 10:52:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5615lb # [ 88.350642] niks3-server[1059]: 2026/09/20 10:52:02 INFO Signed narinfos id=31 count=15616worker1 # [ 88.325908] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.743Z level=INFO msg="Uploading 1 narinfos"5617lb # [ 88.356341] niks3-server[1059]: 2026/09/20 10:52:02 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5618lb # [ 88.369397] niks3-server[1059]: 2026/09/20 10:52:02 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5619worker1 # [ 88.347786] nix-grpc-daemon[1164]: time=2026-09-20T10:52:01.765Z level=INFO msg="Upload complete. (402ms)"5620lb # [ 88.375667] niks3-server[1059]: 2026/09/20 10:52:02 INFO Completed upload id=315621worker1 # [ 88.350179] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=0 paths=1 nar_bytes_in=7685622lb # [ 88.379394] niks3-server[1059]: 2026/09/20 10:52:02 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5623worker1 # [ 88.356287] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905624worker1 # [ 88.366296] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5625worker1 # [ 88.369178] nix-daemon[857]: reaped child process 1247, status = succeeded5626client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5627worker1 # [ 88.374532] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5628client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5629client # farm-top> LOG-farm-top-on-standby5630worker1 # [ 88.986680] nix-daemon[857]: reaped child process 1261, status = succeeded5631lb # [ 89.065960] envoy[931]: conn peer=[2001:db8:1::4]:51998 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2293 tx=512 ms=52135632worker1 # [ 89.115805] nix-daemon[857]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5633worker1 # [ 89.127716] nix-daemon[857]: reaped child process 1279, status = succeeded5634lb # [ 89.171226] niks3-server[1059]: 2026/09/20 10:52:03 INFO Received uploads request method=POST path=/api/pending_closures5635worker1 # [ 89.151146] nix-grpc-daemon[1164]: time=2026-09-20T10:52:02.568Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5636worker1 # [ 89.152785] nix-grpc-daemon[1164]: time=2026-09-20T10:52:02.570Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5637lb # [ 89.206402] niks3-server[1059]: 2026/09/20 10:52:03 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5638lb # [ 89.213193] niks3-server[1059]: 2026/09/20 10:52:03 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5639lb # [ 89.225383] niks3-server[1059]: 2026/09/20 10:52:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5640worker1 # [ 89.202169] nix-grpc-daemon[1164]: time=2026-09-20T10:52:02.619Z level=INFO msg="Uploading 1 narinfos"5641lb # [ 89.229819] niks3-server[1059]: 2026/09/20 10:52:03 INFO Signed narinfos id=32 count=15642lb # [ 89.238444] niks3-server[1059]: 2026/09/20 10:52:03 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5643lb # [ 89.244442] niks3-server[1059]: 2026/09/20 10:52:03 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5644lb # [ 89.249325] niks3-server[1059]: 2026/09/20 10:52:03 INFO Completed upload id=325645worker1 # [ 89.221386] nix-grpc-daemon[1164]: time=2026-09-20T10:52:02.638Z level=WARN msg="Failed to register uploaded object" key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo error="server returned 404: object is not pending upload\n"5646worker1 # [ 89.229157] nix-grpc-daemon[1164]: time=2026-09-20T10:52:02.643Z level=INFO msg="Upload complete. (245ms)"5647worker1 # [ 89.232693] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15648worker1 # [ 89.243822] nix-daemon[857]: reaped child process 1259, status = succeeded5649client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.07 seconds)5650client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5651client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5652worker2 # [ 89.531561] systemd[1]: Listening on Nix worker-protocol over gRPC.5653worker2 # [ 89.537993] systemd[1]: Starting Nix worker-protocol over gRPC...5654worker2 # [ 89.693242] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5655worker2 # [ 89.696674] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15656worker2 # [ 89.703776] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5657worker2 # [ 89.707768] systemd[1]: Started Nix worker-protocol over gRPC.5658lb: must succeed: curl -sf localhost:9901/clusters5659worker2 # [ 89.733242] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:03Z level=info event=scheduler_connected addr=lb:500515660worker1 # [ 89.729172] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05661lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5662worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5663worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5664lb: must succeed: curl -sf localhost:9901/clusters5665lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5666lb: must succeed: curl -sf localhost:9901/clusters5667lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5668worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5669worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5670(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.62 seconds)5671subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5672client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5673client # [ 90.077437] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5674client # Running as unit: rs.service; invocation ID: 8d7df0552eeb4bb59d63abf9f78949975675client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5676worker2 # [ 90.293178] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486885677worker2 # [ 90.304244] nix-daemon[847]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5678client # [ 90.306645] nix[1182]: don't know how to build these paths:5679client # [ 90.307663] nix[1182]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5680worker1 # [ 90.314345] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:03Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05681worker1 # [ 90.316063] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:03Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15682worker1 # [ 90.357603] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905683worker1 # [ 90.365939] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409905684client # [ 90.370325] nix[1182]: copying 1 paths to 'lb:50051'...5685worker1 # [ 90.373521] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409905686client # [ 90.372253] nix[1182]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5687worker1 # [ 90.383117] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5688worker1 # [ 90.527861] nix-daemon[857]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5689worker1 # [ 90.540091] nix-daemon[857]: reaped child process 1295, status = succeeded5690worker1 # [ 90.696031] nix-daemon[857]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5691worker1 # [ 90.707596] nix-daemon[857]: reaped child process 1300, status = succeeded5692lb # [ 90.750461] niks3-server[1059]: 2026/09/20 10:52:04 INFO Received uploads request method=POST path=/api/pending_closures5693worker1 # [ 90.731328] nix-grpc-daemon[1164]: time=2026-09-20T10:52:04.148Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5694worker1 # [ 90.732718] nix-grpc-daemon[1164]: time=2026-09-20T10:52:04.148Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5695lb # [ 90.776431] niks3-server[1059]: 2026/09/20 10:52:04 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5696lb # [ 90.786347] niks3-server[1059]: 2026/09/20 10:52:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5697worker1 # [ 90.763274] nix-grpc-daemon[1164]: time=2026-09-20T10:52:04.180Z level=INFO msg="Uploading 1 narinfos"5698lb # [ 90.791119] niks3-server[1059]: 2026/09/20 10:52:04 INFO Signed narinfos id=33 count=15699lb # [ 90.796474] niks3-server[1059]: 2026/09/20 10:52:04 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5700lb # [ 90.807448] niks3-server[1059]: 2026/09/20 10:52:04 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5701lb # [ 90.810753] niks3-server[1059]: 2026/09/20 10:52:04 INFO Completed upload id=335702worker1 # [ 90.783100] nix-grpc-daemon[1164]: time=2026-09-20T10:52:04.200Z level=INFO msg="Upload complete. (395ms)"5703lb # [ 90.812671] niks3-server[1059]: 2026/09/20 10:52:04 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5704worker1 # [ 90.785056] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=0 paths=1 nar_bytes_in=4165705worker1 # [ 90.788967] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409905706worker1 # [ 90.794796] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5707worker1 # [ 90.797184] nix-daemon[857]: reaped child process 1290, status = succeeded5708worker1 # [ 90.802254] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)5709client # [ 90.798961] nix[1182]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5710lb: must succeed: curl -sf localhost:9901/clusters5711lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5712lb: must succeed: curl -sf localhost:9901/clusters5713lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5714lb: must succeed: curl -sf localhost:9901/clusters5715lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5716lb: must succeed: curl -sf localhost:9901/clusters5717lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5718worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5719worker2 # [ 92.880901] systemd-journald[288]: Received client request to rotate journal, rotating.5720worker2 # [ 92.897782] systemd-journald[288]: Vacuuming done, freed 0B of archived journals from /var/log/journal/f62f87d427dd4777b39d2663d90f953d.5721worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)5722worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5723worker1 # [ 92.805224] systemd[1]: Stopping Nix worker-protocol over gRPC...5724worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5725client: must succeed: sleep 5.05726worker1 # [ 93.270819] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:06Z level=info event=draining5727client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5728worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5729worker1 # [ 98.232744] nix-daemon[857]: reaped child process 1304, status = succeeded5730worker1 # [ 98.392734] nix-daemon[857]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)5731worker1 # [ 98.403087] nix-daemon[857]: reaped child process 1347, status = succeeded5732lb # [ 98.446573] niks3-server[1059]: 2026/09/20 10:52:12 INFO Received uploads request method=POST path=/api/pending_closures5733worker1 # [ 98.425152] nix-grpc-daemon[1164]: time=2026-09-20T10:52:11.842Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5734worker1 # [ 98.426642] nix-grpc-daemon[1164]: time=2026-09-20T10:52:11.844Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5735lb # [ 98.472708] niks3-server[1059]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5736lb # [ 98.480166] niks3-server[1059]: 2026/09/20 10:52:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5737lb # [ 98.482898] niks3-server[1059]: 2026/09/20 10:52:12 INFO Signed narinfos id=34 count=15738worker1 # [ 98.455401] nix-grpc-daemon[1164]: time=2026-09-20T10:52:11.872Z level=INFO msg="Uploading 1 narinfos"5739lb # [ 98.487815] niks3-server[1059]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5740lb # [ 98.491355] niks3-server[1059]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5741lb # [ 98.497524] niks3-server[1059]: 2026/09/20 10:52:12 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5742lb # [ 98.501258] niks3-server[1059]: 2026/09/20 10:52:12 INFO Completed upload id=345743worker1 # [ 98.473546] nix-grpc-daemon[1164]: time=2026-09-20T10:52:11.890Z level=INFO msg="Upload complete. (258ms)"5744lb # [ 98.503183] niks3-server[1059]: 2026/09/20 10:52:12 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5745worker1 # [ 98.476056] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40990 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15746worker1 # [ 98.483508] nix-daemon[857]: reaped child process 1302, status = succeeded5747client # [ 98.512738] systemd[1]: rs.service: Deactivated successfully.5748client # [ 98.516993] systemd[1]: rs.service: Consumed 111ms CPU time over 8.438s wall clock time, 39.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5749lb # [ 99.066174] envoy[931]: conn peer=[2001:db8:1::1]:46706 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9547 tx=3847 ms=58435750lb # [ 99.069837] envoy[931]: conn peer=[2001:db8:1::1]:46714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=1324 ms=82065751worker1 # [ 99.272077] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:12Z level=info event=signal_exit5752worker1 # [ 99.496239] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:12Z level=info event=scheduler_yield5753worker1 # [ 99.498152] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:12Z level=info event=scheduler_restarting addr=lb:500515754worker1 # [ 99.500277] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:473905755worker1 # [ 99.501618] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:12Z level=info event=worker_gone addr=192.168.1.3:500515756worker2 # [ 99.506830] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:13Z level=info event=scheduler_restarting addr=lb:500515757worker1 # [ 99.502782] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:473905758lb # [ 99.527627] niks3-server[1059]: 2026/09/20 10:52:13 INFO lead: released remote=192.168.1.3:358045759worker1 # [ 99.504121] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:12Z level=info event=worker_gone addr=192.168.1.4:500515760worker1 # [ 99.698938] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_connected addr=lb:500515761worker2 # [ 99.706387] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:13Z level=info event=scheduler_connected addr=lb:500515762worker1 # [ 99.703117] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5763worker2 # [ 99.710923] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5764lb # [ 99.734766] niks3-server[1059]: 2026/09/20 10:52:13 INFO lead: acquired remote=192.168.1.4:568245765worker2 # [ 99.717249] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:13Z level=info event=scheduler_take_over5766worker1 # [ 99.904133] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_connected addr=lb:500515767worker2 # [ 99.912111] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_connected addr=lb:500515768worker1 # [ 99.908140] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5769worker2 # [ 99.915260] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5770worker2 # [ 100.112911] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_connected addr=lb:500515771worker1 # [ 100.109132] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_connected addr=lb:500515772worker2 # [ 100.117096] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5773worker1 # [ 100.112995] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5774worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.14 seconds)5775client: waiting for success: ! systemctl is-active rs5776client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5777client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5778worker2 # [ 100.317761] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_connected addr=lb:500515779worker1 # [ 100.313716] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_connected addr=lb:500515780client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5781worker2 # [ 100.321961] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5782client: must fail: journalctl -u rs | grep -q 'reconnecting'5783worker1 # [ 100.317628] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5784client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5785lb: must succeed: curl -sf localhost:9901/clusters5786lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5787worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5788worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5789worker1 # [ 100.500139] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=scheduler_restarting5790worker1 # [ 100.510682] nix-grpc-daemon[1091]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5791worker2 # [ 100.522722] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_connected addr=lb:500515792worker2 # [ 100.524406] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5793worker1 # [ 100.514210] nix-grpc-daemon[1091]: E0000 00:00:1789901533.927692 1134 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5794worker1 # [ 100.523507] nix-grpc-daemon[1091]: E0000 00:00:1789901533.928072 1134 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5795worker1 # [ 100.536937] nix-grpc-daemon[1091]: ts=2026-09-20T10:52:13Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5796worker1 # [ 100.546608] nix-daemon[857]: reaped child process 1155, status = succeeded5797worker2 # [ 100.725227] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_connected addr=lb:500515798worker2 # [ 100.729152] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5799worker2 # [ 100.929854] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:15Z level=info event=scheduler_connected addr=lb:500515800worker2 # [ 100.933771] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5801worker2 # [ 101.134717] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:15Z level=info event=scheduler_connected addr=lb:500515802worker2 # [ 101.141993] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:15Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05803lb: must succeed: curl -sf localhost:9901/clusters5804lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5805worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5806worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5807worker1 # [ 101.884795] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5808worker1 # [ 101.887251] systemd[1]: Stopped Nix worker-protocol over gRPC.5809worker1 # [ 101.889330] systemd[1]: nix-grpc-daemon.service: Consumed 1.239s CPU time over 24.775s wall clock time, 35.3M memory peak, 47.5K incoming IP traffic, 97.8K outgoing IP traffic.5810worker1 # [ 101.904791] systemd[1]: Starting Nix worker-protocol over gRPC...5811worker1 # [ 102.064595] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5812worker1 # [ 102.068432] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15813worker1 # [ 102.074465] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5814worker1 # [ 102.077348] systemd[1]: Started Nix worker-protocol over gRPC.5815worker1 # [ 102.100759] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:15Z level=info event=scheduler_connected addr=lb:500515816worker2 # [ 102.110541] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05817lb: must succeed: curl -sf localhost:9901/clusters5818lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5819worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5820worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5821(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.70 seconds)5822subtest: scheduler restart mid-build: build finishes, no double build5823client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5824client # [ 102.776373] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5825client # Running as unit: mid.service; invocation ID: 3fc74f7e3f85497c950b9331e09a5e555826client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.06 seconds)5827worker1 # [ 102.961377] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524305828worker1 # [ 102.969668] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5829client # [ 102.981390] nix[1218]: don't know how to build these paths:5830client # [ 102.982350] nix[1218]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5831worker2 # [ 103.000250] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:17Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05832worker2 # [ 103.005732] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:17Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15833worker2 # [ 103.048773] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486885834worker2 # [ 103.060806] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 1/5)5835worker2 # [ 103.113381] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 157 ms (attempt 2/5)5836worker2 # [ 103.276264] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 264 ms (attempt 3/5)5837worker2 # [ 103.546228] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 4/5)5838worker2 # [ 103.563320] nix-grpc-daemon[1307]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5839worker2 # [ 103.576989] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486885840client # [ 103.576180] nix[1218]: copying 1 paths to 'lb:50051'...5841worker2 # [ 103.586440] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486885842client # [ 103.579204] nix[1218]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5843worker2 # [ 103.599725] nix-daemon[847]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5844worker2 # [ 103.617930] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:17Z level=info event=niks3_push_started pid=14285845worker2 # [ 103.934622] nix-daemon[847]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5846worker2 # [ 103.945417] nix-daemon[847]: reaped child process 1444, status = succeeded5847worker2 # [ 104.119287] nix-daemon[847]: accepted connection from pid 1446, user nix-grpc-daemon (trusted)5848worker2 # [ 104.133627] nix-daemon[847]: reaped child process 1453, status = succeeded5849lb # [ 104.170453] niks3-server[1059]: 2026/09/20 10:52:18 INFO Received uploads request method=POST path=/api/pending_closures5850worker2 # [ 104.158312] nix-grpc-daemon[1428]: time=2026-09-20T10:52:18.277Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5851worker2 # [ 104.159686] nix-grpc-daemon[1428]: time=2026-09-20T10:52:18.277Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5852lb # [ 104.205407] niks3-server[1059]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5853lb # [ 104.216756] niks3-server[1059]: 2026/09/20 10:52:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5854lb # [ 104.223113] niks3-server[1059]: 2026/09/20 10:52:18 INFO Signed narinfos id=35 count=15855worker2 # [ 104.204906] nix-grpc-daemon[1428]: time=2026-09-20T10:52:18.324Z level=INFO msg="Uploading 1 narinfos"5856lb # [ 104.233532] niks3-server[1059]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5857lb # [ 104.242480] niks3-server[1059]: 2026/09/20 10:52:18 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5858lb # [ 104.247095] niks3-server[1059]: 2026/09/20 10:52:18 INFO Completed upload id=355859lb # [ 104.248937] niks3-server[1059]: 2026/09/20 10:52:18 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5860worker2 # [ 104.227528] nix-grpc-daemon[1428]: time=2026-09-20T10:52:18.347Z level=INFO msg="Upload complete. (436ms)"5861worker2 # [ 104.232547] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48688 duration_s=0 paths=1 nar_bytes_in=4245862worker2 # [ 104.238434] nix-grpc-daemon[1307]: ts=2026-09-20T10:52:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486885863worker2 # [ 104.245941] nix-daemon[847]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5864worker2 # [ 104.251052] nix-daemon[847]: reaped child process 1426, status = succeeded5865client # [ 104.245585] nix[1218]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5866worker2 # [ 104.255482] nix-daemon[847]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5867lb: must succeed: curl -sf localhost:9901/clusters5868lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5869lb: must succeed: curl -sf localhost:9901/clusters5870lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5871worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5872worker2 # [ 106.534636] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1307 (nix-grpc-daemon) on client request.5873worker2 # [ 106.538310] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5874lb # [ 106.566692] niks3-server[1059]: 2026/09/20 10:52:20 INFO lead: released remote=192.168.1.4:568245875client # [ 106.544388] nix[1218]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5876worker1 # [ 106.546274] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:19Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5877worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5878client: must succeed: sleep 5.05879client # [ 106.549141] nix[1218]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5880worker2 # [ 106.566698] nix-daemon[847]: reaped child process 1455, status = succeeded5881worker2 # [ 106.569967] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5882worker2 # [ 106.571480] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5883worker2 # [ 106.572805] systemd[1]: nix-grpc-daemon.service: Consumed 440ms CPU time over 17.031s wall clock time, 33.1M memory peak, 14.7K incoming IP traffic, 30.6K outgoing IP traffic.5884worker2 # [ 106.575793] nix-daemon[847]: reaped child process 1363, status = succeeded5885worker2 # [ 106.583248] nix-daemon[1457]: unexpected Nix daemon error: error: interrupted by the user5886worker2 # [ 106.587722] nix-daemon[847]: reaped child process 1457, status = failed with exit code 15887worker2 # [ 106.716351] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5888worker2 # [ 106.720283] systemd[1]: Starting Nix worker-protocol over gRPC...5889worker1 # [ 106.746541] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:20Z level=info event=scheduler_connected addr=lb:500515890worker2 # [ 106.870084] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5891worker2 # [ 106.873437] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15892worker2 # [ 106.880486] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5893worker2 # [ 106.882096] systemd[1]: Started Nix worker-protocol over gRPC.5894worker2 # [ 106.893170] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:21Z level=info event=scheduler_take_over5895lb # [ 106.913741] niks3-server[1059]: 2026/09/20 10:52:21 INFO lead: acquired remote=192.168.1.4:358685896worker1 # [ 106.890467] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5897worker2 # [ 106.910878] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:21Z level=info event=scheduler_connected addr=lb:500515898worker2 # [ 106.912769] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05899worker2 # [ 107.065538] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:21Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05900worker2 # [ 107.070954] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:21Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15901worker1 # [ 107.091266] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:20Z level=info event=scheduler_connected addr=lb:500515902worker2 # [ 107.098844] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05903worker2 # [ 107.108493] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530845904worker2 # [ 107.119156] nix-daemon[847]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5905worker2 # [ 107.132527] nix-daemon[847]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5906client # [ 107.134646] nix[1218]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5907worker2 # [ 107.145813] nix-daemon[847]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5908lb # [ 109.066563] envoy[931]: conn peer=[2001:db8:1::3]:33952 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5568 tx=1292 ms=232315909lb # [ 109.070507] envoy[931]: conn peer=[2001:db8:1::4]:60336 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2142 tx=567 ms=168235910client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5911client: waiting for success: ! systemctl is-active mid5912worker2 # [ 111.967212] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:26Z level=info event=niks3_push_started pid=15725913worker2 # [ 111.993850] nix-daemon[847]: reaped child process 1548, status = succeeded5914worker2 # [ 112.296795] nix-daemon[847]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)5915worker2 # [ 112.306975] nix-daemon[847]: reaped child process 1588, status = succeeded5916lb # [ 112.345606] niks3-server[1059]: 2026/09/20 10:52:26 INFO Received uploads request method=POST path=/api/pending_closures5917worker2 # [ 112.334755] nix-grpc-daemon[1572]: time=2026-09-20T10:52:26.454Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5918worker2 # [ 112.336238] nix-grpc-daemon[1572]: time=2026-09-20T10:52:26.456Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5919lb # [ 112.391667] niks3-server[1059]: 2026/09/20 10:52:26 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5920lb # [ 112.399689] niks3-server[1059]: 2026/09/20 10:52:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5921worker2 # [ 112.384194] nix-grpc-daemon[1572]: time=2026-09-20T10:52:26.504Z level=INFO msg="Uploading 1 narinfos"5922lb # [ 112.405377] niks3-server[1059]: 2026/09/20 10:52:26 INFO Signed narinfos id=36 count=15923lb # [ 112.414802] niks3-server[1059]: 2026/09/20 10:52:26 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5924lb # [ 112.427034] niks3-server[1059]: 2026/09/20 10:52:26 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5925lb # [ 112.433558] niks3-server[1059]: 2026/09/20 10:52:26 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5926lb # [ 112.440270] niks3-server[1059]: 2026/09/20 10:52:26 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5927lb # [ 112.443969] niks3-server[1059]: 2026/09/20 10:52:26 INFO Completed upload id=365928worker2 # [ 112.423166] nix-grpc-daemon[1572]: time=2026-09-20T10:52:26.543Z level=INFO msg="Upload complete. (266ms)"5929worker2 # [ 112.429744] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15930worker2 # [ 112.446411] nix-daemon[847]: reaped child process 1546, status = succeeded5931client # [ 112.474254] systemd[1]: mid.service: Deactivated successfully.5932client # [ 112.477143] systemd[1]: mid.service: Consumed 117ms CPU time over 9.698s wall clock time, 41.3M memory peak, 9.9K incoming IP traffic, 10.8K outgoing IP traffic.5933client: (finished: waiting for success: ! systemctl is-active mid, in 1.10 seconds)5934client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5935client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5936(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.19 seconds)5937subtest: interrupting the client stops the build on the worker5938client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5939client # [ 112.943660] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5940client # Running as unit: intr.service; invocation ID: f2ea8836916f47c9a00fd207becd220a5941client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5942worker1 # [ 113.139173] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524305943client # [ 113.143152] nix[1254]: don't know how to build these paths:5944client # [ 113.144042] nix[1254]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5945worker2 # [ 113.163715] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:27Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05946worker2 # [ 113.167982] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:27Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15947worker1 # [ 113.201558] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524305948worker1 # [ 113.212754] nix-grpc-daemon[1354]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5)5949worker1 # [ 113.315655] nix-grpc-daemon[1354]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 2/5)5950worker1 # [ 113.491653] nix-grpc-daemon[1354]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 3/5)5951worker1 # [ 113.586612] nix-grpc-daemon[1354]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 352 ms (attempt 4/5)5952worker1 # [ 113.944758] nix-grpc-daemon[1354]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5953worker1 # [ 113.961186] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524305954client # [ 113.968525] nix[1254]: copying 1 paths to 'lb:50051'...5955client # [ 113.972155] nix[1254]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5956worker1 # [ 113.975457] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524305957worker1 # [ 113.986710] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5958worker1 # [ 114.003865] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:27Z level=info event=niks3_push_started pid=14425959worker1 # [ 114.325080] nix-daemon[857]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5960worker1 # [ 114.336119] nix-daemon[857]: reaped child process 1462, status = succeeded5961worker1 # [ 114.491491] nix-daemon[857]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)5962worker1 # [ 114.503235] nix-daemon[857]: reaped child process 1467, status = succeeded5963lb # [ 114.546186] niks3-server[1059]: 2026/09/20 10:52:28 INFO Received uploads request method=POST path=/api/pending_closures5964worker1 # [ 114.528234] nix-grpc-daemon[1442]: time=2026-09-20T10:52:27.945Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5965worker1 # [ 114.531294] nix-grpc-daemon[1442]: time=2026-09-20T10:52:27.947Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5966lb # [ 114.583162] niks3-server[1059]: 2026/09/20 10:52:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5967lb # [ 114.592257] niks3-server[1059]: 2026/09/20 10:52:28 INFO Signed narinfos id=37 count=15968worker1 # [ 114.568705] nix-grpc-daemon[1442]: time=2026-09-20T10:52:27.985Z level=INFO msg="Uploading 1 narinfos"5969lb # [ 114.596312] niks3-server[1059]: 2026/09/20 10:52:28 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5970lb # [ 114.604330] niks3-server[1059]: 2026/09/20 10:52:28 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5971lb # [ 114.615283] niks3-server[1059]: 2026/09/20 10:52:28 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5972lb # [ 114.621098] niks3-server[1059]: 2026/09/20 10:52:28 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5973lb # [ 114.627312] niks3-server[1059]: 2026/09/20 10:52:28 INFO Completed upload id=375974worker1 # [ 114.599764] nix-grpc-daemon[1442]: time=2026-09-20T10:52:28.017Z level=INFO msg="Upload complete. (421ms)"5975worker1 # [ 114.604504] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=4245976worker1 # [ 114.610405] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524305977worker1 # [ 114.619103] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5978worker1 # [ 114.624128] nix-daemon[857]: reaped child process 1440, status = succeeded5979worker1 # [ 114.627887] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5980client # [ 114.625459] nix[1254]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5981client: must succeed: systemctl kill -s INT intr5982client # [ 116.436315] systemd[1]: intr.service: Sent signal SIGINT to main process 1254 (nix) on client request.5983client # [ 116.439649] nix[1254]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5984worker1 # [ 116.443623] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:29Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5985client # [ 116.442764] nix[1254]: error: interrupted by the user5986client: (finished: must succeed: systemctl kill -s INT intr, in 0.03 seconds)5987client # [ 116.466369] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5988client # [ 116.467718] systemd[1]: intr.service: Failed with result 'exit-code'.5989client # [ 116.469770] systemd[1]: intr.service: Consumed 98ms CPU time over 3.525s wall clock time, 37.3M memory peak, 7.8K incoming IP traffic, 8.9K outgoing IP traffic.5990worker1 # [ 116.644790] nix-daemon[857]: reaped child process 1469, status = succeeded5991worker1 # [ 116.650192] nix-daemon[1471]: unexpected Nix daemon error: error: interrupted by the user5992worker1 # [ 116.653462] nix-daemon[857]: reaped child process 1471, status = failed with exit code 15993(finished: subtest: interrupting the client stops the build on the worker, in 4.73 seconds)5994subtest: a deploy mid-build drains: the build finishes, the new generation takes over5995client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw5996client # [ 117.695739] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.5997client # Running as unit: sw.service; invocation ID: 3c9898a8686b4b32bf70d70457c58d6b5998client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.06 seconds)5999client # [ 117.899448] nix[1275]: don't know how to build these paths:6000client # [ 117.900413] nix[1275]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6001worker1 # [ 117.901180] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524306002worker2 # [ 117.919581] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:32Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06003worker2 # [ 117.923547] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:32Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16004worker2 # [ 117.961517] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530846005worker2 # [ 117.972821] nix-grpc-daemon[1493]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5)6006worker2 # [ 117.979507] nix-grpc-daemon[1493]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)6007worker2 # [ 118.008270] nix-grpc-daemon[1493]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 272 ms (attempt 3/5)6008worker2 # [ 118.285915] nix-grpc-daemon[1493]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 727 ms (attempt 4/5)6009worker2 # [ 119.019896] nix-grpc-daemon[1493]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6010worker2 # [ 119.036951] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530846011client # [ 119.038403] nix[1275]: copying 1 paths to 'lb:50051'...6012client # [ 119.039135] nix[1275]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6013lb # [ 119.067622] envoy[931]: conn peer=[2001:db8:1::1]:34468 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3852 tx=1702 ms=94956014lb # [ 119.072818] envoy[931]: conn peer=[2001:db8:1::1]:42438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=730 ms=33216015worker2 # [ 119.052755] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530846016worker2 # [ 119.063749] nix-daemon[847]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)6017worker2 # [ 119.225056] nix-daemon[847]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6018worker2 # [ 119.236945] nix-daemon[847]: reaped child process 1635, status = succeeded6019worker2 # [ 119.390156] nix-daemon[847]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)6020worker2 # [ 119.401468] nix-daemon[847]: reaped child process 1640, status = succeeded6021lb # [ 119.438134] niks3-server[1059]: 2026/09/20 10:52:33 INFO Received uploads request method=POST path=/api/pending_closures6022worker2 # [ 119.422206] nix-grpc-daemon[1572]: time=2026-09-20T10:52:33.542Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6023worker2 # [ 119.424314] nix-grpc-daemon[1572]: time=2026-09-20T10:52:33.543Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6024lb # [ 119.452532] niks3-server[1059]: 2026/09/20 10:52:33 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6025lb # [ 119.458129] niks3-server[1059]: 2026/09/20 10:52:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6026lb # [ 119.459825] niks3-server[1059]: 2026/09/20 10:52:33 INFO Signed narinfos id=38 count=16027worker2 # [ 119.440114] nix-grpc-daemon[1572]: time=2026-09-20T10:52:33.560Z level=INFO msg="Uploading 1 narinfos"6028lb # [ 119.465834] niks3-server[1059]: 2026/09/20 10:52:33 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6029lb # [ 119.468972] niks3-server[1059]: 2026/09/20 10:52:33 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6030lb # [ 119.472093] niks3-server[1059]: 2026/09/20 10:52:33 INFO Completed upload id=386031worker2 # [ 119.452095] nix-grpc-daemon[1572]: time=2026-09-20T10:52:33.572Z level=INFO msg="Upload complete. (377ms)"6032lb # [ 119.473915] niks3-server[1059]: 2026/09/20 10:52:33 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6033worker2 # [ 119.453985] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=0 paths=1 nar_bytes_in=4166034worker2 # [ 119.456866] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530846035worker2 # [ 119.463247] nix-daemon[847]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)6036worker2 # [ 119.466270] nix-daemon[847]: reaped child process 1630, status = succeeded6037client # [ 119.460764] nix[1275]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6038worker2 # [ 119.471055] nix-daemon[847]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)6039worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26040worker2 # Checking switch inhibitors... done6041worker2 # [ 121.904342] nixos[1675]: switching to system configuration /nix/store/dqkfjab4c5znh5yka8mvaak2rm3y1rdh-nixos-system-worker2-test6042worker2 # [ 121.906630] systemd[1]: Stopped target Remote File Systems.6043worker2 # [ 121.909310] systemd[1]: Stopped target Local File Systems.6044worker2 # activating the configuration...6045worker2 # setting up /etc...6046worker2 # [ 122.535513] systemd[1]: Reload requested from client PID 1675 ('.switch-to-conf') (unit backdoor.service)...6047worker2 # [ 122.537101] systemd[1]: Reloading...6048worker2 # [ 123.134308] systemd[1]: Reloading finished in 594 ms.6049worker2 # restarting sysinit-reactivation.target6050worker2 # [ 123.190250] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6051worker2 # [ 123.321819] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6052worker2 # [ 123.323248] systemd[1]: Reached target Reactivate sysinit units.6053worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6054worker2 # [ 123.327695] systemd[1]: Reloading D-Bus System Message Bus...6055worker2 # [ 123.328905] dbus-broker-launch[535]: Caught SIGHUP, trigger reload.6056worker2 # [ 123.333275] systemd[1]: Reloading Nix worker-protocol over gRPC...6057worker2 # [ 123.361415] dbus-broker-launch[535]: Looking up NSS user entry for 'systemd-timesync'...6058worker2 # [ 123.364514] dbus-broker-launch[535]: NSS returned no entry for 'systemd-timesync'6059worker2 # [ 123.365563] dbus-broker-launch[535]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6060worker2 # [ 123.374487] systemd[1]: Reloaded Nix worker-protocol over gRPC.6061worker2 # [ 123.393354] systemd[1]: Reloaded D-Bus System Message Bus.6062worker2 # [ 123.397841] systemd[1]: Reached target Remote File Systems.6063worker2 # [ 123.402803] systemd[1]: Starting linger-users.service...6064worker2 # [ 123.404761] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6065worker2 # [ 123.406655] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6066worker2 # [ 123.412414] systemd[1]: Reached target Local File Systems.6067worker2 # [ 123.413989] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6068worker2 # [ 123.420347] systemd[1]: Starting Create SUID/SGID Wrappers...6069worker2 # [ 123.421520] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6070worker2 # [ 123.422703] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676071worker2 # [ 123.427127] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6072worker2 # [ 123.428585] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6073worker2 # [ 123.431226] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6074worker2 # [ 123.474077] systemd[1]: linger-users.service: Deactivated successfully.6075worker2 # [ 123.475137] systemd[1]: Finished linger-users.service.6076worker2 # [ 123.842666] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6077worker2 # [ 123.844319] systemd[1]: Finished Create SUID/SGID Wrappers.6078worker2 # [ 123.886286] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:38Z level=info event=draining6079worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6080worker2 # [ 124.101386] nixos[1675]: finished switching to system configuration /nix/store/dqkfjab4c5znh5yka8mvaak2rm3y1rdh-nixos-system-worker2-test6081worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.78 seconds)6082worker2: must succeed: pgrep -f 'read -t [9]0 x'6083worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6084client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26085worker1 # [ 124.353749] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524306086client # don't know how to build these paths:6087client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6088worker1 # [ 124.359057] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306089worker2 # [ 124.381317] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:38Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06090worker2 # [ 124.383201] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:38Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16091worker1 # [ 124.422569] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306092worker1 # [ 124.430225] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306093client # copying 1 paths to 'lb:50051'...6094worker1 # [ 124.438172] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306095client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6096worker1 # [ 124.455405] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6097worker1 # [ 124.618528] nix-daemon[857]: accepted connection from pid 1525, user nix-grpc-daemon (trusted)6098worker1 # [ 124.630321] nix-daemon[857]: reaped child process 1528, status = succeeded6099worker1 # [ 124.791190] nix-daemon[857]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6100worker1 # [ 124.802714] nix-daemon[857]: reaped child process 1533, status = succeeded6101lb # [ 124.847452] niks3-server[1059]: 2026/09/20 10:52:38 INFO Received uploads request method=POST path=/api/pending_closures6102worker1 # [ 124.829520] nix-grpc-daemon[1442]: time=2026-09-20T10:52:38.246Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6103worker1 # [ 124.830937] nix-grpc-daemon[1442]: time=2026-09-20T10:52:38.246Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6104lb # [ 124.876216] niks3-server[1059]: 2026/09/20 10:52:38 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6105lb # [ 124.882265] niks3-server[1059]: 2026/09/20 10:52:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6106lb # [ 124.887278] niks3-server[1059]: 2026/09/20 10:52:39 INFO Signed narinfos id=39 count=16107worker1 # [ 124.863110] nix-grpc-daemon[1442]: time=2026-09-20T10:52:38.280Z level=INFO msg="Uploading 1 narinfos"6108lb # [ 124.900113] niks3-server[1059]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6109lb # [ 124.906916] niks3-server[1059]: 2026/09/20 10:52:39 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6110lb # [ 124.912183] niks3-server[1059]: 2026/09/20 10:52:39 INFO Completed upload id=396111worker1 # [ 124.886843] nix-grpc-daemon[1442]: time=2026-09-20T10:52:38.304Z level=INFO msg="Upload complete. (426ms)"6112lb # [ 124.916624] niks3-server[1059]: 2026/09/20 10:52:39 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6113worker1 # [ 124.890070] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=5286114worker1 # [ 124.895398] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306115worker1 # [ 124.899497] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6116worker1 # [ 124.901781] nix-daemon[857]: reaped child process 1523, status = succeeded6117worker1 # [ 124.907137] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6118client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6119client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6120client # farm-a> LOG-farm-a-during-drain6121worker1 # [ 126.534840] nix-daemon[857]: reaped child process 1537, status = succeeded6122worker1 # [ 126.681919] nix-daemon[857]: accepted connection from pid 1552, user nix-grpc-daemon (trusted)6123worker1 # [ 126.693560] nix-daemon[857]: reaped child process 1555, status = succeeded6124lb # [ 126.736774] niks3-server[1059]: 2026/09/20 10:52:40 INFO Received uploads request method=POST path=/api/pending_closures6125worker1 # [ 126.717196] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6126worker1 # [ 126.718679] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.136Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6127lb # [ 126.777981] niks3-server[1059]: 2026/09/20 10:52:40 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6128lb # [ 126.785530] niks3-server[1059]: 2026/09/20 10:52:40 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6129lb # [ 126.797374] niks3-server[1059]: 2026/09/20 10:52:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6130worker1 # [ 126.776266] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.193Z level=INFO msg="Uploading 1 narinfos"6131lb # [ 126.803560] niks3-server[1059]: 2026/09/20 10:52:40 INFO Signed narinfos id=40 count=16132lb # [ 126.808438] niks3-server[1059]: 2026/09/20 10:52:40 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6133lb # [ 126.818056] niks3-server[1059]: 2026/09/20 10:52:40 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6134worker1 # [ 126.793241] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.210Z level=INFO msg="Upload complete. (277ms)"6135lb # [ 126.821050] niks3-server[1059]: 2026/09/20 10:52:40 INFO Completed upload id=406136lb # [ 126.822886] niks3-server[1059]: 2026/09/20 10:52:40 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6137worker1 # [ 126.795674] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16138worker2 # [ 126.810863] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:40Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06139worker1 # [ 126.805788] nix-daemon[857]: reaped child process 1535, status = succeeded6140worker2 # [ 126.812990] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:40Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16141worker1 # [ 126.854065] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306142worker1 # [ 126.866654] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306143worker1 # [ 126.879886] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.297Z level=INFO msg="All 1 paths already cached"6144client # copying 1 paths to 'lb:50051'...6145worker1 # [ 126.887861] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306146client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6147worker1 # [ 126.907251] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6148worker1 # [ 127.057069] nix-daemon[857]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6149worker1 # [ 127.069195] nix-daemon[857]: reaped child process 1562, status = succeeded6150worker1 # [ 127.222542] nix-daemon[857]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6151worker1 # [ 127.234276] nix-daemon[857]: reaped child process 1567, status = succeeded6152lb # [ 127.276202] niks3-server[1059]: 2026/09/20 10:52:41 INFO Received uploads request method=POST path=/api/pending_closures6153worker1 # [ 127.256433] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.673Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6154worker1 # [ 127.257906] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.675Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6155lb # [ 127.304320] niks3-server[1059]: 2026/09/20 10:52:41 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6156lb # [ 127.316152] niks3-server[1059]: 2026/09/20 10:52:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6157worker1 # [ 127.292770] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.710Z level=INFO msg="Uploading 1 narinfos"6158lb # [ 127.320266] niks3-server[1059]: 2026/09/20 10:52:41 INFO Signed narinfos id=41 count=16159lb # [ 127.326207] niks3-server[1059]: 2026/09/20 10:52:41 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6160lb # [ 127.337053] niks3-server[1059]: 2026/09/20 10:52:41 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6161lb # [ 127.340463] niks3-server[1059]: 2026/09/20 10:52:41 INFO Completed upload id=416162lb # [ 127.342059] niks3-server[1059]: 2026/09/20 10:52:41 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6163worker1 # [ 127.313915] nix-grpc-daemon[1442]: time=2026-09-20T10:52:40.731Z level=INFO msg="Upload complete. (395ms)"6164worker1 # [ 127.317061] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=6406165worker1 # [ 127.322277] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306166worker1 # [ 127.330266] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6167worker1 # [ 127.334935] nix-daemon[857]: reaped child process 1557, status = succeeded6168worker1 # [ 127.339837] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6169client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6170client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6171client # farm-b> LOG-farm-b-during-drain6172worker1 # [ 128.967852] nix-daemon[857]: reaped child process 1571, status = succeeded6173worker1 # [ 129.088251] nix-daemon[857]: accepted connection from pid 1586, user nix-grpc-daemon (trusted)6174worker1 # [ 129.100429] nix-daemon[857]: reaped child process 1589, status = succeeded6175lb # [ 129.141159] niks3-server[1059]: 2026/09/20 10:52:43 INFO Received uploads request method=POST path=/api/pending_closures6176worker1 # [ 129.118346] nix-grpc-daemon[1442]: time=2026-09-20T10:52:42.535Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6177worker1 # [ 129.120233] nix-grpc-daemon[1442]: time=2026-09-20T10:52:42.537Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6178lb # [ 129.161307] niks3-server[1059]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6179lb # [ 129.164221] niks3-server[1059]: 2026/09/20 10:52:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6180lb # [ 129.166267] niks3-server[1059]: 2026/09/20 10:52:43 INFO Signed narinfos id=42 count=16181worker1 # [ 129.140865] nix-grpc-daemon[1442]: time=2026-09-20T10:52:42.558Z level=INFO msg="Uploading 1 narinfos"6182lb # [ 129.169504] niks3-server[1059]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6183lb # [ 129.175964] niks3-server[1059]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6184lb # [ 129.182241] niks3-server[1059]: 2026/09/20 10:52:43 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6185lb # [ 129.184675] niks3-server[1059]: 2026/09/20 10:52:43 INFO Completed upload id=426186worker1 # [ 129.158181] nix-grpc-daemon[1442]: time=2026-09-20T10:52:42.575Z level=INFO msg="Upload complete. (209ms)"6187lb # [ 129.186406] niks3-server[1059]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6188worker1 # [ 129.161230] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16189worker1 # [ 129.167091] nix-daemon[857]: reaped child process 1569, status = succeeded6190worker2 # [ 129.173362] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:43Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06191worker2 # [ 129.175124] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:43Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16192worker1 # [ 129.210698] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306193worker1 # [ 129.221689] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306194worker1 # [ 129.233374] nix-grpc-daemon[1442]: time=2026-09-20T10:52:42.650Z level=INFO msg="All 2 paths already cached"6195client # copying 1 paths to 'lb:50051'...6196worker1 # [ 129.241508] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306197client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6198worker1 # [ 129.253911] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6199worker1 # [ 129.413740] nix-daemon[857]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6200worker1 # [ 129.425717] nix-daemon[857]: reaped child process 1596, status = succeeded6201worker1 # [ 129.580394] nix-daemon[857]: accepted connection from pid 1598, user nix-grpc-daemon (trusted)6202worker1 # [ 129.591981] nix-daemon[857]: reaped child process 1601, status = succeeded6203lb # [ 129.634594] niks3-server[1059]: 2026/09/20 10:52:43 INFO Received uploads request method=POST path=/api/pending_closures6204worker1 # [ 129.610636] nix-grpc-daemon[1442]: time=2026-09-20T10:52:43.028Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6205worker1 # [ 129.612484] nix-grpc-daemon[1442]: time=2026-09-20T10:52:43.029Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6206lb # [ 129.649155] niks3-server[1059]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6207lb # [ 129.651908] niks3-server[1059]: 2026/09/20 10:52:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6208lb # [ 129.654543] niks3-server[1059]: 2026/09/20 10:52:43 INFO Signed narinfos id=43 count=16209worker1 # [ 129.626599] nix-grpc-daemon[1442]: time=2026-09-20T10:52:43.043Z level=INFO msg="Uploading 1 narinfos"6210lb # [ 129.657545] niks3-server[1059]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6211lb # [ 129.665349] niks3-server[1059]: 2026/09/20 10:52:43 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6212worker1 # [ 129.644238] nix-grpc-daemon[1442]: time=2026-09-20T10:52:43.061Z level=INFO msg="Upload complete. (377ms)"6213lb # [ 129.672069] niks3-server[1059]: 2026/09/20 10:52:43 INFO Completed upload id=436214worker1 # [ 129.646158] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=7686215lb # [ 129.676608] niks3-server[1059]: 2026/09/20 10:52:43 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6216worker1 # [ 129.651390] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306217worker1 # [ 129.655800] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6218worker1 # [ 129.657985] nix-daemon[857]: reaped child process 1591, status = succeeded6219worker1 # [ 129.663155] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6220client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6221client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6222client # farm-top> LOG-farm-top-during-drain6223worker1 # [ 130.691216] nix-daemon[857]: reaped child process 1605, status = succeeded6224worker1 # [ 130.819225] nix-daemon[857]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6225worker1 # [ 130.830853] nix-daemon[857]: reaped child process 1623, status = succeeded6226lb # [ 130.868317] niks3-server[1059]: 2026/09/20 10:52:44 INFO Received uploads request method=POST path=/api/pending_closures6227worker1 # [ 130.844455] nix-grpc-daemon[1442]: time=2026-09-20T10:52:44.261Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6228worker1 # [ 130.845918] nix-grpc-daemon[1442]: time=2026-09-20T10:52:44.263Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6229lb # [ 130.887367] niks3-server[1059]: 2026/09/20 10:52:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6230worker1 # [ 130.859862] nix-grpc-daemon[1442]: time=2026-09-20T10:52:44.277Z level=INFO msg="Uploading 1 narinfos"6231lb # [ 130.888900] niks3-server[1059]: 2026/09/20 10:52:45 INFO Signed narinfos id=44 count=16232lb # [ 130.893136] niks3-server[1059]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6233lb # [ 130.898459] niks3-server[1059]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6234lb # [ 130.903907] niks3-server[1059]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6235lb # [ 130.908318] niks3-server[1059]: 2026/09/20 10:52:45 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6236lb # [ 130.911600] niks3-server[1059]: 2026/09/20 10:52:45 INFO Completed upload id=446237worker1 # [ 130.883925] nix-grpc-daemon[1442]: time=2026-09-20T10:52:44.301Z level=INFO msg="Upload complete. (213ms)"6238lb # [ 130.914337] niks3-server[1059]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6239worker1 # [ 130.886222] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16240worker1 # [ 130.892495] nix-daemon[857]: reaped child process 1603, status = succeeded6241client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.77 seconds)6242client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6243client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6244worker2: must succeed: pkill -f 'read -t [9]0 x'6245worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6246client: waiting for success: ! systemctl is-active sw6247worker2 # [ 131.313078] nix-daemon[847]: reaped child process 1644, status = succeeded6248worker2 # [ 131.480104] nix-daemon[847]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6249worker2 # [ 131.493501] nix-daemon[847]: reaped child process 1824, status = succeeded6250lb # [ 131.531133] niks3-server[1059]: 2026/09/20 10:52:45 INFO Received uploads request method=POST path=/api/pending_closures6251worker2 # [ 131.518128] nix-grpc-daemon[1572]: time=2026-09-20T10:52:45.637Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6252worker2 # [ 131.520205] nix-grpc-daemon[1572]: time=2026-09-20T10:52:45.639Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6253lb # [ 131.568294] niks3-server[1059]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6254lb # [ 131.578371] niks3-server[1059]: 2026/09/20 10:52:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6255lb # [ 131.582475] niks3-server[1059]: 2026/09/20 10:52:45 INFO Signed narinfos id=45 count=16256worker2 # [ 131.564960] nix-grpc-daemon[1572]: time=2026-09-20T10:52:45.684Z level=INFO msg="Uploading 1 narinfos"6257lb # [ 131.591083] niks3-server[1059]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6258lb # [ 131.594953] niks3-server[1059]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6259lb # [ 131.604770] niks3-server[1059]: 2026/09/20 10:52:45 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6260lb # [ 131.609609] niks3-server[1059]: 2026/09/20 10:52:45 INFO Completed upload id=456261worker2 # [ 131.590956] nix-grpc-daemon[1572]: time=2026-09-20T10:52:45.711Z level=INFO msg="Upload complete. (299ms)"6262lb # [ 131.614378] niks3-server[1059]: 2026/09/20 10:52:45 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6263worker2 # [ 131.600142] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53084 duration_s=12 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16264worker2 # [ 131.611720] nix-daemon[847]: reaped child process 1642, status = succeeded6265client # [ 131.630527] systemd[1]: sw.service: Deactivated successfully.6266client # [ 131.634387] systemd[1]: sw.service: Consumed 119ms CPU time over 13.938s wall clock time, 39.1M memory peak, 8.7K incoming IP traffic, 9.2K outgoing IP traffic.6267worker2 # [ 131.890066] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=signal_exit6268lb # [ 131.914951] niks3-server[1059]: 2026/09/20 10:52:46 INFO lead: released remote=192.168.1.4:358686269worker2 # [ 131.895229] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_yield6270worker1 # [ 131.892521] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:45Z level=info event=scheduler_restarting addr=lb:500516271worker2 # [ 131.900938] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:530586272worker2 # [ 131.904379] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=worker_gone addr=192.168.1.3:500516273worker2 # [ 131.907417] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_restarting addr=lb:500516274worker2 # [ 131.910586] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:530586275worker2 # [ 131.913865] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=worker_gone addr=192.168.1.4:500516276lb # [ 132.111272] niks3-server[1059]: 2026/09/20 10:52:46 INFO lead: acquired remote=192.168.1.3:447546277worker1 # [ 132.087332] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:45Z level=info event=scheduler_take_over6278worker1 # [ 132.092421] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:45Z level=info event=scheduler_connected addr=lb:500516279worker2 # [ 132.101158] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516280worker1 # [ 132.096229] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6281worker2 # [ 132.105166] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6282client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6283client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6284worker1 # [ 132.297124] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:45Z level=info event=scheduler_connected addr=lb:500516285worker2 # [ 132.306516] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516286worker1 # [ 132.301286] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6287worker2 # [ 132.310796] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6288client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6289worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6290worker1 # [ 132.501974] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:45Z level=info event=scheduler_connected addr=lb:500516291worker2 # [ 132.511876] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516292worker1 # [ 132.505899] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6293worker2 # [ 132.514931] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6294worker1 # [ 132.706706] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516295worker2 # [ 132.717479] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516296worker1 # [ 132.710659] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6297worker2 # [ 132.720681] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6298worker2 # [ 132.896189] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:47Z level=info event=scheduler_restarting6299worker1 # [ 132.912887] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516300worker1 # [ 132.916132] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6301worker2 # [ 132.930315] nix-grpc-daemon[1493]: ts=2026-09-20T10:52:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6302worker2 # [ 132.935505] nix-daemon[847]: reaped child process 1544, status = succeeded6303worker2 # [ 133.075758] nix-grpc-daemon[1493]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6304worker2 # [ 133.079361] nix-grpc-daemon[1493]: E0000 00:00:1789901567.195448 1536 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6305worker2 # [ 133.084134] nix-grpc-daemon[1493]: E0000 00:00:1789901567.195659 1536 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6306worker1 # [ 133.113839] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516307worker1 # [ 133.117661] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6308worker1 # [ 133.318274] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516309worker1 # [ 133.321986] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6310worker1 # [ 133.522562] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_connected addr=lb:500516311worker1 # [ 133.524338] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6312worker1 # [ 133.724914] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:47Z level=info event=scheduler_connected addr=lb:500516313worker1 # [ 133.726974] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6314worker1 # [ 133.927907] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:47Z level=info event=scheduler_connected addr=lb:500516315worker1 # [ 133.929930] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6316worker1 # [ 134.130460] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:47Z level=info event=scheduler_connected addr=lb:500516317worker1 # [ 134.138183] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06318worker2 # [ 134.922315] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6319worker2 # [ 134.923438] systemd[1]: nix-grpc-daemon.service: Consumed 673ms CPU time over 28.351s wall clock time, 33.6M memory peak, 23.8K incoming IP traffic, 32.6K outgoing IP traffic.6320worker2 # [ 134.931749] systemd[1]: Starting Nix worker-protocol over gRPC...6321worker2 # [ 135.098654] nix-grpc-daemon[1838]: ts=2026-09-20T10:52:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6322worker2 # [ 135.101961] nix-grpc-daemon[1838]: ts=2026-09-20T10:52:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16323worker2 # [ 135.108077] nix-grpc-daemon[1838]: ts=2026-09-20T10:52:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6324worker2 # [ 135.110858] systemd[1]: Started Nix worker-protocol over gRPC.6325worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.82 seconds)6326lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6327lb: must succeed: curl -sf localhost:9901/clusters6328worker2 # [ 135.140274] nix-grpc-daemon[1838]: ts=2026-09-20T10:52:49Z level=info event=scheduler_connected addr=lb:500516329worker1 # [ 135.137128] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6331lb: must succeed: curl -sf localhost:9901/clusters6332lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6333lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.13 seconds)6334lb: must succeed: curl -sf localhost:9901/clusters6335lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6336worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6337worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6338(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.71 seconds)6339subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6340client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16341worker2 # [ 136.559542] nix-grpc-daemon[1838]: ts=2026-09-20T10:52:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:599706342worker2 # [ 136.569357] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6343worker2 # [ 136.583583] nix-grpc-daemon[1838]: ts=2026-09-20T10:52:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599706344worker2 # [ 136.594910] nix-grpc-daemon[1838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)6345worker2 # [ 136.650947] nix-grpc-daemon[1838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 2/5)6346worker2 # [ 136.779276] nix-grpc-daemon[1838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 252 ms (attempt 3/5)6347worker2 # [ 137.037615] nix-grpc-daemon[1838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 626 ms (attempt 4/5)6348worker2 # [ 137.665570] nix-grpc-daemon[1838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6349worker1 # [ 137.704077] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06350worker1 # [ 137.708066] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16351worker1 # [ 137.712297] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06352worker1 # [ 137.716258] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06353worker1 # [ 137.747509] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306354worker1 # [ 137.759579] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306355worker1 # [ 137.772409] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306356worker1 # [ 137.780887] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6357worker1 # [ 137.953460] nix-daemon[857]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6358worker1 # [ 137.965797] nix-daemon[857]: reaped child process 1635, status = succeeded6359worker1 # [ 138.123486] nix-daemon[857]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)6360worker1 # [ 138.134907] nix-daemon[857]: reaped child process 1640, status = succeeded6361lb # [ 138.179506] niks3-server[1059]: 2026/09/20 10:52:52 INFO Received uploads request method=POST path=/api/pending_closures6362worker1 # [ 138.161694] nix-grpc-daemon[1442]: time=2026-09-20T10:52:51.578Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6363worker1 # [ 138.163187] nix-grpc-daemon[1442]: time=2026-09-20T10:52:51.580Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6364lb # [ 138.209755] niks3-server[1059]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6365lb # [ 138.224902] niks3-server[1059]: 2026/09/20 10:52:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6366worker1 # [ 138.202385] nix-grpc-daemon[1442]: time=2026-09-20T10:52:51.619Z level=INFO msg="Uploading 1 narinfos"6367lb # [ 138.231127] niks3-server[1059]: 2026/09/20 10:52:52 INFO Signed narinfos id=46 count=16368lb # [ 138.235825] niks3-server[1059]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6369lb # [ 138.247703] niks3-server[1059]: 2026/09/20 10:52:52 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6370lb # [ 138.253906] niks3-server[1059]: 2026/09/20 10:52:52 INFO Completed upload id=466371worker1 # [ 138.226303] nix-grpc-daemon[1442]: time=2026-09-20T10:52:51.643Z level=INFO msg="Upload complete. (432ms)"6372worker1 # [ 138.229522] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=5046373lb # [ 138.259499] niks3-server[1059]: 2026/09/20 10:52:52 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6374worker1 # [ 138.235725] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306375worker1 # [ 138.248723] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6376worker1 # [ 138.251036] nix-daemon[857]: reaped child process 1630, status = succeeded6377worker1 # [ 138.255805] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6378lb # [ 139.067738] envoy[931]: conn peer=[2001:db8:1::1]:58642 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9598 tx=3825 ms=65486379lb # [ 139.072418] envoy[931]: conn peer=[2001:db8:1::1]:42448 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1377 ms=137026380lb # [ 139.076820] envoy[931]: conn peer=[2001:db8:1::4]:59916 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1721 tx=539 ms=260146381worker1 # [ 139.082792] nix-daemon[857]: reaped child process 1644, status = succeeded6382worker1 # [ 139.235157] nix-daemon[857]: accepted connection from pid 1659, user nix-grpc-daemon (trusted)6383worker1 # [ 139.246968] nix-daemon[857]: reaped child process 1662, status = succeeded6384lb # [ 139.291470] niks3-server[1059]: 2026/09/20 10:52:53 INFO Received uploads request method=POST path=/api/pending_closures6385worker1 # [ 139.271626] nix-grpc-daemon[1442]: time=2026-09-20T10:52:52.688Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6386worker1 # [ 139.273164] nix-grpc-daemon[1442]: time=2026-09-20T10:52:52.690Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6387lb # [ 139.325083] niks3-server[1059]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6388lb # [ 139.337969] niks3-server[1059]: 2026/09/20 10:52:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6389worker1 # [ 139.315439] nix-grpc-daemon[1442]: time=2026-09-20T10:52:52.732Z level=INFO msg="Uploading 1 narinfos"6390lb # [ 139.344269] niks3-server[1059]: 2026/09/20 10:52:53 INFO Signed narinfos id=47 count=16391lb # [ 139.347334] niks3-server[1059]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6392lb # [ 139.354692] niks3-server[1059]: 2026/09/20 10:52:53 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6393lb # [ 139.358139] niks3-server[1059]: 2026/09/20 10:52:53 INFO Completed upload id=476394worker1 # [ 139.330420] nix-grpc-daemon[1442]: time=2026-09-20T10:52:52.747Z level=INFO msg="Upload complete. (267ms)"6395lb # [ 139.361768] niks3-server[1059]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6396worker1 # [ 139.334552] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16397lb # [ 139.365684] niks3-server[1059]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6398worker1 # [ 139.342151] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:52Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16399worker1 # [ 139.347404] nix-daemon[857]: reaped child process 1642, status = succeeded6400worker1 # [ 139.378563] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306401worker1 # [ 139.383919] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:52Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06402worker1 # [ 139.391122] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306403worker1 # [ 139.400392] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306404worker1 # [ 139.406947] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6405worker1 # [ 139.565274] nix-daemon[857]: accepted connection from pid 1668, user nix-grpc-daemon (trusted)6406worker1 # [ 139.576924] nix-daemon[857]: reaped child process 1671, status = succeeded6407worker1 # [ 139.734646] nix-daemon[857]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6408worker1 # [ 139.746743] nix-daemon[857]: reaped child process 1676, status = succeeded6409lb # [ 139.789914] niks3-server[1059]: 2026/09/20 10:52:53 INFO Received uploads request method=POST path=/api/pending_closures6410worker1 # [ 139.767209] nix-grpc-daemon[1442]: time=2026-09-20T10:52:53.184Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6411worker1 # [ 139.769153] nix-grpc-daemon[1442]: time=2026-09-20T10:52:53.186Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6412lb # [ 139.804973] niks3-server[1059]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6413lb # [ 139.808640] niks3-server[1059]: 2026/09/20 10:52:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6414lb # [ 139.810992] niks3-server[1059]: 2026/09/20 10:52:53 INFO Signed narinfos id=48 count=16415worker1 # [ 139.783139] nix-grpc-daemon[1442]: time=2026-09-20T10:52:53.200Z level=INFO msg="Uploading 1 narinfos"6416lb # [ 139.814249] niks3-server[1059]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6417lb # [ 139.817943] niks3-server[1059]: 2026/09/20 10:52:53 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6418lb # [ 139.823248] niks3-server[1059]: 2026/09/20 10:52:53 INFO Completed upload id=486419worker1 # [ 139.795510] nix-grpc-daemon[1442]: time=2026-09-20T10:52:53.212Z level=INFO msg="Upload complete. (380ms)"6420lb # [ 139.824941] niks3-server[1059]: 2026/09/20 10:52:53 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6421worker1 # [ 139.797560] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=5046422worker1 # [ 139.801157] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306423worker1 # [ 139.808074] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6424worker1 # [ 139.809538] nix-daemon[857]: reaped child process 1666, status = succeeded6425worker1 # [ 139.814405] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6426worker1 # [ 140.828164] nix-daemon[857]: reaped child process 1680, status = succeeded6427worker1 # [ 140.964244] nix-daemon[857]: accepted connection from pid 1695, user nix-grpc-daemon (trusted)6428worker1 # [ 140.975979] nix-daemon[857]: reaped child process 1698, status = succeeded6429lb # [ 141.016321] niks3-server[1059]: 2026/09/20 10:52:55 INFO Received uploads request method=POST path=/api/pending_closures6430worker1 # [ 140.997273] nix-grpc-daemon[1442]: time=2026-09-20T10:52:54.413Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6431worker1 # [ 140.998620] nix-grpc-daemon[1442]: time=2026-09-20T10:52:54.413Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6432lb # [ 141.052624] niks3-server[1059]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6433lb # [ 141.065167] niks3-server[1059]: 2026/09/20 10:52:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6434lb # [ 141.069512] niks3-server[1059]: 2026/09/20 10:52:55 INFO Signed narinfos id=49 count=16435worker1 # [ 141.044930] nix-grpc-daemon[1442]: time=2026-09-20T10:52:54.462Z level=INFO msg="Uploading 1 narinfos"6436lb # [ 141.076868] niks3-server[1059]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6437lb # [ 141.084095] niks3-server[1059]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6438lb # [ 141.090928] niks3-server[1059]: 2026/09/20 10:52:55 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6439lb # [ 141.098716] niks3-server[1059]: 2026/09/20 10:52:55 INFO Completed upload id=496440worker1 # [ 141.071082] nix-grpc-daemon[1442]: time=2026-09-20T10:52:54.488Z level=INFO msg="Upload complete. (250ms)"6441lb # [ 141.100502] niks3-server[1059]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6442worker1 # [ 141.081755] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16443worker1 # [ 141.088316] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:54Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16444worker1 # [ 141.092203] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306445worker1 # [ 141.095588] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:54Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06446worker1 # [ 141.104778] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306447worker1 # [ 141.106862] nix-daemon[857]: reaped child process 1678, status = succeeded6448worker1 # [ 141.110668] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306449worker1 # [ 141.114498] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6450worker1 # [ 141.263929] nix-daemon[857]: accepted connection from pid 1704, user nix-grpc-daemon (trusted)6451worker1 # [ 141.276563] nix-daemon[857]: reaped child process 1707, status = succeeded6452worker1 # [ 141.425096] nix-daemon[857]: accepted connection from pid 1709, user nix-grpc-daemon (trusted)6453worker1 # [ 141.436598] nix-daemon[857]: reaped child process 1712, status = succeeded6454lb # [ 141.479916] niks3-server[1059]: 2026/09/20 10:52:55 INFO Received uploads request method=POST path=/api/pending_closures6455worker1 # [ 141.459150] nix-grpc-daemon[1442]: time=2026-09-20T10:52:54.876Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6456worker1 # [ 141.460692] nix-grpc-daemon[1442]: time=2026-09-20T10:52:54.878Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6457lb # [ 141.509564] niks3-server[1059]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6458lb # [ 141.517973] niks3-server[1059]: 2026/09/20 10:52:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6459worker1 # [ 141.494719] nix-grpc-daemon[1442]: time=2026-09-20T10:52:54.912Z level=INFO msg="Uploading 1 narinfos"6460lb # [ 141.522930] niks3-server[1059]: 2026/09/20 10:52:55 INFO Signed narinfos id=50 count=16461lb # [ 141.528391] niks3-server[1059]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6462lb # [ 141.539893] niks3-server[1059]: 2026/09/20 10:52:55 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6463worker1 # [ 141.518236] nix-grpc-daemon[1442]: time=2026-09-20T10:52:54.935Z level=INFO msg="Upload complete. (398ms)"6464lb # [ 141.545986] niks3-server[1059]: 2026/09/20 10:52:55 INFO Completed upload id=506465worker1 # [ 141.520188] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=5046466lb # [ 141.550622] niks3-server[1059]: 2026/09/20 10:52:55 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6467worker1 # [ 141.524653] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306468worker1 # [ 141.534059] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6469worker1 # [ 141.538126] nix-daemon[857]: reaped child process 1702, status = succeeded6470worker1 # [ 141.542888] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6471worker1 # [ 142.769926] nix-daemon[857]: reaped child process 1716, status = succeeded6472worker1 # [ 142.931806] nix-daemon[857]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6473worker1 # [ 142.942271] nix-daemon[857]: reaped child process 1734, status = succeeded6474lb # [ 142.987645] niks3-server[1059]: 2026/09/20 10:52:57 INFO Received uploads request method=POST path=/api/pending_closures6475worker1 # [ 142.968164] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.385Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6476worker1 # [ 142.969758] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.387Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6477lb # [ 143.024519] niks3-server[1059]: 2026/09/20 10:52:57 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6478lb # [ 143.032211] niks3-server[1059]: 2026/09/20 10:52:57 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6479lb # [ 143.042322] niks3-server[1059]: 2026/09/20 10:52:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6480lb # [ 143.048237] niks3-server[1059]: 2026/09/20 10:52:57 INFO Signed narinfos id=51 count=16481worker1 # [ 143.023329] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.440Z level=INFO msg="Uploading 1 narinfos"6482lb # [ 143.053675] niks3-server[1059]: 2026/09/20 10:52:57 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6483lb # [ 143.061465] niks3-server[1059]: 2026/09/20 10:52:57 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6484lb # [ 143.067477] niks3-server[1059]: 2026/09/20 10:52:57 INFO Completed upload id=516485worker1 # [ 143.039905] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.457Z level=INFO msg="Upload complete. (288ms)"6486lb # [ 143.070094] niks3-server[1059]: 2026/09/20 10:52:57 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6487worker1 # [ 143.047697] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16488worker1 # [ 143.056506] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:56Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16489worker1 # [ 143.061241] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:56Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06490worker1 # [ 143.069591] nix-daemon[857]: reaped child process 1714, status = succeeded6491worker1 # [ 143.090781] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306492worker1 # [ 143.102509] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306493worker1 # [ 143.115094] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.532Z level=INFO msg="All 1 paths already cached"6494worker1 # [ 143.121558] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306495worker1 # [ 143.131112] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6496worker1 # [ 143.271328] nix-daemon[857]: accepted connection from pid 1740, user nix-grpc-daemon (trusted)6497worker1 # [ 143.283140] nix-daemon[857]: reaped child process 1743, status = succeeded6498worker1 # [ 143.424570] nix-daemon[857]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6499worker1 # [ 143.436329] nix-daemon[857]: reaped child process 1748, status = succeeded6500lb # [ 143.479221] niks3-server[1059]: 2026/09/20 10:52:57 INFO Received uploads request method=POST path=/api/pending_closures6501worker1 # [ 143.458517] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.875Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6502worker1 # [ 143.460064] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.877Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6503lb # [ 143.504146] niks3-server[1059]: 2026/09/20 10:52:57 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6504lb # [ 143.516345] niks3-server[1059]: 2026/09/20 10:52:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6505worker1 # [ 143.493654] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.910Z level=INFO msg="Uploading 1 narinfos"6506lb # [ 143.521620] niks3-server[1059]: 2026/09/20 10:52:57 INFO Signed narinfos id=52 count=16507lb # [ 143.526532] niks3-server[1059]: 2026/09/20 10:52:57 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6508lb # [ 143.538975] niks3-server[1059]: 2026/09/20 10:52:57 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6509lb # [ 143.545100] niks3-server[1059]: 2026/09/20 10:52:57 INFO Completed upload id=526510worker1 # [ 143.517310] nix-grpc-daemon[1442]: time=2026-09-20T10:52:56.934Z level=INFO msg="Upload complete. (377ms)"6511worker1 # [ 143.519471] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=6246512lb # [ 143.548422] niks3-server[1059]: 2026/09/20 10:52:57 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6513worker1 # [ 143.524860] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306514worker1 # [ 143.533570] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6515worker1 # [ 143.537435] nix-daemon[857]: reaped child process 1738, status = succeeded6516worker1 # [ 143.542207] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6517worker1 # [ 144.759289] nix-daemon[857]: reaped child process 1752, status = succeeded6518worker1 # [ 144.897143] nix-daemon[857]: accepted connection from pid 1767, user nix-grpc-daemon (trusted)6519worker1 # [ 144.911893] nix-daemon[857]: reaped child process 1770, status = succeeded6520lb # [ 144.956636] niks3-server[1059]: 2026/09/20 10:52:59 INFO Received uploads request method=POST path=/api/pending_closures6521worker1 # [ 144.939483] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.356Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6522worker1 # [ 144.941070] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.358Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6523lb # [ 144.993314] niks3-server[1059]: 2026/09/20 10:52:59 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6524lb # [ 145.009753] niks3-server[1059]: 2026/09/20 10:52:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6525worker1 # [ 144.986278] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.403Z level=INFO msg="Uploading 1 narinfos"6526lb # [ 145.015437] niks3-server[1059]: 2026/09/20 10:52:59 INFO Signed narinfos id=53 count=16527lb # [ 145.018532] niks3-server[1059]: 2026/09/20 10:52:59 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6528lb # [ 145.026494] niks3-server[1059]: 2026/09/20 10:52:59 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6529lb # [ 145.029357] niks3-server[1059]: 2026/09/20 10:52:59 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6530lb # [ 145.033646] niks3-server[1059]: 2026/09/20 10:52:59 INFO Completed upload id=536531worker1 # [ 145.007048] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.424Z level=INFO msg="Upload complete. (256ms)"6532lb # [ 145.035219] niks3-server[1059]: 2026/09/20 10:52:59 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6533worker1 # [ 145.010614] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16534worker1 # [ 145.014689] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:58Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16535worker1 # [ 145.017772] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306536worker1 # [ 145.020131] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:58Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06537worker1 # [ 145.027823] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306538worker1 # [ 145.029675] nix-daemon[857]: reaped child process 1750, status = succeeded6539worker1 # [ 145.035334] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.452Z level=INFO msg="All 1 paths already cached"6540worker1 # [ 145.039059] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306541worker1 # [ 145.043839] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6542worker1 # [ 145.225839] nix-daemon[857]: accepted connection from pid 1776, user nix-grpc-daemon (trusted)6543worker1 # [ 145.241665] nix-daemon[857]: reaped child process 1779, status = succeeded6544worker1 # [ 145.398412] nix-daemon[857]: accepted connection from pid 1781, user nix-grpc-daemon (trusted)6545worker1 # [ 145.412743] nix-daemon[857]: reaped child process 1784, status = succeeded6546lb # [ 145.456543] niks3-server[1059]: 2026/09/20 10:52:59 INFO Received uploads request method=POST path=/api/pending_closures6547worker1 # [ 145.435712] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.853Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6548worker1 # [ 145.437277] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.854Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6549lb # [ 145.475233] niks3-server[1059]: 2026/09/20 10:52:59 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6550lb # [ 145.481604] niks3-server[1059]: 2026/09/20 10:52:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6551lb # [ 145.483975] niks3-server[1059]: 2026/09/20 10:52:59 INFO Signed narinfos id=54 count=16552worker1 # [ 145.456289] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.873Z level=INFO msg="Uploading 1 narinfos"6553lb # [ 145.486559] niks3-server[1059]: 2026/09/20 10:52:59 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6554lb # [ 145.492054] niks3-server[1059]: 2026/09/20 10:52:59 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6555lb # [ 145.494818] niks3-server[1059]: 2026/09/20 10:52:59 INFO Completed upload id=546556worker1 # [ 145.467122] nix-grpc-daemon[1442]: time=2026-09-20T10:52:58.884Z level=INFO msg="Upload complete. (416ms)"6557lb # [ 145.497254] niks3-server[1059]: 2026/09/20 10:52:59 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6558worker1 # [ 145.468657] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=6246559worker1 # [ 145.474626] nix-grpc-daemon[1354]: ts=2026-09-20T10:52:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306560worker1 # [ 145.480651] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6561worker1 # [ 145.483133] nix-daemon[857]: reaped child process 1774, status = succeeded6562worker1 # [ 145.488812] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6563worker1 # [ 146.514801] nix-daemon[857]: reaped child process 1788, status = succeeded6564worker1 # [ 146.671340] nix-daemon[857]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)6565worker1 # [ 146.682191] nix-daemon[857]: reaped child process 1806, status = succeeded6566lb # [ 146.725653] niks3-server[1059]: 2026/09/20 10:53:00 INFO Received uploads request method=POST path=/api/pending_closures6567worker1 # [ 146.702313] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.119Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6568worker1 # [ 146.703795] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.121Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6569lb # [ 146.747820] niks3-server[1059]: 2026/09/20 10:53:00 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6570lb # [ 146.750670] niks3-server[1059]: 2026/09/20 10:53:00 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6571lb # [ 146.753227] niks3-server[1059]: 2026/09/20 10:53:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6572lb # [ 146.755317] niks3-server[1059]: 2026/09/20 10:53:00 INFO Signed narinfos id=55 count=16573worker1 # [ 146.727548] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.144Z level=INFO msg="Uploading 1 narinfos"6574lb # [ 146.757987] niks3-server[1059]: 2026/09/20 10:53:00 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6575lb # [ 146.762086] niks3-server[1059]: 2026/09/20 10:53:00 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6576lb # [ 146.765142] niks3-server[1059]: 2026/09/20 10:53:00 INFO Completed upload id=556577worker1 # [ 146.738612] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.156Z level=INFO msg="Upload complete. (242ms)"6578lb # [ 146.766987] niks3-server[1059]: 2026/09/20 10:53:00 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6579worker1 # [ 146.741838] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16580worker1 # [ 146.746995] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:00Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16581worker1 # [ 146.749823] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306582worker1 # [ 146.752839] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:00Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06583worker1 # [ 146.758284] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306584worker1 # [ 146.761104] nix-daemon[857]: reaped child process 1786, status = succeeded6585worker1 # [ 146.765040] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.182Z level=INFO msg="All 1 paths already cached"6586worker1 # [ 146.768787] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306587worker1 # [ 146.772413] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6588worker1 # [ 146.916375] nix-daemon[857]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6589worker1 # [ 146.928121] nix-daemon[857]: reaped child process 1815, status = succeeded6590worker1 # [ 147.074049] nix-daemon[857]: accepted connection from pid 1817, user nix-grpc-daemon (trusted)6591worker1 # [ 147.085680] nix-daemon[857]: reaped child process 1820, status = succeeded6592lb # [ 147.129084] niks3-server[1059]: 2026/09/20 10:53:01 INFO Received uploads request method=POST path=/api/pending_closures6593worker1 # [ 147.109210] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.526Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6594worker1 # [ 147.110560] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.526Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6595lb # [ 147.152660] niks3-server[1059]: 2026/09/20 10:53:01 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6596lb # [ 147.162191] niks3-server[1059]: 2026/09/20 10:53:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6597lb # [ 147.166616] niks3-server[1059]: 2026/09/20 10:53:01 INFO Signed narinfos id=56 count=16598worker1 # [ 147.141830] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.559Z level=INFO msg="Uploading 1 narinfos"6599lb # [ 147.176152] niks3-server[1059]: 2026/09/20 10:53:01 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6600lb # [ 147.185116] niks3-server[1059]: 2026/09/20 10:53:01 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6601worker1 # [ 147.163868] nix-grpc-daemon[1442]: time=2026-09-20T10:53:00.581Z level=INFO msg="Upload complete. (385ms)"6602lb # [ 147.191871] niks3-server[1059]: 2026/09/20 10:53:01 INFO Completed upload id=566603worker1 # [ 147.165764] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=6246604lb # [ 147.195462] niks3-server[1059]: 2026/09/20 10:53:01 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6605worker1 # [ 147.172129] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306606worker1 # [ 147.177145] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6607worker1 # [ 147.179455] nix-daemon[857]: reaped child process 1810, status = succeeded6608worker1 # [ 147.184462] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6609worker1 # [ 148.597884] nix-daemon[857]: reaped child process 1824, status = succeeded6610worker1 # [ 148.739807] nix-daemon[857]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6611worker1 # [ 148.750580] nix-daemon[857]: reaped child process 1842, status = succeeded6612lb # [ 148.795311] niks3-server[1059]: 2026/09/20 10:53:02 INFO Received uploads request method=POST path=/api/pending_closures6613worker1 # [ 148.775146] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.192Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6614worker1 # [ 148.776672] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.194Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6615lb # [ 148.827125] niks3-server[1059]: 2026/09/20 10:53:02 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6616lb # [ 148.837771] niks3-server[1059]: 2026/09/20 10:53:02 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6617lb # [ 148.846797] niks3-server[1059]: 2026/09/20 10:53:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6618lb # [ 148.851665] niks3-server[1059]: 2026/09/20 10:53:02 INFO Signed narinfos id=57 count=16619worker1 # [ 148.826787] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.244Z level=INFO msg="Uploading 1 narinfos"6620lb # [ 148.863133] niks3-server[1059]: 2026/09/20 10:53:02 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6621worker1 # [ 148.841338] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.258Z level=INFO msg="Upload complete. (248ms)"6622lb # [ 148.868983] niks3-server[1059]: 2026/09/20 10:53:02 INFO Completed upload id=576623worker1 # [ 148.843751] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16624lb # [ 148.872365] niks3-server[1059]: 2026/09/20 10:53:02 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6625worker1 # [ 148.847551] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:02Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16626lb # [ 148.876226] niks3-server[1059]: 2026/09/20 10:53:02 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6627worker1 # [ 148.850899] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306628worker1 # [ 148.853948] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:02Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06629worker1 # [ 148.858208] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306630worker1 # [ 148.862697] nix-daemon[857]: reaped child process 1822, status = succeeded6631worker1 # [ 148.864574] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.282Z level=INFO msg="All 2 paths already cached"6632worker1 # [ 148.868262] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306633worker1 # [ 148.872144] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6634worker1 # [ 149.025768] nix-daemon[857]: accepted connection from pid 1848, user nix-grpc-daemon (trusted)6635worker1 # [ 149.037785] nix-daemon[857]: reaped child process 1851, status = succeeded6636worker1 # [ 149.194729] nix-daemon[857]: accepted connection from pid 1853, user nix-grpc-daemon (trusted)6637worker1 # [ 149.205153] nix-daemon[857]: reaped child process 1856, status = succeeded6638lb # [ 149.248363] niks3-server[1059]: 2026/09/20 10:53:03 INFO Received uploads request method=POST path=/api/pending_closures6639worker1 # [ 149.226299] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.643Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6640worker1 # [ 149.227756] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.645Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6641lb # [ 149.265686] niks3-server[1059]: 2026/09/20 10:53:03 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6642lb # [ 149.268431] niks3-server[1059]: 2026/09/20 10:53:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6643lb # [ 149.269960] niks3-server[1059]: 2026/09/20 10:53:03 INFO Signed narinfos id=58 count=16644worker1 # [ 149.243451] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.660Z level=INFO msg="Uploading 1 narinfos"6645lb # [ 149.273506] niks3-server[1059]: 2026/09/20 10:53:03 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6646lb # [ 149.278430] niks3-server[1059]: 2026/09/20 10:53:03 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6647lb # [ 149.280942] niks3-server[1059]: 2026/09/20 10:53:03 INFO Completed upload id=586648worker1 # [ 149.253225] nix-grpc-daemon[1442]: time=2026-09-20T10:53:02.670Z level=INFO msg="Upload complete. (375ms)"6649lb # [ 149.283463] niks3-server[1059]: 2026/09/20 10:53:03 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6650worker1 # [ 149.255162] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=7526651worker1 # [ 149.257894] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306652worker1 # [ 149.263351] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6653worker1 # [ 149.267186] nix-daemon[857]: reaped child process 1846, status = succeeded6654worker1 # [ 149.272029] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6655worker1 # [ 150.085401] nix-daemon[857]: reaped child process 1860, status = succeeded6656worker1 # [ 150.211131] nix-daemon[857]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6657worker1 # [ 150.222901] nix-daemon[857]: reaped child process 1878, status = succeeded6658lb # [ 150.262391] niks3-server[1059]: 2026/09/20 10:53:04 INFO Received uploads request method=POST path=/api/pending_closures6659worker1 # [ 150.239362] nix-grpc-daemon[1442]: time=2026-09-20T10:53:03.656Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6660worker1 # [ 150.240818] nix-grpc-daemon[1442]: time=2026-09-20T10:53:03.658Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6661lb # [ 150.280115] niks3-server[1059]: 2026/09/20 10:53:04 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6662lb # [ 150.286404] niks3-server[1059]: 2026/09/20 10:53:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6663lb # [ 150.288905] niks3-server[1059]: 2026/09/20 10:53:04 INFO Signed narinfos id=59 count=16664worker1 # [ 150.261280] nix-grpc-daemon[1442]: time=2026-09-20T10:53:03.678Z level=INFO msg="Uploading 1 narinfos"6665lb # [ 150.291673] niks3-server[1059]: 2026/09/20 10:53:04 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6666lb # [ 150.296059] niks3-server[1059]: 2026/09/20 10:53:04 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6667lb # [ 150.298981] niks3-server[1059]: 2026/09/20 10:53:04 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6668lb # [ 150.302061] niks3-server[1059]: 2026/09/20 10:53:04 INFO Completed upload id=596669worker1 # [ 150.275489] nix-grpc-daemon[1442]: time=2026-09-20T10:53:03.692Z level=INFO msg="Upload complete. (197ms)"6670lb # [ 150.303729] niks3-server[1059]: 2026/09/20 10:53:04 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6671worker1 # [ 150.277927] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16672worker1 # [ 150.281533] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:03Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16673worker1 # [ 150.286715] nix-daemon[857]: reaped child process 1858, status = succeeded6674worker1 # [ 150.323753] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306675worker1 # [ 150.330064] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306676worker1 # [ 150.338577] nix-grpc-daemon[1442]: time=2026-09-20T10:53:03.755Z level=INFO msg="All 2 paths already cached"6677worker1 # [ 150.343725] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306678worker1 # [ 150.350252] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6679worker1 # [ 150.484411] nix-daemon[857]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6680worker1 # [ 150.496296] nix-daemon[857]: reaped child process 1885, status = succeeded6681worker1 # [ 150.628345] nix-daemon[857]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6682worker1 # [ 150.639769] nix-daemon[857]: reaped child process 1890, status = succeeded6683lb # [ 150.681732] niks3-server[1059]: 2026/09/20 10:53:04 INFO Received uploads request method=POST path=/api/pending_closures6684worker1 # [ 150.658827] nix-grpc-daemon[1442]: time=2026-09-20T10:53:04.076Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6685worker1 # [ 150.660351] nix-grpc-daemon[1442]: time=2026-09-20T10:53:04.077Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6686lb # [ 150.698728] niks3-server[1059]: 2026/09/20 10:53:04 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6687lb # [ 150.702522] niks3-server[1059]: 2026/09/20 10:53:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6688lb # [ 150.704170] niks3-server[1059]: 2026/09/20 10:53:04 INFO Signed narinfos id=60 count=16689worker1 # [ 150.677801] nix-grpc-daemon[1442]: time=2026-09-20T10:53:04.095Z level=INFO msg="Uploading 1 narinfos"6690lb # [ 150.708490] niks3-server[1059]: 2026/09/20 10:53:04 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6691lb # [ 150.712542] niks3-server[1059]: 2026/09/20 10:53:04 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6692lb # [ 150.715822] niks3-server[1059]: 2026/09/20 10:53:04 INFO Completed upload id=606693worker1 # [ 150.688069] nix-grpc-daemon[1442]: time=2026-09-20T10:53:04.105Z level=INFO msg="Upload complete. (329ms)"6694lb # [ 150.717432] niks3-server[1059]: 2026/09/20 10:53:04 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6695worker1 # [ 150.689873] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=7526696worker1 # [ 150.693901] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306697worker1 # [ 150.699484] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6698worker1 # [ 150.701705] nix-daemon[857]: reaped child process 1880, status = succeeded6699worker1 # [ 150.706896] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6700worker1 # [ 151.922557] nix-daemon[857]: reaped child process 1894, status = succeeded6701worker1 # [ 152.063035] nix-daemon[857]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)6702worker1 # [ 152.074810] nix-daemon[857]: reaped child process 1912, status = succeeded6703lb # [ 152.119585] niks3-server[1059]: 2026/09/20 10:53:06 INFO Received uploads request method=POST path=/api/pending_closures6704worker1 # [ 152.099445] nix-grpc-daemon[1442]: time=2026-09-20T10:53:05.516Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6705worker1 # [ 152.100920] nix-grpc-daemon[1442]: time=2026-09-20T10:53:05.518Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6706lb # [ 152.150151] niks3-server[1059]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6707lb # [ 152.160794] niks3-server[1059]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6708lb # [ 152.171528] niks3-server[1059]: 2026/09/20 10:53:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6709lb # [ 152.176352] niks3-server[1059]: 2026/09/20 10:53:06 INFO Signed narinfos id=61 count=16710worker1 # [ 152.151330] nix-grpc-daemon[1442]: time=2026-09-20T10:53:05.568Z level=INFO msg="Uploading 1 narinfos"6711lb # [ 152.187472] niks3-server[1059]: 2026/09/20 10:53:06 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6712lb # [ 152.193104] niks3-server[1059]: 2026/09/20 10:53:06 INFO Completed upload id=616713worker1 # [ 152.167571] nix-grpc-daemon[1442]: time=2026-09-20T10:53:05.584Z level=INFO msg="Upload complete. (253ms)"6714lb # [ 152.196471] niks3-server[1059]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6715worker1 # [ 152.169892] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16716lb # [ 152.200061] niks3-server[1059]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6717worker1 # [ 152.174238] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:05Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16718worker1 # [ 152.176733] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306719worker1 # [ 152.182634] nix-daemon[857]: reaped child process 1892, status = succeeded6720worker1 # [ 152.184226] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306721worker1 # [ 152.190588] nix-grpc-daemon[1442]: time=2026-09-20T10:53:05.608Z level=INFO msg="All 2 paths already cached"6722worker1 # [ 152.194657] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:524306723worker1 # [ 152.198471] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6724worker1 # [ 152.344158] nix-daemon[857]: accepted connection from pid 1916, user nix-grpc-daemon (trusted)6725worker1 # [ 152.356478] nix-daemon[857]: reaped child process 1919, status = succeeded6726worker1 # [ 152.518044] nix-daemon[857]: accepted connection from pid 1921, user nix-grpc-daemon (trusted)6727worker1 # [ 152.529676] nix-daemon[857]: reaped child process 1924, status = succeeded6728lb # [ 152.573722] niks3-server[1059]: 2026/09/20 10:53:06 INFO Received uploads request method=POST path=/api/pending_closures6729worker1 # [ 152.553312] nix-grpc-daemon[1442]: time=2026-09-20T10:53:05.970Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6730worker1 # [ 152.554775] nix-grpc-daemon[1442]: time=2026-09-20T10:53:05.972Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6731lb # [ 152.605288] niks3-server[1059]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6732lb # [ 152.609390] niks3-server[1059]: 2026/09/20 10:53:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6733lb # [ 152.614391] niks3-server[1059]: 2026/09/20 10:53:06 INFO Signed narinfos id=62 count=16734worker1 # [ 152.590799] nix-grpc-daemon[1442]: time=2026-09-20T10:53:06.008Z level=INFO msg="Uploading 1 narinfos"6735lb # [ 152.622522] niks3-server[1059]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6736lb # [ 152.634247] niks3-server[1059]: 2026/09/20 10:53:06 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6737worker1 # [ 152.612296] nix-grpc-daemon[1442]: time=2026-09-20T10:53:06.029Z level=INFO msg="Upload complete. (408ms)"6738lb # [ 152.640347] niks3-server[1059]: 2026/09/20 10:53:06 INFO Completed upload id=626739lb # [ 152.643697] niks3-server[1059]: 2026/09/20 10:53:06 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6740worker1 # [ 152.614737] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=0 paths=1 nar_bytes_in=7526741worker1 # [ 152.620207] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:524306742worker1 # [ 152.630065] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6743worker1 # [ 152.633500] nix-daemon[857]: reaped child process 1914, status = succeeded6744worker1 # [ 152.638547] nix-daemon[857]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)6745worker1 # [ 153.451682] nix-daemon[857]: reaped child process 1928, status = succeeded6746worker1 # [ 153.600084] nix-daemon[857]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)6747worker1 # [ 153.610778] nix-daemon[857]: reaped child process 1946, status = succeeded6748lb # [ 153.655657] niks3-server[1059]: 2026/09/20 10:53:07 INFO Received uploads request method=POST path=/api/pending_closures6749worker1 # [ 153.635908] nix-grpc-daemon[1442]: time=2026-09-20T10:53:07.053Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6750worker1 # [ 153.637610] nix-grpc-daemon[1442]: time=2026-09-20T10:53:07.054Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6751lb # [ 153.697425] niks3-server[1059]: 2026/09/20 10:53:07 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6752lb # [ 153.701234] niks3-server[1059]: 2026/09/20 10:53:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6753lb # [ 153.706537] niks3-server[1059]: 2026/09/20 10:53:07 INFO Signed narinfos id=63 count=16754worker1 # [ 153.681851] nix-grpc-daemon[1442]: time=2026-09-20T10:53:07.099Z level=INFO msg="Uploading 1 narinfos"6755lb # [ 153.712838] niks3-server[1059]: 2026/09/20 10:53:07 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6756lb # [ 153.724163] niks3-server[1059]: 2026/09/20 10:53:07 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6757lb # [ 153.730519] niks3-server[1059]: 2026/09/20 10:53:07 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6758lb # [ 153.736430] niks3-server[1059]: 2026/09/20 10:53:07 INFO Completed upload id=636759worker1 # [ 153.711284] nix-grpc-daemon[1442]: time=2026-09-20T10:53:07.128Z level=INFO msg="Upload complete. (266ms)"6760lb # [ 153.739422] niks3-server[1059]: 2026/09/20 10:53:07 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6761worker1 # [ 153.719290] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52430 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16762worker1 # [ 153.731130] nix-daemon[857]: reaped child process 1926, status = succeeded6763client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.42 seconds)6764worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6765worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6766client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'6767client # [ 153.884961] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6768client # Running as unit: gpu.service; invocation ID: e698331c78dc42e5b3aa400391f1ac7b6769client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.07 seconds)6770lb: must succeed: curl -sf localhost:9901/clusters6771lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6772lb: must succeed: curl -sf localhost:9901/clusters6773lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6774worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6775client # [ 154.082860] nix[1355]: don't know how to build these paths:6776client # [ 154.083760] nix[1355]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6777worker1 # [ 154.084409] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524306778worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6779worker2 # [ 154.094418] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599706780client # [ 154.106940] nix[1355]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6781worker1 # [ 154.108929] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:07Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06782lb: must succeed: curl -sf localhost:9901/clusters6783lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6784lb: must succeed: curl -sf localhost:9901/clusters6785lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6786worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6787worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6788client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6789lb # [ 159.068801] envoy[931]: conn peer=[2001:db8:1::1]:43062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27085 tx=10714 ms=171906790client # [ 184.108115] nix[1355]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6791client # [ 184.115895] nix[1355]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6792client # [ 184.149637] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6793client # [ 184.152970] systemd[1]: gpu.service: Failed with result 'exit-code'.6794client # [ 184.157837] systemd[1]: gpu.service: Consumed 118ms CPU time over 30.271s wall clock time, 33.4M memory peak, 6.7K incoming IP traffic, 5.8K outgoing IP traffic.6795client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.55 seconds)6796client: must succeed: journalctl -u gpu -o cat6797client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6798lb: must succeed: curl -sf localhost:9901/clusters6799lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6800lb: must succeed: curl -sf localhost:9901/clusters6801lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6802worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6803worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6804(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.63 seconds)6805subtest: low disk drains a worker and builds go to the other6806worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6807worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.94 seconds)6808worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6809worker1 # [ 186.111506] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:39Z level=info event=unhealthy reason=min_free6810worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6811client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26812worker1 # [ 187.220768] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:524306813client # don't know how to build these paths:6814client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6815worker1 # [ 187.229071] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:524306816worker1 # [ 187.249143] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:40Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06817worker1 # [ 187.253239] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:40Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16818worker2 # [ 187.301970] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599706819worker2 # [ 187.313434] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599706820client # copying 1 paths to 'lb:50051'...6821client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6822worker2 # [ 187.326206] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599706823worker2 # [ 187.338266] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6824worker2 # [ 187.353865] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:41Z level=info event=niks3_push_started pid=18946825worker2 # [ 187.622494] nix-daemon[847]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)6826worker2 # [ 187.633255] nix-daemon[847]: reaped child process 1910, status = succeeded6827worker2 # [ 187.764457] nix-daemon[847]: accepted connection from pid 1912, user nix-grpc-daemon (trusted)6828worker2 # [ 187.775848] nix-daemon[847]: reaped child process 1915, status = succeeded6829lb # [ 187.814737] niks3-server[1059]: 2026/09/20 10:53:41 INFO Received uploads request method=POST path=/api/pending_closures6830worker2 # [ 187.803117] nix-grpc-daemon[1894]: time=2026-09-20T10:53:41.923Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6831worker2 # [ 187.806207] nix-grpc-daemon[1894]: time=2026-09-20T10:53:41.924Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6832lb # [ 187.851304] niks3-server[1059]: 2026/09/20 10:53:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6833worker2 # [ 187.840669] nix-grpc-daemon[1894]: time=2026-09-20T10:53:41.960Z level=INFO msg="Uploading 1 narinfos"6834lb # [ 187.862587] niks3-server[1059]: 2026/09/20 10:53:41 INFO Signed narinfos id=64 count=16835lb # [ 187.868083] niks3-server[1059]: 2026/09/20 10:53:41 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6836lb # [ 187.873916] niks3-server[1059]: 2026/09/20 10:53:41 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6837lb # [ 187.886604] niks3-server[1059]: 2026/09/20 10:53:42 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6838lb # [ 187.891829] niks3-server[1059]: 2026/09/20 10:53:42 INFO Completed upload id=646839worker2 # [ 187.873584] nix-grpc-daemon[1894]: time=2026-09-20T10:53:41.993Z level=INFO msg="Upload complete. (374ms)"6840lb # [ 187.895344] niks3-server[1059]: 2026/09/20 10:53:42 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6841worker2 # [ 187.875975] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59970 duration_s=0 paths=1 nar_bytes_in=5126842worker2 # [ 187.878977] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599706843worker2 # [ 187.885289] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6844worker2 # [ 187.888313] nix-daemon[847]: reaped child process 1892, status = succeeded6845worker2 # [ 187.893076] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6846client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6847client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6848client # farm-a> LOG-farm-a-drain6849lb # [ 189.069813] envoy[931]: conn peer=[2001:db8:1::1]:54268 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300456850worker2 # [ 189.117790] nix-daemon[847]: reaped child process 1919, status = succeeded6851worker2 # [ 189.270806] nix-daemon[847]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)6852worker2 # [ 189.282417] nix-daemon[847]: reaped child process 1937, status = succeeded6853lb # [ 189.319071] niks3-server[1059]: 2026/09/20 10:53:43 INFO Received uploads request method=POST path=/api/pending_closures6854worker2 # [ 189.306812] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.426Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6855worker2 # [ 189.308507] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.428Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6856lb # [ 189.351580] niks3-server[1059]: 2026/09/20 10:53:43 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6857lb # [ 189.366421] niks3-server[1059]: 2026/09/20 10:53:43 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6858lb # [ 189.374733] niks3-server[1059]: 2026/09/20 10:53:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6859worker2 # [ 189.359173] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.478Z level=INFO msg="Uploading 1 narinfos"6860lb # [ 189.379786] niks3-server[1059]: 2026/09/20 10:53:43 INFO Signed narinfos id=65 count=16861lb # [ 189.383119] niks3-server[1059]: 2026/09/20 10:53:43 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6862lb # [ 189.387533] niks3-server[1059]: 2026/09/20 10:53:43 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6863lb # [ 189.390817] niks3-server[1059]: 2026/09/20 10:53:43 INFO Completed upload id=656864worker2 # [ 189.369831] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.489Z level=INFO msg="Upload complete. (268ms)"6865lb # [ 189.392791] niks3-server[1059]: 2026/09/20 10:53:43 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6866worker1 # [ 189.371424] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:42Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06867worker2 # [ 189.374473] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59970 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16868worker1 # [ 189.373347] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:42Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16869worker2 # [ 189.386748] nix-daemon[847]: reaped child process 1917, status = succeeded6870worker2 # [ 189.423869] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599706871worker2 # [ 189.432081] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599706872worker2 # [ 189.440052] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.560Z level=INFO msg="All 1 paths already cached"6873client # copying 1 paths to 'lb:50051'...6874worker2 # [ 189.445325] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599706875client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6876worker2 # [ 189.451882] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6877worker2 # [ 189.574942] nix-daemon[847]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)6878worker2 # [ 189.586777] nix-daemon[847]: reaped child process 1944, status = succeeded6879worker2 # [ 189.710297] nix-daemon[847]: accepted connection from pid 1946, user nix-grpc-daemon (trusted)6880worker2 # [ 189.721747] nix-daemon[847]: reaped child process 1949, status = succeeded6881lb # [ 189.752506] niks3-server[1059]: 2026/09/20 10:53:43 INFO Received uploads request method=POST path=/api/pending_closures6882worker2 # [ 189.739681] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.859Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6883worker2 # [ 189.741041] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.859Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6884lb # [ 189.776283] niks3-server[1059]: 2026/09/20 10:53:43 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6885lb # [ 189.787642] niks3-server[1059]: 2026/09/20 10:53:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6886worker2 # [ 189.771343] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.891Z level=INFO msg="Uploading 1 narinfos"6887lb # [ 189.792600] niks3-server[1059]: 2026/09/20 10:53:43 INFO Signed narinfos id=66 count=16888lb # [ 189.797756] niks3-server[1059]: 2026/09/20 10:53:43 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6889lb # [ 189.809644] niks3-server[1059]: 2026/09/20 10:53:43 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6890worker2 # [ 189.793027] nix-grpc-daemon[1894]: time=2026-09-20T10:53:43.913Z level=INFO msg="Upload complete. (336ms)"6891lb # [ 189.814409] niks3-server[1059]: 2026/09/20 10:53:43 INFO Completed upload id=666892worker2 # [ 189.794844] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59970 duration_s=0 paths=1 nar_bytes_in=6246893lb # [ 189.817661] niks3-server[1059]: 2026/09/20 10:53:43 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6894worker2 # [ 189.800947] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599706895worker2 # [ 189.802664] nix-daemon[847]: reaped child process 1939, status = succeeded6896worker2 # [ 189.805468] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6897worker2 # [ 189.811780] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6898client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6899client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6900client # farm-b> LOG-farm-b-drain6901worker2 # [ 190.636980] nix-daemon[847]: reaped child process 1953, status = succeeded6902worker2 # [ 190.749365] nix-daemon[847]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)6903worker2 # [ 190.759752] nix-daemon[847]: reaped child process 1971, status = succeeded6904lb # [ 190.792934] niks3-server[1059]: 2026/09/20 10:53:44 INFO Received uploads request method=POST path=/api/pending_closures6905worker2 # [ 190.777991] nix-grpc-daemon[1894]: time=2026-09-20T10:53:44.897Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6906worker2 # [ 190.779649] nix-grpc-daemon[1894]: time=2026-09-20T10:53:44.899Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6907lb # [ 190.822587] niks3-server[1059]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6908lb # [ 190.835067] niks3-server[1059]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6909lb # [ 190.846248] niks3-server[1059]: 2026/09/20 10:53:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6910lb # [ 190.851651] niks3-server[1059]: 2026/09/20 10:53:44 INFO Signed narinfos id=67 count=16911worker2 # [ 190.833146] nix-grpc-daemon[1894]: time=2026-09-20T10:53:44.953Z level=INFO msg="Uploading 1 narinfos"6912lb # [ 190.854462] niks3-server[1059]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6913lb # [ 190.865074] niks3-server[1059]: 2026/09/20 10:53:44 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6914lb # [ 190.869558] niks3-server[1059]: 2026/09/20 10:53:44 INFO Completed upload id=676915lb # [ 190.872562] niks3-server[1059]: 2026/09/20 10:53:44 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6916worker2 # [ 190.848567] nix-grpc-daemon[1894]: time=2026-09-20T10:53:44.968Z level=INFO msg="Upload complete. (228ms)"6917worker1 # [ 190.851634] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:44Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06918worker2 # [ 190.855119] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59970 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16919worker1 # [ 190.853571] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:44Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16920worker2 # [ 190.863352] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599706921worker2 # [ 190.873492] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599706922worker2 # [ 190.880390] nix-daemon[847]: reaped child process 1951, status = succeeded6923worker2 # [ 190.882954] nix-grpc-daemon[1894]: time=2026-09-20T10:53:45.003Z level=INFO msg="All 2 paths already cached"6924client # copying 1 paths to 'lb:50051'...6925worker2 # [ 190.889522] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599706926client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6927worker2 # [ 190.895471] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6928worker2 # [ 191.007353] nix-daemon[847]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)6929worker2 # [ 191.020459] nix-daemon[847]: reaped child process 1978, status = succeeded6930worker2 # [ 191.144036] nix-daemon[847]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)6931worker2 # [ 191.155897] nix-daemon[847]: reaped child process 1983, status = succeeded6932lb # [ 191.185106] niks3-server[1059]: 2026/09/20 10:53:45 INFO Received uploads request method=POST path=/api/pending_closures6933worker2 # [ 191.171626] nix-grpc-daemon[1894]: time=2026-09-20T10:53:45.291Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6934worker2 # [ 191.173025] nix-grpc-daemon[1894]: time=2026-09-20T10:53:45.291Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6935lb # [ 191.208647] niks3-server[1059]: 2026/09/20 10:53:45 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6936lb # [ 191.219576] niks3-server[1059]: 2026/09/20 10:53:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6937lb # [ 191.224985] niks3-server[1059]: 2026/09/20 10:53:45 INFO Signed narinfos id=68 count=16938worker2 # [ 191.206837] nix-grpc-daemon[1894]: time=2026-09-20T10:53:45.326Z level=INFO msg="Uploading 1 narinfos"6939lb # [ 191.231661] niks3-server[1059]: 2026/09/20 10:53:45 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6940lb # [ 191.241243] niks3-server[1059]: 2026/09/20 10:53:45 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6941lb # [ 191.247297] niks3-server[1059]: 2026/09/20 10:53:45 INFO Completed upload id=686942worker2 # [ 191.228318] nix-grpc-daemon[1894]: time=2026-09-20T10:53:45.348Z level=INFO msg="Upload complete. (328ms)"6943lb # [ 191.250122] niks3-server[1059]: 2026/09/20 10:53:45 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6944worker2 # [ 191.230556] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59970 duration_s=0 paths=1 nar_bytes_in=7526945worker2 # [ 191.235832] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599706946worker2 # [ 191.243560] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6947worker2 # [ 191.251719] nix-daemon[847]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6948client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6949worker2 # [ 191.253806] nix-daemon[847]: reaped child process 1973, status = succeeded6950client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6951client # farm-top> LOG-farm-top-drain6952worker2 # [ 192.070932] nix-daemon[847]: reaped child process 1987, status = succeeded6953worker2 # [ 192.192276] nix-daemon[847]: accepted connection from pid 2002, user nix-grpc-daemon (trusted)6954worker2 # [ 192.204138] nix-daemon[847]: reaped child process 2005, status = succeeded6955lb # [ 192.237437] niks3-server[1059]: 2026/09/20 10:53:46 INFO Received uploads request method=POST path=/api/pending_closures6956worker2 # [ 192.223264] nix-grpc-daemon[1894]: time=2026-09-20T10:53:46.342Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6957worker2 # [ 192.224640] nix-grpc-daemon[1894]: time=2026-09-20T10:53:46.342Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6958lb # [ 192.275990] niks3-server[1059]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6959lb # [ 192.281769] niks3-server[1059]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6960lb # [ 192.289694] niks3-server[1059]: 2026/09/20 10:53:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6961worker2 # [ 192.272545] nix-grpc-daemon[1894]: time=2026-09-20T10:53:46.392Z level=INFO msg="Uploading 1 narinfos"6962lb # [ 192.293921] niks3-server[1059]: 2026/09/20 10:53:46 INFO Signed narinfos id=69 count=16963lb # [ 192.298781] niks3-server[1059]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6964lb # [ 192.306794] niks3-server[1059]: 2026/09/20 10:53:46 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6965lb # [ 192.309490] niks3-server[1059]: 2026/09/20 10:53:46 INFO Completed upload id=696966worker2 # [ 192.288539] nix-grpc-daemon[1894]: time=2026-09-20T10:53:46.408Z level=INFO msg="Upload complete. (230ms)"6967lb # [ 192.311736] niks3-server[1059]: 2026/09/20 10:53:46 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6968worker2 # [ 192.293520] nix-grpc-daemon[1838]: ts=2026-09-20T10:53:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59970 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16969worker2 # [ 192.304573] nix-daemon[847]: reaped child process 1985, status = succeeded6970client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.29 seconds)6971client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6972client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.17 seconds)6973worker1: must succeed: rm /nix/.rw-store/fill6974worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)6975worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6976worker1 # [ 193.117794] nix-grpc-daemon[1354]: ts=2026-09-20T10:53:46Z level=info event=healthy reason=min_free6977worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)6978(finished: subtest: low disk drains a worker and builds go to the other, in 8.78 seconds)6979(finished: run the VM test script, in 194.62 seconds)6980test script finished in 194.83s6981cleanup6982kill QemuMachine (pid 49)6983client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6984kill QemuMachine (pid 50)6985lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6986kill QemuMachine (pid 52)6987worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6988kill QemuMachine (pid 51)6989worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6990(finished: cleanup, in 0.90 seconds)6991additionally exposed symbols:6992 client, lb, worker1, worker2,6993 vlan1,6994 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh