vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #346
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.5JmUP0IOZ6', fmt=raw size=107374182414worker2 # mke2fs 1.47.4 (6-Mar-2025)15worker2: QEMU running (pid 52)16worker2 # Discarding device blocks: 0/262144 done17worker1 # Disk image does not exist, creating the virtualisation disk image...18worker1: QEMU running (pid 50)19worker1 # Formatting '/build/vm-state-worker1/tmp.bYazoDgZRM', fmt=raw size=107374182420worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes21worker1 # mke2fs 1.47.4 (6-Mar-2025)22worker2 # Filesystem UUID: f8c76065-f7e2-484b-bb47-291c85ba7af923client # Disk image does not exist, creating the virtualisation disk image...24worker2 # Superblock backups stored on blocks:25client: QEMU running (pid 51)26worker1 # Discarding device blocks: 0/262144 done27worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes28client # Formatting '/build/vm-state-client/tmp.W8TqUm7ZH0', fmt=raw size=107374182429worker1 # Filesystem UUID: d859bcac-5995-48ce-890c-cb000cff588d30client # mke2fs 1.47.4 (6-Mar-2025)31worker1 # Superblock backups stored on blocks:32client # Discarding device blocks: 0/262144 done33worker1 # 32768, 98304, 163840, 22937634client # Creating filesystem with 262144 4k blocks and 65536 inodes35worker1 # 36client # Filesystem UUID: be84c316-f6d5-4087-babe-dc9a9e2268e637worker1 # Allocating group tables: 0/8 done38client # Superblock backups stored on blocks:39worker1 # Writing inode tables: 0/8 done40client # 32768, 98304, 163840, 22937641worker1 # Creating journal (8192 blocks): done42client # 43worker1 # Writing superblocks and filesystem accounting information: 0/8 done44client # Allocating group tables: 0/8 done45worker1 # 46client # Writing inode tables: 0/8 done47worker1 # Virtualisation disk image created.48client # Creating journal (8192 blocks): done49worker2 # 32768, 98304, 163840, 22937650client # Writing superblocks and filesystem accounting information: 0/8 done51worker2 # 52worker2 # Allocating group tables: 0/8 done53client # 54worker2 # Writing inode tables: 0/8 done55client # Virtualisation disk image created.56worker2 # Creating journal (8192 blocks): done57worker2 # Writing superblocks and filesystem accounting information: 0/8 done58worker2 # 59worker2 # Virtualisation disk image created.60lb # Disk image does not exist, creating the virtualisation disk image...61lb: QEMU running (pid 49)62lb # Formatting '/build/vm-state-lb/tmp.FDZgXgQI96', fmt=raw size=107374182463lb # mke2fs 1.47.4 (6-Mar-2025)64lb # Discarding device blocks: 0/262144 done65lb # Creating filesystem with 262144 4k blocks and 65536 inodes66lb # Filesystem UUID: 9288f693-c413-4d06-b6cf-ec41a4990e6c67lb # Superblock backups stored on blocks:68lb # 32768, 98304, 163840, 22937669lb # 70lb # Allocating group tables: 0/8 done71lb # Writing inode tables: 0/8 done72lb # Creating journal (8192 blocks): done73(finished: start all VMs, in 0.26 seconds)74lb # Writing superblocks and filesystem accounting information: 0/8 done75lb # 76lb # Virtualisation disk image created.77client: waiting for unit network-addresses-eth1.service78client: waiting for the VM to finish booting79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # 87worker1 # 88worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90worker2 # 91worker2 # 92worker2 # 93worker2 # 94client # 95client # 96worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0097client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0098worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...99worker1 # 100worker1 # 101worker1 # 102worker1 # 103worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00104client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...105client # 106client # 107client # 108client # 109worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...110worker2 # 111worker2 # 112client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114worker1 # 115worker1 # 116client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117client # 118client # 119lb # 120lb # 121lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123lb # 124lb # 125lb # 126lb # 127lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129lb # 130lb # 131worker2 # Booting from ROM...132worker1 # Booting from ROM...133client # Booting from ROM...134lb # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fh6xxw8xvlmj68akgmw7adnq10pgz83n-nixos-system-worker2-test/init regInfo=/nix/store/hp0lp7z9dyfhxzcc2ijpbcgr9h7ymn04-closure-info/registration console=ttyS0,115200n8 console=tty0137worker2 # [ 0.000000] BIOS-provided physical RAM map:138worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026143worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved144worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026145worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved146worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved147worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved148client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test/init regInfo=/nix/store/99fnjaydk0fikl0m8dra9464h914kxma-closure-info/registration console=ttyS0,115200n8 console=tty0149worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved150client # [ 0.000000] BIOS-provided physical RAM map:151worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jng12si38v0hgj5yxfi834jahap2r1pg-nixos-system-worker1-test/init regInfo=/nix/store/sfjjg89c4i2p0lxif17jmh4yccykig41-closure-info/registration console=ttyS0,115200n8 console=tty0152worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved153worker1 # [ 0.000000] BIOS-provided physical RAM map:154client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable155worker2 # [ 0.000000] NX (Execute Disable) protection: active156worker2 # [ 0.000000] APIC: Static calls initialized157worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable158client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved159worker2 # [ 0.000000] SMBIOS 2.8 present.160worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved161client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved162worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014163worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved164client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable165worker2 # [ 0.000000] DMI: Memory slots populated: 1/1166worker2 # [ 0.000000] Hypervisor detected: KVM167worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable168client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved169worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000170worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved171client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved172worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00173worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved174worker2 # [ 0.000000] kvm-clock: using sched offset of 460917566 cycles175client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved176worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved177client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved178worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns179worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved180client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved181worker2 # [ 0.000006] tsc: Detected 2400.012 MHz processor182worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved183worker2 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000184client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved185worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved186client # [ 0.000000] NX (Execute Disable) protection: active187worker2 # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs188worker1 # [ 0.000000] NX (Execute Disable) protection: active189client # [ 0.000000] APIC: Static calls initialized190worker1 # [ 0.000000] APIC: Static calls initialized191client # [ 0.000000] SMBIOS 2.8 present.192worker2 # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT193worker1 # [ 0.000000] SMBIOS 2.8 present.194worker2 # [ 0.002739] found SMP MP-table at [mem 0x000f5450-0x000f545f]195worker2 # [ 0.002755] Using GB pages for direct mapping196client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014197worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014198client # [ 0.000000] DMI: Memory slots populated: 1/1199worker2 # [ 0.002828] RAMDISK: [mem 0x5e367000-0x5ffcffff]200worker1 # [ 0.000000] DMI: Memory slots populated: 1/1201client # [ 0.000000] Hypervisor detected: KVM202worker2 # [ 0.002837] ACPI: Early table checksum verification disabled203worker1 # [ 0.000000] Hypervisor detected: KVM204client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000205worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000206worker2 # [ 0.002841] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )207client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00208worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00209worker2 # [ 0.002846] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)210client # [ 0.000000] kvm-clock: using sched offset of 458497126 cycles211worker1 # [ 0.000000] kvm-clock: using sched offset of 466468987 cycles212worker2 # [ 0.002850] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)213client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns214worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns215client # [ 0.000004] tsc: Detected 2400.012 MHz processor216worker2 # [ 0.002858] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)217worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor218worker2 # [ 0.002860] ACPI: FACS 0x000000005FFE0000 000040219client # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000220worker1 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000221worker2 # [ 0.002862] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)222worker1 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs223client # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs224worker2 # [ 0.002864] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)225worker1 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT226client # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT227worker1 # [ 0.002734] found SMP MP-table at [mem 0x000f5450-0x000f545f]228client # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]229worker2 # [ 0.002865] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)230worker1 # [ 0.002745] Using GB pages for direct mapping231client # [ 0.002744] Using GB pages for direct mapping232worker1 # [ 0.002846] RAMDISK: [mem 0x5e367000-0x5ffcffff]233worker2 # [ 0.002867] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)234client # [ 0.002879] RAMDISK: [mem 0x5e366000-0x5ffcffff]235worker1 # [ 0.002855] ACPI: Early table checksum verification disabled236client # [ 0.002887] ACPI: Early table checksum verification disabled237worker2 # [ 0.002868] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]238worker1 # [ 0.002858] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )239client # [ 0.002890] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )240worker2 # [ 0.002869] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]241worker1 # [ 0.002862] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)242worker2 # [ 0.002870] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]243client # [ 0.002893] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)244worker1 # [ 0.002867] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)245worker2 # [ 0.002870] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]246client # [ 0.002898] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)247worker2 # [ 0.002871] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]248worker1 # [ 0.002874] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)249client # [ 0.002904] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)250worker1 # [ 0.002876] ACPI: FACS 0x000000005FFE0000 000040251worker2 # [ 0.002871] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]252client # [ 0.002906] ACPI: FACS 0x000000005FFE0000 000040253lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026254worker1 # [ 0.002877] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)255worker2 # [ 0.002872] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]256client # [ 0.002908] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)257worker2 # [ 0.003113] No NUMA configuration found258worker1 # [ 0.002879] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)259worker2 # [ 0.003114] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]260client # [ 0.002909] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)261worker2 # [ 0.003117] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]262worker1 # [ 0.002880] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)263worker2 # [ 0.005505] Zone ranges:264client # [ 0.002911] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)265worker2 # [ 0.005506] DMA [mem 0x0000000000001000-0x0000000000ffffff]266worker1 # [ 0.002882] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)267lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0268client # [ 0.002912] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)269lb # [ 0.000000] BIOS-provided physical RAM map:270worker2 # [ 0.005507] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]271worker1 # [ 0.002883] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]272worker2 # [ 0.005509] Normal empty273client # [ 0.002914] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]274lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable275worker2 # [ 0.005510] Device empty276worker1 # [ 0.002884] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]277client # [ 0.002915] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]278worker2 # [ 0.005510] Movable zone start for each node279lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved280worker2 # [ 0.005511] Early memory node ranges281worker1 # [ 0.002885] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]282client # [ 0.002915] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]283lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved284worker2 # [ 0.005511] node 0: [mem 0x0000000000001000-0x000000000009efff]285worker1 # [ 0.002885] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]286client # [ 0.002916] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]287worker2 # [ 0.005513] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]288lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable289worker1 # [ 0.002886] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]290client # [ 0.002916] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]291worker2 # [ 0.005514] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]292lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved293worker1 # [ 0.002886] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]294client # [ 0.002917] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]295worker2 # [ 0.005538] On node 0, zone DMA: 1 pages in unavailable ranges296lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved297worker1 # [ 0.002887] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]298worker2 # [ 0.005816] On node 0, zone DMA: 97 pages in unavailable ranges299client # [ 0.002917] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]300lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved301worker1 # [ 0.003117] No NUMA configuration found302client # [ 0.003138] No NUMA configuration found303worker2 # [ 0.034245] On node 0, zone DMA32: 40 pages in unavailable ranges304worker1 # [ 0.003118] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]305worker2 # [ 0.034710] ACPI: PM-Timer IO Port: 0x608306lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved307client # [ 0.003139] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]308worker1 # [ 0.003120] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]309lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved310worker2 # [ 0.034723] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])311client # [ 0.003142] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]312worker1 # [ 0.005537] Zone ranges:313client # [ 0.005451] Zone ranges:314lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved315worker1 # [ 0.005538] DMA [mem 0x0000000000001000-0x0000000000ffffff]316worker2 # [ 0.034753] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23317client # [ 0.005453] DMA [mem 0x0000000000001000-0x0000000000ffffff]318lb # [ 0.000000] NX (Execute Disable) protection: active319worker2 # [ 0.034756] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)320worker1 # [ 0.005540] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]321worker1 # [ 0.005541] Normal empty322lb # [ 0.000000] APIC: Static calls initialized323client # [ 0.005455] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]324lb # [ 0.000000] SMBIOS 2.8 present.325worker1 # [ 0.005542] Device empty326worker2 # [ 0.034758] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)327client # [ 0.005456] Normal empty328worker1 # [ 0.005542] Movable zone start for each node329client # [ 0.005457] Device empty330worker2 # [ 0.034759] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)331worker1 # [ 0.005543] Early memory node ranges332client # [ 0.005457] Movable zone start for each node333lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014334client # [ 0.005458] Early memory node ranges335worker1 # [ 0.005544] node 0: [mem 0x0000000000001000-0x000000000009efff]336lb # [ 0.000000] DMI: Memory slots populated: 1/1337worker2 # [ 0.034760] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)338lb # [ 0.000000] Hypervisor detected: KVM339client # [ 0.005458] node 0: [mem 0x0000000000001000-0x000000000009efff]340worker1 # [ 0.005545] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]341worker2 # [ 0.034761] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)342lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000343client # [ 0.005459] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]344worker2 # [ 0.034764] ACPI: Using ACPI (MADT) for SMP configuration information345worker1 # [ 0.005546] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]346lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00347worker2 # [ 0.034765] ACPI: HPET id: 0x8086a201 base: 0xfed00000348client # [ 0.005461] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]349worker1 # [ 0.005568] On node 0, zone DMA: 1 pages in unavailable ranges350lb # [ 0.000000] kvm-clock: using sched offset of 472689717 cycles351worker2 # [ 0.034769] TSC deadline timer available352client # [ 0.005481] On node 0, zone DMA: 1 pages in unavailable ranges353worker1 # [ 0.005846] On node 0, zone DMA: 97 pages in unavailable ranges354worker2 # [ 0.034773] CPU topo: Max. logical packages: 1355worker1 # [ 0.034576] On node 0, zone DMA32: 40 pages in unavailable ranges356client # [ 0.005747] On node 0, zone DMA: 97 pages in unavailable ranges357lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns358worker2 # [ 0.034774] CPU topo: Max. logical dies: 1359worker1 # [ 0.035042] ACPI: PM-Timer IO Port: 0x608360worker2 # [ 0.034775] CPU topo: Max. dies per package: 1361lb # [ 0.000005] tsc: Detected 2400.012 MHz processor362client # [ 0.033262] On node 0, zone DMA32: 40 pages in unavailable ranges363worker1 # [ 0.035055] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])364client # [ 0.033720] ACPI: PM-Timer IO Port: 0x608365worker2 # [ 0.034778] CPU topo: Max. threads per core: 1366lb # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000367worker2 # [ 0.034779] CPU topo: Num. cores per package: 1368client # [ 0.033732] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])369worker1 # [ 0.035081] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23370lb # [ 0.000833] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs371worker2 # [ 0.034780] CPU topo: Num. threads per package: 1372client # [ 0.033758] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23373worker1 # [ 0.035084] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)374lb # [ 0.000836] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT375worker2 # [ 0.034781] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs376client # [ 0.033761] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)377worker1 # [ 0.035085] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)378lb # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]379worker2 # [ 0.034804] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()380lb # [ 0.002749] Using GB pages for direct mapping381client # [ 0.033763] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)382worker1 # [ 0.035087] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)383lb # [ 0.002905] RAMDISK: [mem 0x5e367000-0x5ffcffff]384worker2 # [ 0.034838] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]385client # [ 0.033764] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)386worker1 # [ 0.035088] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)387lb # [ 0.002914] ACPI: Early table checksum verification disabled388worker2 # [ 0.034840] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]389client # [ 0.033765] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)390worker1 # [ 0.035089] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)391lb # [ 0.002918] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )392worker2 # [ 0.034841] [mem 0x60000000-0xafffffff] available for PCI devices393client # [ 0.033765] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)394worker1 # [ 0.035091] ACPI: Using ACPI (MADT) for SMP configuration information395worker2 # [ 0.034843] Booting paravirtualized kernel on KVM396lb # [ 0.002922] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)397worker1 # [ 0.035092] ACPI: HPET id: 0x8086a201 base: 0xfed00000398client # [ 0.033768] ACPI: Using ACPI (MADT) for SMP configuration information399worker1 # [ 0.035097] TSC deadline timer available400lb # [ 0.002927] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)401client # [ 0.033769] ACPI: HPET id: 0x8086a201 base: 0xfed00000402worker2 # [ 0.034847] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns403worker1 # [ 0.035101] CPU topo: Max. logical packages: 1404client # [ 0.033774] TSC deadline timer available405worker1 # [ 0.035102] CPU topo: Max. logical dies: 1406lb # [ 0.002934] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)407client # [ 0.033778] CPU topo: Max. logical packages: 1408worker2 # [ 0.039358] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1409worker1 # [ 0.035102] CPU topo: Max. dies per package: 1410lb # [ 0.002936] ACPI: FACS 0x000000005FFE0000 000040411client # [ 0.033779] CPU topo: Max. logical dies: 1412worker1 # [ 0.035106] CPU topo: Max. threads per core: 1413worker2 # [ 0.041654] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152414client # [ 0.033780] CPU topo: Max. dies per package: 1415worker2 # [ 0.041701] kvm-guest: PV spinlocks disabled, single CPU416worker1 # [ 0.035106] CPU topo: Num. cores per package: 1417lb # [ 0.002938] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)418client # [ 0.033783] CPU topo: Max. threads per core: 1419worker1 # [ 0.035107] CPU topo: Num. threads per package: 1420client # [ 0.033784] CPU topo: Num. cores per package: 1421lb # [ 0.002939] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)422worker1 # [ 0.035107] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs423client # [ 0.033784] CPU topo: Num. threads per package: 1424worker1 # [ 0.035128] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()425lb # [ 0.002941] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)426client # [ 0.033785] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs427client # [ 0.033805] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()428worker1 # [ 0.035163] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]429lb # [ 0.002943] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)430worker2 # [ 0.041702] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fh6xxw8xvlmj68akgmw7adnq10pgz83n-nixos-system-worker2-test/init regInfo=/nix/store/hp0lp7z9dyfhxzcc2ijpbcgr9h7ymn04-closure-info/registration console=ttyS0,115200n8 console=tty0431client # [ 0.033838] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]432lb # [ 0.002944] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]433worker1 # [ 0.035165] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]434worker1 # [ 0.035166] [mem 0x60000000-0xafffffff] available for PCI devices435client # [ 0.033840] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]436lb # [ 0.002945] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]437worker1 # [ 0.035168] Booting paravirtualized kernel on KVM438worker2 # [ 0.041803] Unknown kernel command line parameters "regInfo=/nix/store/hp0lp7z9dyfhxzcc2ijpbcgr9h7ymn04-closure-info/registration", will be passed to user space.439client # [ 0.033841] [mem 0x60000000-0xafffffff] available for PCI devices440lb # [ 0.002946] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]441worker2 # [ 0.041816] random: crng init done442client # [ 0.033843] Booting paravirtualized kernel on KVM443lb # [ 0.002946] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]444worker1 # [ 0.035171] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns445worker2 # [ 0.041817] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes446lb # [ 0.002946] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]447worker1 # [ 0.039646] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1448client # [ 0.033847] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns449worker2 # [ 0.044346] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)450worker1 # [ 0.041980] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152451lb # [ 0.002947] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]452client # [ 0.038320] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1453worker2 # [ 0.045472] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)454worker1 # [ 0.042025] kvm-guest: PV spinlocks disabled, single CPU455lb # [ 0.002947] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]456client # [ 0.040832] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152457worker2 # [ 0.045504] Fallback order for Node 0: 0458lb # [ 0.003185] No NUMA configuration found459client # [ 0.040876] kvm-guest: PV spinlocks disabled, single CPU460worker2 # [ 0.045507] Built 1 zonelists, mobility grouping on. Total pages: 393078461lb # [ 0.003185] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]462worker2 # [ 0.045508] Policy zone: DMA32463lb # [ 0.003188] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]464worker2 # [ 0.048140] mem auto-init: stack:all(zero), heap alloc:on, heap free:off465lb # [ 0.005599] Zone ranges:466worker2 # [ 0.051653] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1467lb # [ 0.005600] DMA [mem 0x0000000000001000-0x0000000000ffffff]468worker2 # [ 0.055447] allocated 3145728 bytes of page_ext469worker1 # [ 0.042027] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jng12si38v0hgj5yxfi834jahap2r1pg-nixos-system-worker1-test/init regInfo=/nix/store/sfjjg89c4i2p0lxif17jmh4yccykig41-closure-info/registration console=ttyS0,115200n8 console=tty0470lb # [ 0.005602] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]471worker2 # [ 0.066058] ftrace: allocating 48731 entries in 192 pages472lb # [ 0.005604] Normal empty473lb # [ 0.005605] Device empty474client # [ 0.040878] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test/init regInfo=/nix/store/99fnjaydk0fikl0m8dra9464h914kxma-closure-info/registration console=ttyS0,115200n8 console=tty0475worker2 # [ 0.066060] ftrace: allocated 192 pages with 2 groups476lb # [ 0.005605] Movable zone start for each node477worker2 # [ 0.066985] Dynamic Preempt: lazy478worker1 # [ 0.042128] Unknown kernel command line parameters "regInfo=/nix/store/sfjjg89c4i2p0lxif17jmh4yccykig41-closure-info/registration", will be passed to user space.479lb # [ 0.005606] Early memory node ranges480worker1 # [ 0.042143] random: crng init done481worker2 # [ 0.067137] rcu: Preemptible hierarchical RCU implementation.482lb # [ 0.005606] node 0: [mem 0x0000000000001000-0x000000000009efff]483client # [ 0.040972] Unknown kernel command line parameters "regInfo=/nix/store/99fnjaydk0fikl0m8dra9464h914kxma-closure-info/registration", will be passed to user space.484worker2 # [ 0.067138] rcu: RCU event tracing is enabled.485worker1 # [ 0.042144] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes486client # [ 0.040986] random: crng init done487lb # [ 0.005608] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]488worker2 # [ 0.067139] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.489client # [ 0.040986] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes490worker1 # [ 0.044412] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)491lb # [ 0.005609] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]492worker2 # [ 0.067141] Trampoline variant of Tasks RCU enabled.493worker2 # [ 0.067141] Rude variant of Tasks RCU enabled.494worker1 # [ 0.045828] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)495lb # [ 0.005629] On node 0, zone DMA: 1 pages in unavailable ranges496client # [ 0.043193] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)497worker2 # [ 0.067142] Tracing variant of Tasks RCU enabled.498worker1 # [ 0.045865] Fallback order for Node 0: 0499lb # [ 0.005908] On node 0, zone DMA: 97 pages in unavailable ranges500client # [ 0.044332] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)501worker1 # [ 0.045868] Built 1 zonelists, mobility grouping on. Total pages: 393078502worker2 # [ 0.067142] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.503lb # [ 0.034535] On node 0, zone DMA32: 40 pages in unavailable ranges504client # [ 0.044366] Fallback order for Node 0: 0505worker1 # [ 0.045870] Policy zone: DMA32506lb # [ 0.035010] ACPI: PM-Timer IO Port: 0x608507worker2 # [ 0.067143] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1508client # [ 0.044368] Built 1 zonelists, mobility grouping on. Total pages: 393078509worker1 # [ 0.048560] mem auto-init: stack:all(zero), heap alloc:on, heap free:off510lb # [ 0.035024] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])511client # [ 0.044369] Policy zone: DMA32512worker1 # [ 0.052124] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1513worker2 # [ 0.067205] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.514client # [ 0.046963] mem auto-init: stack:all(zero), heap alloc:on, heap free:off515lb # [ 0.035055] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23516worker1 # [ 0.055647] allocated 3145728 bytes of page_ext517lb # [ 0.035058] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)518client # [ 0.050377] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1519worker1 # [ 0.066171] ftrace: allocating 48731 entries in 192 pages520worker2 # [ 0.067207] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.521client # [ 0.054131] allocated 3145728 bytes of page_ext522lb # [ 0.035060] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)523worker1 # [ 0.066173] ftrace: allocated 192 pages with 2 groups524worker1 # [ 0.067040] Dynamic Preempt: lazy525worker2 # [ 0.067208] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.526client # [ 0.064466] ftrace: allocating 48731 entries in 192 pages527lb # [ 0.035061] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)528client # [ 0.064468] ftrace: allocated 192 pages with 2 groups529worker1 # [ 0.067182] rcu: Preemptible hierarchical RCU implementation.530worker2 # [ 0.071657] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16531client # [ 0.065380] Dynamic Preempt: lazy532lb # [ 0.035062] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)533worker1 # [ 0.067182] rcu: RCU event tracing is enabled.534worker2 # [ 0.071949] rcu: srcu_init: Setting srcu_struct sizes based on contention.535client # [ 0.065563] rcu: Preemptible hierarchical RCU implementation.536lb # [ 0.035063] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)537worker1 # [ 0.067183] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.538client # [ 0.065564] rcu: RCU event tracing is enabled.539worker1 # [ 0.067185] Trampoline variant of Tasks RCU enabled.540worker2 # [ 0.071958] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns541lb # [ 0.035065] ACPI: Using ACPI (MADT) for SMP configuration information542client # [ 0.065565] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.543worker1 # [ 0.067185] Rude variant of Tasks RCU enabled.544lb # [ 0.035066] ACPI: HPET id: 0x8086a201 base: 0xfed00000545worker1 # [ 0.067186] Tracing variant of Tasks RCU enabled.546client # [ 0.065566] Trampoline variant of Tasks RCU enabled.547lb # [ 0.035071] TSC deadline timer available548worker2 # [ 0.072081] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)549worker2 # [ 0.075619] Console: colour VGA+ 80x25550lb # [ 0.035076] CPU topo: Max. logical packages: 1551client # [ 0.065567] Rude variant of Tasks RCU enabled.552worker1 # [ 0.067187] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.553worker2 # [ 0.075622] printk: legacy console [tty0] enabled554client # [ 0.065567] Tracing variant of Tasks RCU enabled.555lb # [ 0.035076] CPU topo: Max. logical dies: 1556worker2 # [ 0.105215] printk: legacy console [ttyS0] enabled557lb # [ 0.035077] CPU topo: Max. dies per package: 1558worker1 # [ 0.067187] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1559client # [ 0.065568] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.560lb # [ 0.035082] CPU topo: Max. threads per core: 1561worker2 # [ 0.201401] ACPI: Core revision 20250807562worker1 # [ 0.067248] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.563lb # [ 0.035083] CPU topo: Num. cores per package: 1564client # [ 0.065569] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1565lb # [ 0.035084] CPU topo: Num. threads per package: 1566worker1 # [ 0.067250] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.567client # [ 0.065600] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.568lb # [ 0.035084] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs569worker2 # [ 0.202305] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns570worker1 # [ 0.067251] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.571client # [ 0.065602] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.572worker2 # [ 0.203865] APIC: Switch to symmetric I/O mode setup573lb # [ 0.035106] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()574worker1 # [ 0.071718] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16575worker2 # [ 0.204788] x2apic enabled576lb # [ 0.035143] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]577client # [ 0.065603] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.578worker1 # [ 0.072315] rcu: srcu_init: Setting srcu_struct sizes based on contention.579lb # [ 0.035144] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]580worker2 # [ 0.205515] APIC: Switched APIC routing to: physical x2apic581client # [ 0.070203] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16582lb # [ 0.035146] [mem 0x60000000-0xafffffff] available for PCI devices583worker1 # [ 0.072322] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns584client # [ 0.070487] rcu: srcu_init: Setting srcu_struct sizes based on contention.585lb # [ 0.035148] Booting paravirtualized kernel on KVM586worker1 # [ 0.072433] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)587worker2 # [ 0.207314] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1588client # [ 0.070493] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns589worker1 # [ 0.075980] Console: colour VGA+ 80x25590lb # [ 0.035152] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns591worker1 # [ 0.075983] printk: legacy console [tty0] enabled592client # [ 0.070600] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)593lb # [ 0.039635] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1594worker2 # [ 0.208270] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns595worker1 # [ 0.105455] printk: legacy console [ttyS0] enabled596client # [ 0.074331] Console: colour VGA+ 80x25597lb # [ 0.041949] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152598worker1 # [ 0.201304] ACPI: Core revision 20250807599client # [ 0.074334] printk: legacy console [tty0] enabled600lb # [ 0.042001] kvm-guest: PV spinlocks disabled, single CPU601worker2 # [ 0.209872] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)602client # [ 0.104410] printk: legacy console [ttyS0] enabled603client # [ 0.202606] ACPI: Core revision 20250807604worker1 # [ 0.202136] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns605worker2 # [ 0.210957] x86/cpu: User Mode Instruction Prevention (UMIP) activated606worker1 # [ 0.203638] APIC: Switch to symmetric I/O mode setup607worker2 # [ 0.212004] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127608client # [ 0.203543] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns609worker1 # [ 0.204610] x2apic enabled610worker2 # [ 0.212869] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0611client # [ 0.205083] APIC: Switch to symmetric I/O mode setup612lb # [ 0.042003] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0613worker1 # [ 0.205376] APIC: Switched APIC routing to: physical x2apic614client # [ 0.206028] x2apic enabled615worker2 # [ 0.213872] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto616lb # [ 0.042102] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.617client # [ 0.206782] APIC: Switched APIC routing to: physical x2apic618worker1 # [ 0.207143] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1619lb # [ 0.042116] random: crng init done620worker2 # [ 0.214869] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl621lb # [ 0.042116] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes622worker1 # [ 0.208100] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns623worker2 # [ 0.215869] Transient Scheduler Attacks: Vulnerable: No microcode624lb # [ 0.044399] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)625client # [ 0.208797] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1626worker2 # [ 0.216869] Spectre V2 : Mitigation: Enhanced / Automatic IBRS627lb # [ 0.045526] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)628worker1 # [ 0.209764] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)629lb # [ 0.045559] Fallback order for Node 0: 0630worker2 # [ 0.217869] Speculative Return Stack Overflow: Mitigation: Safe RET631client # [ 0.209932] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns632worker1 # [ 0.210847] x86/cpu: User Mode Instruction Prevention (UMIP) activated633lb # [ 0.045562] Built 1 zonelists, mobility grouping on. Total pages: 393078634lb # [ 0.045563] Policy zone: DMA32635client # [ 0.211547] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)636worker1 # [ 0.211914] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127637lb # [ 0.048495] mem auto-init: stack:all(zero), heap alloc:on, heap free:off638lb # [ 0.051987] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1639client # [ 0.212632] x86/cpu: User Mode Instruction Prevention (UMIP) activated640worker1 # [ 0.212761] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0641lb # [ 0.055518] allocated 3145728 bytes of page_ext642client # [ 0.214626] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127643lb # [ 0.066039] ftrace: allocating 48731 entries in 192 pages644worker2 # [ 0.218868] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization645lb # [ 0.066041] ftrace: allocated 192 pages with 2 groups646worker1 # [ 0.213765] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto647lb # [ 0.066925] Dynamic Preempt: lazy648worker2 # [ 0.220876] Spectre V2 : Enabling IBPB for BPF649client # [ 0.215526] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0650lb # [ 0.067147] rcu: Preemptible hierarchical RCU implementation.651worker1 # [ 0.215762] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl652lb # [ 0.067148] rcu: RCU event tracing is enabled.653worker2 # [ 0.221869] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier654client # [ 0.216550] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto655worker1 # [ 0.216761] Transient Scheduler Attacks: Vulnerable: No microcode656lb # [ 0.067148] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.657worker2 # [ 0.222869] active return thunk: srso_alias_return_thunk658lb # [ 0.067150] Trampoline variant of Tasks RCU enabled.659worker1 # [ 0.217761] Spectre V2 : Mitigation: Enhanced / Automatic IBRS660lb # [ 0.067150] Rude variant of Tasks RCU enabled.661client # [ 0.217545] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl662worker2 # [ 0.223889] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'663lb # [ 0.067151] Tracing variant of Tasks RCU enabled.664worker1 # [ 0.218761] Speculative Return Stack Overflow: Mitigation: Safe RET665client # [ 0.219545] Transient Scheduler Attacks: Vulnerable: No microcode666worker2 # [ 0.224869] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'667lb # [ 0.067151] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.668client # [ 0.220545] Spectre V2 : Mitigation: Enhanced / Automatic IBRS669lb # [ 0.067152] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1670worker2 # [ 0.225869] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'671client # [ 0.221491] Speculative Return Stack Overflow: Mitigation: Safe RET672lb # [ 0.067213] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.673worker2 # [ 0.226869] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'674worker2 # [ 0.227868] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'675lb # [ 0.067216] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.676worker1 # [ 0.219761] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization677worker2 # [ 0.228869] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'678worker1 # [ 0.221774] Spectre V2 : Enabling IBPB for BPF679lb # [ 0.067217] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.680lb # [ 0.071646] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16681worker2 # [ 0.229868] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'682worker1 # [ 0.222762] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier683client # [ 0.222545] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization684lb # [ 0.071931] rcu: srcu_init: Setting srcu_struct sizes based on contention.685worker2 # [ 0.230869] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256686worker1 # [ 0.223762] active return thunk: srso_alias_return_thunk687client # [ 0.224553] Spectre V2 : Enabling IBPB for BPF688lb # [ 0.071938] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns689worker2 # [ 0.231869] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64690worker1 # [ 0.224782] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'691client # [ 0.225298] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier692lb # [ 0.072062] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)693worker2 # [ 0.232869] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512694worker1 # [ 0.225761] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'695lb # [ 0.075587] Console: colour VGA+ 80x25696client # [ 0.226547] active return thunk: srso_alias_return_thunk697worker2 # [ 0.233868] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024698lb # [ 0.075593] printk: legacy console [tty0] enabled699worker1 # [ 0.226761] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'700lb # [ 0.105102] printk: legacy console [ttyS0] enabled701client # [ 0.227566] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'702worker2 # [ 0.234869] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8703lb # [ 0.200947] ACPI: Core revision 20250807704worker1 # [ 0.227761] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'705client # [ 0.228544] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'706worker1 # [ 0.228761] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'707worker2 # [ 0.235868] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.708client # [ 0.229544] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'709lb # [ 0.201826] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns710worker1 # [ 0.229762] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'711client # [ 0.230545] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'712lb # [ 0.203457] APIC: Switch to symmetric I/O mode setup713lb # [ 0.204525] x2apic enabled714client # [ 0.231546] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'715worker1 # [ 0.230761] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'716worker1 # [ 0.231763] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256717lb # [ 0.205337] APIC: Switched APIC routing to: physical x2apic718client # [ 0.232546] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'719worker1 # [ 0.232760] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64720client # [ 0.233545] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'721worker1 # [ 0.233761] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512722lb # [ 0.207575] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1723client # [ 0.234546] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256724worker1 # [ 0.234763] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024725client # [ 0.236277] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64726lb # [ 0.208599] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns727worker1 # [ 0.235762] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8728client # [ 0.237260] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512729client # [ 0.238254] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024730lb # [ 0.210266] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)731worker1 # [ 0.236761] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.732client # [ 0.239262] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8733lb # [ 0.212352] x86/cpu: User Mode Instruction Prevention (UMIP) activated734lb # [ 0.213381] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127735client # [ 0.239546] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.736lb # [ 0.214263] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0737lb # [ 0.215268] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto738lb # [ 0.216263] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl739lb # [ 0.217263] Transient Scheduler Attacks: Vulnerable: No microcode740lb # [ 0.219044] Spectre V2 : Mitigation: Enhanced / Automatic IBRS741lb # [ 0.220014] Speculative Return Stack Overflow: Mitigation: Safe RET742lb # [ 0.221078] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization743lb # [ 0.222272] Spectre V2 : Enabling IBPB for BPF744lb # [ 0.223824] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier745lb # [ 0.225263] active return thunk: srso_alias_return_thunk746lb # [ 0.226286] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'747lb # [ 0.227262] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748lb # [ 0.228262] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749lb # [ 0.229263] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750lb # [ 0.230262] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751lb # [ 0.231263] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752lb # [ 0.232263] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753lb # [ 0.234263] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754worker2 # [ 0.272632] Freeing SMP alternatives memory: 44K755lb # [ 0.235262] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64756worker2 # [ 0.272873] pid_max: default: 32768 minimum: 301757lb # [ 0.236263] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512758worker2 # [ 0.273963] LSM: initializing lsm=capability,landlock,yama,bpf,ima759lb # [ 0.237262] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024760worker2 # [ 0.274980] landlock: Up and running.761worker2 # [ 0.275869] Yama: becoming mindful.762lb # [ 0.238262] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8763worker2 # [ 0.276913] LSM support for eBPF active764lb # [ 0.239262] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.765worker2 # [ 0.277681] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker2 # [ 0.278897] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker1 # [ 0.272994] Freeing SMP alternatives memory: 44K768worker2 # [ 0.281193] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)769worker1 # [ 0.273764] pid_max: default: 32768 minimum: 301770worker1 # [ 0.274627] LSM: initializing lsm=capability,landlock,yama,bpf,ima771client # [ 0.276136] Freeing SMP alternatives memory: 44K772worker2 # [ 0.282469] Performance Events: Fam17h+ core perfctr, AMD PMU driver.773worker1 # [ 0.275802] landlock: Up and running.774worker2 # [ 0.282878] ... version: 2775client # [ 0.276547] pid_max: default: 32768 minimum: 301776worker1 # [ 0.276401] Yama: becoming mindful.777worker2 # [ 0.283560] ... bit width: 48778client # [ 0.277645] LSM: initializing lsm=capability,landlock,yama,bpf,ima779worker1 # [ 0.276975] LSM support for eBPF active780worker2 # [ 0.283871] ... generic counters: 6781client # [ 0.278648] landlock: Up and running.782worker2 # [ 0.284573] ... generic bitmap: 000000000000003f783client # [ 0.279247] Yama: becoming mindful.784worker1 # [ 0.277891] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)785worker2 # [ 0.284871] ... fixed-purpose counters: 0786client # [ 0.280161] LSM support for eBPF active787worker2 # [ 0.285560] ... fixed-purpose bitmap: 0000000000000000788worker1 # [ 0.278802] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)789client # [ 0.280671] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)790worker2 # [ 0.285871] ... value mask: 0000ffffffffffff791worker2 # [ 0.286739] ... max period: 00007fffffffffff792client # [ 0.281582] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)793worker2 # [ 0.287535] ... global_ctrl mask: 000000000000003f794worker1 # [ 0.281472] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)795worker2 # [ 0.287969] signal: max sigframe size: 3376796worker1 # [ 0.282289] Performance Events: Fam17h+ core perfctr, AMD PMU driver.797worker2 # [ 0.288755] rcu: Hierarchical SRCU implementation.798client # [ 0.284581] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)799worker1 # [ 0.282766] ... version: 2800worker2 # [ 0.289438] rcu: Max phase no-delay instances is 400.801worker1 # [ 0.283463] ... bit width: 48802client # [ 0.286113] Performance Events: Fam17h+ core perfctr, AMD PMU driver.803worker1 # [ 0.283763] ... generic counters: 6804client # [ 0.286553] ... version: 2805worker1 # [ 0.284467] ... generic bitmap: 000000000000003f806client # [ 0.287271] ... bit width: 48807worker1 # [ 0.284812] ... fixed-purpose counters: 0808client # [ 0.287548] ... generic counters: 6809worker1 # [ 0.285510] ... fixed-purpose bitmap: 0000000000000000810client # [ 0.288292] ... generic bitmap: 000000000000003f811worker1 # [ 0.285763] ... value mask: 0000ffffffffffff812client # [ 0.288547] ... fixed-purpose counters: 0813worker2 # [ 0.294504] smp: Bringing up secondary CPUs ...814worker1 # [ 0.286677] ... max period: 00007fffffffffff815client # [ 0.289222] ... fixed-purpose bitmap: 0000000000000000816worker2 # [ 0.294886] smp: Brought up 1 node, 1 CPU817worker1 # [ 0.287448] ... global_ctrl mask: 000000000000003f818client # [ 0.289546] ... value mask: 0000ffffffffffff819worker2 # [ 0.295520] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)820worker1 # [ 0.287886] signal: max sigframe size: 3376821client # [ 0.290420] ... max period: 00007fffffffffff822worker1 # [ 0.288637] rcu: Hierarchical SRCU implementation.823client # [ 0.291287] ... global_ctrl mask: 000000000000003f824worker1 # [ 0.289332] rcu: Max phase no-delay instances is 400.825worker2 # [ 0.296049] Memory: 1454580K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)826client # [ 0.291647] signal: max sigframe size: 3376827worker2 # [ 0.297181] devtmpfs: initialized828client # [ 0.292486] rcu: Hierarchical SRCU implementation.829worker2 # [ 0.297997] x86/mm: Memory block size: 128MB830client # [ 0.293248] rcu: Max phase no-delay instances is 400.831worker2 # [ 0.299858] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)832worker1 # [ 0.294375] smp: Bringing up secondary CPUs ...833worker2 # [ 0.300753] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).834worker1 # [ 0.294776] smp: Brought up 1 node, 1 CPU835worker2 # [ 0.301962] pinctrl core: initialized pinctrl subsystem836worker1 # [ 0.295464] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)837client # [ 0.298115] smp: Bringing up secondary CPUs ...838worker2 # [ 0.302994] PM: RTC time: 12:45:00, date: 2026-09-20839client # [ 0.298561] smp: Brought up 1 node, 1 CPU840client # [ 0.299243] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)841worker1 # [ 0.295988] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)842worker1 # [ 0.297091] devtmpfs: initialized843lb # [ 0.275781] Freeing SMP alternatives memory: 44K844worker1 # [ 0.297929] x86/mm: Memory block size: 128MB845worker2 # [ 0.306633] NET: Registered PF_NETLINK/PF_ROUTE protocol family846client # [ 0.299764] Memory: 1454532K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)847lb # [ 0.276265] pid_max: default: 32768 minimum: 301848client # [ 0.300773] devtmpfs: initialized849lb # [ 0.277362] LSM: initializing lsm=capability,landlock,yama,bpf,ima850worker2 # [ 0.307405] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations851client # [ 0.301552] x86/mm: Memory block size: 128MB852lb # [ 0.278375] landlock: Up and running.853worker1 # [ 0.299738] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)854lb # [ 0.278979] Yama: becoming mindful.855worker2 # [ 0.307897] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations856lb # [ 0.279895] LSM support for eBPF active857worker1 # [ 0.300656] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).858client # [ 0.303418] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)859worker2 # [ 0.309161] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations860worker1 # [ 0.301849] pinctrl core: initialized pinctrl subsystem861lb # [ 0.280392] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)862worker2 # [ 0.309882] audit: initializing netlink subsys (disabled)863client # [ 0.304429] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).864worker1 # [ 0.302942] PM: RTC time: 12:45:00, date: 2026-09-20865lb # [ 0.281301] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)866client # [ 0.305640] pinctrl core: initialized pinctrl subsystem867worker2 # [ 0.311004] thermal_sys: Registered thermal governor 'fair_share'868worker2 # [ 0.311007] thermal_sys: Registered thermal governor 'bang_bang'869client # [ 0.306787] PM: RTC time: 12:45:00, date: 2026-09-20870worker2 # [ 0.311872] thermal_sys: Registered thermal governor 'step_wise'871lb # [ 0.284226] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)872worker2 # [ 0.312796] thermal_sys: Registered thermal governor 'user_space'873worker1 # [ 0.306637] NET: Registered PF_NETLINK/PF_ROUTE protocol family874lb # [ 0.284855] Performance Events: Fam17h+ core perfctr, AMD PMU driver.875worker2 # [ 0.313569] audit: type=2000 audit(1789908300.936:1): state=initialized audit_enabled=0 res=1876lb # [ 0.285267] ... version: 2877worker1 # [ 0.308058] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations878lb # [ 0.285948] ... bit width: 48879worker2 # [ 0.314875] thermal_sys: Registered thermal governor 'power_allocator'880client # [ 0.310726] NET: Registered PF_NETLINK/PF_ROUTE protocol family881worker2 # [ 0.314898] cpuidle: using governor menu882lb # [ 0.286265] ... generic counters: 6883worker1 # [ 0.308789] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations884lb # [ 0.286969] ... generic bitmap: 000000000000003f885client # [ 0.312098] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations886worker2 # [ 0.317076] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5887lb # [ 0.287313] ... fixed-purpose counters: 0888worker1 # [ 0.310065] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations889lb # [ 0.288008] ... fixed-purpose bitmap: 0000000000000000890client # [ 0.312576] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations891worker1 # [ 0.310774] audit: initializing netlink subsys (disabled)892worker2 # [ 0.318132] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]893lb # [ 0.288266] ... value mask: 0000ffffffffffff894worker2 # [ 0.318875] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry895client # [ 0.313843] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations896worker1 # [ 0.311765] audit: type=2000 audit(1789908300.946:1): state=initialized audit_enabled=0 res=1897lb # [ 0.289135] ... max period: 00007fffffffffff898client # [ 0.314559] audit: initializing netlink subsys (disabled)899worker2 # [ 0.319861] PCI: Using configuration type 1 for base access900lb # [ 0.289910] ... global_ctrl mask: 000000000000003f901worker1 # [ 0.312895] thermal_sys: Registered thermal governor 'fair_share'902lb # [ 0.290385] signal: max sigframe size: 3376903worker1 # [ 0.312898] thermal_sys: Registered thermal governor 'bang_bang'904client # [ 0.315655] thermal_sys: Registered thermal governor 'fair_share'905worker2 # [ 0.320717] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.906lb # [ 0.291161] rcu: Hierarchical SRCU implementation.907worker1 # [ 0.313765] thermal_sys: Registered thermal governor 'step_wise'908client # [ 0.315657] thermal_sys: Registered thermal governor 'bang_bang'909lb # [ 0.291818] rcu: Max phase no-delay instances is 400.910client # [ 0.316548] thermal_sys: Registered thermal governor 'step_wise'911worker1 # [ 0.314697] thermal_sys: Registered thermal governor 'user_space'912client # [ 0.317448] thermal_sys: Registered thermal governor 'user_space'913worker1 # [ 0.315497] thermal_sys: Registered thermal governor 'power_allocator'914worker1 # [ 0.315777] cpuidle: using governor menu915client # [ 0.318271] audit: type=2000 audit(1789908300.947:1): state=initialized audit_enabled=0 res=1916client # [ 0.319550] thermal_sys: Registered thermal governor 'power_allocator'917worker1 # [ 0.318020] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5918worker2 # [ 0.326145] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages919client # [ 0.319566] cpuidle: using governor menu920lb # [ 0.296945] smp: Bringing up secondary CPUs ...921worker1 # [ 0.319007] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]922worker2 # [ 0.326873] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page923lb # [ 0.297280] smp: Brought up 1 node, 1 CPU924client # [ 0.321806] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5925worker1 # [ 0.319766] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry926lb # [ 0.297918] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)927worker1 # [ 0.320774] PCI: Using configuration type 1 for base access928client # [ 0.322793] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]929client # [ 0.323552] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry930worker1 # [ 0.321879] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.931lb # [ 0.298434] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)932client # [ 0.324560] PCI: Using configuration type 1 for base access933lb # [ 0.299552] devtmpfs: initialized934worker2 # [ 0.331873] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages935lb # [ 0.300301] x86/mm: Memory block size: 128MB936worker2 # [ 0.332871] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page937client # [ 0.325728] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.938lb # [ 0.301994] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)939lb # [ 0.303150] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).940lb # [ 0.304345] pinctrl core: initialized pinctrl subsystem941worker1 # [ 0.328792] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages942lb # [ 0.305422] PM: RTC time: 12:45:00, date: 2026-09-20943client # [ 0.330817] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages944worker1 # [ 0.329764] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page945client # [ 0.331548] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page946lb # [ 0.309203] NET: Registered PF_NETLINK/PF_ROUTE protocol family947worker2 # [ 0.343291] ACPI: Added _OSI(Module Device)948lb # [ 0.310560] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations949worker1 # [ 0.334765] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages950worker2 # [ 0.343873] ACPI: Added _OSI(Processor Device)951worker1 # [ 0.335764] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page952worker2 # [ 0.344599] ACPI: Added _OSI(Processor Aggregator Device)953client # [ 0.336551] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages954lb # [ 0.311290] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations955client # [ 0.337548] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page956lb # [ 0.312558] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations957lb # [ 0.313276] audit: initializing netlink subsys (disabled)958lb # [ 0.314267] audit: type=2000 audit(1789908300.977:1): state=initialized audit_enabled=0 res=1959lb # [ 0.315426] thermal_sys: Registered thermal governor 'fair_share'960lb # [ 0.315428] thermal_sys: Registered thermal governor 'bang_bang'961lb # [ 0.316266] thermal_sys: Registered thermal governor 'step_wise'962lb # [ 0.317198] thermal_sys: Registered thermal governor 'user_space'963lb # [ 0.317991] thermal_sys: Registered thermal governor 'power_allocator'964lb # [ 0.318279] cpuidle: using governor menu965worker2 # [ 0.353778] ACPI: 1 ACPI AML tables successfully acquired and loaded966worker1 # [ 0.347079] ACPI: Added _OSI(Module Device)967client # [ 0.348033] ACPI: Added _OSI(Module Device)968lb # [ 0.320484] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5969worker1 # [ 0.347765] ACPI: Added _OSI(Processor Device)970client # [ 0.348550] ACPI: Added _OSI(Processor Device)971worker1 # [ 0.348519] ACPI: Added _OSI(Processor Aggregator Device)972client # [ 0.349294] ACPI: Added _OSI(Processor Aggregator Device)973lb # [ 0.321567] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]974worker2 # [ 0.357706] ACPI: Interpreter enabled975lb # [ 0.322269] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry976worker2 # [ 0.358360] ACPI: PM: (supports S0 S3 S4 S5)977lb # [ 0.323277] PCI: Using configuration type 1 for base access978worker2 # [ 0.358872] ACPI: Using IOAPIC for interrupt routing979lb # [ 0.324403] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.980worker1 # [ 0.354453] ACPI: 1 ACPI AML tables successfully acquired and loaded981worker2 # [ 0.363914] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug982worker2 # [ 0.364872] PCI: Using E820 reservations for host bridge windows983worker1 # [ 0.358181] ACPI: Interpreter enabled984client # [ 0.359161] ACPI: 1 ACPI AML tables successfully acquired and loaded985lb # [ 0.331284] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages986worker1 # [ 0.358780] ACPI: PM: (supports S0 S3 S4 S5)987lb # [ 0.332266] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page988worker2 # [ 0.367937] ACPI: Enabled 2 GPEs in block 00 to 3F989worker1 # [ 0.359480] ACPI: Using IOAPIC for interrupt routing990lb # [ 0.333255] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages991client # [ 0.362975] ACPI: Interpreter enabled992client # [ 0.363563] ACPI: PM: (supports S0 S3 S4 S5)993worker1 # [ 0.361833] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug994client # [ 0.364241] ACPI: Using IOAPIC for interrupt routing995worker1 # [ 0.362764] PCI: Using E820 reservations for host bridge windows996client # [ 0.364602] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug997worker2 # [ 0.373594] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])998lb # [ 0.338267] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page999client # [ 0.365547] PCI: Using E820 reservations for host bridge windows1000worker2 # [ 0.373878] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1001worker1 # [ 0.365890] ACPI: Enabled 2 GPEs in block 00 to 3F1002worker2 # [ 0.374948] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1003client # [ 0.368620] ACPI: Enabled 2 GPEs in block 00 to 3F1004worker2 # [ 0.376003] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1005worker2 # [ 0.377243] PCI host bridge to bus 0000:001006worker2 # [ 0.377876] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1007worker2 # [ 0.378872] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1008lb # [ 0.346660] ACPI: Added _OSI(Module Device)1009worker2 # [ 0.379871] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1010lb # [ 0.347267] ACPI: Added _OSI(Processor Device)1011worker1 # [ 0.373361] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1012worker2 # [ 0.380871] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1013lb # [ 0.347970] ACPI: Added _OSI(Processor Aggregator Device)1014worker2 # [ 0.381871] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1015worker1 # [ 0.373770] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1016worker2 # [ 0.382871] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1017worker1 # [ 0.374842] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1018client # [ 0.377279] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1019worker2 # [ 0.383872] pci_bus 0000:00: root bus resource [bus 00-ff]1020worker1 # [ 0.375886] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1021client # [ 0.377553] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1022worker2 # [ 0.384836] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1023worker1 # [ 0.377131] PCI host bridge to bus 0000:001024worker1 # [ 0.377768] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1025client # [ 0.379185] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1026worker2 # [ 0.386321] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1027worker1 # [ 0.378763] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1028client # [ 0.379678] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1029client # [ 0.380906] PCI host bridge to bus 0000:001030worker1 # [ 0.379763] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1031lb # [ 0.355863] ACPI: 1 ACPI AML tables successfully acquired and loaded1032client # [ 0.381553] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1033worker1 # [ 0.380764] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1034client # [ 0.382550] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1035worker1 # [ 0.381763] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1036client # [ 0.383547] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1037worker1 # [ 0.382764] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1038lb # [ 0.359671] ACPI: Interpreter enabled1039worker1 # [ 0.383763] pci_bus 0000:00: root bus resource [bus 00-ff]1040client # [ 0.384548] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1041lb # [ 0.360281] ACPI: PM: (supports S0 S3 S4 S5)1042lb # [ 0.360964] ACPI: Using IOAPIC for interrupt routing1043client # [ 0.385549] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1044worker1 # [ 0.384692] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1045client # [ 0.386549] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1046worker2 # [ 0.388987] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1047worker1 # [ 0.386206] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1048client # [ 0.387548] pci_bus 0000:00: root bus resource [bus 00-ff]1049worker2 # [ 0.389951] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1050lb # [ 0.363312] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1051client # [ 0.388509] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1052worker2 # [ 0.390893] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1053lb # [ 0.364266] PCI: Using E820 reservations for host bridge windows1054client # [ 0.389970] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1055worker2 # [ 0.392237] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1056lb # [ 0.367334] ACPI: Enabled 2 GPEs in block 00 to 3F1057worker2 # [ 0.393592] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1058worker2 # [ 0.395879] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1059worker1 # [ 0.388847] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1060worker2 # [ 0.396758] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1061worker1 # [ 0.389843] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1062worker2 # [ 0.397662] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1063worker1 # [ 0.390786] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1064worker2 # [ 0.398726] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1065worker1 # [ 0.392143] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1066lb # [ 0.375772] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1067client # [ 0.392620] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1068worker2 # [ 0.400436] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1069lb # [ 0.376272] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1070client # [ 0.393560] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1071worker1 # [ 0.393494] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1072client # [ 0.394464] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1073lb # [ 0.377344] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1074lb # [ 0.378388] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1075client # [ 0.395600] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1076worker2 # [ 0.401888] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1077worker1 # [ 0.395792] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1078lb # [ 0.379611] PCI host bridge to bus 0000:001079worker2 # [ 0.402698] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1080worker1 # [ 0.396633] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1081worker2 # [ 0.403563] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1082lb # [ 0.380269] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1083client # [ 0.397281] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1084worker1 # [ 0.397475] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1085lb # [ 0.381265] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1086worker2 # [ 0.404433] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1087lb # [ 0.382265] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1088worker1 # [ 0.397770] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1089client # [ 0.399566] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1090lb # [ 0.383265] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1091worker1 # [ 0.399321] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1092client # [ 0.400447] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1093lb # [ 0.384265] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1094worker2 # [ 0.406879] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1095client # [ 0.401273] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1096lb # [ 0.385265] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1097worker2 # [ 0.407706] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1098client # [ 0.401553] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1099lb # [ 0.386265] pci_bus 0000:00: root bus resource [bus 00-ff]1100worker1 # [ 0.400792] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1101worker2 # [ 0.408591] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1102worker1 # [ 0.401621] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1103lb # [ 0.387233] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1104client # [ 0.403055] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1105worker1 # [ 0.402502] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1106worker2 # [ 0.409408] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1107lb # [ 0.388663] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1108client # [ 0.404555] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1109worker1 # [ 0.403330] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1110client # [ 0.405372] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1111worker2 # [ 0.410889] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1112client # [ 0.406298] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1113worker2 # [ 0.411706] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1114worker2 # [ 0.412583] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1115worker1 # [ 0.405708] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1116client # [ 0.407119] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1117worker1 # [ 0.406396] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1118worker2 # [ 0.413966] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1119worker1 # [ 0.406785] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1120client # [ 0.409529] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1121worker2 # [ 0.415886] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1122worker1 # [ 0.408305] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1123client # [ 0.410237] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1124worker2 # [ 0.416713] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1125lb # [ 0.391385] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1126client # [ 0.410568] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1127worker2 # [ 0.417580] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1128lb # [ 0.392320] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1129worker1 # [ 0.409805] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1130lb # [ 0.393235] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1131worker1 # [ 0.410658] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1132client # [ 0.412088] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1133worker2 # [ 0.418432] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1134lb # [ 0.394310] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1135worker1 # [ 0.411482] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1136client # [ 0.413569] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1137lb # [ 0.395979] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1138worker1 # [ 0.412328] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1139client # [ 0.414407] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1140worker2 # [ 0.420879] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1141worker2 # [ 0.421878] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1142client # [ 0.415281] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1143worker1 # [ 0.413779] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1144worker2 # [ 0.422810] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1145worker1 # [ 0.414645] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1146lb # [ 0.398274] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1147client # [ 0.416486] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1148lb # [ 0.399098] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1149worker1 # [ 0.415490] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1150worker2 # [ 0.424288] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1151lb # [ 0.399973] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1152client # [ 0.418584] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1153worker1 # [ 0.416324] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1154lb # [ 0.400271] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1155worker2 # [ 0.425878] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1156client # [ 0.419426] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1157worker2 # [ 0.426715] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1158client # [ 0.420287] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1159lb # [ 0.401847] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1160worker2 # [ 0.427591] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1161worker2 # [ 0.427878] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1162worker1 # [ 0.418930] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1163client # [ 0.421120] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1164lb # [ 0.403282] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1165worker1 # [ 0.419770] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1166lb # [ 0.404111] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1167worker2 # [ 0.429827] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1168worker1 # [ 0.420751] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1169lb # [ 0.404982] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1170worker2 # [ 0.431698] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1171worker1 # [ 0.422214] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1172client # [ 0.423747] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1173worker2 # [ 0.432595] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1174lb # [ 0.405874] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1175client # [ 0.424553] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1176client # [ 0.425472] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1177worker1 # [ 0.423770] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1178worker2 # [ 0.433414] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1179worker1 # [ 0.424611] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1180client # [ 0.427008] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1181lb # [ 0.408223] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1182worker1 # [ 0.425556] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1183lb # [ 0.408880] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1184worker2 # [ 0.435583] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1185worker1 # [ 0.426571] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1186client # [ 0.428554] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1187lb # [ 0.409287] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1188worker2 # [ 0.435879] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1189client # [ 0.429379] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1190worker2 # [ 0.436795] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1191worker1 # [ 0.428440] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1192lb # [ 0.410822] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1193client # [ 0.430326] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1194client # [ 0.431343] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1195worker2 # [ 0.438427] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1196worker1 # [ 0.429627] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1197lb # [ 0.413196] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1198worker1 # [ 0.430613] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1199client # [ 0.433180] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1200lb # [ 0.413875] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1201worker2 # [ 0.439882] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1202lb # [ 0.414287] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1203worker2 # [ 0.440714] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1204worker1 # [ 0.432130] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1205client # [ 0.434426] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1206worker2 # [ 0.441599] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1207client # [ 0.435288] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1208lb # [ 0.416164] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1209worker1 # [ 0.433805] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1210worker2 # [ 0.442443] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1211client # [ 0.436124] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1212worker1 # [ 0.434653] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1213lb # [ 0.418273] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1214worker2 # [ 0.443466] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1215lb # [ 0.419089] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1216worker1 # [ 0.435513] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1217client # [ 0.437579] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1218lb # [ 0.419967] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1219worker2 # [ 0.444632] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1220client # [ 0.438463] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1221worker1 # [ 0.437212] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1222worker2 # [ 0.446377] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1223lb # [ 0.420841] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1224client # [ 0.439274] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1225worker1 # [ 0.438792] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1226worker1 # [ 0.439697] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1227client # [ 0.440931] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1228worker1 # [ 0.440604] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1229lb # [ 0.423283] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1230client # [ 0.442557] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1231lb # [ 0.424272] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1232worker1 # [ 0.442249] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1233client # [ 0.443506] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1234lb # [ 0.425207] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1235client # [ 0.444272] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1236worker1 # [ 0.443413] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1237lb # [ 0.426711] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1238client # [ 0.445157] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1239worker1 # [ 0.443980] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1240lb # [ 0.428274] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1241worker1 # [ 0.445359] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1242client # [ 0.446278] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1243lb # [ 0.429190] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1244lb # [ 0.429976] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1245client # [ 0.446788] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1246worker2 # [ 0.447063] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1247lb # [ 0.430271] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1248client # [ 0.448087] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1249worker2 # [ 0.449381] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1250lb # [ 0.432052] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1251worker2 # [ 0.450104] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1252lb # [ 0.434082] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1253worker2 # [ 0.451404] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1254lb # [ 0.434991] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1255worker2 # [ 0.452147] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1256lb # [ 0.435800] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1257worker2 # [ 0.453172] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1258worker2 # [ 0.454129] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1259lb # [ 0.437296] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1260lb # [ 0.438137] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1261worker1 # [ 0.445973] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1262worker2 # [ 0.455732] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1263lb # [ 0.438978] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1264worker2 # [ 0.456489] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1265worker1 # [ 0.448338] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1266worker2 # [ 0.457184] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1267lb # [ 0.439850] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1268worker1 # [ 0.449019] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1269client # [ 0.448777] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1270worker2 # [ 0.458494] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1271worker1 # [ 0.450293] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1272client # [ 0.451119] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1273lb # [ 0.441272] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1274lb # [ 0.442089] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1275worker1 # [ 0.451047] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1276client # [ 0.451809] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1277lb # [ 0.442988] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1278worker1 # [ 0.452045] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1279client # [ 0.453120] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1280lb # [ 0.443828] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1281worker2 # [ 0.462579] ACPI: PCI: Interrupt link LNKA configured for IRQ 101282worker1 # [ 0.453032] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1283client # [ 0.454564] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1284worker2 # [ 0.462982] ACPI: PCI: Interrupt link LNKB configured for IRQ 101285lb # [ 0.444855] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1286worker2 # [ 0.463973] ACPI: PCI: Interrupt link LNKC configured for IRQ 111287worker1 # [ 0.454599] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1288client # [ 0.455848] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1289lb # [ 0.445483] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1290worker1 # [ 0.455395] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1291worker2 # [ 0.464971] ACPI: PCI: Interrupt link LNKD configured for IRQ 111292client # [ 0.456819] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1293worker2 # [ 0.465967] ACPI: PCI: Interrupt link LNKE configured for IRQ 101294lb # [ 0.446793] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1295worker1 # [ 0.456085] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1296worker2 # [ 0.466969] ACPI: PCI: Interrupt link LNKF configured for IRQ 101297client # [ 0.458359] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1298worker1 # [ 0.457364] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1299worker2 # [ 0.467971] ACPI: PCI: Interrupt link LNKG configured for IRQ 111300client # [ 0.459181] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1301worker2 # [ 0.468975] ACPI: PCI: Interrupt link LNKH configured for IRQ 111302worker2 # [ 0.469912] ACPI: PCI: Interrupt link GSIA configured for IRQ 161303client # [ 0.459924] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1304worker2 # [ 0.470853] ACPI: PCI: Interrupt link GSIB configured for IRQ 171305client # [ 0.461400] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1306worker1 # [ 0.460747] ACPI: PCI: Interrupt link LNKA configured for IRQ 101307worker2 # [ 0.471603] ACPI: PCI: Interrupt link GSIC configured for IRQ 181308worker2 # [ 0.471887] ACPI: PCI: Interrupt link GSID configured for IRQ 191309worker1 # [ 0.461593] ACPI: PCI: Interrupt link LNKB configured for IRQ 101310worker2 # [ 0.472823] ACPI: PCI: Interrupt link GSIE configured for IRQ 201311worker1 # [ 0.462578] ACPI: PCI: Interrupt link LNKC configured for IRQ 111312worker2 # [ 0.473586] ACPI: PCI: Interrupt link GSIF configured for IRQ 211313worker1 # [ 0.463569] ACPI: PCI: Interrupt link LNKD configured for IRQ 111314worker2 # [ 0.473884] ACPI: PCI: Interrupt link GSIG configured for IRQ 221315client # [ 0.465190] ACPI: PCI: Interrupt link LNKA configured for IRQ 101316worker1 # [ 0.464570] ACPI: PCI: Interrupt link LNKE configured for IRQ 101317worker2 # [ 0.474829] ACPI: PCI: Interrupt link GSIH configured for IRQ 231318client # [ 0.465676] ACPI: PCI: Interrupt link LNKB configured for IRQ 101319worker1 # [ 0.465585] ACPI: PCI: Interrupt link LNKF configured for IRQ 101320client # [ 0.466659] ACPI: PCI: Interrupt link LNKC configured for IRQ 111321worker1 # [ 0.466579] ACPI: PCI: Interrupt link LNKG configured for IRQ 111322lb # [ 0.447460] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1323worker2 # [ 0.476836] iommu: Default domain type: Translated1324client # [ 0.467654] ACPI: PCI: Interrupt link LNKD configured for IRQ 111325worker1 # [ 0.467706] ACPI: PCI: Interrupt link LNKH configured for IRQ 111326lb # [ 0.449773] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1327worker2 # [ 0.477542] iommu: DMA domain TLB invalidation policy: lazy mode1328client # [ 0.468658] ACPI: PCI: Interrupt link LNKE configured for IRQ 101329worker1 # [ 0.468509] ACPI: PCI: Interrupt link GSIA configured for IRQ 161330worker2 # [ 0.478144] ACPI: bus type USB registered1331worker1 # [ 0.468776] ACPI: PCI: Interrupt link GSIB configured for IRQ 171332lb # [ 0.450495] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1333client # [ 0.469668] ACPI: PCI: Interrupt link LNKF configured for IRQ 101334worker2 # [ 0.478864] usbcore: registered new interface driver usbfs1335worker1 # [ 0.469699] ACPI: PCI: Interrupt link GSIC configured for IRQ 181336client # [ 0.470655] ACPI: PCI: Interrupt link LNKG configured for IRQ 111337worker2 # [ 0.479760] usbcore: registered new interface driver hub1338lb # [ 0.451762] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1339worker1 # [ 0.470486] ACPI: PCI: Interrupt link GSID configured for IRQ 191340worker2 # [ 0.480506] usbcore: registered new device driver usb1341client # [ 0.471657] ACPI: PCI: Interrupt link LNKH configured for IRQ 111342worker1 # [ 0.470779] ACPI: PCI: Interrupt link GSIE configured for IRQ 201343lb # [ 0.452551] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1344client # [ 0.472596] ACPI: PCI: Interrupt link GSIA configured for IRQ 161345worker2 # [ 0.481762] NetLabel: Initializing1346worker1 # [ 0.471705] ACPI: PCI: Interrupt link GSIF configured for IRQ 211347client # [ 0.473519] ACPI: PCI: Interrupt link GSIB configured for IRQ 171348worker2 # [ 0.482270] NetLabel: domain hash size = 1281349worker1 # [ 0.472472] ACPI: PCI: Interrupt link GSIG configured for IRQ 221350lb # [ 0.453558] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1351client # [ 0.474287] ACPI: PCI: Interrupt link GSIC configured for IRQ 181352worker2 # [ 0.482871] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1353worker1 # [ 0.472781] ACPI: PCI: Interrupt link GSIH configured for IRQ 231354client # [ 0.474566] ACPI: PCI: Interrupt link GSID configured for IRQ 191355worker2 # [ 0.483773] NetLabel: unlabeled traffic allowed by default1356lb # [ 0.454521] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1357worker2 # [ 0.484521] PCI: Using ACPI for IRQ routing1358client # [ 0.475498] ACPI: PCI: Interrupt link GSIE configured for IRQ 201359worker1 # [ 0.474814] iommu: Default domain type: Translated1360lb # [ 0.456046] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1361client # [ 0.476289] ACPI: PCI: Interrupt link GSIF configured for IRQ 211362worker1 # [ 0.475588] iommu: DMA domain TLB invalidation policy: lazy mode1363lb # [ 0.456272] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1364client # [ 0.476562] ACPI: PCI: Interrupt link GSIG configured for IRQ 221365worker1 # [ 0.476722] ACPI: bus type USB registered1366client # [ 0.477524] ACPI: PCI: Interrupt link GSIH configured for IRQ 231367worker1 # [ 0.477292] usbcore: registered new interface driver usbfs1368lb # [ 0.457481] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1369worker1 # [ 0.477775] usbcore: registered new interface driver hub1370lb # [ 0.458828] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1371client # [ 0.479371] iommu: Default domain type: Translated1372worker1 # [ 0.478609] usbcore: registered new device driver usb1373client # [ 0.480118] iommu: DMA domain TLB invalidation policy: lazy mode1374worker1 # [ 0.480231] NetLabel: Initializing1375client # [ 0.480803] ACPI: bus type USB registered1376worker1 # [ 0.480764] NetLabel: domain hash size = 1281377client # [ 0.481567] usbcore: registered new interface driver usbfs1378worker1 # [ 0.481454] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1379client # [ 0.482456] usbcore: registered new interface driver hub1380lb # [ 0.462350] ACPI: PCI: Interrupt link LNKA configured for IRQ 101381worker1 # [ 0.481804] NetLabel: unlabeled traffic allowed by default1382client # [ 0.483224] usbcore: registered new device driver usb1383worker1 # [ 0.482707] PCI: Using ACPI for IRQ routing1384lb # [ 0.463369] ACPI: PCI: Interrupt link LNKB configured for IRQ 101385lb # [ 0.464364] ACPI: PCI: Interrupt link LNKC configured for IRQ 111386client # [ 0.484429] NetLabel: Initializing1387client # [ 0.484974] NetLabel: domain hash size = 1281388lb # [ 0.465363] ACPI: PCI: Interrupt link LNKD configured for IRQ 111389client # [ 0.485548] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1390lb # [ 0.466362] ACPI: PCI: Interrupt link LNKE configured for IRQ 101391lb # [ 0.467364] ACPI: PCI: Interrupt link LNKF configured for IRQ 101392client # [ 0.486575] NetLabel: unlabeled traffic allowed by default1393client # [ 0.487565] PCI: Using ACPI for IRQ routing1394lb # [ 0.468365] ACPI: PCI: Interrupt link LNKG configured for IRQ 111395lb # [ 0.469365] ACPI: PCI: Interrupt link LNKH configured for IRQ 111396lb # [ 0.470303] ACPI: PCI: Interrupt link GSIA configured for IRQ 161397lb # [ 0.471283] ACPI: PCI: Interrupt link GSIB configured for IRQ 171398lb # [ 0.472240] ACPI: PCI: Interrupt link GSIC configured for IRQ 181399lb # [ 0.472992] ACPI: PCI: Interrupt link GSID configured for IRQ 191400lb # [ 0.473277] ACPI: PCI: Interrupt link GSIE configured for IRQ 201401lb # [ 0.474207] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402lb # [ 0.474972] ACPI: PCI: Interrupt link GSIG configured for IRQ 221403lb # [ 0.475276] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404lb # [ 0.477246] iommu: Default domain type: Translated1405lb # [ 0.477835] iommu: DMA domain TLB invalidation policy: lazy mode1406lb # [ 0.478520] ACPI: bus type USB registered1407lb # [ 0.479253] usbcore: registered new interface driver usbfs1408lb # [ 0.479929] usbcore: registered new interface driver hub1409lb # [ 0.480275] usbcore: registered new device driver usb1410lb # [ 0.482351] NetLabel: Initializing1411lb # [ 0.482934] NetLabel: domain hash size = 1281412lb # [ 0.483265] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413lb # [ 0.484188] NetLabel: unlabeled traffic allowed by default1414lb # [ 0.484933] PCI: Using ACPI for IRQ routing1415worker2 # [ 0.528117] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker2 # [ 0.528867] pci 0000:00:01.0: vgaarb: bridge control possible1417worker2 # [ 0.528867] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker2 # [ 0.528875] vgaarb: loaded1419worker2 # [ 0.529596] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420worker2 # [ 0.529873] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421worker2 # [ 0.533962] clocksource: Switched to clocksource kvm-clock1422worker1 # [ 0.527653] pci 0000:00:01.0: vgaarb: setting as boot VGA device1423worker1 # [ 0.527760] pci 0000:00:01.0: vgaarb: bridge control possible1424worker2 # [ 0.537710] VFS: Disk quotas dquot_6.6.01425worker1 # [ 0.527760] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1426worker1 # [ 0.527769] vgaarb: loaded1427worker2 # [ 0.538412] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1428client # [ 0.531864] pci 0000:00:01.0: vgaarb: setting as boot VGA device1429worker2 # [ 0.539713] pnp: PnP ACPI init1430worker1 # [ 0.528517] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01431client # [ 0.532543] pci 0000:00:01.0: vgaarb: bridge control possible1432worker1 # [ 0.529336] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1433worker2 # [ 0.540518] ACPI: IRQ 4 override to edge(!), high(!)1434client # [ 0.532543] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1435worker2 # [ 0.541447] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1436client # [ 0.532553] vgaarb: loaded1437worker2 # [ 0.542947] pnp: PnP ACPI: found 6 devices1438client # [ 0.533267] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01439client # [ 0.533550] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1440worker1 # [ 0.532850] clocksource: Switched to clocksource kvm-clock1441worker1 # [ 0.536446] VFS: Disk quotas dquot_6.6.01442worker1 # [ 0.537198] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1443worker1 # [ 0.538603] pnp: PnP ACPI init1444client # [ 0.538633] clocksource: Switched to clocksource kvm-clock1445worker2 # [ 0.550527] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1446worker1 # [ 0.539465] ACPI: IRQ 4 override to edge(!), high(!)1447worker2 # [ 0.551993] clocksource: Switched to clocksource acpi_pm1448worker1 # [ 0.540497] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1449worker2 # [ 0.552932] NET: Registered PF_INET protocol family1450worker1 # [ 0.542089] pnp: PnP ACPI: found 6 devices1451client # [ 0.542219] VFS: Disk quotas dquot_6.6.01452worker2 # [ 0.554091] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1453client # [ 0.542938] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1454client # [ 0.544364] pnp: PnP ACPI init1455client # [ 0.545313] ACPI: IRQ 4 override to edge(!), high(!)1456client # [ 0.546395] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1457client # [ 0.547986] pnp: PnP ACPI: found 6 devices1458lb # [ 0.528757] pci 0000:00:01.0: vgaarb: setting as boot VGA device1459lb # [ 0.529261] pci 0000:00:01.0: vgaarb: bridge control possible1460lb # [ 0.529261] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1461worker1 # [ 0.549963] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1462lb # [ 0.529270] vgaarb: loaded1463worker1 # [ 0.551520] clocksource: Switched to clocksource acpi_pm1464lb # [ 0.530009] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01465worker1 # [ 0.552528] NET: Registered PF_INET protocol family1466lb # [ 0.530265] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1467worker1 # [ 0.553757] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1468client # [ 0.555672] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1469lb # [ 0.533357] clocksource: Switched to clocksource kvm-clock1470client # [ 0.557182] clocksource: Switched to clocksource acpi_pm1471client # [ 0.558206] NET: Registered PF_INET protocol family1472client # [ 0.559452] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1473lb # [ 0.536996] VFS: Disk quotas dquot_6.6.01474worker2 # [ 0.571721] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1475lb # [ 0.537708] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1476worker2 # [ 0.573232] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1477lb # [ 0.539039] pnp: PnP ACPI init1478lb # [ 0.539805] ACPI: IRQ 4 override to edge(!), high(!)1479worker2 # [ 0.574485] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1480worker2 # [ 0.575775] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1481lb # [ 0.540796] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1482worker2 # [ 0.576986] TCP: Hash tables configured (established 16384 bind 16384)1483lb # [ 0.542366] pnp: PnP ACPI: found 6 devices1484worker2 # [ 0.578064] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1485worker2 # [ 0.579272] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1486worker2 # [ 0.580349] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1487worker2 # [ 0.581571] NET: Registered PF_UNIX/PF_LOCAL protocol family1488worker2 # [ 0.582542] NET: Registered PF_XDP protocol family1489worker2 # [ 0.583405] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1490worker1 # [ 0.571418] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1491worker2 # [ 0.584389] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1492worker1 # [ 0.573113] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1493worker2 # [ 0.585369] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1494lb # [ 0.549994] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1495lb # [ 0.551423] clocksource: Switched to clocksource acpi_pm1496worker1 # [ 0.574406] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1497worker2 # [ 0.586445] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1498lb # [ 0.552395] NET: Registered PF_INET protocol family1499worker2 # [ 0.587505] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1500worker1 # [ 0.575772] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1501client # [ 0.576436] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1502worker1 # [ 0.577017] TCP: Hash tables configured (established 16384 bind 16384)1503worker2 # [ 0.588606] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1504lb # [ 0.553611] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1505client # [ 0.577929] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1506worker1 # [ 0.578078] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1507worker2 # [ 0.590269] ACPI: \_SB_.GSIA: Enabled at IRQ 161508worker1 # [ 0.579319] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1509client # [ 0.579282] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1510worker2 # [ 0.592472] ACPI: \_SB_.GSIB: Enabled at IRQ 171511worker1 # [ 0.580534] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1512client # [ 0.580685] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1513worker1 # [ 0.581861] NET: Registered PF_UNIX/PF_LOCAL protocol family1514client # [ 0.581979] TCP: Hash tables configured (established 16384 bind 16384)1515worker1 # [ 0.582867] NET: Registered PF_XDP protocol family1516worker2 # [ 0.594751] ACPI: \_SB_.GSIC: Enabled at IRQ 181517client # [ 0.583159] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1518worker1 # [ 0.583755] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1519client # [ 0.584420] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1520worker1 # [ 0.584762] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1521worker2 # [ 0.596786] ACPI: \_SB_.GSID: Enabled at IRQ 191522client # [ 0.585535] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1523worker1 # [ 0.585776] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1524worker2 # [ 0.598552] PCI: CLS 0 bytes, default 641525client # [ 0.586823] NET: Registered PF_UNIX/PF_LOCAL protocol family1526worker1 # [ 0.586886] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1527client # [ 0.587828] NET: Registered PF_XDP protocol family1528worker1 # [ 0.587959] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1529worker2 # [ 0.599446] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1530client # [ 0.588740] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1531worker1 # [ 0.589014] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1532worker2 # [ 0.601127] Trying to unpack rootfs image as initramfs...1533client # [ 0.589784] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1534worker1 # [ 0.590719] ACPI: \_SB_.GSIA: Enabled at IRQ 161535client # [ 0.590875] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1536client # [ 0.592031] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1537worker1 # [ 0.593372] ACPI: \_SB_.GSIB: Enabled at IRQ 171538client # [ 0.593215] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1539lb # [ 0.570687] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1540client # [ 0.594342] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1541worker1 # [ 0.595408] ACPI: \_SB_.GSIC: Enabled at IRQ 181542lb # [ 0.572194] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1543client # [ 0.596100] ACPI: \_SB_.GSIA: Enabled at IRQ 161544lb # [ 0.573480] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1545worker1 # [ 0.597410] ACPI: \_SB_.GSID: Enabled at IRQ 191546client # [ 0.598447] ACPI: \_SB_.GSIB: Enabled at IRQ 171547lb # [ 0.574795] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1548worker1 # [ 0.599221] PCI: CLS 0 bytes, default 641549lb # [ 0.575975] TCP: Hash tables configured (established 16384 bind 16384)1550client # [ 0.600540] ACPI: \_SB_.GSIC: Enabled at IRQ 181551lb # [ 0.577063] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1552worker1 # [ 0.600100] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1553lb # [ 0.578285] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1554worker1 # [ 0.601822] Trying to unpack rootfs image as initramfs...1555client # [ 0.602529] ACPI: \_SB_.GSID: Enabled at IRQ 191556lb # [ 0.579354] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1557lb # [ 0.580533] NET: Registered PF_UNIX/PF_LOCAL protocol family1558client # [ 0.604333] PCI: CLS 0 bytes, default 641559lb # [ 0.581476] NET: Registered PF_XDP protocol family1560lb # [ 0.582318] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1561client # [ 0.605267] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1562lb # [ 0.583311] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1563client # [ 0.607115] Trying to unpack rootfs image as initramfs...1564lb # [ 0.584287] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1565lb # [ 0.585384] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1566lb # [ 0.586446] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1567lb # [ 0.587513] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1568lb # [ 0.589203] ACPI: \_SB_.GSIA: Enabled at IRQ 161569lb # [ 0.591402] ACPI: \_SB_.GSIB: Enabled at IRQ 171570lb # [ 0.593637] ACPI: \_SB_.GSIC: Enabled at IRQ 181571lb # [ 0.595647] ACPI: \_SB_.GSID: Enabled at IRQ 191572lb # [ 0.597311] PCI: CLS 0 bytes, default 641573lb # [ 0.598241] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574lb # [ 0.599994] Trying to unpack rootfs image as initramfs...1575worker2 # [ 0.647372] Initialise system trusted keyrings1576worker2 # [ 0.650985] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker1 # [ 0.645893] Initialise system trusted keyrings1578worker1 # [ 0.649506] workingset: timestamp_bits=40 max_order=19 bucket_order=01579client # [ 0.653645] Initialise system trusted keyrings1580client # [ 0.657119] workingset: timestamp_bits=40 max_order=19 bucket_order=01581worker2 # [ 0.670299] Key type asymmetric registered1582worker2 # [ 0.674965] Asymmetric key parser 'x509' registered1583worker2 # [ 0.675883] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1584lb # [ 0.642734] Initialise system trusted keyrings1585worker2 # [ 0.679071] io scheduler mq-deadline registered1586worker2 # [ 0.679850] io scheduler kyber registered1587lb # [ 0.646514] workingset: timestamp_bits=40 max_order=19 bucket_order=01588worker1 # [ 0.670820] Key type asymmetric registered1589worker2 # [ 0.683955] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1590worker1 # [ 0.673472] Asymmetric key parser 'x509' registered1591worker2 # [ 0.685278] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1592worker1 # [ 0.674352] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1593worker1 # [ 0.677580] io scheduler mq-deadline registered1594worker1 # [ 0.678376] io scheduler kyber registered1595client # [ 0.678431] Key type asymmetric registered1596worker2 # [ 0.691775] Linux agpgart interface v0.1031597worker2 # [ 0.692559] ACPI: bus type drm_connector registered1598client # [ 0.681108] Asymmetric key parser 'x509' registered1599client # [ 0.681955] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1600worker1 # [ 0.682679] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1601worker2 # [ 0.695367] usbcore: registered new interface driver usbserial_generic1602worker1 # [ 0.684009] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1603worker2 # [ 0.696501] usbserial: USB Serial support registered for generic1604client # [ 0.685198] io scheduler mq-deadline registered1605client # [ 0.685972] io scheduler kyber registered1606client # [ 0.688712] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1607worker2 # [ 0.699934] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1608worker1 # [ 0.689411] Linux agpgart interface v0.1031609worker2 # [ 0.699934] Please enable it if your BIOS has the CPPC option.1610worker1 # [ 0.690216] ACPI: bus type drm_connector registered1611client # [ 0.689992] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1612worker2 # [ 0.702150] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1613lb # [ 0.667859] Key type asymmetric registered1614worker2 # [ 0.705114] drop_monitor: Initializing network drop monitor service1615lb # [ 0.670366] Asymmetric key parser 'x509' registered1616worker2 # [ 0.706303] NET: Registered PF_INET6 protocol family1617worker1 # [ 0.694595] usbcore: registered new interface driver usbserial_generic1618lb # [ 0.671213] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1619client # [ 0.695547] Linux agpgart interface v0.1031620worker1 # [ 0.695721] usbserial: USB Serial support registered for generic1621client # [ 0.696311] ACPI: bus type drm_connector registered1622lb # [ 0.674518] io scheduler mq-deadline registered1623lb # [ 0.675267] io scheduler kyber registered1624worker1 # [ 0.698473] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1625worker2 # [ 0.711241] Segment Routing with IPv61626client # [ 0.699512] usbcore: registered new interface driver usbserial_generic1627worker1 # [ 0.698473] Please enable it if your BIOS has the CPPC option.1628worker2 # [ 0.711898] In-situ OAM (IOAM) with IPv61629client # [ 0.700560] usbserial: USB Serial support registered for generic1630worker1 # [ 0.700751] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1631lb # [ 0.677958] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1632lb # [ 0.679209] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1633worker2 # [ 0.715222] IPI shorthand broadcast: enabled1634client # [ 0.703103] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1635client # [ 0.703103] Please enable it if your BIOS has the CPPC option.1636worker1 # [ 0.704648] drop_monitor: Initializing network drop monitor service1637client # [ 0.705352] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1638worker1 # [ 0.705836] NET: Registered PF_INET6 protocol family1639lb # [ 0.684644] Linux agpgart interface v0.1031640worker1 # [ 0.708912] Segment Routing with IPv61641lb # [ 0.685427] ACPI: bus type drm_connector registered1642client # [ 0.709490] drop_monitor: Initializing network drop monitor service1643worker2 # [ 0.721626] sched_clock: Marking stable (587016037, 133910481)->(795361633, -74435115)1644client # [ 0.710710] NET: Registered PF_INET6 protocol family1645worker1 # [ 0.711480] In-situ OAM (IOAM) with IPv61646lb # [ 0.688762] usbcore: registered new interface driver usbserial_generic1647worker1 # [ 0.712585] IPI shorthand broadcast: enabled1648lb # [ 0.689824] usbserial: USB Serial support registered for generic1649worker2 # [ 0.725115] registered taskstats version 11650client # [ 0.714540] Segment Routing with IPv61651worker2 # [ 0.726091] Loading compiled-in X.509 certificates1652lb # [ 0.692370] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1653client # [ 0.717103] In-situ OAM (IOAM) with IPv61654lb # [ 0.692370] Please enable it if your BIOS has the CPPC option.1655client # [ 0.718162] IPI shorthand broadcast: enabled1656lb # [ 0.694529] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1657worker1 # [ 0.720885] sched_clock: Marking stable (587013546, 133438763)->(798005540, -77553231)1658lb # [ 0.698498] drop_monitor: Initializing network drop monitor service1659lb # [ 0.699679] NET: Registered PF_INET6 protocol family1660worker1 # [ 0.725657] registered taskstats version 11661lb # [ 0.702816] Segment Routing with IPv61662worker1 # [ 0.726667] Loading compiled-in X.509 certificates1663client # [ 0.726562] sched_clock: Marking stable (590015941, 136050924)->(803421863, -77354998)1664lb # [ 0.705382] In-situ OAM (IOAM) with IPv61665lb # [ 0.706443] IPI shorthand broadcast: enabled1666client # [ 0.732329] registered taskstats version 11667worker2 # [ 0.744947] Demotion targets for Node 0: null1668client # [ 0.733363] Loading compiled-in X.509 certificates1669worker2 # [ 0.746974] Key type .fscrypt registered1670worker2 # [ 0.747706] Key type fscrypt-provisioning registered1671worker2 # [ 0.748707] ima: No TPM chip found, activating TPM-bypass!1672lb # [ 0.714790] sched_clock: Marking stable (580013684, 134323998)->(785801570, -71463888)1673worker2 # [ 0.752941] ima: Allocated hash algorithm: sha11674worker2 # [ 0.753749] ima: No architecture policies found1675lb # [ 0.719626] registered taskstats version 11676lb # [ 0.720609] Loading compiled-in X.509 certificates1677worker2 # [ 0.755938] PM: Magic number: 10:31:7851678worker1 # [ 0.745472] Demotion targets for Node 0: null1679worker2 # [ 0.757694] RAS: Correctable Errors collector initialized.1680worker1 # [ 0.746386] Key type .fscrypt registered1681worker1 # [ 0.747109] Key type fscrypt-provisioning registered1682worker1 # [ 0.748073] ima: No TPM chip found, activating TPM-bypass!1683client # [ 0.752090] Demotion targets for Node 0: null1684worker1 # [ 0.752478] ima: Allocated hash algorithm: sha11685client # [ 0.752989] Key type .fscrypt registered1686worker1 # [ 0.753277] ima: No architecture policies found1687client # [ 0.753668] Key type fscrypt-provisioning registered1688worker1 # [ 0.755465] PM: Magic number: 10:31:7851689worker2 # [ 0.767658] clk: Disabling unused clocks1690client # [ 0.756194] ima: No TPM chip found, activating TPM-bypass!1691client # [ 0.757143] ima: Allocated hash algorithm: sha11692worker1 # [ 0.757174] RAS: Correctable Errors collector initialized.1693client # [ 0.757922] ima: No architecture policies found1694worker2 # [ 0.769952] PM: genpd: Disabling unused power domains1695client # [ 0.762243] PM: Magic number: 10:31:7851696lb # [ 0.739353] Demotion targets for Node 0: null1697lb # [ 0.740219] Key type .fscrypt registered1698client # [ 0.763858] RAS: Correctable Errors collector initialized.1699lb # [ 0.740887] Key type fscrypt-provisioning registered1700lb # [ 0.741791] ima: No TPM chip found, activating TPM-bypass!1701worker1 # [ 0.767171] clk: Disabling unused clocks1702lb # [ 0.744353] ima: Allocated hash algorithm: sha11703lb # [ 0.745141] ima: No architecture policies found1704worker1 # [ 0.769466] PM: genpd: Disabling unused power domains1705client # [ 0.771738] clk: Disabling unused clocks1706lb # [ 0.749361] PM: Magic number: 10:31:7851707client # [ 0.774084] PM: genpd: Disabling unused power domains1708lb # [ 0.751115] RAS: Correctable Errors collector initialized.1709lb # [ 0.759130] clk: Disabling unused clocks1710lb # [ 0.763358] PM: genpd: Disabling unused power domains1711worker2 # [ 0.938202] Freeing initrd memory: 29092K1712worker2 # [ 0.941404] Freeing unused decrypted memory: 2028K1713worker2 # [ 0.944103] Freeing unused kernel image (initmem) memory: 3644K1714worker2 # [ 0.945155] Write protecting the kernel read-only data: 32768k1715worker2 # [ 0.947030] Freeing unused kernel image (text/rodata gap) memory: 1208K1716worker2 # [ 0.948510] Freeing unused kernel image (rodata/data gap) memory: 752K1717client # [ 0.937951] Freeing initrd memory: 29096K1718client # [ 0.941246] Freeing unused decrypted memory: 2028K1719worker1 # [ 0.942865] Freeing initrd memory: 29092K1720client # [ 0.943904] Freeing unused kernel image (initmem) memory: 3644K1721client # [ 0.945023] Write protecting the kernel read-only data: 32768k1722worker1 # [ 0.946070] Freeing unused decrypted memory: 2028K1723client # [ 0.946941] Freeing unused kernel image (text/rodata gap) memory: 1208K1724client # [ 0.948538] Freeing unused kernel image (rodata/data gap) memory: 752K1725worker1 # [ 0.948777] Freeing unused kernel image (initmem) memory: 3644K1726worker1 # [ 0.949845] Write protecting the kernel read-only data: 32768k1727worker1 # [ 0.951813] Freeing unused kernel image (text/rodata gap) memory: 1208K1728worker1 # [ 0.953372] Freeing unused kernel image (rodata/data gap) memory: 752K1729lb # [ 0.937391] Freeing initrd memory: 29092K1730lb # [ 0.940641] Freeing unused decrypted memory: 2028K1731lb # [ 0.943367] Freeing unused kernel image (initmem) memory: 3644K1732lb # [ 0.944435] Write protecting the kernel read-only data: 32768k1733lb # [ 0.946307] Freeing unused kernel image (text/rodata gap) memory: 1208K1734lb # [ 0.947824] Freeing unused kernel image (rodata/data gap) memory: 752K1735worker2 # [ 0.999735] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736worker2 # [ 1.000769] Run /init as init process1737client # [ 0.999603] x86/mm: Checked W+X mappings: passed, no W+X pages found.1738worker2 # [ 1.011717] systemd[1]: Inserted module 'autofs4'1739client # [ 1.000634] Run /init as init process1740worker1 # [ 1.004503] x86/mm: Checked W+X mappings: passed, no W+X pages found.1741worker1 # [ 1.005549] Run /init as init process1742client # [ 1.010889] systemd[1]: Inserted module 'autofs4'1743worker1 # [ 1.015983] systemd[1]: Inserted module 'autofs4'1744worker2 # [ 1.028047] fuse: init (API version 7.45)1745worker2 # [ 1.033837] ACPI: \_SB_.GSIG: Enabled at IRQ 221746lb # [ 0.998900] x86/mm: Checked W+X mappings: passed, no W+X pages found.1747lb # [ 0.999956] Run /init as init process1748worker2 # [ 1.036257] ACPI: \_SB_.GSIH: Enabled at IRQ 231749client # [ 1.027483] fuse: init (API version 7.45)1750worker2 # [ 1.039275] ACPI: \_SB_.GSIE: Enabled at IRQ 201751worker2 # [ 1.041471] ACPI: \_SB_.GSIF: Enabled at IRQ 211752worker1 # [ 1.032361] fuse: init (API version 7.45)1753client # [ 1.033432] ACPI: \_SB_.GSIG: Enabled at IRQ 221754lb # [ 1.010827] systemd[1]: Inserted module 'autofs4'1755client # [ 1.035862] ACPI: \_SB_.GSIH: Enabled at IRQ 231756worker1 # [ 1.038241] ACPI: \_SB_.GSIG: Enabled at IRQ 221757client # [ 1.039284] ACPI: \_SB_.GSIE: Enabled at IRQ 201758worker1 # [ 1.040893] ACPI: \_SB_.GSIH: Enabled at IRQ 231759client # [ 1.041577] ACPI: \_SB_.GSIF: Enabled at IRQ 211760worker1 # [ 1.044291] ACPI: \_SB_.GSIE: Enabled at IRQ 201761worker1 # [ 1.047076] ACPI: \_SB_.GSIF: Enabled at IRQ 211762lb # [ 1.027290] fuse: init (API version 7.45)1763worker2 # [ 1.065696] systemd[1]: Successfully made /usr/ read-only.1764lb # [ 1.033178] ACPI: \_SB_.GSIG: Enabled at IRQ 221765lb # [ 1.035655] ACPI: \_SB_.GSIH: Enabled at IRQ 231766lb # [ 1.038482] ACPI: \_SB_.GSIE: Enabled at IRQ 201767lb # [ 1.040710] ACPI: \_SB_.GSIF: Enabled at IRQ 211768client # [ 1.071330] systemd[1]: Successfully made /usr/ read-only.1769worker1 # [ 1.076599] systemd[1]: Successfully made /usr/ read-only.1770lb # [ 1.068578] systemd[1]: Successfully made /usr/ read-only.1771worker2 # [ 1.401261] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772worker2 # [ 1.405942] systemd[1]: Detected virtualization kvm.1773worker2 # [ 1.406752] systemd[1]: Detected architecture x86-64.1774worker2 # [ 1.407578] systemd[1]: Running in initrd.1775worker2 # [ 1.408652] systemd[1]: Initializing machine ID from random generator.1776worker2 # [ 1.409852] systemd[1]: Hostname set to <worker2>.1777client # [ 1.407070] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778client # [ 1.412186] systemd[1]: Detected virtualization kvm.1779client # [ 1.413039] systemd[1]: Detected architecture x86-64.1780client # [ 1.413888] systemd[1]: Running in initrd.1781client # [ 1.415138] systemd[1]: Initializing machine ID from random generator.1782client # [ 1.416363] systemd[1]: Hostname set to <client>.1783worker1 # [ 1.413296] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784worker1 # [ 1.425580] systemd[1]: Detected virtualization kvm.1785worker1 # [ 1.427713] systemd[1]: Detected architecture x86-64.1786worker1 # [ 1.429780] systemd[1]: Running in initrd.1787lb # [ 1.404241] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1788lb # [ 1.408998] systemd[1]: Detected virtualization kvm.1789lb # [ 1.409827] systemd[1]: Detected architecture x86-64.1790lb # [ 1.410656] systemd[1]: Running in initrd.1791worker1 # [ 1.432403] systemd[1]: Initializing machine ID from random generator.1792lb # [ 1.411846] systemd[1]: Initializing machine ID from random generator.1793lb # [ 1.412993] systemd[1]: Hostname set to <lb>.1794worker1 # [ 1.435550] systemd[1]: Hostname set to <worker1>.1795worker2 # [ 1.600195] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796client # [ 1.609708] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797worker2 # [ 1.638654] systemd[1]: Queued start job for default target Initrd Default Target.1798worker2 # [ 1.643279] systemd[1]: Created slice Slice /system/modprobe.1799lb # [ 1.608905] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker2 # [ 1.644425] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1801worker2 # [ 1.645776] systemd[1]: Expecting device /dev/disk/by-label/nixos...1802worker2 # [ 1.646827] systemd[1]: Reached target Path Units.1803worker2 # [ 1.647665] systemd[1]: Reached target Slice Units.1804worker2 # [ 1.648550] systemd[1]: Reached target Swaps.1805worker2 # [ 1.649314] systemd[1]: Reached target Timer Units.1806worker2 # [ 1.650236] systemd[1]: Listening on D-Bus System Message Bus Socket.1807worker2 # [ 1.651393] systemd[1]: Listening on Journal Socket (/dev/log).1808worker2 # [ 1.652483] systemd[1]: Listening on Journal Sockets.1809worker2 # [ 1.653432] systemd[1]: Listening on udev Control Socket.1810worker2 # [ 1.654405] systemd[1]: Listening on udev Kernel Socket.1811worker2 # [ 1.655313] systemd[1]: Reached target Socket Units.1812worker2 # [ 1.656989] systemd[1]: Starting Create List of Static Device Nodes...1813worker2 # [ 1.660744] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1814client # [ 1.650462] systemd[1]: Queued start job for default target Initrd Default Target.1815client # [ 1.654496] systemd[1]: Created slice Slice /system/modprobe.1816worker2 # [ 1.667480] systemd[1]: Starting Load Kernel Module configfs...1817client # [ 1.655674] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1818client # [ 1.657091] systemd[1]: Expecting device /dev/disk/by-label/nixos...1819client # [ 1.658170] systemd[1]: Reached target Path Units.1820client # [ 1.659008] systemd[1]: Reached target Slice Units.1821client # [ 1.659866] systemd[1]: Reached target Swaps.1822client # [ 1.660628] systemd[1]: Reached target Timer Units.1823client # [ 1.661588] systemd[1]: Listening on D-Bus System Message Bus Socket.1824worker1 # [ 1.661039] systemd[1]: bpf-restrict-fs: LSM BPF program attached1825client # [ 1.662758] systemd[1]: Listening on Journal Socket (/dev/log).1826client # [ 1.663890] systemd[1]: Listening on Journal Sockets.1827client # [ 1.664854] systemd[1]: Listening on udev Control Socket.1828client # [ 1.665847] systemd[1]: Listening on udev Kernel Socket.1829client # [ 1.666787] systemd[1]: Reached target Socket Units.1830client # [ 1.668606] systemd[1]: Starting Create List of Static Device Nodes...1831worker2 # [ 1.681113] systemd[1]: Starting Journal Service...1832client # [ 1.672136] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1833lb # [ 1.649634] systemd[1]: Queued start job for default target Initrd Default Target.1834lb # [ 1.653761] systemd[1]: Created slice Slice /system/modprobe.1835lb # [ 1.654930] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1836lb # [ 1.656343] systemd[1]: Expecting device /dev/disk/by-label/nixos...1837client # [ 1.680250] systemd[1]: Starting Load Kernel Module configfs...1838lb # [ 1.657411] systemd[1]: Reached target Path Units.1839lb # [ 1.658257] systemd[1]: Reached target Slice Units.1840lb # [ 1.659122] systemd[1]: Reached target Swaps.1841lb # [ 1.659889] systemd[1]: Reached target Timer Units.1842worker2 # [ 1.695028] systemd[1]: Starting Load Kernel Modules...1843lb # [ 1.660904] systemd[1]: Listening on D-Bus System Message Bus Socket.1844lb # [ 1.662060] systemd[1]: Listening on Journal Socket (/dev/log).1845lb # [ 1.663178] systemd[1]: Listening on Journal Sockets.1846lb # [ 1.664169] systemd[1]: Listening on udev Control Socket.1847lb # [ 1.665178] systemd[1]: Listening on udev Kernel Socket.1848lb # [ 1.666107] systemd[1]: Reached target Socket Units.1849lb # [ 1.667926] systemd[1]: Starting Create List of Static Device Nodes...1850client # [ 1.695741] systemd[1]: Starting Journal Service...1851lb # [ 1.672407] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1852worker2 # [ 1.701010] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1853lb # [ 1.680406] systemd[1]: Starting Load Kernel Module configfs...1854worker1 # [ 1.705897] systemd[1]: Queued start job for default target Initrd Default Target.1855worker2 # [ 1.718555] netfs: FS-Cache loaded1856worker1 # [ 1.709834] systemd[1]: Created slice Slice /system/modprobe.1857client # [ 1.710136] systemd[1]: Starting Load Kernel Modules...1858worker1 # [ 1.711059] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1859worker2 # [ 1.723373] 9pnet: Installing 9P2000 support1860worker1 # [ 1.712487] systemd[1]: Expecting device /dev/disk/by-label/nixos...1861worker1 # [ 1.713591] systemd[1]: Reached target Path Units.1862worker1 # [ 1.714480] systemd[1]: Reached target Slice Units.1863worker1 # [ 1.715343] systemd[1]: Reached target Swaps.1864worker1 # [ 1.716155] systemd[1]: Reached target Timer Units.1865lb # [ 1.693593] systemd[1]: Starting Journal Service...1866worker1 # [ 1.717103] systemd[1]: Listening on D-Bus System Message Bus Socket.1867worker1 # [ 1.718379] systemd[1]: Listening on Journal Socket (/dev/log).1868worker1 # [ 1.719551] systemd[1]: Listening on Journal Sockets.1869worker1 # [ 1.720542] systemd[1]: Listening on udev Control Socket.1870worker1 # [ 1.721568] systemd[1]: Listening on udev Kernel Socket.1871worker1 # [ 1.722507] systemd[1]: Reached target Socket Units.1872worker2 # [ 1.735038] systemd[1]: Starting Coldplug All udev Devices...1873worker1 # [ 1.724329] systemd[1]: Starting Create List of Static Device Nodes...1874client # [ 1.715139] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1875worker1 # [ 1.728329] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1876lb # [ 1.705757] systemd[1]: Starting Load Kernel Modules...1877client # [ 1.731922] netfs: FS-Cache loaded1878worker1 # [ 1.735196] systemd[1]: Starting Load Kernel Module configfs...1879client # [ 1.736829] 9pnet: Installing 9P2000 support1880worker2 # [ 1.750373] systemd[1]: Finished Create List of Static Device Nodes.1881worker2 # [ 1.757671] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1882lb # [ 1.712434] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1883client # [ 1.749155] systemd[1]: Starting Coldplug All udev Devices...1884worker1 # [ 1.752173] systemd[1]: Starting Journal Service...1885lb # [ 1.731009] netfs: FS-Cache loaded1886worker2 # [ 1.767370] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1887lb # [ 1.735910] 9pnet: Installing 9P2000 support1888worker2 # [ 1.771577] systemd-journald[67]: Collecting audit messages is disabled.1889worker2 # [ 1.773229] systemd[1]: modprobe@configfs.service: Deactivated successfully.1890client # [ 1.765151] systemd[1]: Finished Create List of Static Device Nodes.1891worker1 # [ 1.766529] systemd[1]: Starting Load Kernel Modules...1892worker2 # [ 1.781311] systemd[1]: Finished Load Kernel Module configfs.1893client # [ 1.769913] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1894lb # [ 1.747421] systemd[1]: Starting Coldplug All udev Devices...1895worker2 # [ 1.785476] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1896client # [ 1.778532] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1897worker2 # [ 1.790282] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1898worker1 # [ 1.771536] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1899client # [ 1.784544] systemd[1]: modprobe@configfs.service: Deactivated successfully.1900lb # [ 1.762440] systemd[1]: Finished Create List of Static Device Nodes.1901client # [ 1.785905] systemd-journald[67]: Collecting audit messages is disabled.1902worker2 # [ 1.798163] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1903worker1 # [ 1.788035] netfs: FS-Cache loaded1904worker2 # [ 1.799514] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1905lb # [ 1.768012] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1906client # [ 1.792340] systemd[1]: Finished Load Kernel Module configfs.1907worker1 # [ 1.792883] 9pnet: Installing 9P2000 support1908client # [ 1.797578] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1909lb # [ 1.775783] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1910worker1 # [ 1.802292] systemd[1]: Starting Coldplug All udev Devices...1911client # [ 1.802376] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1912lb # [ 1.780823] systemd[1]: modprobe@configfs.service: Deactivated successfully.1913lb # [ 1.785756] systemd-journald[67]: Collecting audit messages is disabled.1914client # [ 1.809132] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1915lb # [ 1.788768] systemd[1]: Finished Load Kernel Module configfs.1916client # [ 1.812087] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1917lb # [ 1.794952] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1918worker2 # [ 1.832592] systemd[1]: Finished Load Kernel Modules.1919worker1 # [ 1.821730] systemd[1]: Finished Create List of Static Device Nodes.1920lb # [ 1.798779] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1921worker1 # [ 1.827116] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1922worker2 # [ 1.840153] systemd[1]: Starting Apply Kernel Variables...1923lb # [ 1.807593] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1924lb # [ 1.810531] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1925worker1 # [ 1.834878] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1926worker1 # [ 1.836823] systemd-journald[67]: Collecting audit messages is disabled.1927worker2 # [ 1.852054] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1928worker1 # [ 1.840897] systemd[1]: modprobe@configfs.service: Deactivated successfully.1929client # [ 1.844613] systemd[1]: Finished Load Kernel Modules.1930worker1 # [ 1.847714] systemd[1]: Finished Load Kernel Module configfs.1931client # [ 1.851269] systemd[1]: Starting Apply Kernel Variables...1932worker1 # [ 1.851076] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1933worker2 # [ 1.863665] systemd[1]: Starting Create Static Device Nodes in /dev...1934worker1 # [ 1.854974] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1935worker1 # [ 1.860617] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1936client # [ 1.861227] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1937worker1 # [ 1.865539] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1938lb # [ 1.843440] systemd[1]: Finished Load Kernel Modules.1939worker2 # [ 1.883267] systemd[1]: Finished Apply Kernel Variables.1940client # [ 1.872293] systemd[1]: Starting Create Static Device Nodes in /dev...1941lb # [ 1.852576] systemd[1]: Starting Apply Kernel Variables...1942worker2 # [ 1.759472] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.894528] systemd[1]: Started Journal Service.1943worker2 # 1944worker2 # [ 1.762321] systemd-modules-load[68]: Inserted module 'virtio_balloon'1945worker2 # [ 1.764305] systemd-modules-load[68]: Inserted module 'virtio_gpu'1946lb # [ 1.866754] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1947worker1 # [ 1.891566] systemd[1]: Finished Load Kernel Modules.1948client # [ 1.893216] systemd[1]: Finished Apply Kernel Variables.1949worker2 # [ 1.773148] systemd[1]: Finished Create Static Device Nodes in /dev.1950worker2 # [ 1.776866] systemd[1]: Reached target Preparation for Local File Systems.1951worker1 # [ 1.899675] systemd[1]: Starting Apply Kernel Variables...1952lb # [ 1.877573] systemd[1]: Starting Create Static Device Nodes in /dev...1953worker2 # [ 1.779860] systemd[1]: Reached target Local File Systems.1954client # [ 1.902800] systemd[1]: Started Journal Service.1955worker2 # [ 1.783077] systemd[1]: Starting Create System Files and Directories...1956client # [ 1.769244] systemd-modules-load[68]: Inserted module 'dm_mod'1957client # [ 1.772275] systemd-modules-load[68]: Inserted module 'virtio_balloon'1958worker2 # [ 1.788146] systemd[1]: Starting Rule-based Manager for Device Events and Files...1959client # [ 1.775186] systemd-modules-load[68]: Inserted module 'virtio_gpu'1960client # [ 1.777272] systemd[1]: Finished Create Static Device Nodes in /dev.1961client # [ 1.778428] systemd[1]: Reached target Preparation for Local File Systems.1962worker1 # [ 1.914536] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1963client # [ 1.780370] systemd[1]: Reached target Local File Systems.1964client # [ 1.782270] systemd[1]: Starting Create System Files and Directories...1965lb # [ 1.896464] systemd[1]: Finished Apply Kernel Variables.1966worker1 # [ 1.923679] systemd[1]: Starting Create Static Device Nodes in /dev...1967client # [ 1.789923] systemd[1]: Starting Rule-based Manager for Device Events and Files...1968lb # [ 1.905677] systemd[1]: Started Journal Service.1969lb # [ 1.769739] systemd-modules-load[68]: Inserted module 'dm_mod'1970lb # [ 1.774976] systemd-modules-load[68]: Inserted module 'virtio_balloon'1971lb # [ 1.776054] systemd-modules-load[68]: Inserted module 'virtio_gpu'1972worker1 # [ 1.937065] systemd[1]: Finished Apply Kernel Variables.1973worker2 # [ 1.818428] systemd[1]: Finished Create System Files and Directories.1974lb # [ 1.787132] systemd[1]: Finished Create Static Device Nodes in /dev.1975lb # [ 1.790076] systemd[1]: Reached target Preparation for Local File Systems.1976client # [ 1.812154] systemd[1]: Finished Create System Files and Directories.1977lb # [ 1.791073] systemd[1]: Reached target Local File Systems.1978lb # [ 1.794073] systemd[1]: Starting Create System Files and Directories...1979lb # [ 1.799071] systemd[1]: Starting Rule-based Manager for Device Events and Files...1980worker1 # [ 1.822585] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.957119] systemd[1]: Started Journal Service.1981worker1 # 1982worker1 # [ 1.825953] systemd-modules-load[68]: Inserted module 'virtio_balloon'1983worker1 # [ 1.827291] systemd-modules-load[68]: Inserted module 'virtio_gpu'1984worker2 # [ 1.840440] systemd-udevd[83]: Using default interface naming scheme 'v261'.1985worker1 # [ 1.831934] systemd[1]: Finished Create Static Device Nodes in /dev.1986worker1 # [ 1.839133] systemd[1]: Reached target Preparation for Local File Systems.1987worker1 # [ 1.840145] systemd[1]: Reached target Local File Systems.1988worker1 # [ 1.843062] systemd[1]: Starting Create System Files and Directories...1989client # [ 1.842413] systemd-udevd[77]: Using default interface naming scheme 'v261'.1990worker1 # [ 1.847978] systemd[1]: Starting Rule-based Manager for Device Events and Files...1991worker2 # [ 1.865362] systemd[1]: Started Rule-based Manager for Device Events and Files.1992lb # [ 1.830729] systemd[1]: Finished Create System Files and Directories.1993client # [ 1.866875] systemd[1]: Started Rule-based Manager for Device Events and Files.1994worker1 # [ 1.876472] systemd[1]: Finished Create System Files and Directories.1995lb # [ 1.854301] systemd-udevd[83]: Using default interface naming scheme 'v261'.1996worker1 # [ 1.898616] systemd-udevd[83]: Using default interface naming scheme 'v261'.1997lb # [ 1.879170] systemd[1]: Started Rule-based Manager for Device Events and Files.1998worker2 # [ 1.915084] systemd[1]: Finished Coldplug All udev Devices.1999worker2 # [ 1.917815] systemd[1]: Reached target System Initialization.2000worker2 # [ 1.918645] systemd[1]: Reached target Basic System.
loading 1,933 hidden lines…
3934lb # [ 13.894496] postgresql-pre-start[802]: initdb: warning: enabling "trust" authentication for local connections3935lb # [ 13.895743] postgresql-pre-start[802]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3936lb # [ 13.897630] postgresql-pre-start[802]: Success. You can now start the database server using:3937lb # [ 13.898671] postgresql-pre-start[802]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3938lb # [ 14.007471] postgres[867]: [867] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3939lb # [ 14.012027] postgres[867]: [867] LOG: listening on IPv6 address "::1", port 54323940lb # [ 14.013139] postgres[867]: [867] LOG: listening on IPv4 address "127.0.0.1", port 54323941lb # [ 14.015482] postgres[867]: [867] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3942lb # [ 14.028336] postgres[876]: [876] LOG: database system was shut down at 2026-09-20 12:45:14 GMT3943lb # [ 14.033711] postgres[867]: [867] LOG: database system is ready to accept connections3944lb # [ 14.039337] systemd[1]: Started PostgreSQL Server.3945lb # [ 14.045098] systemd[1]: Starting PostgreSQL Setup Scripts...3946lb # [ 14.245388] postgresql-setup-start[887]: CREATE DATABASE3947lb # [ 14.288470] postgresql-setup-start[893]: CREATE ROLE3948lb # [ 14.306462] postgresql-setup-start[895]: ALTER DATABASE3949lb # [ 14.311251] systemd[1]: Finished PostgreSQL Setup Scripts.3950lb # [ 14.313335] systemd[1]: Reached target PostgreSQL.3951client # [ 14.811828] dhcpcd[565]: eth0: leased 10.0.2.15 for 86400 seconds3952client # [ 14.815570] dhcpcd[565]: eth0: adding route to 10.0.2.0/243953client # [ 14.817744] dhcpcd[565]: eth0: adding default route via 10.0.2.23954client # [ 14.927401] systemd[1]: Started DHCP Client.3955client # [ 14.929554] systemd[1]: Reached target Multi-User System.3956client # [ 14.930601] systemd[1]: Startup finished in 867ms (kernel) + 3.846s (initrd) + 10.216s (userspace) = 14.930s.3957worker2 # [ 15.075920] dhcpcd[691]: eth0: leased 10.0.2.15 for 86400 seconds3958worker2 # [ 15.077101] dhcpcd[691]: eth0: adding route to 10.0.2.0/243959worker2 # [ 15.078285] dhcpcd[691]: eth0: adding default route via 10.0.2.23960worker2 # [ 15.160628] systemd[1]: Started DHCP Client.3961worker2 # [ 15.162817] systemd[1]: Reached target Multi-User System.3962worker2 # [ 15.163907] systemd[1]: Startup finished in 870ms (kernel) + 4.023s (initrd) + 10.268s (userspace) = 15.162s.3963lb # [ 17.253867] dhcpcd[785]: eth0: leased 10.0.2.15 for 86400 seconds3964lb # [ 17.256908] dhcpcd[785]: eth0: adding route to 10.0.2.0/243965lb # [ 17.259412] dhcpcd[785]: eth0: adding default route via 10.0.2.23966lb # [ 17.360207] systemd[1]: Started DHCP Client.3967lb # [ 17.362421] systemd[1]: Reached target Network is Online.3968lb # [ 17.366661] systemd[1]: Started Envoy reverse proxy.3969lb # [ 17.370871] systemd[1]: Started mock-oidc.service.3970lb # [ 17.482435] mock-oidc-server[986]: Mock OIDC Server running3971lb # [ 17.484269] mock-oidc-server[986]: OIDC Address: 192.168.1.2:80803972lb # [ 17.485578] mock-oidc-server[986]: Issue Address: 0.0.0.0:80813973lb # [ 17.486891] mock-oidc-server[986]: Issuer: http://192.168.1.2:8080/oidc3974lb # [ 17.488315] mock-oidc-server[986]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3975lb # [ 17.489903] mock-oidc-server[986]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3976lb # [ 17.491698] mock-oidc-server[986]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3977worker1 # [ 17.812229] dhcpcd[691]: eth0: leased 10.0.2.15 for 86400 seconds3978worker1 # [ 17.813279] dhcpcd[691]: eth0: adding route to 10.0.2.0/243979worker1 # [ 17.814145] dhcpcd[691]: eth0: adding default route via 10.0.2.23980worker1 # [ 17.912501] systemd[1]: Started DHCP Client.3981worker1 # [ 17.914436] systemd[1]: Reached target Multi-User System.3982worker1 # [ 17.915729] systemd[1]: Startup finished in 875ms (kernel) + 4.923s (initrd) + 12.116s (userspace) = 17.915s.3983lb # [ 17.902033] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3984lb # [ 17.903815] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:469] statically linked extensions:3985lb # [ 17.905427] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3986lb # [ 17.907983] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3987lb # [ 17.910547] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3988lb # [ 17.913142] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3989lb # [ 17.916079] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3990lb # [ 17.917581] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3991lb # [ 17.919141] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23992lb # [ 17.920761] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3993lb # [ 17.922353] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3994lb # [ 17.924626] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3995lb # [ 17.926890] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3996lb # [ 17.928701] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3997lb # [ 17.930433] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3998lb # [ 17.947955] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3999lb # [ 17.949584] envoy[985]: [2026-09-20 12:45:19.048][985][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4000lb # [ 17.951156] envoy[985]: [2026-09-20 12:45:19.049][985][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4001lb # [ 17.952866] envoy[985]: [2026-09-20 12:45:19.049][985][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4002lb # [ 17.958525] envoy[985]: [2026-09-20 12:45:19.049][985][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4003lb # [ 17.960125] envoy[985]: [2026-09-20 12:45:19.049][985][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4004lb # [ 17.962943] envoy[985]: [2026-09-20 12:45:19.049][985][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4005lb # [ 17.964676] envoy[985]: [2026-09-20 12:45:19.049][985][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4006lb # [ 17.966801] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4007lb # [ 17.968901] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4008lb # [ 17.970617] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4009lb # [ 17.975130] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4010lb # [ 17.976841] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4011lb # [ 17.978530] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4012lb # [ 17.980862] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4013lb # [ 17.982443] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4014lb # [ 17.984065] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4015lb # [ 17.987542] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4016lb # [ 17.989250] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4017lb # [ 17.991381] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4018lb # [ 17.993253] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4019lb # [ 17.995273] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4020lb # [ 17.997241] envoy[985]: [2026-09-20 12:45:19.052][985][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4021lb # [ 17.998817] envoy[985]: [2026-09-20 12:45:19.054][985][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4022lb # [ 18.001432] envoy[985]: [2026-09-20 12:45:19.054][985][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4023lb # [ 18.005227] envoy[985]: [2026-09-20 12:45:19.054][985][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4024lb # [ 18.008393] envoy[985]: [2026-09-20 12:45:19.054][985][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4025lb # [ 18.010675] envoy[985]: [2026-09-20 12:45:19.054][985][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4026lb # [ 18.012770] envoy[985]: [2026-09-20 12:45:19.054][985][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4027lb # [ 18.016282] envoy[985]: [2026-09-20 12:45:19.055][985][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4028lb # [ 18.018085] envoy[985]: [2026-09-20 12:45:19.055][985][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4029lb # [ 18.020124] envoy[985]: [2026-09-20 12:45:19.055][985][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4030lb # [ 18.026811] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4031lb # [ 18.029392] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4032lb # [ 18.036951] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4033lb # [ 18.039683] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4034lb # [ 18.044449] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4035lb # [ 18.046799] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4036lb # [ 18.049676] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4037lb # [ 18.054655] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4038lb # [ 18.056634] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4039lb # [ 18.058340] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4040lb # [ 18.060144] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4041lb # [ 18.062303] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4042lb # [ 18.065498] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4043lb # [ 18.068134] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4044lb # [ 18.069946] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4045lb # [ 18.075528] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4046lb # [ 18.077092] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4047lb # [ 18.081634] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4048lb # [ 18.083546] envoy[985]: [2026-09-20 12:45:19.057][985][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4049lb # [ 18.086238] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4050lb # [ 18.092391] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4051lb # [ 18.095100] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4052lb # [ 18.096810] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4053lb # [ 18.100805] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4054lb # [ 18.102543] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4055lb # [ 18.104564] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4056lb # [ 18.106507] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4057lb # [ 18.108165] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4058lb # [ 18.109965] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4059lb # [ 18.112442] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4060lb # [ 18.117848] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84061lb # [ 18.119814] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4062lb # [ 18.123090] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4063lb # [ 18.147947] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4064lb # [ 18.150347] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4065lb # [ 18.152937] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4066lb # [ 18.163790] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24067lb # [ 18.165652] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4068lb # [ 18.167628] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4069lb # [ 18.169640] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4070lb # [ 18.172158] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4071lb # [ 18.174537] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14072lb # [ 18.176873] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4073lb # [ 18.178531] envoy[985]: [2026-09-20 12:45:19.062][985][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4074lb # [ 18.180457] envoy[985]: [2026-09-20 12:45:19.063][985][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4075lb # [ 18.183090] envoy[985]: [2026-09-20 12:45:19.063][985][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4076lb # [ 18.185944] envoy[985]: [2026-09-20 12:45:19.063][985][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4077lb # [ 18.190123] envoy[985]: [2026-09-20 12:45:19.063][985][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24078lb # [ 18.192160] envoy[985]: [2026-09-20 12:45:19.063][985][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4079lb # [ 18.195280] envoy[985]: [2026-09-20 12:45:19.063][985][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4080lb # [ 18.197650] envoy[985]: [2026-09-20 12:45:19.063][985][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4081lb # [ 18.199384] envoy[985]: [2026-09-20 12:45:19.063][985][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4082lb # [ 18.201492] envoy[985]: [2026-09-20 12:45:19.067][985][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4083lb # [ 18.204089] envoy[985]: [2026-09-20 12:45:19.067][985][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4084lb # [ 18.208130] envoy[985]: [2026-09-20 12:45:19.067][985][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4085lb # [ 18.210625] envoy[985]: [2026-09-20 12:45:19.067][985][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4086lb # [ 18.217116] envoy[985]: [2026-09-20 12:45:19.067][985][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4087lb # [ 18.218661] envoy[985]: [2026-09-20 12:45:19.067][985][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4088lb # [ 18.220610] envoy[985]: [2026-09-20 12:45:19.069][985][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4089lb # [ 18.222021] envoy[985]: [2026-09-20 12:45:19.165][985][info][main] [source/server/server.cc:531] HTTP header map info:4090lb # [ 18.223218] envoy[985]: [2026-09-20 12:45:19.207][985][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4091lb # [ 18.235497] envoy[985]: [2026-09-20 12:45:19.207][985][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4092lb # [ 18.236844] envoy[985]: [2026-09-20 12:45:19.208][985][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4093lb # [ 18.244659] envoy[985]: [2026-09-20 12:45:19.208][985][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4094lb # [ 18.282454] envoy[985]: [2026-09-20 12:45:19.429][985][info][main] [source/server/server.cc:968] runtime: {}4095lb # [ 18.288883] envoy[985]: [2026-09-20 12:45:19.435][985][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4096lb # [ 18.291168] envoy[985]: [2026-09-20 12:45:19.438][985][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014097lb # [ 18.293970] envoy[985]: [2026-09-20 12:45:19.441][985][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014098lb # [ 18.297226] envoy[985]: [2026-09-20 12:45:19.444][985][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4099lb # [ 18.298829] envoy[985]: [2026-09-20 12:45:19.445][985][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4100lb # [ 18.300531] envoy[985]: [2026-09-20 12:45:19.445][985][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4101lb # [ 18.349445] envoy[985]: [2026-09-20 12:45:19.496][985][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4102worker1 # [ 18.373662] systemd[1]: Starting Nix worker-protocol over gRPC...4103worker2 # [ 18.389691] systemd[1]: Reached target Network is Online.4104worker2 # [ 18.396107] systemd[1]: Starting scheduler-token.service...4105lb # [ 18.367729] envoy[985]: [2026-09-20 12:45:19.514][985][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4106lb # [ 18.370101] envoy[985]: [2026-09-20 12:45:19.517][985][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4107lb # [ 18.373037] envoy[985]: [2026-09-20 12:45:19.517][985][info][main] [source/server/server.cc:1077] starting main dispatch loop4108worker2 # [ 18.507634] systemd[1]: scheduler-token.service: Deactivated successfully.4109worker2 # [ 18.509279] systemd[1]: Finished scheduler-token.service.4110worker2 # [ 18.511272] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 113ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4111worker2 # [ 18.515791] systemd[1]: Starting Nix worker-protocol over gRPC...4112worker1 # [ 18.577599] nix-grpc-daemon[784]: ts=2026-09-20T12:45:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4113worker1 # [ 18.587116] nix-grpc-daemon[784]: ts=2026-09-20T12:45:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4114worker1 # [ 18.590789] nix-grpc-daemon[784]: ts=2026-09-20T12:45:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14115worker1 # [ 18.623994] nix-grpc-daemon[784]: ts=2026-09-20T12:45:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4116worker1 # [ 18.627157] systemd[1]: Started Nix worker-protocol over gRPC.4117worker1 # [ 18.658633] nix-grpc-daemon[784]: ts=2026-09-20T12:45:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4118lb # [ 18.646840] envoy[985]: [2026-09-20 12:45:19.793][985][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4119lb # [ 18.648777] envoy[985]: [2026-09-20 12:45:19.795][985][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4120lb # [ 18.650710] envoy[985]: [2026-09-20 12:45:19.795][985][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4121lb # [ 18.652508] envoy[985]: [2026-09-20 12:45:19.795][985][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4122worker2 # [ 18.696821] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4123worker2 # [ 18.707565] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4124worker2 # [ 18.711217] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14125worker2 # [ 18.749376] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4126worker2 # [ 18.752125] systemd[1]: Started Nix worker-protocol over gRPC.4127worker2 # [ 18.805729] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_connected addr=lb:500514128worker2 # [ 18.818461] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4129worker1 # [ 18.859667] nix-grpc-daemon[784]: ts=2026-09-20T12:45:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4130worker2 # [ 19.020966] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_connected addr=lb:500514131worker2 # [ 19.024780] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4132worker2 # [ 19.225899] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_connected addr=lb:500514133worker2 # [ 19.229805] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4134worker1 # [ 19.260308] nix-grpc-daemon[784]: ts=2026-09-20T12:45:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4135worker2 # [ 19.431344] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_connected addr=lb:500514136worker2 # [ 19.434475] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker2 # [ 19.635229] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_connected addr=lb:500514138worker2 # [ 19.636979] nix-grpc-daemon[786]: ts=2026-09-20T12:45:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker2 # [ 19.838341] nix-grpc-daemon[786]: ts=2026-09-20T12:45:20Z level=info event=scheduler_connected addr=lb:500514140worker2 # [ 19.840216] nix-grpc-daemon[786]: ts=2026-09-20T12:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4141worker2 # [ 20.040977] nix-grpc-daemon[786]: ts=2026-09-20T12:45:20Z level=info event=scheduler_connected addr=lb:500514142worker2 # [ 20.042676] nix-grpc-daemon[786]: ts=2026-09-20T12:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker1 # [ 20.065465] nix-grpc-daemon[784]: ts=2026-09-20T12:45:20Z level=info event=scheduler_connected addr=lb:500514144worker1 # [ 20.069722] nix-grpc-daemon[784]: ts=2026-09-20T12:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 20.243453] nix-grpc-daemon[786]: ts=2026-09-20T12:45:20Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 20.245118] nix-grpc-daemon[786]: ts=2026-09-20T12:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker1 # [ 20.271084] nix-grpc-daemon[784]: ts=2026-09-20T12:45:20Z level=info event=scheduler_connected addr=lb:500514148worker1 # [ 20.275539] nix-grpc-daemon[784]: ts=2026-09-20T12:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 20.445780] nix-grpc-daemon[786]: ts=2026-09-20T12:45:20Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 20.447548] nix-grpc-daemon[786]: ts=2026-09-20T12:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 20.477436] nix-grpc-daemon[784]: ts=2026-09-20T12:45:20Z level=info event=scheduler_connected addr=lb:500514152worker1 # [ 20.480427] nix-grpc-daemon[784]: ts=2026-09-20T12:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker2 # [ 20.648566] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 20.650366] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker1 # [ 20.678246] nix-grpc-daemon[784]: ts=2026-09-20T12:45:20Z level=info event=scheduler_connected addr=lb:500514156worker1 # [ 20.682099] nix-grpc-daemon[784]: ts=2026-09-20T12:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 20.852647] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 20.855810] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 20.882769] nix-grpc-daemon[784]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514160worker1 # [ 20.886692] nix-grpc-daemon[784]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker2 # [ 21.053465] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 21.057228] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker1 # [ 21.088707] nix-grpc-daemon[784]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 21.092424] nix-grpc-daemon[784]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 21.258294] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 21.262155] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker1 # [ 21.293420] nix-grpc-daemon[784]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514168worker1 # [ 21.297865] nix-grpc-daemon[784]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker2 # [ 21.463399] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 21.467394] nix-grpc-daemon[786]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker1 # [ 21.498150] nix-grpc-daemon[784]: ts=2026-09-20T12:45:21Z level=info event=scheduler_connected addr=lb:500514172worker1 # [ 21.500242] nix-grpc-daemon[784]: ts=2026-09-20T12:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker2 # [ 21.668594] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 21.672421] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker1 # [ 21.701095] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514176worker1 # [ 21.705439] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 21.874710] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514178worker2 # [ 21.877615] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker1 # [ 21.906358] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514180worker1 # [ 21.910773] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker2 # [ 22.075541] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 22.079583] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker1 # [ 22.113095] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 22.116391] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker2 # [ 22.281119] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 22.285154] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker1 # [ 22.314073] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 22.317877] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker2 # [ 22.486240] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514190worker2 # [ 22.490170] nix-grpc-daemon[786]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker1 # [ 22.518912] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 22.522880] nix-grpc-daemon[784]: ts=2026-09-20T12:45:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 22.690621] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 22.692585] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 22.723671] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514196worker1 # [ 22.727518] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker2 # [ 22.894599] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514198worker2 # [ 22.896284] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker1 # [ 22.928370] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 22.932336] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker2 # [ 23.095845] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 23.098185] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker1 # [ 23.133158] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 23.134851] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker2 # [ 23.299427] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 23.303305] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 23.335560] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514208worker1 # [ 23.337703] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker2 # [ 23.504425] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 23.508343] nix-grpc-daemon[786]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 23.538158] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 23.539960] nix-grpc-daemon[784]: ts=2026-09-20T12:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 23.708901] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 23.710801] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker1 # [ 23.740450] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 23.742245] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 23.912343] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514218worker2 # [ 23.916448] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker1 # [ 23.942890] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 23.944945] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 24.117483] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 24.121507] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 24.145901] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514224worker1 # [ 24.147679] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 24.322382] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514226worker2 # [ 24.324518] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker1 # [ 24.348811] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 24.352721] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 24.525713] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 24.529545] nix-grpc-daemon[786]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker1 # [ 24.553255] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 24.555401] nix-grpc-daemon[784]: ts=2026-09-20T12:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker2 # [ 24.730651] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 24.734568] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker1 # [ 24.756190] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 24.760122] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 24.936609] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 24.940136] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker1 # [ 24.961387] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514240worker1 # [ 24.965275] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker2 # [ 25.140803] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 25.142858] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker1 # [ 25.167472] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514244worker1 # [ 25.170654] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker2 # [ 25.344264] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514246worker2 # [ 25.348485] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker1 # [ 25.368072] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 25.369777] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker2 # [ 25.548853] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514250worker2 # [ 25.550550] nix-grpc-daemon[786]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker1 # [ 25.570297] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_connected addr=lb:500514252worker1 # [ 25.572064] nix-grpc-daemon[784]: ts=2026-09-20T12:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker2 # [ 25.751367] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 25.755427] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker1 # [ 25.773231] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514256worker1 # [ 25.778183] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker2 # [ 25.957175] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514258worker2 # [ 25.961449] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker1 # [ 25.978869] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514260worker1 # [ 25.983331] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker2 # [ 26.162741] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514262worker2 # [ 26.165826] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker1 # [ 26.185685] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514264worker1 # [ 26.188804] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker2 # [ 26.366969] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514266worker2 # [ 26.370440] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker1 # [ 26.386643] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514268worker1 # [ 26.390613] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker2 # [ 26.571625] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514270worker2 # [ 26.575458] nix-grpc-daemon[786]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker1 # [ 26.591466] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_connected addr=lb:500514272worker1 # [ 26.595564] nix-grpc-daemon[784]: ts=2026-09-20T12:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker2 # [ 26.776693] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514274worker2 # [ 26.780526] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker1 # [ 26.796706] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514276worker1 # [ 26.801386] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker2 # [ 26.983132] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514278worker2 # [ 26.986351] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker1 # [ 27.001679] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514280worker1 # [ 27.005416] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker2 # [ 27.183960] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514282worker2 # [ 27.187845] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker1 # [ 27.206457] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 27.208064] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker2 # [ 27.388909] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514286worker2 # [ 27.392949] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker1 # [ 27.409069] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514288worker1 # [ 27.412939] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker2 # [ 27.593610] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514290worker2 # [ 27.595335] nix-grpc-daemon[786]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker1 # [ 27.613302] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_connected addr=lb:500514292worker1 # [ 27.615110] nix-grpc-daemon[784]: ts=2026-09-20T12:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker2 # [ 27.796483] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 27.800481] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker1 # [ 27.815952] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514296worker1 # [ 27.819962] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker2 # [ 28.002917] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514298worker2 # [ 28.006513] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker1 # [ 28.020336] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514300worker1 # [ 28.022048] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker2 # [ 28.207477] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514302worker2 # [ 28.211409] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker1 # [ 28.223410] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514304worker1 # [ 28.224704] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305lb # [ 28.205107] rustfs-bucket-start[1096]: mb s3://farm4306lb # [ 28.210623] systemd[1]: Finished rustfs-bucket.service.4307lb # [ 28.216432] systemd[1]: Starting niks3 server...4308worker2 # [ 28.411880] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514309worker2 # [ 28.413978] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310worker1 # [ 28.423934] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514311worker1 # [ 28.425681] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4312lb # [ 28.402399] postgres[1112]: [1112] ERROR: relation "goose_db_version" does not exist at character 364313lb # [ 28.403876] postgres[1112]: [1112] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4314lb # [ 28.437893] niks3-server[1107]: 2026/09/20 12:45:29 OK 20241026095416_initial_model.sql (19.14ms)4315lb # [ 28.441422] niks3-server[1107]: 2026/09/20 12:45:29 OK 20251210153512_drop_unused_gin_index.sql (3.69ms)4316lb # [ 28.446545] niks3-server[1107]: 2026/09/20 12:45:29 OK 20251218171726_add_pins.sql (3.67ms)4317lb # [ 28.451521] niks3-server[1107]: 2026/09/20 12:45:29 OK 20260628120000_add_object_size_and_stats.sql (5.05ms)4318lb # [ 28.456712] niks3-server[1107]: 2026/09/20 12:45:29 OK 20260905000000_add_claims.sql (3.53ms)4319lb # [ 28.461129] niks3-server[1107]: 2026/09/20 12:45:29 OK 20260920000000_drop_claims.sql (4.35ms)4320lb # [ 28.462308] niks3-server[1107]: 2026/09/20 12:45:29 goose: successfully migrated database to version: 202609200000004321lb # [ 28.467165] niks3-server[1107]: 2026/09/20 12:45:29 OK 1_commit_pending_closure.sql (6.14ms)4322lb # [ 28.469631] niks3-server[1107]: 2026/09/20 12:45:29 OK 2_object_stats_trigger.sql (1.23ms)4323lb # [ 28.470814] niks3-server[1107]: 2026/09/20 12:45:29 goose: up to current file version: 24324lb # [ 28.478749] niks3-server[1107]: 2026/09/20 12:45:29 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4325lb # [ 28.545317] niks3-server[1107]: 2026/09/20 12:45:29 INFO Read proxy enabled — serving cache objects from S34326lb # [ 28.547336] niks3-server[1107]: 2026/09/20 12:45:29 INFO Using socket-activated listener address=0.0.0.0:57514327lb # [ 28.553907] niks3-server[1107]: 2026/09/20 12:45:29 INFO systemd watchdog enabled interval=15s4328lb # [ 28.555122] systemd[1]: Started niks3 server.4329lb # [ 28.555788] systemd[1]: Reached target Multi-User System.4330lb # [ 28.558205] niks3-server[1107]: 2026/09/20 12:45:29 INFO Starting HTTP server address=0.0.0.0:57514331lb # [ 28.559910] systemd[1]: Startup finished in 869ms (kernel) + 4.880s (initrd) + 22.809s (userspace) = 28.559s.4332lb # [ 28.565283] niks3-server[1107]: 2026/09/20 12:45:29 INFO lead: acquired remote=192.168.1.4:462184333worker2 # [ 28.605887] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_take_over4334worker2 # [ 28.615111] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514335worker2 # [ 28.618175] nix-grpc-daemon[786]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4336worker1 # [ 28.626105] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_connected addr=lb:500514337worker1 # [ 28.627871] nix-grpc-daemon[784]: ts=2026-09-20T12:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4338worker2 # [ 28.818609] nix-grpc-daemon[786]: ts=2026-09-20T12:45:29Z level=info event=scheduler_connected addr=lb:500514339worker2 # [ 28.835219] nix-grpc-daemon[786]: ts=2026-09-20T12:45:29Z level=info event=oidc_jwks provider=mock keys=14340worker2 # [ 28.837105] nix-grpc-daemon[786]: ts=2026-09-20T12:45:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:329104341worker1 # [ 28.828436] nix-grpc-daemon[784]: ts=2026-09-20T12:45:29Z level=info event=scheduler_connected addr=lb:500514342worker2 # [ 28.839311] nix-grpc-daemon[786]: ts=2026-09-20T12:45:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04343worker2 # [ 28.842161] nix-grpc-daemon[786]: ts=2026-09-20T12:45:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:329104344worker2 # [ 28.843644] nix-grpc-daemon[786]: ts=2026-09-20T12:45:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04345lb: (finished: waiting for unit niks3.service, in 17.48 seconds)4346lb: waiting for TCP port 5751 on localhost4347lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4348lb: (finished: waiting for TCP port 5751 on localhost, in 0.05 seconds)4349lb: waiting for unit envoy.service4350lb: (finished: waiting for unit envoy.service, in 0.04 seconds)4351lb: waiting for TCP port 50051 on localhost4352lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4353lb # [ 29.142860] envoy[985]: conn peer=[::1]:46094 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24354lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4355worker2 # [ 29.277510] systemd[1]: Starting scheduler-token.service...4356worker2 # [ 29.333542] systemd[1]: scheduler-token.service: Deactivated successfully.4357worker2 # [ 29.335108] systemd[1]: Finished scheduler-token.service.4358worker2 # [ 29.336701] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 59ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4359lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4360??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4361 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284362lb: must succeed: curl -sf localhost:9901/clusters4363lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4364lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.06 seconds)4365subtest: one node schedules and both builders hold a WorkerSession on it4366lb: must succeed: curl -sf localhost:9901/clusters4367lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4368worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4369worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4370lb: must succeed: curl -sf localhost:9901/clusters4371lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4372lb: must succeed: curl -sf localhost:9901/clusters4373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4374worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4375worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.03 seconds)4376(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.20 seconds)4377subtest: balancer auth: client cert, bearer token, nothing, unknown CN4378worker1 # [ 29.918602] nix-grpc-daemon[784]: ts=2026-09-20T12:45:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447644379worker1 # [ 29.924154] systemd[1]: Started Nix Daemon.4380worker1 # [ 30.141555] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4381worker1 # [ 30.187850] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)4382worker1 # [ 30.279789] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 2/5)4383worker1 # [ 30.329769] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 279 ms (attempt 3/5)4384worker1 # [ 30.614625] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 209 ms (attempt 4/5)4385worker1 # [ 30.827634] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4386worker2 # [ 30.876712] nix-grpc-daemon[786]: ts=2026-09-20T12:45:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329144387worker2 # [ 30.888922] systemd[1]: Started Nix Daemon.4388worker2 # [ 31.117202] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4389worker2 # [ 31.144760] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5)4390worker2 # [ 31.148517] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 160 ms (attempt 2/5)4391worker2 # [ 31.311781] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 240 ms (attempt 3/5)4392worker2 # [ 31.557573] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 4/5)4393worker2 # [ 31.628296] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4394worker2 # [ 31.648124] nix-grpc-daemon[786]: ts=2026-09-20T12:45:32Z level=debug event=path_info_miss path=937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test error="error: path '/nix/store/937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test' is required, but there is no substituter that can build it"4395worker1 # [ 31.645879] nix-grpc-daemon[784]: ts=2026-09-20T12:45:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:447644396worker1 # [ 31.663168] nix-grpc-daemon[784]: ts=2026-09-20T12:45:31Z level=debug event=path_info_miss path=937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test error="error: path '/nix/store/937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test' is required, but there is no substituter that can build it"4397client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4398client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4399client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4400client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4401worker2 # [ 32.030819] nix-grpc-daemon[786]: ts=2026-09-20T12:45:32Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:329144402worker1 # [ 32.049233] nix-grpc-daemon[784]: ts=2026-09-20T12:45:32Z level=info event=oidc_jwks provider=mock keys=14403worker1 # [ 32.053964] nix-grpc-daemon[784]: ts=2026-09-20T12:45:32Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:447644404worker2 # [ 32.077075] nix-grpc-daemon[786]: ts=2026-09-20T12:45:32Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:329144405worker1 # [ 32.062509] nix-grpc-daemon[784]: ts=2026-09-20T12:45:32Z level=debug event=path_info_miss path=937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test error="error: path '/nix/store/937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test' is required, but there is no substituter that can build it"4406worker2 # [ 32.085432] nix-grpc-daemon[786]: ts=2026-09-20T12:45:32Z level=debug event=path_info_miss path=937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test error="error: path '/nix/store/937srf9qjadzml345gkxgn0gqprgv042-nixos-system-client-test' is required, but there is no substituter that can build it"4407client: must succeed: rm -r /var/lib/nix-grpc-store4408client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4409worker2 # [ 34.341599] nix-grpc-daemon[786]: ts=2026-09-20T12:45:34Z level=info event=denied method=QueryMissing cn=- role=none4410worker2 # [ 34.578935] nix-grpc-daemon[786]: ts=2026-09-20T12:45:34Z level=info event=denied method=QueryMissing cn=stranger role=none4411(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.00 seconds)4412subtest: DAG build is scheduled across workers and lands in the cache4413client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24414worker2 # [ 34.790515] nix-grpc-daemon[786]: ts=2026-09-20T12:45:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329144415client # don't know how to build these paths:4416client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4417worker1 # [ 34.792841] nix-grpc-daemon[784]: ts=2026-09-20T12:45:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447644418worker2 # [ 34.829958] nix-grpc-daemon[786]: ts=2026-09-20T12:45:35Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04419worker2 # [ 34.832051] nix-grpc-daemon[786]: ts=2026-09-20T12:45:35Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14420worker2 # [ 34.881508] nix-grpc-daemon[786]: ts=2026-09-20T12:45:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144421worker2 # [ 34.903611] nix-grpc-daemon[786]: ts=2026-09-20T12:45:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144422client # copying 1 paths to 'lb:50051'...4423worker2 # [ 34.915936] nix-grpc-daemon[786]: ts=2026-09-20T12:45:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329144424client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4425worker2 # [ 34.931334] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4426worker2 # [ 34.957632] nix-grpc-daemon[786]: ts=2026-09-20T12:45:35Z level=info event=niks3_push_started pid=8674427worker2 # [ 35.205235] nix-grpc-daemon[867]: time=2026-09-20T12:45:35.559Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4428worker2 # [ 35.393646] nix-daemon[855]: accepted connection from pid 880, user nix-grpc-daemon (trusted)4429worker2 # [ 35.412263] nix-daemon[855]: reaped child process 883, status = succeeded4430worker2 # [ 35.570753] nix-daemon[855]: accepted connection from pid 886, user nix-grpc-daemon (trusted)4431worker2 # [ 35.583768] nix-daemon[855]: reaped child process 889, status = succeeded4432lb # [ 35.565324] niks3-server[1107]: 2026/09/20 12:45:36 INFO Received uploads request method=POST path=/api/pending_closures4433worker2 # [ 35.624884] nix-grpc-daemon[867]: time=2026-09-20T12:45:35.982Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4434worker2 # [ 35.627836] nix-grpc-daemon[867]: time=2026-09-20T12:45:35.984Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4435lb # [ 35.637155] niks3-server[1107]: 2026/09/20 12:45:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4436lb # [ 35.649257] niks3-server[1107]: 2026/09/20 12:45:36 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4437lb # [ 35.664245] niks3-server[1107]: 2026/09/20 12:45:36 INFO Signed narinfos id=1 count=14438worker2 # [ 35.702269] nix-grpc-daemon[867]: time=2026-09-20T12:45:36.059Z level=INFO msg="Uploading 1 narinfos"4439lb # [ 35.672417] niks3-server[1107]: 2026/09/20 12:45:36 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4440lb # [ 35.680640] niks3-server[1107]: 2026/09/20 12:45:36 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4441lb # [ 35.694034] niks3-server[1107]: 2026/09/20 12:45:36 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4442lb # [ 35.700088] niks3-server[1107]: 2026/09/20 12:45:36 INFO Completed upload id=14443worker2 # [ 35.735716] nix-grpc-daemon[867]: time=2026-09-20T12:45:36.092Z level=INFO msg="Upload complete. (519ms)"4444worker2 # [ 35.738331] nix-grpc-daemon[786]: ts=2026-09-20T12:45:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=5044445worker2 # [ 35.743348] nix-grpc-daemon[786]: ts=2026-09-20T12:45:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144446worker2 # [ 35.753182] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4447worker2 # [ 35.762139] nix-daemon[855]: reaped child process 865, status = succeeded4448client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4449worker2 # [ 35.767115] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4450client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4451client # farm-a> LOG-farm-a-t14452worker2 # [ 36.579564] nix-daemon[855]: reaped child process 893, status = succeeded4453worker2 # [ 36.712341] nix-daemon[855]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4454worker2 # [ 36.725062] nix-daemon[855]: reaped child process 911, status = succeeded4455lb # [ 36.702066] niks3-server[1107]: 2026/09/20 12:45:37 INFO Received uploads request method=POST path=/api/pending_closures4456worker2 # [ 36.743440] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.100Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4457worker2 # [ 36.745267] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.102Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4458lb # [ 36.730595] niks3-server[1107]: 2026/09/20 12:45:37 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4459lb # [ 36.736246] niks3-server[1107]: 2026/09/20 12:45:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4460lb # [ 36.737868] niks3-server[1107]: 2026/09/20 12:45:37 INFO Signed narinfos id=2 count=14461worker2 # [ 36.774384] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.130Z level=INFO msg="Uploading 1 narinfos"4462lb # [ 36.741366] niks3-server[1107]: 2026/09/20 12:45:37 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4463lb # [ 36.745283] niks3-server[1107]: 2026/09/20 12:45:37 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4464lb # [ 36.750084] niks3-server[1107]: 2026/09/20 12:45:37 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4465lb # [ 36.752981] niks3-server[1107]: 2026/09/20 12:45:37 INFO Completed upload id=24466worker2 # [ 36.790258] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.147Z level=INFO msg="Upload complete. (216ms)"4467lb # [ 36.755093] niks3-server[1107]: 2026/09/20 12:45:37 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4468worker2 # [ 36.794280] nix-grpc-daemon[786]: ts=2026-09-20T12:45:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14469worker2 # [ 36.806212] nix-daemon[855]: reaped child process 891, status = succeeded4470worker2 # [ 36.808286] nix-grpc-daemon[786]: ts=2026-09-20T12:45:37Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04471worker2 # [ 36.810315] nix-grpc-daemon[786]: ts=2026-09-20T12:45:37Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14472worker2 # [ 36.858257] nix-grpc-daemon[786]: ts=2026-09-20T12:45:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144473worker2 # [ 36.869297] nix-grpc-daemon[786]: ts=2026-09-20T12:45:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144474worker2 # [ 36.882158] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.239Z level=INFO msg="All 1 paths already cached"4475client # copying 1 paths to 'lb:50051'...4476worker2 # [ 36.889163] nix-grpc-daemon[786]: ts=2026-09-20T12:45:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329144477client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4478worker2 # [ 36.909167] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4479worker2 # [ 37.070704] nix-daemon[855]: accepted connection from pid 917, user nix-grpc-daemon (trusted)4480worker2 # [ 37.085852] nix-daemon[855]: reaped child process 920, status = succeeded4481worker2 # [ 37.213434] nix-daemon[855]: accepted connection from pid 922, user nix-grpc-daemon (trusted)4482worker2 # [ 37.227752] nix-daemon[855]: reaped child process 925, status = succeeded4483lb # [ 37.200227] niks3-server[1107]: 2026/09/20 12:45:38 INFO Received uploads request method=POST path=/api/pending_closures4484worker2 # [ 37.257104] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.614Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4485worker2 # [ 37.258660] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.615Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4486lb # [ 37.245239] niks3-server[1107]: 2026/09/20 12:45:38 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4487lb # [ 37.253465] niks3-server[1107]: 2026/09/20 12:45:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4488worker2 # [ 37.297636] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.654Z level=INFO msg="Uploading 1 narinfos"4489lb # [ 37.261755] niks3-server[1107]: 2026/09/20 12:45:38 INFO Signed narinfos id=3 count=14490lb # [ 37.268058] niks3-server[1107]: 2026/09/20 12:45:38 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4491lb # [ 37.276575] niks3-server[1107]: 2026/09/20 12:45:38 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4492lb # [ 37.286489] niks3-server[1107]: 2026/09/20 12:45:38 INFO Completed upload id=34493lb # [ 37.288141] niks3-server[1107]: 2026/09/20 12:45:38 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4494worker2 # [ 37.322358] nix-grpc-daemon[867]: time=2026-09-20T12:45:37.679Z level=INFO msg="Upload complete. (399ms)"4495worker2 # [ 37.328181] nix-grpc-daemon[786]: ts=2026-09-20T12:45:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=6244496worker2 # [ 37.333915] nix-grpc-daemon[786]: ts=2026-09-20T12:45:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144497worker2 # [ 37.344523] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4498worker2 # [ 37.355421] nix-daemon[855]: reaped child process 915, status = succeeded4499client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4500worker2 # [ 37.362120] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4501client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4502client # farm-b> LOG-farm-b-t14503worker2 # [ 38.376418] nix-daemon[855]: reaped child process 929, status = succeeded4504lb # [ 38.356647] envoy[985]: conn peer=[2001:db8:1::1]:38864 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=17914505lb # [ 38.360565] envoy[985]: conn peer=[2001:db8:1::1]:38866 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=363 ms=694506lb # [ 38.363856] envoy[985]: conn peer=[2001:db8:1::1]:49940 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=104507lb # [ 38.367214] envoy[985]: conn peer=[2001:db8:1::1]:49948 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=64508worker2 # [ 38.518993] nix-daemon[855]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4509worker2 # [ 38.533504] nix-daemon[855]: reaped child process 947, status = succeeded4510lb # [ 38.515410] niks3-server[1107]: 2026/09/20 12:45:39 INFO Received uploads request method=POST path=/api/pending_closures4511worker2 # [ 38.559032] nix-grpc-daemon[867]: time=2026-09-20T12:45:38.915Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4512worker2 # [ 38.561849] nix-grpc-daemon[867]: time=2026-09-20T12:45:38.916Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4513lb # [ 38.546877] niks3-server[1107]: 2026/09/20 12:45:39 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4514lb # [ 38.560431] niks3-server[1107]: 2026/09/20 12:45:39 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4515lb # [ 38.570849] niks3-server[1107]: 2026/09/20 12:45:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4516worker2 # [ 38.612906] nix-grpc-daemon[867]: time=2026-09-20T12:45:38.969Z level=INFO msg="Uploading 1 narinfos"4517lb # [ 38.576857] niks3-server[1107]: 2026/09/20 12:45:39 INFO Signed narinfos id=4 count=14518lb # [ 38.588033] niks3-server[1107]: 2026/09/20 12:45:39 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4519lb # [ 38.628707] niks3-server[1107]: 2026/09/20 12:45:39 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4520lb # [ 38.631672] niks3-server[1107]: 2026/09/20 12:45:39 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4521lb # [ 38.635335] niks3-server[1107]: 2026/09/20 12:45:39 INFO Completed upload id=44522worker2 # [ 38.671466] nix-grpc-daemon[867]: time=2026-09-20T12:45:39.028Z level=INFO msg="Upload complete. (302ms)"4523worker2 # [ 38.679418] nix-grpc-daemon[786]: ts=2026-09-20T12:45:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14524worker2 # [ 38.688978] nix-grpc-daemon[786]: ts=2026-09-20T12:45:39Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04525worker2 # [ 38.693616] nix-grpc-daemon[786]: ts=2026-09-20T12:45:39Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14526worker2 # [ 38.699124] nix-daemon[855]: reaped child process 927, status = succeeded4527worker2 # [ 38.736715] nix-grpc-daemon[786]: ts=2026-09-20T12:45:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144528worker2 # [ 38.748896] nix-grpc-daemon[786]: ts=2026-09-20T12:45:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144529worker2 # [ 38.761572] nix-grpc-daemon[867]: time=2026-09-20T12:45:39.118Z level=INFO msg="All 2 paths already cached"4530client # copying 1 paths to 'lb:50051'...4531client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4532worker2 # [ 38.769469] nix-grpc-daemon[786]: ts=2026-09-20T12:45:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329144533worker2 # [ 38.779280] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4534worker2 # [ 38.914549] nix-daemon[855]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4535worker2 # [ 38.929288] nix-daemon[855]: reaped child process 957, status = succeeded4536worker2 # [ 39.068420] nix-daemon[855]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4537worker2 # [ 39.081316] nix-daemon[855]: reaped child process 962, status = succeeded4538lb # [ 39.058847] niks3-server[1107]: 2026/09/20 12:45:40 INFO Received uploads request method=POST path=/api/pending_closures4539worker2 # [ 39.102637] nix-grpc-daemon[867]: time=2026-09-20T12:45:39.459Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4540worker2 # [ 39.103987] nix-grpc-daemon[867]: time=2026-09-20T12:45:39.459Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4541lb # [ 39.084594] niks3-server[1107]: 2026/09/20 12:45:40 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4542lb # [ 39.093111] niks3-server[1107]: 2026/09/20 12:45:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4543lb # [ 39.098065] niks3-server[1107]: 2026/09/20 12:45:40 INFO Signed narinfos id=5 count=14544worker2 # [ 39.136585] nix-grpc-daemon[867]: time=2026-09-20T12:45:39.493Z level=INFO msg="Uploading 1 narinfos"4545lb # [ 39.106494] niks3-server[1107]: 2026/09/20 12:45:40 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4546lb # [ 39.116910] niks3-server[1107]: 2026/09/20 12:45:40 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4547lb # [ 39.124276] niks3-server[1107]: 2026/09/20 12:45:40 INFO Completed upload id=54548worker2 # [ 39.162315] nix-grpc-daemon[867]: time=2026-09-20T12:45:39.519Z level=INFO msg="Upload complete. (370ms)"4549lb # [ 39.127204] niks3-server[1107]: 2026/09/20 12:45:40 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4550worker2 # [ 39.166320] nix-grpc-daemon[786]: ts=2026-09-20T12:45:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=7524551worker2 # [ 39.170848] nix-grpc-daemon[786]: ts=2026-09-20T12:45:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144552worker2 # [ 39.180956] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4553worker2 # [ 39.186898] nix-daemon[855]: reaped child process 952, status = succeeded4554client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4555client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4556worker2 # [ 39.190085] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4557client # farm-top> LOG-farm-top-t14558worker2 # [ 40.218385] nix-daemon[855]: reaped child process 966, status = succeeded4559worker2 # [ 40.348319] nix-daemon[855]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4560worker2 # [ 40.361625] nix-daemon[855]: reaped child process 984, status = succeeded4561lb # [ 40.341597] niks3-server[1107]: 2026/09/20 12:45:41 INFO Received uploads request method=POST path=/api/pending_closures4562worker2 # [ 40.388168] nix-grpc-daemon[867]: time=2026-09-20T12:45:40.744Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4563worker2 # [ 40.391099] nix-grpc-daemon[867]: time=2026-09-20T12:45:40.746Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4564lb # [ 40.364750] niks3-server[1107]: 2026/09/20 12:45:41 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4565lb # [ 40.371097] niks3-server[1107]: 2026/09/20 12:45:41 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4566lb # [ 40.374660] niks3-server[1107]: 2026/09/20 12:45:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4567lb # [ 40.376975] niks3-server[1107]: 2026/09/20 12:45:41 INFO Signed narinfos id=6 count=14568worker2 # [ 40.412735] nix-grpc-daemon[867]: time=2026-09-20T12:45:40.769Z level=INFO msg="Uploading 1 narinfos"4569lb # [ 40.379624] niks3-server[1107]: 2026/09/20 12:45:41 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4570lb # [ 40.383984] niks3-server[1107]: 2026/09/20 12:45:41 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4571lb # [ 40.387807] niks3-server[1107]: 2026/09/20 12:45:41 INFO Completed upload id=64572worker2 # [ 40.423444] nix-grpc-daemon[867]: time=2026-09-20T12:45:40.780Z level=INFO msg="Upload complete. (224ms)"4573lb # [ 40.389059] niks3-server[1107]: 2026/09/20 12:45:41 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4574worker2 # [ 40.425925] nix-grpc-daemon[786]: ts=2026-09-20T12:45:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14575worker2 # [ 40.433255] nix-daemon[855]: reaped child process 964, status = succeeded4576client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 5.85 seconds)4577client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4578client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.15 seconds)4579client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4580client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4581client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4582client # copying 3 paths...4583client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4584client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4585client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4586client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.22 seconds)4587(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.28 seconds)4588subtest: repeat is answered Cached by the scheduler without building4589worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4590worker1 # finding garbage collector roots...4591worker1 # deleting specified paths...4592worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4593worker1 # deleting unused links...4594worker1 # note: hard linking is currently saving 0.0 KiB4595worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4596worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4597worker2 # finding garbage collector roots...4598worker2 # deleting specified paths...4599worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4600worker2 # deleting unused links...4601worker2 # note: hard linking is currently saving 0.0 KiB4602worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.21 seconds)4603client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24604client # don't know how to build these paths:4605client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4606worker1 # [ 41.513653] nix-grpc-daemon[784]: ts=2026-09-20T12:45:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447644607worker2 # [ 41.528601] nix-grpc-daemon[786]: ts=2026-09-20T12:45:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144608worker2 # [ 41.536094] nix-grpc-daemon[867]: time=2026-09-20T12:45:41.892Z level=INFO msg="All 2 paths already cached"4609worker2 # [ 41.545623] nix-grpc-daemon[786]: ts=2026-09-20T12:45:41Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14610client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.24 seconds)4611client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4612client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4613lb: must succeed: curl -sf localhost:9901/clusters4614lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4615lb: must succeed: curl -sf localhost:9901/clusters4616lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4617worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4618worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4619(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.02 seconds)4620subtest: inputs already in the cache are not sent to the scheduler4621lb: must succeed: curl -sf localhost:9901/clusters4622lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4623lb: must succeed: curl -sf localhost:9901/clusters4624lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4625worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4626worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4627client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24628worker1 # [ 42.181152] nix-grpc-daemon[784]: ts=2026-09-20T12:45:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447644629client # don't know how to build these paths:4630client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4631worker2 # [ 42.201375] nix-grpc-daemon[786]: ts=2026-09-20T12:45:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144632worker2 # [ 42.209426] nix-grpc-daemon[867]: time=2026-09-20T12:45:42.566Z level=INFO msg="All 2 paths already cached"4633worker2 # [ 42.222112] nix-grpc-daemon[786]: ts=2026-09-20T12:45:42Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04634worker2 # [ 42.228354] nix-grpc-daemon[786]: ts=2026-09-20T12:45:42Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14635worker2 # [ 42.233284] nix-grpc-daemon[786]: ts=2026-09-20T12:45:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144636worker2 # [ 42.245063] nix-grpc-daemon[786]: ts=2026-09-20T12:45:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144637worker2 # [ 42.257196] nix-grpc-daemon[867]: time=2026-09-20T12:45:42.614Z level=INFO msg="All 2 paths already cached"4638client # copying 1 paths to 'lb:50051'...4639client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4640worker2 # [ 42.264737] nix-grpc-daemon[786]: ts=2026-09-20T12:45:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329144641worker2 # [ 42.275944] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4642worker2 # [ 42.419196] nix-daemon[855]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4643worker2 # [ 42.433362] nix-daemon[855]: reaped child process 1017, status = succeeded4644worker2 # [ 42.573178] nix-daemon[855]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4645worker2 # [ 42.585599] nix-daemon[855]: reaped child process 1022, status = succeeded4646lb # [ 42.560198] niks3-server[1107]: 2026/09/20 12:45:43 INFO Received uploads request method=POST path=/api/pending_closures4647worker2 # [ 42.604338] nix-grpc-daemon[867]: time=2026-09-20T12:45:42.960Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4648worker2 # [ 42.605835] nix-grpc-daemon[867]: time=2026-09-20T12:45:42.960Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4649lb # [ 42.591394] niks3-server[1107]: 2026/09/20 12:45:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4650worker2 # [ 42.631774] nix-grpc-daemon[867]: time=2026-09-20T12:45:42.988Z level=INFO msg="Uploading 1 narinfos"4651lb # [ 42.597137] niks3-server[1107]: 2026/09/20 12:45:43 INFO Signed narinfos id=7 count=14652lb # [ 42.603460] niks3-server[1107]: 2026/09/20 12:45:43 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4653lb # [ 42.613821] niks3-server[1107]: 2026/09/20 12:45:43 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4654lb # [ 42.620150] niks3-server[1107]: 2026/09/20 12:45:43 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4655lb # [ 42.630650] niks3-server[1107]: 2026/09/20 12:45:43 INFO Completed upload id=74656worker2 # [ 42.666243] nix-grpc-daemon[867]: time=2026-09-20T12:45:43.023Z level=INFO msg="Upload complete. (380ms)"4657lb # [ 42.633747] niks3-server[1107]: 2026/09/20 12:45:43 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4658worker2 # [ 42.670482] nix-grpc-daemon[786]: ts=2026-09-20T12:45:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=7524659worker2 # [ 42.676429] nix-grpc-daemon[786]: ts=2026-09-20T12:45:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144660worker2 # [ 42.684098] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4661worker2 # [ 42.690264] nix-daemon[855]: reaped child process 1012, status = succeeded4662client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4663client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4664worker2 # [ 42.695385] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4665client # farm-top> LOG-farm-top-t1b4666worker2 # [ 43.308783] nix-daemon[855]: reaped child process 1026, status = succeeded4667worker2 # [ 43.457081] nix-daemon[855]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4668worker2 # [ 43.470134] nix-daemon[855]: reaped child process 1044, status = succeeded4669lb # [ 43.449665] niks3-server[1107]: 2026/09/20 12:45:44 INFO Received uploads request method=POST path=/api/pending_closures4670worker2 # [ 43.493158] nix-grpc-daemon[867]: time=2026-09-20T12:45:43.850Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4671worker2 # [ 43.494808] nix-grpc-daemon[867]: time=2026-09-20T12:45:43.851Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4672lb # [ 43.491057] niks3-server[1107]: 2026/09/20 12:45:44 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4673lb # [ 43.498639] niks3-server[1107]: 2026/09/20 12:45:44 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4674lb # [ 43.504889] niks3-server[1107]: 2026/09/20 12:45:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4675lb # [ 43.509114] niks3-server[1107]: 2026/09/20 12:45:44 INFO Signed narinfos id=8 count=14676worker2 # [ 43.547684] nix-grpc-daemon[867]: time=2026-09-20T12:45:43.904Z level=INFO msg="Uploading 1 narinfos"4677lb # [ 43.521384] niks3-server[1107]: 2026/09/20 12:45:44 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4678lb # [ 43.528344] niks3-server[1107]: 2026/09/20 12:45:44 INFO Completed upload id=84679worker2 # [ 43.563877] nix-grpc-daemon[867]: time=2026-09-20T12:45:43.920Z level=INFO msg="Upload complete. (262ms)"4680lb # [ 43.532802] niks3-server[1107]: 2026/09/20 12:45:44 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4681lb # [ 43.535111] niks3-server[1107]: 2026/09/20 12:45:44 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4682worker2 # [ 43.568729] nix-grpc-daemon[786]: ts=2026-09-20T12:45:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14683worker2 # [ 43.580836] nix-daemon[855]: reaped child process 1024, status = succeeded4684client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.64 seconds)4685client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4686client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.16 seconds)4687lb: must succeed: curl -sf localhost:9901/clusters4688lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4689lb: must succeed: curl -sf localhost:9901/clusters4690lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4691worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4692worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4693(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.05 seconds)4694subtest: two clients wanting the same drv build it once4695client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4696client # [ 43.968723] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4697client # Running as unit: dedup1.service; invocation ID: 2cf1902948244dc088d8b58218538ce34698client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.04 seconds)4699client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4700client # [ 44.146773] nix[889]: don't know how to build these paths:4701worker1 # [ 44.148617] nix-grpc-daemon[784]: ts=2026-09-20T12:45:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447644702client # [ 44.147770] nix[889]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4703worker1 # [ 44.159275] nix-grpc-daemon[784]: ts=2026-09-20T12:45:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447644704worker2 # [ 44.168816] nix-grpc-daemon[786]: ts=2026-09-20T12:45:44Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04705worker2 # [ 44.172809] nix-grpc-daemon[786]: ts=2026-09-20T12:45:44Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14706worker1 # [ 44.169155] nix-grpc-daemon[784]: ts=2026-09-20T12:45:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447644707client # [ 44.173086] nix[889]: copying 1 paths to 'lb:50051'...4708worker1 # [ 44.176320] nix-grpc-daemon[784]: ts=2026-09-20T12:45:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447644709client # [ 44.173967] nix[889]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4710worker1 # [ 44.181563] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4711worker1 # [ 44.193557] nix-grpc-daemon[784]: ts=2026-09-20T12:45:44Z level=info event=niks3_push_started pid=8684712worker1 # [ 44.434160] nix-grpc-daemon[868]: time=2026-09-20T12:45:44.738Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4713worker1 # [ 44.613849] nix-daemon[841]: accepted connection from pid 881, user nix-grpc-daemon (trusted)4714worker1 # [ 44.629672] nix-daemon[841]: reaped child process 884, status = succeeded4715worker1 # [ 44.769992] nix-daemon[841]: accepted connection from pid 886, user nix-grpc-daemon (trusted)4716worker1 # [ 44.783570] nix-daemon[841]: reaped child process 889, status = succeeded4717lb # [ 44.767894] niks3-server[1107]: 2026/09/20 12:45:45 INFO Received uploads request method=POST path=/api/pending_closures4718worker1 # [ 44.800741] nix-grpc-daemon[868]: time=2026-09-20T12:45:45.108Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4719worker1 # [ 44.802777] nix-grpc-daemon[868]: time=2026-09-20T12:45:45.110Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4720lb # [ 44.808796] niks3-server[1107]: 2026/09/20 12:45:45 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4721lb # [ 44.814444] niks3-server[1107]: 2026/09/20 12:45:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4722worker1 # [ 44.843928] nix-grpc-daemon[868]: time=2026-09-20T12:45:45.152Z level=INFO msg="Uploading 1 narinfos"4723lb # [ 44.819835] niks3-server[1107]: 2026/09/20 12:45:45 INFO Signed narinfos id=9 count=14724lb # [ 44.833114] niks3-server[1107]: 2026/09/20 12:45:45 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4725lb # [ 44.839515] niks3-server[1107]: 2026/09/20 12:45:45 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4726lb # [ 44.844946] niks3-server[1107]: 2026/09/20 12:45:45 INFO Completed upload id=94727worker1 # [ 44.871039] nix-grpc-daemon[868]: time=2026-09-20T12:45:45.179Z level=INFO msg="Upload complete. (422ms)"4728lb # [ 44.847758] niks3-server[1107]: 2026/09/20 12:45:45 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4729worker1 # [ 44.873249] nix-grpc-daemon[784]: ts=2026-09-20T12:45:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=0 paths=1 nar_bytes_in=4324730worker1 # [ 44.876199] nix-grpc-daemon[784]: ts=2026-09-20T12:45:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447644731worker1 # [ 44.882746] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4732worker1 # [ 44.886242] nix-daemon[841]: reaped child process 866, status = succeeded4733worker1 # [ 44.891760] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4734client # [ 44.887937] nix[889]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4735client # [ 45.023577] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4736client # Running as unit: dedup2.service; invocation ID: d744a531282f44668e287563d50a65784737client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.06 seconds)4738worker2 # [ 45.271151] nix-grpc-daemon[786]: ts=2026-09-20T12:45:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329144739client # [ 45.261203] nix[904]: don't know how to build these paths:4740client # [ 45.262449] nix[904]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4741worker2 # [ 45.278309] nix-grpc-daemon[786]: ts=2026-09-20T12:45:45Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04742worker1 # [ 45.270290] nix-grpc-daemon[784]: ts=2026-09-20T12:45:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447644743client # [ 45.272899] nix[904]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4744client: must succeed: sleep 24745client: (finished: must succeed: sleep 2, in 2.05 seconds)4746lb # [ 48.357875] envoy[985]: conn peer=[2001:db8:1::1]:49958 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=3847 ms=56744747lb # [ 48.361699] envoy[985]: conn peer=[2001:db8:1::1]:49966 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=354748lb # [ 48.365176] envoy[985]: conn peer=[2001:db8:1::1]:49976 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4152 tx=1689 ms=14284749??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4750 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394751client: waiting for success: ! systemctl is-active dedup1 dedup24752??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4753 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394754worker1 # [ 48.508935] nix-daemon[841]: reaped child process 893, status = succeeded4755worker1 # [ 48.658847] nix-daemon[841]: accepted connection from pid 922, user nix-grpc-daemon (trusted)4756worker1 # [ 48.670907] nix-daemon[841]: reaped child process 925, status = succeeded4757lb # [ 48.660596] niks3-server[1107]: 2026/09/20 12:45:49 INFO Received uploads request method=POST path=/api/pending_closures4758worker1 # [ 48.690599] nix-grpc-daemon[868]: time=2026-09-20T12:45:48.998Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4759worker1 # [ 48.692274] nix-grpc-daemon[868]: time=2026-09-20T12:45:49.000Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4760lb # [ 48.704847] niks3-server[1107]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4761lb # [ 48.711886] niks3-server[1107]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4762lb # [ 48.724580] niks3-server[1107]: 2026/09/20 12:45:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4763lb # [ 48.728428] niks3-server[1107]: 2026/09/20 12:45:49 INFO Signed narinfos id=10 count=14764worker1 # [ 48.753491] nix-grpc-daemon[868]: time=2026-09-20T12:45:49.060Z level=INFO msg="Uploading 1 narinfos"4765lb # [ 48.733361] niks3-server[1107]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4766lb # [ 48.742108] niks3-server[1107]: 2026/09/20 12:45:49 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4767lb # [ 48.745116] niks3-server[1107]: 2026/09/20 12:45:49 INFO Completed upload id=104768worker1 # [ 48.771184] nix-grpc-daemon[868]: time=2026-09-20T12:45:49.077Z level=INFO msg="Upload complete. (265ms)"4769lb # [ 48.748076] niks3-server[1107]: 2026/09/20 12:45:49 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4770worker1 # [ 48.775692] nix-grpc-daemon[784]: ts=2026-09-20T12:45:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14771worker1 # [ 48.782258] nix-grpc-daemon[784]: ts=2026-09-20T12:45:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14772worker1 # [ 48.792417] nix-daemon[841]: reaped child process 891, status = succeeded4773client # [ 48.795391] systemd[1]: dedup1.service: Deactivated successfully.4774client # [ 48.799256] systemd[1]: dedup1.service: Consumed 98ms CPU time over 4.829s wall clock time, 37.3M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.4775client # [ 48.841121] systemd[1]: dedup2.service: Deactivated successfully.4776client # [ 48.842591] systemd[1]: dedup2.service: Consumed 109ms CPU time over 3.818s wall clock time, 31.2M memory peak, 7.5K incoming IP traffic, 6.5K outgoing IP traffic.4777client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds)4778client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4779client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.03 seconds)4780worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4781worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4782worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4783worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4784worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4785worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4786worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4787worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4788(finished: subtest: two clients wanting the same drv build it once, in 5.83 seconds)4789subtest: build hook: nix-daemon with builders = grpc://lb4790client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4791client # [ 49.944038] systemd[1]: Started Nix Daemon.4792client # [ 50.104489] nix-daemon[945]: accepted connection from pid 942, user root (trusted)4793worker1 # [ 50.281922] nix-grpc-daemon[784]: ts=2026-09-20T12:45:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:447644794worker1 # [ 50.287110] nix-grpc-daemon[784]: ts=2026-09-20T12:45:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447644795worker2 # [ 50.313899] nix-grpc-daemon[786]: ts=2026-09-20T12:45:50Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04796worker2 # [ 50.319566] nix-grpc-daemon[786]: ts=2026-09-20T12:45:50Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14797worker2 # [ 50.361718] nix-grpc-daemon[786]: ts=2026-09-20T12:45:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144798worker2 # [ 50.374611] nix-grpc-daemon[786]: ts=2026-09-20T12:45:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144799worker2 # [ 50.386161] nix-grpc-daemon[786]: ts=2026-09-20T12:45:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329144800worker2 # [ 50.407200] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4801worker2 # [ 50.578817] nix-daemon[855]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)4802worker2 # [ 50.592548] nix-daemon[855]: reaped child process 1083, status = succeeded4803worker2 # [ 50.729054] nix-daemon[855]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)4804worker2 # [ 50.742392] nix-daemon[855]: reaped child process 1088, status = succeeded4805lb # [ 50.718430] niks3-server[1107]: 2026/09/20 12:45:51 INFO Received uploads request method=POST path=/api/pending_closures4806worker2 # [ 50.762168] nix-grpc-daemon[867]: time=2026-09-20T12:45:51.118Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4807worker2 # [ 50.763722] nix-grpc-daemon[867]: time=2026-09-20T12:45:51.118Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4808lb # [ 50.742533] niks3-server[1107]: 2026/09/20 12:45:51 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4809lb # [ 50.753632] niks3-server[1107]: 2026/09/20 12:45:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4810lb # [ 50.759258] niks3-server[1107]: 2026/09/20 12:45:51 INFO Signed narinfos id=11 count=14811worker2 # [ 50.797733] nix-grpc-daemon[867]: time=2026-09-20T12:45:51.154Z level=INFO msg="Uploading 1 narinfos"4812lb # [ 50.766140] niks3-server[1107]: 2026/09/20 12:45:51 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4813lb # [ 50.776205] niks3-server[1107]: 2026/09/20 12:45:51 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4814lb # [ 50.782531] niks3-server[1107]: 2026/09/20 12:45:51 INFO Completed upload id=114815worker2 # [ 50.818085] nix-grpc-daemon[867]: time=2026-09-20T12:45:51.175Z level=INFO msg="Upload complete. (399ms)"4816lb # [ 50.785201] niks3-server[1107]: 2026/09/20 12:45:51 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4817worker2 # [ 50.821855] nix-grpc-daemon[786]: ts=2026-09-20T12:45:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=5124818worker2 # [ 50.827116] nix-grpc-daemon[786]: ts=2026-09-20T12:45:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144819worker2 # [ 50.835493] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4820worker2 # [ 50.840561] nix-daemon[855]: reaped child process 1078, status = succeeded4821worker2 # [ 50.845410] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4822worker2 # [ 51.870850] nix-daemon[855]: reaped child process 1092, status = succeeded4823worker2 # [ 52.004254] nix-daemon[855]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4824worker2 # [ 52.017400] nix-daemon[855]: reaped child process 1110, status = succeeded4825lb # [ 51.992250] niks3-server[1107]: 2026/09/20 12:45:53 INFO Received uploads request method=POST path=/api/pending_closures4826worker2 # [ 52.130436] nix-grpc-daemon[867]: time=2026-09-20T12:45:52.486Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4827worker2 # [ 52.140789] nix-grpc-daemon[867]: time=2026-09-20T12:45:52.491Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4828lb # [ 52.143444] niks3-server[1107]: 2026/09/20 12:45:53 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4829lb # [ 52.148498] niks3-server[1107]: 2026/09/20 12:45:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4830lb # [ 52.153949] niks3-server[1107]: 2026/09/20 12:45:53 INFO Signed narinfos id=12 count=14831worker2 # [ 52.193085] nix-grpc-daemon[867]: time=2026-09-20T12:45:52.549Z level=INFO msg="Uploading 1 narinfos"4832lb # [ 52.166293] niks3-server[1107]: 2026/09/20 12:45:53 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4833lb # [ 52.171091] niks3-server[1107]: 2026/09/20 12:45:53 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4834lb # [ 52.180039] niks3-server[1107]: 2026/09/20 12:45:53 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4835lb # [ 52.193587] niks3-server[1107]: 2026/09/20 12:45:53 INFO Completed upload id=124836worker2 # [ 52.232175] nix-grpc-daemon[867]: time=2026-09-20T12:45:52.589Z level=INFO msg="Upload complete. (378ms)"4837lb # [ 52.199519] niks3-server[1107]: 2026/09/20 12:45:53 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4838worker2 # [ 52.238810] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14839worker2 # [ 52.254371] nix-daemon[855]: reaped child process 1090, status = succeeded4840worker1 # [ 52.278167] nix-grpc-daemon[784]: ts=2026-09-20T12:45:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:447644841worker1 # [ 52.288180] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4842worker2 # [ 52.330738] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329144843worker2 # [ 52.335860] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_out=1284844worker2 # [ 52.522877] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329144845worker1 # [ 52.514426] nix-grpc-daemon[784]: ts=2026-09-20T12:45:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447644846worker1 # [ 52.523945] nix-grpc-daemon[868]: time=2026-09-20T12:45:52.831Z level=INFO msg="All 1 paths already cached"4847worker2 # [ 52.549224] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04848worker2 # [ 52.554610] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14849worker2 # [ 52.559465] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144850worker2 # [ 52.570280] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144851worker2 # [ 52.582270] nix-grpc-daemon[867]: time=2026-09-20T12:45:52.939Z level=INFO msg="All 1 paths already cached"4852worker2 # [ 52.588275] nix-grpc-daemon[786]: ts=2026-09-20T12:45:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329144853worker2 # [ 52.593072] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4854worker2 # [ 52.740245] nix-daemon[855]: accepted connection from pid 1116, user nix-grpc-daemon (trusted)4855worker2 # [ 52.754269] nix-daemon[855]: reaped child process 1119, status = succeeded4856worker2 # [ 52.907585] nix-daemon[855]: accepted connection from pid 1121, user nix-grpc-daemon (trusted)4857worker2 # [ 52.920100] nix-daemon[855]: reaped child process 1124, status = succeeded4858lb # [ 52.898297] niks3-server[1107]: 2026/09/20 12:45:54 INFO Received uploads request method=POST path=/api/pending_closures4859worker2 # [ 53.079395] nix-grpc-daemon[867]: time=2026-09-20T12:45:53.436Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4860worker2 # [ 53.083839] nix-grpc-daemon[867]: time=2026-09-20T12:45:53.440Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4861lb # [ 53.067379] niks3-server[1107]: 2026/09/20 12:45:54 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4862lb # [ 53.076956] niks3-server[1107]: 2026/09/20 12:45:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4863lb # [ 53.080946] niks3-server[1107]: 2026/09/20 12:45:54 INFO Signed narinfos id=13 count=14864worker2 # [ 53.120713] nix-grpc-daemon[867]: time=2026-09-20T12:45:53.477Z level=INFO msg="Uploading 1 narinfos"4865lb # [ 53.089716] niks3-server[1107]: 2026/09/20 12:45:54 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4866lb # [ 53.099728] niks3-server[1107]: 2026/09/20 12:45:54 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4867lb # [ 53.113581] niks3-server[1107]: 2026/09/20 12:45:54 INFO Completed upload id=134868worker2 # [ 53.149891] nix-grpc-daemon[867]: time=2026-09-20T12:45:53.506Z level=INFO msg="Upload complete. (550ms)"4869worker2 # [ 53.155280] nix-grpc-daemon[786]: ts=2026-09-20T12:45:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=6244870worker2 # [ 53.160979] nix-grpc-daemon[786]: ts=2026-09-20T12:45:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144871lb # [ 53.127536] niks3-server[1107]: 2026/09/20 12:45:54 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4872worker2 # [ 53.174192] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4873worker2 # [ 53.184541] nix-daemon[855]: reaped child process 1114, status = succeeded4874worker2 # [ 53.194689] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4875worker2 # [ 54.412279] nix-daemon[855]: reaped child process 1128, status = succeeded4876worker2 # [ 54.552356] nix-daemon[855]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4877worker2 # [ 54.564445] nix-daemon[855]: reaped child process 1146, status = succeeded4878lb # [ 54.545633] niks3-server[1107]: 2026/09/20 12:45:55 INFO Received uploads request method=POST path=/api/pending_closures4879worker2 # [ 54.716959] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.073Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4880worker2 # [ 54.723632] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.077Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4881lb # [ 54.723198] niks3-server[1107]: 2026/09/20 12:45:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4882lb # [ 54.733198] niks3-server[1107]: 2026/09/20 12:45:55 INFO Signed narinfos id=14 count=14883worker2 # [ 54.769434] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.126Z level=INFO msg="Uploading 1 narinfos"4884lb # [ 54.737956] niks3-server[1107]: 2026/09/20 12:45:55 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4885lb # [ 54.741814] niks3-server[1107]: 2026/09/20 12:45:55 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4886lb # [ 54.748275] niks3-server[1107]: 2026/09/20 12:45:55 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4887lb # [ 54.759723] niks3-server[1107]: 2026/09/20 12:45:55 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4888lb # [ 54.762936] niks3-server[1107]: 2026/09/20 12:45:55 INFO Completed upload id=144889lb # [ 54.764313] niks3-server[1107]: 2026/09/20 12:45:55 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4890worker2 # [ 54.798702] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.155Z level=INFO msg="Upload complete. (393ms)"4891worker2 # [ 54.808132] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14892worker2 # [ 54.815260] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329144893worker2 # [ 54.825839] nix-daemon[855]: reaped child process 1126, status = succeeded4894worker2 # [ 54.830169] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329144895worker2 # [ 54.833884] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_out=1764896worker2 # [ 55.100363] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329144897worker2 # [ 55.106979] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144898worker2 # [ 55.112122] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.468Z level=INFO msg="All 2 paths already cached"4899worker2 # [ 55.121324] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04900worker2 # [ 55.125916] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14901worker2 # [ 55.130960] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144902worker2 # [ 55.140521] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144903worker2 # [ 55.150348] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.507Z level=INFO msg="All 2 paths already cached"4904worker2 # [ 55.156459] nix-grpc-daemon[786]: ts=2026-09-20T12:45:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329144905worker2 # [ 55.164852] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4906worker2 # [ 55.331441] nix-daemon[855]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)4907worker2 # [ 55.343978] nix-daemon[855]: reaped child process 1155, status = succeeded4908worker2 # [ 55.486367] nix-daemon[855]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4909worker2 # [ 55.499737] nix-daemon[855]: reaped child process 1160, status = succeeded4910lb # [ 55.477586] niks3-server[1107]: 2026/09/20 12:45:56 INFO Received uploads request method=POST path=/api/pending_closures4911worker2 # [ 55.521538] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.878Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4912worker2 # [ 55.522985] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.878Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4913lb # [ 55.510705] niks3-server[1107]: 2026/09/20 12:45:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4914lb # [ 55.516422] niks3-server[1107]: 2026/09/20 12:45:56 INFO Signed narinfos id=15 count=14915worker2 # [ 55.555154] nix-grpc-daemon[867]: time=2026-09-20T12:45:55.912Z level=INFO msg="Uploading 1 narinfos"4916lb # [ 55.528251] niks3-server[1107]: 2026/09/20 12:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4917lb # [ 55.614439] niks3-server[1107]: 2026/09/20 12:45:56 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4918lb # [ 55.621154] niks3-server[1107]: 2026/09/20 12:45:56 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4919worker2 # [ 55.664111] nix-grpc-daemon[867]: time=2026-09-20T12:45:56.021Z level=INFO msg="Upload complete. (487ms)"4920lb # [ 55.628543] niks3-server[1107]: 2026/09/20 12:45:56 INFO Completed upload id=154921worker2 # [ 55.666365] nix-grpc-daemon[786]: ts=2026-09-20T12:45:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=7524922lb # [ 55.632582] niks3-server[1107]: 2026/09/20 12:45:56 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4923worker2 # [ 55.673323] nix-grpc-daemon[786]: ts=2026-09-20T12:45:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329144924worker2 # [ 55.675340] nix-daemon[855]: reaped child process 1150, status = succeeded4925worker2 # [ 55.679256] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4926worker2 # [ 55.686487] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4927worker2 # [ 56.917905] nix-daemon[855]: reaped child process 1164, status = succeeded4928worker2 # [ 57.066462] nix-daemon[855]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)4929worker2 # [ 57.079532] nix-daemon[855]: reaped child process 1182, status = succeeded4930lb # [ 57.057992] niks3-server[1107]: 2026/09/20 12:45:58 INFO Received uploads request method=POST path=/api/pending_closures4931worker2 # [ 57.481634] nix-grpc-daemon[867]: time=2026-09-20T12:45:57.837Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4932worker2 # [ 57.494654] nix-grpc-daemon[867]: time=2026-09-20T12:45:57.837Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4933lb # [ 57.490301] niks3-server[1107]: 2026/09/20 12:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4934lb # [ 57.493774] niks3-server[1107]: 2026/09/20 12:45:58 INFO Signed narinfos id=16 count=14935worker2 # [ 57.530059] nix-grpc-daemon[867]: time=2026-09-20T12:45:57.886Z level=INFO msg="Uploading 1 narinfos"4936lb # [ 57.499245] niks3-server[1107]: 2026/09/20 12:45:58 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4937lb # [ 57.504693] niks3-server[1107]: 2026/09/20 12:45:58 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4938lb # [ 57.528984] niks3-server[1107]: 2026/09/20 12:45:58 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4939lb # [ 57.547650] niks3-server[1107]: 2026/09/20 12:45:58 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4940lb # [ 57.568490] niks3-server[1107]: 2026/09/20 12:45:58 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4941worker2 # [ 57.616035] nix-grpc-daemon[867]: time=2026-09-20T12:45:57.973Z level=INFO msg="Upload complete. (718ms)"4942lb # [ 57.580438] niks3-server[1107]: 2026/09/20 12:45:58 INFO Completed upload id=164943worker2 # [ 57.618567] nix-grpc-daemon[786]: ts=2026-09-20T12:45:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14944worker2 # [ 57.626047] nix-daemon[855]: reaped child process 1162, status = succeeded4945worker1 # [ 57.653937] nix-grpc-daemon[784]: ts=2026-09-20T12:45:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:447644946worker1 # [ 57.667617] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4947worker1 # [ 57.675910] nix-grpc-daemon[784]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4948worker1 # [ 57.695458] nix-grpc-daemon[784]: ts=2026-09-20T12:45:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:447644949worker1 # [ 57.698538] nix-grpc-daemon[784]: ts=2026-09-20T12:45:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=0 paths=1 nar_bytes_out=2324950client # [ 57.788633] nix-daemon[945]: reaped child process 947, status = succeeded4951client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.04 seconds)4952client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4953client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4954client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4955client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)4956client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4957client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4958(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.12 seconds)4959subtest: an input only one worker has still reaches the builder4960client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4961client # this derivation will be built:4962client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4963client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4964client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.37 seconds)4965client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4966lb # [ 58.359128] envoy[985]: conn peer=[2001:db8:1::1]:43588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1345 ms=46384967lb # [ 58.362878] envoy[985]: conn peer=[2001:db8:1::1]:43602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1085 ms=35724968lb # [ 58.366344] envoy[985]: conn peer=[2001:db8:1::1]:43626 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=333 ms=174969lb # [ 58.369699] envoy[985]: conn peer=[2001:db8:1::1]:43614 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3413 tx=1751 ms=20554970lb # [ 58.373173] envoy[985]: conn peer=[2001:db8:1::1]:34120 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=534971lb # [ 58.376618] envoy[985]: conn peer=[2001:db8:1::1]:43628 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3880 tx=1880 ms=23594972lb # [ 58.380816] envoy[985]: conn peer=[2001:db8:1::1]:34130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2199 ms=26604973lb # [ 58.384341] envoy[985]: conn peer=[2001:db8:1::1]:34132 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=534974client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.16 seconds)4975worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4976worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.18 seconds)4977worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4978worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4979client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24980client # [ 58.749629] nix-daemon[945]: accepted connection from pid 1024, user root (trusted)4981client # this derivation will be built:4982client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4983worker1 # [ 58.920932] nix-grpc-daemon[784]: ts=2026-09-20T12:45:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:447644984worker2 # [ 58.935044] nix-grpc-daemon[786]: ts=2026-09-20T12:45:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329144985client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...4986client # copying 1 paths...4987client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4988worker2 # [ 58.943642] nix-grpc-daemon[786]: ts=2026-09-20T12:45:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329144989worker2 # [ 58.949136] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4990worker2 # [ 59.099720] nix-daemon[855]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)4991worker2 # [ 59.112924] nix-daemon[855]: reaped child process 1192, status = succeeded4992lb # [ 59.088971] niks3-server[1107]: 2026/09/20 12:46:00 INFO Received uploads request method=POST path=/api/pending_closures4993worker2 # [ 59.133405] nix-grpc-daemon[867]: time=2026-09-20T12:45:59.490Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4994worker2 # [ 59.135241] nix-grpc-daemon[867]: time=2026-09-20T12:45:59.492Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4995lb # [ 59.117554] niks3-server[1107]: 2026/09/20 12:46:00 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4996lb # [ 59.130695] niks3-server[1107]: 2026/09/20 12:46:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4997worker2 # [ 59.173828] nix-grpc-daemon[867]: time=2026-09-20T12:45:59.530Z level=INFO msg="Uploading 1 narinfos"4998lb # [ 59.137799] niks3-server[1107]: 2026/09/20 12:46:00 INFO Signed narinfos id=17 count=14999lb # [ 59.141754] niks3-server[1107]: 2026/09/20 12:46:00 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5000lb # [ 59.151117] niks3-server[1107]: 2026/09/20 12:46:00 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5001lb # [ 59.158881] niks3-server[1107]: 2026/09/20 12:46:00 INFO Completed upload id=175002worker2 # [ 59.194792] nix-grpc-daemon[867]: time=2026-09-20T12:45:59.551Z level=INFO msg="Upload complete. (241ms)"5003lb # [ 59.160679] niks3-server[1107]: 2026/09/20 12:46:00 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5004worker2 # [ 59.196909] nix-grpc-daemon[786]: ts=2026-09-20T12:45:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=1205005worker2 # [ 59.204528] nix-daemon[855]: reaped child process 1187, status = succeeded5006worker2 # [ 59.207350] nix-grpc-daemon[786]: ts=2026-09-20T12:45:59Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05007worker2 # [ 59.209160] nix-grpc-daemon[786]: ts=2026-09-20T12:45:59Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15008worker1 # [ 59.244790] nix-grpc-daemon[784]: ts=2026-09-20T12:45:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447645009worker1 # [ 59.254670] nix-grpc-daemon[784]: ts=2026-09-20T12:45:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447645010client # copying 2 paths to 'lb:50051'...5011worker1 # [ 59.268942] nix-grpc-daemon[784]: ts=2026-09-20T12:45:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447645012client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5013client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5014worker1 # [ 59.282756] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5015worker1 # [ 59.454799] nix-daemon[841]: accepted connection from pid 946, user nix-grpc-daemon (trusted)5016worker1 # [ 59.476657] nix-daemon[841]: reaped child process 949, status = succeeded5017worker1 # [ 59.620075] nix-daemon[841]: accepted connection from pid 951, user nix-grpc-daemon (trusted)5018worker1 # [ 59.636743] nix-daemon[841]: reaped child process 954, status = succeeded5019lb # [ 59.624427] niks3-server[1107]: 2026/09/20 12:46:00 INFO Received uploads request method=POST path=/api/pending_closures5020lb # [ 59.980376] niks3-server[1107]: 2026/09/20 12:46:01 INFO Received uploads request method=POST path=/api/pending_closures5021worker1 # [ 60.055903] nix-grpc-daemon[868]: time=2026-09-20T12:46:00.362Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5022worker1 # [ 60.057375] nix-grpc-daemon[868]: time=2026-09-20T12:46:00.362Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5023worker1 # [ 60.059218] nix-grpc-daemon[868]: time=2026-09-20T12:46:00.363Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5024lb # [ 60.062839] niks3-server[1107]: 2026/09/20 12:46:01 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5025lb # [ 60.080841] niks3-server[1107]: 2026/09/20 12:46:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5026lb # [ 60.085239] niks3-server[1107]: 2026/09/20 12:46:01 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5027lb # [ 60.092808] niks3-server[1107]: 2026/09/20 12:46:01 INFO Signed narinfos id=19 count=25028lb # [ 60.096505] niks3-server[1107]: 2026/09/20 12:46:01 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5029lb # [ 60.099391] niks3-server[1107]: 2026/09/20 12:46:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5030lb # [ 60.102049] niks3-server[1107]: 2026/09/20 12:46:01 INFO Signed narinfos id=18 count=15031worker1 # [ 60.127076] nix-grpc-daemon[868]: time=2026-09-20T12:46:00.434Z level=INFO msg="Uploading 3 narinfos"5032lb # [ 60.105332] niks3-server[1107]: 2026/09/20 12:46:01 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5033lb # [ 60.118168] niks3-server[1107]: 2026/09/20 12:46:01 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5034lb # [ 60.121039] niks3-server[1107]: 2026/09/20 12:46:01 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5035lb # [ 60.125394] niks3-server[1107]: 2026/09/20 12:46:01 INFO Completed upload id=185036lb # [ 60.127495] niks3-server[1107]: 2026/09/20 12:46:01 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5037lb # [ 60.129391] niks3-server[1107]: 2026/09/20 12:46:01 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5038lb # [ 60.131710] niks3-server[1107]: 2026/09/20 12:46:01 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5039lb # [ 60.133844] niks3-server[1107]: 2026/09/20 12:46:01 INFO Completed upload id=195040worker1 # [ 60.158587] nix-grpc-daemon[868]: time=2026-09-20T12:46:00.466Z level=INFO msg="Upload complete. (867ms)"5041worker1 # [ 60.164631] nix-grpc-daemon[784]: ts=2026-09-20T12:46:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=0 paths=2 nar_bytes_in=9765042worker1 # [ 60.171588] nix-grpc-daemon[784]: ts=2026-09-20T12:46:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447645043worker1 # [ 60.180337] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5044worker1 # [ 60.187470] nix-daemon[841]: reaped child process 944, status = succeeded5045worker1 # [ 60.192206] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5046client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5047client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5048worker1 # [ 61.406496] nix-daemon[841]: reaped child process 958, status = succeeded5049worker1 # [ 61.576336] nix-daemon[841]: accepted connection from pid 973, user nix-grpc-daemon (trusted)5050worker1 # [ 61.590292] nix-daemon[841]: reaped child process 976, status = succeeded5051lb # [ 61.579200] niks3-server[1107]: 2026/09/20 12:46:02 INFO Received uploads request method=POST path=/api/pending_closures5052worker1 # [ 61.611609] nix-grpc-daemon[868]: time=2026-09-20T12:46:01.919Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5053lb # [ 61.607676] niks3-server[1107]: 2026/09/20 12:46:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5054worker1 # [ 61.638280] nix-grpc-daemon[868]: time=2026-09-20T12:46:01.946Z level=INFO msg="Uploading 1 narinfos"5055lb # [ 61.613831] niks3-server[1107]: 2026/09/20 12:46:02 INFO Signed narinfos id=20 count=15056lb # [ 61.623710] niks3-server[1107]: 2026/09/20 12:46:02 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5057lb # [ 61.634225] niks3-server[1107]: 2026/09/20 12:46:02 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5058lb # [ 61.643178] niks3-server[1107]: 2026/09/20 12:46:02 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5059lb # [ 61.649353] niks3-server[1107]: 2026/09/20 12:46:02 INFO Completed upload id=205060worker1 # [ 61.673619] nix-grpc-daemon[868]: time=2026-09-20T12:46:01.981Z level=WARN msg="Failed to register uploaded object" key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo error="server returned 404: object is not pending upload\n"5061worker1 # [ 61.681265] nix-grpc-daemon[868]: time=2026-09-20T12:46:01.986Z level=INFO msg="Upload complete. (279ms)"5062worker1 # [ 61.684227] nix-grpc-daemon[784]: ts=2026-09-20T12:46:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15063worker1 # [ 61.694412] nix-daemon[841]: reaped child process 956, status = succeeded5064client # copying 1 paths...5065worker1 # [ 61.726165] nix-grpc-daemon[784]: ts=2026-09-20T12:46:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:447645066client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5067worker1 # [ 61.746192] nix-grpc-daemon[784]: ts=2026-09-20T12:46:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:447645068worker1 # [ 61.750857] nix-grpc-daemon[784]: ts=2026-09-20T12:46:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=0 paths=1 nar_bytes_out=1205069client # [ 61.848155] nix-daemon[945]: reaped child process 1027, status = succeeded5070client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.25 seconds)5071client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25072client # [ 62.001831] nix-daemon[945]: accepted connection from pid 1045, user root (trusted)5073client # this derivation will be built:5074client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5075client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5076worker2 # [ 62.184511] nix-grpc-daemon[786]: ts=2026-09-20T12:46:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329145077worker2 # [ 62.190290] nix-grpc-daemon[786]: ts=2026-09-20T12:46:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329145078worker2 # [ 62.195648] nix-grpc-daemon[867]: time=2026-09-20T12:46:02.551Z level=INFO msg="All 1 paths already cached"5079worker2 # [ 62.205167] nix-grpc-daemon[786]: ts=2026-09-20T12:46:02Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05080worker2 # [ 62.210733] nix-grpc-daemon[786]: ts=2026-09-20T12:46:02Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15081worker2 # [ 62.216300] nix-grpc-daemon[786]: ts=2026-09-20T12:46:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329145082worker2 # [ 62.227812] nix-grpc-daemon[786]: ts=2026-09-20T12:46:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329145083client # copying 1 paths to 'lb:50051'...5084client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5085worker2 # [ 62.244920] nix-grpc-daemon[786]: ts=2026-09-20T12:46:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329145086worker2 # [ 62.253794] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5087worker2 # [ 62.269951] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)5088worker2 # [ 62.362882] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 2/5)5089worker2 # [ 62.510145] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 319 ms (attempt 3/5)5090worker2 # [ 62.835340] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 4/5)5091worker2 # [ 62.849854] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5092worker2 # [ 62.855877] nix-grpc-daemon[786]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5093worker2 # [ 63.032510] nix-daemon[855]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5094worker2 # [ 63.047105] nix-daemon[855]: reaped child process 1207, status = succeeded5095worker2 # [ 63.186506] nix-daemon[855]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5096worker2 # [ 63.200058] nix-daemon[855]: reaped child process 1212, status = succeeded5097lb # [ 63.180687] niks3-server[1107]: 2026/09/20 12:46:04 INFO Received uploads request method=POST path=/api/pending_closures5098worker2 # [ 63.303417] nix-grpc-daemon[867]: time=2026-09-20T12:46:03.659Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5099worker2 # [ 63.306858] nix-grpc-daemon[867]: time=2026-09-20T12:46:03.663Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5100lb # [ 63.299072] niks3-server[1107]: 2026/09/20 12:46:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5101lb # [ 63.303550] niks3-server[1107]: 2026/09/20 12:46:04 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5102lb # [ 63.308217] niks3-server[1107]: 2026/09/20 12:46:04 INFO Signed narinfos id=21 count=15103worker2 # [ 63.347056] nix-grpc-daemon[867]: time=2026-09-20T12:46:03.703Z level=INFO msg="Uploading 1 narinfos"5104lb # [ 63.322248] niks3-server[1107]: 2026/09/20 12:46:04 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5105lb # [ 63.326911] niks3-server[1107]: 2026/09/20 12:46:04 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5106lb # [ 63.332660] niks3-server[1107]: 2026/09/20 12:46:04 INFO Completed upload id=215107worker2 # [ 63.367978] nix-grpc-daemon[867]: time=2026-09-20T12:46:03.725Z level=INFO msg="Upload complete. (493ms)"5108lb # [ 63.336136] niks3-server[1107]: 2026/09/20 12:46:04 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5109worker2 # [ 63.373703] nix-grpc-daemon[786]: ts=2026-09-20T12:46:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=1 paths=1 nar_bytes_in=5605110worker2 # [ 63.380333] nix-grpc-daemon[786]: ts=2026-09-20T12:46:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329145111worker2 # [ 63.397272] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5112worker2 # [ 63.406238] nix-daemon[855]: reaped child process 1198, status = succeeded5113client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5114worker2 # [ 63.412160] nix-daemon[855]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5115client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5116worker2 # [ 64.646982] nix-daemon[855]: reaped child process 1216, status = succeeded5117worker2 # [ 64.799615] nix-daemon[855]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5118worker2 # [ 64.812070] nix-daemon[855]: reaped child process 1234, status = succeeded5119lb # [ 64.785421] niks3-server[1107]: 2026/09/20 12:46:05 INFO Received uploads request method=POST path=/api/pending_closures5120worker2 # [ 64.831199] nix-grpc-daemon[867]: time=2026-09-20T12:46:05.188Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5121lb # [ 64.816444] niks3-server[1107]: 2026/09/20 12:46:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5122worker2 # [ 64.858865] nix-grpc-daemon[867]: time=2026-09-20T12:46:05.215Z level=INFO msg="Uploading 1 narinfos"5123lb # [ 64.824116] niks3-server[1107]: 2026/09/20 12:46:05 INFO Signed narinfos id=22 count=15124lb # [ 64.828645] niks3-server[1107]: 2026/09/20 12:46:05 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5125lb # [ 64.835896] niks3-server[1107]: 2026/09/20 12:46:05 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5126lb # [ 64.843852] niks3-server[1107]: 2026/09/20 12:46:05 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5127lb # [ 64.846239] niks3-server[1107]: 2026/09/20 12:46:05 INFO Completed upload id=225128lb # [ 64.848790] niks3-server[1107]: 2026/09/20 12:46:05 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5129worker2 # [ 64.883456] nix-grpc-daemon[867]: time=2026-09-20T12:46:05.240Z level=INFO msg="Upload complete. (256ms)"5130worker2 # [ 64.886863] nix-grpc-daemon[786]: ts=2026-09-20T12:46:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15131client # copying 1 paths...5132worker2 # [ 64.891938] nix-grpc-daemon[786]: ts=2026-09-20T12:46:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329145133client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5134worker2 # [ 64.896322] nix-daemon[855]: reaped child process 1214, status = succeeded5135worker1 # [ 64.897371] nix-grpc-daemon[784]: ts=2026-09-20T12:46:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:447645136worker1 # [ 64.902709] nix-grpc-daemon[784]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5137worker1 # [ 64.909505] nix-grpc-daemon[784]: ts=2026-09-20T12:46:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=0 paths=1 nar_bytes_out=1205138client # [ 65.015338] nix-daemon[945]: reaped child process 1048, status = succeeded5139client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.17 seconds)5140client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25141client # [ 65.188915] nix-daemon[945]: accepted connection from pid 1066, user root (trusted)5142client # this derivation will be built:5143client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5144worker2 # [ 65.348792] nix-grpc-daemon[786]: ts=2026-09-20T12:46:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329145145client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5146worker1 # [ 65.343130] nix-grpc-daemon[784]: ts=2026-09-20T12:46:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447645147worker1 # [ 65.353410] nix-grpc-daemon[868]: time=2026-09-20T12:46:05.660Z level=INFO msg="All 1 paths already cached"5148worker2 # [ 65.379590] nix-grpc-daemon[786]: ts=2026-09-20T12:46:05Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05149worker2 # [ 65.384413] nix-grpc-daemon[786]: ts=2026-09-20T12:46:05Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15150worker1 # [ 65.412349] nix-grpc-daemon[784]: ts=2026-09-20T12:46:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447645151worker1 # [ 65.421160] nix-grpc-daemon[784]: ts=2026-09-20T12:46:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447645152worker1 # [ 65.429379] nix-grpc-daemon[868]: time=2026-09-20T12:46:05.737Z level=INFO msg="All 1 paths already cached"5153client # copying 1 paths to 'lb:50051'...5154worker1 # [ 65.435197] nix-grpc-daemon[784]: ts=2026-09-20T12:46:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447645155client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5156worker1 # [ 65.442989] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5157worker1 # [ 65.594243] nix-daemon[841]: accepted connection from pid 981, user nix-grpc-daemon (trusted)5158worker1 # [ 65.607294] nix-daemon[841]: reaped child process 984, status = succeeded5159worker1 # [ 65.754068] nix-daemon[841]: accepted connection from pid 986, user nix-grpc-daemon (trusted)5160worker1 # [ 65.766090] nix-daemon[841]: reaped child process 989, status = succeeded5161lb # [ 65.757759] niks3-server[1107]: 2026/09/20 12:46:06 INFO Received uploads request method=POST path=/api/pending_closures5162worker1 # [ 65.790350] nix-grpc-daemon[868]: time=2026-09-20T12:46:06.098Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5163worker1 # [ 65.791673] nix-grpc-daemon[868]: time=2026-09-20T12:46:06.098Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5164lb # [ 65.781333] niks3-server[1107]: 2026/09/20 12:46:06 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5165lb # [ 65.793230] niks3-server[1107]: 2026/09/20 12:46:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5166worker1 # [ 65.822335] nix-grpc-daemon[868]: time=2026-09-20T12:46:06.130Z level=INFO msg="Uploading 1 narinfos"5167lb # [ 65.797848] niks3-server[1107]: 2026/09/20 12:46:06 INFO Signed narinfos id=23 count=15168lb # [ 65.811264] niks3-server[1107]: 2026/09/20 12:46:06 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5169lb # [ 65.816967] niks3-server[1107]: 2026/09/20 12:46:06 INFO Completed upload id=235170worker1 # [ 65.843922] nix-grpc-daemon[868]: time=2026-09-20T12:46:06.152Z level=INFO msg="Upload complete. (395ms)"5171lb # [ 65.820250] niks3-server[1107]: 2026/09/20 12:46:06 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5172worker1 # [ 65.847123] nix-grpc-daemon[784]: ts=2026-09-20T12:46:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=0 paths=1 nar_bytes_in=5605173lb # [ 65.825509] niks3-server[1107]: 2026/09/20 12:46:06 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5174worker1 # [ 65.852598] nix-daemon[841]: reaped child process 979, status = succeeded5175worker1 # [ 65.854113] nix-grpc-daemon[784]: ts=2026-09-20T12:46:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447645176worker1 # [ 65.857889] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5177worker1 # [ 65.865814] nix-daemon[841]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5178client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5179client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5180worker1 # [ 66.491653] nix-daemon[841]: reaped child process 993, status = succeeded5181worker1 # [ 66.641547] nix-daemon[841]: accepted connection from pid 1008, user nix-grpc-daemon (trusted)5182worker1 # [ 66.656264] nix-daemon[841]: reaped child process 1011, status = succeeded5183lb # [ 66.645368] niks3-server[1107]: 2026/09/20 12:46:07 INFO Received uploads request method=POST path=/api/pending_closures5184worker1 # [ 66.681318] nix-grpc-daemon[868]: time=2026-09-20T12:46:06.987Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5185lb # [ 66.677701] niks3-server[1107]: 2026/09/20 12:46:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5186lb # [ 66.682970] niks3-server[1107]: 2026/09/20 12:46:07 INFO Signed narinfos id=24 count=15187worker1 # [ 66.710848] nix-grpc-daemon[868]: time=2026-09-20T12:46:07.018Z level=INFO msg="Uploading 1 narinfos"5188lb # [ 66.692886] niks3-server[1107]: 2026/09/20 12:46:07 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5189lb # [ 66.697739] niks3-server[1107]: 2026/09/20 12:46:07 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5190lb # [ 66.710581] niks3-server[1107]: 2026/09/20 12:46:07 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5191worker1 # [ 66.749570] nix-grpc-daemon[868]: time=2026-09-20T12:46:07.057Z level=INFO msg="Upload complete. (276ms)"5192lb # [ 66.725297] niks3-server[1107]: 2026/09/20 12:46:07 INFO Completed upload id=245193worker1 # [ 66.753344] nix-grpc-daemon[784]: ts=2026-09-20T12:46:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44764 duration_s=0 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15194lb # [ 66.732694] niks3-server[1107]: 2026/09/20 12:46:07 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5195worker1 # [ 66.759316] nix-daemon[841]: reaped child process 991, status = succeeded5196client # copying 1 paths...5197worker1 # [ 66.798857] nix-grpc-daemon[784]: ts=2026-09-20T12:46:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:447645198client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5199worker2 # [ 66.829991] nix-grpc-daemon[786]: ts=2026-09-20T12:46:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329145200worker2 # [ 66.837600] nix-grpc-daemon[786]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5201worker2 # [ 66.846441] nix-grpc-daemon[786]: ts=2026-09-20T12:46:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_out=1205202client # [ 66.938545] nix-daemon[945]: reaped child process 1069, status = succeeded5203client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.93 seconds)5204(finished: subtest: an input only one worker has still reaches the builder, in 9.07 seconds)5205subtest: upload whose reference the worker lacks is completed from the cache5206client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5207client # this derivation will be built:5208client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5209client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5210client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.37 seconds)5211client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5212client # this derivation will be built:5213client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5214client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5215client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5216client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5217client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.17 seconds)5218worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5219worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.22 seconds)5220worker2 # [ 68.155576] systemd[1]: Stopping Nix worker-protocol over gRPC...5221lb # [ 68.359966] envoy[985]: conn peer=[2001:db8:1::1]:34144 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=515222lb # [ 68.363750] envoy[985]: conn peer=[2001:db8:1::1]:34134 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4223 tx=1765 ms=28825223lb # [ 68.367419] envoy[985]: conn peer=[2001:db8:1::1]:34146 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3700 tx=1805 ms=27895224lb # [ 68.370950] envoy[985]: conn peer=[2001:db8:1::1]:45708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=655225lb # [ 68.374500] envoy[985]: conn peer=[2001:db8:1::1]:45714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3714 tx=1774 ms=15485226lb # [ 68.378096] envoy[985]: conn peer=[2001:db8:1::1]:45722 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=695227worker2 # [ 68.773060] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=signal_exit5228worker2 # [ 69.018456] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=scheduler_yield5229worker1 # [ 69.010821] nix-grpc-daemon[784]: ts=2026-09-20T12:46:09Z level=info event=scheduler_restarting addr=lb:500515230lb # [ 68.986882] niks3-server[1107]: 2026/09/20 12:46:10 INFO lead: released remote=192.168.1.4:462185231worker2 # [ 69.027887] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:329105232worker2 # [ 69.031652] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=worker_gone addr=192.168.1.3:500515233worker2 # [ 69.035927] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=scheduler_restarting addr=lb:500515234worker2 # [ 69.038972] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:329105235worker2 # [ 69.042326] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=worker_gone addr=192.168.1.4:500515236worker1 # [ 69.210643] nix-grpc-daemon[784]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500515237worker1 # [ 69.214578] nix-grpc-daemon[784]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5238worker2 # [ 69.233114] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500515239worker2 # [ 69.237196] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5240worker1 # [ 69.415398] nix-grpc-daemon[784]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500515241worker1 # [ 69.419364] nix-grpc-daemon[784]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5242worker2 # [ 69.437940] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500515243worker2 # [ 69.441786] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5244worker1 # [ 69.619691] nix-grpc-daemon[784]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500515245worker1 # [ 69.621880] nix-grpc-daemon[784]: ts=2026-09-20T12:46:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5246worker2 # [ 69.642905] nix-grpc-daemon[786]: ts=2026-09-20T12:46:09Z level=info event=scheduler_connected addr=lb:500515247worker2 # [ 69.646936] nix-grpc-daemon[786]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5248worker1 # [ 69.822773] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500515249worker1 # [ 69.826742] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5250worker2 # [ 69.849582] nix-grpc-daemon[786]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500515251worker2 # [ 69.852778] nix-grpc-daemon[786]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5252worker2 # [ 70.022854] nix-grpc-daemon[786]: ts=2026-09-20T12:46:10Z level=info event=scheduler_restarting5253worker1 # [ 70.028503] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500515254worker2 # [ 70.042603] nix-daemon[855]: reaped child process 857, status = succeeded5255worker1 # [ 70.032742] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5256worker2 # [ 70.055225] nix-grpc-daemon[786]: ts=2026-09-20T12:46:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5257worker1 # [ 70.233245] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500515258worker1 # [ 70.237152] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5259worker1 # [ 70.438073] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500515260worker1 # [ 70.442921] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5261worker1 # [ 70.643679] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_connected addr=lb:500515262worker1 # [ 70.648157] nix-grpc-daemon[784]: ts=2026-09-20T12:46:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5263worker1 # [ 70.849078] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500515264worker1 # [ 70.853337] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5265worker2 # [ 70.879365] nix-grpc-daemon[786]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5266worker2 # [ 70.882963] nix-grpc-daemon[786]: E0000 00:00:1789908371.234918 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5267worker2 # [ 70.887808] nix-grpc-daemon[786]: E0000 00:00:1789908371.239809 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5268worker1 # [ 71.054674] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500515269worker1 # [ 71.058738] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5270worker1 # [ 71.259628] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500515271worker1 # [ 71.263647] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5272worker1 # [ 71.464665] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500515273worker1 # [ 71.468603] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5274worker1 # [ 71.669501] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_connected addr=lb:500515275worker1 # [ 71.673708] nix-grpc-daemon[784]: ts=2026-09-20T12:46:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5276worker2 # [ 71.833696] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5277worker2 # [ 71.837887] systemd[1]: Stopped Nix worker-protocol over gRPC.5278worker2 # [ 71.842564] systemd[1]: nix-grpc-daemon.service: Consumed 2.654s CPU time over 53.316s wall clock time, 60.8M memory peak, 95.3K incoming IP traffic, 218.1K outgoing IP traffic.5279worker2 # [ 71.855264] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5280worker2 # [ 71.859572] systemd[1]: Closed Nix worker-protocol over gRPC.5281worker2 # [ 71.863265] systemd[1]: nix-grpc-daemon.socket: Consumed 303us CPU time over 1min 2.810s wall clock time, 256K memory peak, 90.9K incoming IP traffic, 71.6K outgoing IP traffic.5282lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5283lb: must succeed: curl -sf localhost:9901/clusters5284worker1 # [ 71.874198] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500515285worker1 # [ 71.876310] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5286lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5287lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5288client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25289worker1 # [ 72.078287] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500515290worker1 # [ 72.082221] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5291worker1 # [ 72.124959] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447645292worker1 # [ 72.132144] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:447645293client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5294client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.27 seconds)5295worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5296worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5297worker2 # [ 72.270807] systemd[1]: Listening on Nix worker-protocol over gRPC.5298worker1 # [ 72.282775] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500515299worker1 # [ 72.284725] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5300worker1 # [ 72.300298] systemd[1]: Stopping Nix worker-protocol over gRPC...5301worker1 # [ 72.485252] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500515302worker1 # [ 72.487118] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303worker1 # [ 72.648678] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=signal_exit5304worker1 # [ 72.687861] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_connected addr=lb:500515305worker1 # [ 72.691641] nix-grpc-daemon[784]: ts=2026-09-20T12:46:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5306worker2 # [ 72.815613] systemd[1]: Starting scheduler-token.service...5307worker2 # [ 72.867565] systemd[1]: scheduler-token.service: Deactivated successfully.5308worker2 # [ 72.869348] systemd[1]: Finished scheduler-token.service.5309worker2 # [ 72.870571] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 53ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5310worker2 # [ 72.876509] systemd[1]: Starting Nix worker-protocol over gRPC...5311worker1 # [ 72.892650] nix-grpc-daemon[784]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500515312worker1 # [ 72.897287] nix-grpc-daemon[784]: ts=2026-09-20T12:46:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker2 # [ 73.007225] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5314worker2 # [ 73.009233] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5315worker2 # [ 73.013413] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15316worker2 # [ 73.019846] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5317worker2 # [ 73.021393] systemd[1]: Started Nix worker-protocol over gRPC.5318worker2 # [ 73.031934] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=scheduler_take_over5319lb # [ 72.998169] niks3-server[1107]: 2026/09/20 12:46:14 INFO lead: acquired remote=192.168.1.4:501365320worker2 # [ 73.044725] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500515321worker2 # [ 73.051817] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=oidc_jwks provider=mock keys=15322worker2 # [ 73.053172] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:430305323worker2 # [ 73.054847] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05324worker2 # [ 73.110324] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:430305325worker1 # [ 73.098754] nix-grpc-daemon[784]: ts=2026-09-20T12:46:13Z level=info event=scheduler_connected addr=lb:500515326worker2 # [ 73.112358] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05327worker1 # [ 73.897914] nix-grpc-daemon[784]: ts=2026-09-20T12:46:14Z level=info event=scheduler_restarting5328worker2 # [ 73.914994] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:430305329worker1 # [ 73.907862] nix-grpc-daemon[784]: ts=2026-09-20T12:46:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5330worker2 # [ 73.919077] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:14Z level=info event=worker_gone addr=192.168.1.3:500515331worker1 # [ 73.917337] nix-daemon[841]: reaped child process 844, status = succeeded5332worker1 # [ 74.757261] nix-grpc-daemon[784]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5333worker1 # [ 74.760610] nix-grpc-daemon[784]: E0000 00:00:1789908375.063646 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5334worker1 # [ 74.765450] nix-grpc-daemon[784]: E0000 00:00:1789908375.064570 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5335worker1 # [ 75.759503] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5336worker1 # [ 75.762722] systemd[1]: Stopped Nix worker-protocol over gRPC.5337worker1 # [ 75.767360] systemd[1]: nix-grpc-daemon.service: Consumed 1.336s CPU time over 57.385s wall clock time, 47.9M memory peak, 51K incoming IP traffic, 93.5K outgoing IP traffic.5338worker1 # [ 75.778276] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5339worker1 # [ 75.782691] systemd[1]: Closed Nix worker-protocol over gRPC.5340worker1 # [ 75.785549] systemd[1]: nix-grpc-daemon.socket: Consumed 295us CPU time over 1min 5.639s wall clock time, 256K memory peak, 58.8K incoming IP traffic, 52.7K outgoing IP traffic.5341lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5342lb: must succeed: curl -sf localhost:9901/clusters5343lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5344lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5345client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25346worker2 # [ 76.063450] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:430445347worker2 # [ 76.070273] nix-daemon[855]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5348worker2 # [ 76.080145] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)5349worker2 # [ 76.109138] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 133 ms (attempt 2/5)5350worker2 # [ 76.245082] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 305 ms (attempt 3/5)5351worker2 # [ 76.553244] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 609 ms (attempt 4/5)5352worker2 # [ 77.163812] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5353client # copying 2 paths...5354client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5355worker2 # [ 77.186361] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:430445356client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5357worker2 # [ 77.199330] nix-daemon[855]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5358worker2 # [ 77.220454] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:17Z level=info event=niks3_push_started pid=13175359worker2 # [ 77.384410] nix-grpc-daemon[1317]: time=2026-09-20T12:46:17.740Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5360worker2 # [ 77.525181] nix-daemon[855]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)5361worker2 # [ 77.537643] nix-daemon[855]: reaped child process 1333, status = succeeded5362lb # [ 77.516635] niks3-server[1107]: 2026/09/20 12:46:18 INFO Received uploads request method=POST path=/api/pending_closures5363lb # [ 77.521704] niks3-server[1107]: 2026/09/20 12:46:18 INFO Received uploads request method=POST path=/api/pending_closures5364worker2 # [ 77.561225] nix-grpc-daemon[1317]: time=2026-09-20T12:46:17.918Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5365worker2 # [ 77.562877] nix-grpc-daemon[1317]: time=2026-09-20T12:46:17.919Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5366worker2 # [ 77.564976] nix-grpc-daemon[1317]: time=2026-09-20T12:46:17.921Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5367lb # [ 77.557903] niks3-server[1107]: 2026/09/20 12:46:18 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5368lb # [ 77.564994] niks3-server[1107]: 2026/09/20 12:46:18 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5369lb # [ 77.578671] niks3-server[1107]: 2026/09/20 12:46:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5370lb # [ 77.583057] niks3-server[1107]: 2026/09/20 12:46:18 INFO Signed narinfos id=25 count=15371lb # [ 77.584457] niks3-server[1107]: 2026/09/20 12:46:18 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5372lb # [ 77.588167] niks3-server[1107]: 2026/09/20 12:46:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5373worker2 # [ 77.624573] nix-grpc-daemon[1317]: time=2026-09-20T12:46:17.980Z level=INFO msg="Uploading 3 narinfos"5374lb # [ 77.589932] niks3-server[1107]: 2026/09/20 12:46:18 INFO Signed narinfos id=26 count=25375lb # [ 77.606066] niks3-server[1107]: 2026/09/20 12:46:18 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5376lb # [ 77.609877] niks3-server[1107]: 2026/09/20 12:46:18 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5377lb # [ 77.615203] niks3-server[1107]: 2026/09/20 12:46:18 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5378lb # [ 77.619429] niks3-server[1107]: 2026/09/20 12:46:18 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5379lb # [ 77.623397] niks3-server[1107]: 2026/09/20 12:46:18 INFO Completed upload id=255380lb # [ 77.624822] niks3-server[1107]: 2026/09/20 12:46:18 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5381lb # [ 77.626817] niks3-server[1107]: 2026/09/20 12:46:18 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5382lb # [ 77.630087] niks3-server[1107]: 2026/09/20 12:46:18 INFO Completed upload id=265383worker2 # [ 77.665925] nix-grpc-daemon[1317]: time=2026-09-20T12:46:18.022Z level=INFO msg="Upload complete. (278ms)"5384worker2 # [ 77.669083] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43044 duration_s=0 paths=2 nar_bytes_in=3125385worker2 # [ 77.674976] nix-daemon[855]: reaped child process 1315, status = succeeded5386client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.85 seconds)5387worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5388worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5389worker1 # [ 77.748126] systemd[1]: Listening on Nix worker-protocol over gRPC.5390worker1 # [ 77.752660] systemd[1]: Starting Nix worker-protocol over gRPC...5391worker1 # [ 77.898364] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5392worker1 # [ 77.900754] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5393worker1 # [ 77.904065] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15394worker1 # [ 77.910363] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5395worker1 # [ 77.913680] systemd[1]: Started Nix worker-protocol over gRPC.5396lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5397lb: must succeed: curl -sf localhost:9901/clusters5398worker1 # [ 77.926434] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:18Z level=info event=scheduler_connected addr=lb:500515399worker2 # [ 77.940521] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:430305400worker2 # [ 77.945090] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05401lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5402lb # [ 78.360344] envoy[985]: conn peer=[2001:db8:1::4]:60498 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=46484 tx=2728 ms=512685403lb # [ 78.364387] envoy[985]: conn peer=[2001:db8:1::1]:45732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=225404lb # [ 78.367866] envoy[985]: conn peer=[2001:db8:1::3]:46310 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13730 tx=2523 ms=544105405lb # [ 78.371512] envoy[985]: conn peer=[2001:db8:1::1]:37984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1007 tx=282 ms=16165406lb: must succeed: curl -sf localhost:9901/clusters5407lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5408lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.13 seconds)5409(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.10 seconds)5410subtest: scheduler down: the other node takes the lock and keeps it5411lb: must succeed: curl -sf localhost:9901/clusters5412lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5413lb: must succeed: curl -sf localhost:9901/clusters5414lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5415lb: must succeed: curl -sf localhost:9901/clusters5416lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5417lb: must succeed: curl -sf localhost:9901/clusters5418lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5419worker2 # [ 79.293380] systemd[1]: Stopping Nix worker-protocol over gRPC...5420worker2 # [ 80.023815] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=signal_exit5421worker2 # [ 80.038776] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=scheduler_yield5422lb # [ 80.006593] niks3-server[1107]: 2026/09/20 12:46:21 INFO lead: released remote=192.168.1.4:501365423worker1 # [ 80.032423] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:20Z level=info event=scheduler_restarting addr=lb:500515424worker2 # [ 80.044602] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:430305425worker2 # [ 80.050350] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=worker_gone addr=192.168.1.3:500515426worker2 # [ 80.053463] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=scheduler_restarting addr=lb:500515427worker2 # [ 80.058301] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:430305428worker2 # [ 80.061612] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=worker_gone addr=192.168.1.4:500515429worker1 # [ 80.232831] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500515430worker1 # [ 80.235195] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker2 # [ 80.251340] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500515432worker2 # [ 80.255368] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker1 # [ 80.436260] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500515434worker1 # [ 80.440458] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker2 # [ 80.456437] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500515436worker2 # [ 80.460358] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker1 # [ 80.641062] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:20Z level=info event=scheduler_connected addr=lb:500515438worker1 # [ 80.642971] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5439worker2 # [ 80.661917] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500515440worker2 # [ 80.665205] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5441worker1 # [ 80.843958] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500515442worker1 # [ 80.848312] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5443worker2 # [ 80.863285] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500515444worker2 # [ 80.867353] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5445worker2 # [ 81.043523] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:21Z level=info event=scheduler_restarting5446worker1 # [ 81.049536] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500515447worker2 # [ 81.060446] nix-grpc-daemon[1261]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5448worker1 # [ 81.051651] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5449worker2 # [ 81.064440] nix-grpc-daemon[1261]: E0000 00:00:1789908381.417385 1304 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5450worker2 # [ 81.080448] nix-grpc-daemon[1261]: E0000 00:00:1789908381.427902 1304 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5451worker2 # [ 81.089410] nix-daemon[855]: reaped child process 1310, status = succeeded5452worker2 # [ 81.092226] nix-grpc-daemon[1261]: ts=2026-09-20T12:46:21Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5453worker1 # [ 81.252759] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500515454worker1 # [ 81.254908] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5455worker1 # [ 81.456126] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500515456worker1 # [ 81.460565] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5457worker1 # [ 81.661761] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_connected addr=lb:500515458worker1 # [ 81.666265] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5459worker1 # [ 81.866883] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500515460worker1 # [ 81.869187] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5461worker1 # [ 82.070972] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500515462worker1 # [ 82.074863] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5463worker1 # [ 82.272705] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500515464worker1 # [ 82.274708] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5465worker1 # [ 82.475973] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500515466worker1 # [ 82.480358] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5467worker1 # [ 82.681401] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_connected addr=lb:500515468worker1 # [ 82.685560] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5469worker1 # [ 82.886304] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500515470worker1 # [ 82.888664] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5471lb # [ 82.895728] niks3-server[1107]: 2026/09/20 12:46:24 INFO lead: acquired remote=192.168.1.3:372285472worker1 # [ 82.923116] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_take_over5473worker2 # [ 83.050735] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5474worker2 # [ 83.056246] systemd[1]: Stopped Nix worker-protocol over gRPC.5475worker2 # [ 83.059323] systemd[1]: nix-grpc-daemon.service: Consumed 337ms CPU time over 10.172s wall clock time, 30.6M memory peak, 19.3K incoming IP traffic, 25.8K outgoing IP traffic.5476worker2 # [ 83.073115] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5477worker2 # [ 83.077378] systemd[1]: Closed Nix worker-protocol over gRPC.5478worker2 # [ 83.080400] systemd[1]: nix-grpc-daemon.socket: Consumed 285us CPU time over 10.808s wall clock time, 256K memory peak, 20.9K incoming IP traffic, 15.2K outgoing IP traffic.5479worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5480worker1 # [ 83.089817] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500515481worker1 # [ 83.091734] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)5483worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5484worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5485worker1 # [ 83.292710] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500515486worker1 # [ 83.295087] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5487worker1 # [ 83.495777] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_connected addr=lb:500515488worker1 # [ 83.497944] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5489worker1 # [ 83.699157] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500515490worker1 # [ 83.703341] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5491worker1 # [ 83.904255] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:24Z level=info event=scheduler_connected addr=lb:500515492worker1 # [ 83.912242] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:365225493worker1 # [ 83.917465] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05494worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5495worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5496client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25497worker1 # [ 84.433776] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:365385498worker1 # [ 84.445493] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5499client # don't know how to build these paths:5500client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5501worker1 # [ 84.459971] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:365385502worker1 # [ 84.469176] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 1/5)5503worker1 # [ 84.519585] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)5504worker1 # [ 84.544939] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 3/5)5505worker1 # [ 84.563302] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 114 ms (attempt 4/5)5506worker1 # [ 84.680451] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5507worker1 # [ 84.703760] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:25Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05508worker1 # [ 84.709306] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:25Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15509worker1 # [ 84.751805] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:365385510worker1 # [ 84.763113] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:365385511client # copying 1 paths to 'lb:50051'...5512client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5513worker1 # [ 84.775384] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:365385514worker1 # [ 84.783878] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5515worker1 # [ 84.805397] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:25Z level=info event=niks3_push_started pid=11045516worker1 # [ 84.990314] nix-grpc-daemon[1104]: time=2026-09-20T12:46:25.297Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5517worker1 # [ 85.148374] nix-daemon[841]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)5518worker1 # [ 85.161572] nix-daemon[841]: reaped child process 1120, status = succeeded5519worker1 # [ 85.321274] nix-daemon[841]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)5520worker1 # [ 85.333676] nix-daemon[841]: reaped child process 1125, status = succeeded5521lb # [ 85.325706] niks3-server[1107]: 2026/09/20 12:46:26 INFO Received uploads request method=POST path=/api/pending_closures5522worker1 # [ 85.357835] nix-grpc-daemon[1104]: time=2026-09-20T12:46:25.665Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5523worker1 # [ 85.360868] nix-grpc-daemon[1104]: time=2026-09-20T12:46:25.667Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5524lb # [ 85.360584] niks3-server[1107]: 2026/09/20 12:46:26 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5525lb # [ 85.372083] niks3-server[1107]: 2026/09/20 12:46:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5526lb # [ 85.376671] niks3-server[1107]: 2026/09/20 12:46:26 INFO Signed narinfos id=27 count=15527worker1 # [ 85.403801] nix-grpc-daemon[1104]: time=2026-09-20T12:46:25.711Z level=INFO msg="Uploading 1 narinfos"5528lb # [ 85.383459] niks3-server[1107]: 2026/09/20 12:46:26 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5529lb # [ 85.388351] niks3-server[1107]: 2026/09/20 12:46:26 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5530lb # [ 85.391908] niks3-server[1107]: 2026/09/20 12:46:26 INFO Completed upload id=275531worker1 # [ 85.416393] nix-grpc-daemon[1104]: time=2026-09-20T12:46:25.724Z level=INFO msg="Upload complete. (421ms)"5532lb # [ 85.393101] niks3-server[1107]: 2026/09/20 12:46:26 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5533worker1 # [ 85.418713] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36538 duration_s=0 paths=1 nar_bytes_in=5205534worker1 # [ 85.421754] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:365385535worker1 # [ 85.428065] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5536worker1 # [ 85.431379] nix-daemon[841]: reaped child process 1102, status = succeeded5537client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5538worker1 # [ 85.436118] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5539client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5540client # farm-a> LOG-farm-a-on-standby5541worker1 # [ 86.262228] nix-daemon[841]: reaped child process 1129, status = succeeded5542worker1 # [ 86.407211] nix-daemon[841]: accepted connection from pid 1144, user nix-grpc-daemon (trusted)5543worker1 # [ 86.419649] nix-daemon[841]: reaped child process 1147, status = succeeded5544lb # [ 86.406102] niks3-server[1107]: 2026/09/20 12:46:27 INFO Received uploads request method=POST path=/api/pending_closures5545worker1 # [ 86.440769] nix-grpc-daemon[1104]: time=2026-09-20T12:46:26.748Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5546worker1 # [ 86.442546] nix-grpc-daemon[1104]: time=2026-09-20T12:46:26.750Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5547lb # [ 86.442125] niks3-server[1107]: 2026/09/20 12:46:27 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5548lb # [ 86.445330] niks3-server[1107]: 2026/09/20 12:46:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5549worker1 # [ 86.476306] nix-grpc-daemon[1104]: time=2026-09-20T12:46:26.784Z level=INFO msg="Uploading 1 narinfos"5550lb # [ 86.451642] niks3-server[1107]: 2026/09/20 12:46:27 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5551lb # [ 86.455127] niks3-server[1107]: 2026/09/20 12:46:27 INFO Signed narinfos id=28 count=15552lb # [ 86.465504] niks3-server[1107]: 2026/09/20 12:46:27 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5553lb # [ 86.469852] niks3-server[1107]: 2026/09/20 12:46:27 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5554lb # [ 86.471972] niks3-server[1107]: 2026/09/20 12:46:27 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5555lb # [ 86.476782] niks3-server[1107]: 2026/09/20 12:46:27 INFO Completed upload id=285556worker1 # [ 86.501243] nix-grpc-daemon[1104]: time=2026-09-20T12:46:26.809Z level=INFO msg="Upload complete. (257ms)"5557worker1 # [ 86.506434] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36538 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15558worker1 # [ 86.514633] nix-daemon[841]: reaped child process 1127, status = succeeded5559worker1 # [ 86.519771] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:26Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05560worker1 # [ 86.521447] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:26Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15561worker1 # [ 86.567832] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:365385562worker1 # [ 86.576511] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:365385563worker1 # [ 86.585544] nix-grpc-daemon[1104]: time=2026-09-20T12:46:26.893Z level=INFO msg="All 1 paths already cached"5564client # copying 1 paths to 'lb:50051'...5565client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5566worker1 # [ 86.590186] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:365385567worker1 # [ 86.594575] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5568worker1 # [ 86.737777] nix-daemon[841]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5569worker1 # [ 86.750394] nix-daemon[841]: reaped child process 1156, status = succeeded5570worker1 # [ 86.908966] nix-daemon[841]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)5571worker1 # [ 86.921199] nix-daemon[841]: reaped child process 1161, status = succeeded5572lb # [ 86.911701] niks3-server[1107]: 2026/09/20 12:46:28 INFO Received uploads request method=POST path=/api/pending_closures5573worker1 # [ 86.946980] nix-grpc-daemon[1104]: time=2026-09-20T12:46:27.254Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5574worker1 # [ 86.948341] nix-grpc-daemon[1104]: time=2026-09-20T12:46:27.254Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5575lb # [ 86.942066] niks3-server[1107]: 2026/09/20 12:46:28 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5576lb # [ 86.951471] niks3-server[1107]: 2026/09/20 12:46:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5577lb # [ 86.955638] niks3-server[1107]: 2026/09/20 12:46:28 INFO Signed narinfos id=29 count=15578worker1 # [ 86.983234] nix-grpc-daemon[1104]: time=2026-09-20T12:46:27.291Z level=INFO msg="Uploading 1 narinfos"5579lb # [ 86.964881] niks3-server[1107]: 2026/09/20 12:46:28 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5580lb # [ 86.974269] niks3-server[1107]: 2026/09/20 12:46:28 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5581lb # [ 86.980876] niks3-server[1107]: 2026/09/20 12:46:28 INFO Completed upload id=295582worker1 # [ 87.005497] nix-grpc-daemon[1104]: time=2026-09-20T12:46:27.313Z level=INFO msg="Upload complete. (403ms)"5583lb # [ 86.983345] niks3-server[1107]: 2026/09/20 12:46:28 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5584worker1 # [ 87.010141] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36538 duration_s=0 paths=1 nar_bytes_in=6405585worker1 # [ 87.014520] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:365385586worker1 # [ 87.024149] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5587worker1 # [ 87.028530] nix-daemon[841]: reaped child process 1151, status = succeeded5588worker1 # [ 87.033677] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5589client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5590client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5591client # farm-b> LOG-farm-b-on-standby5592worker1 # [ 88.259251] nix-daemon[841]: reaped child process 1165, status = succeeded5593lb # [ 88.360839] envoy[985]: conn peer=[2001:db8:1::4]:54786 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4478 tx=326 ms=80405594worker1 # [ 88.408296] nix-daemon[841]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)5595worker1 # [ 88.420513] nix-daemon[841]: reaped child process 1183, status = succeeded5596lb # [ 88.409828] niks3-server[1107]: 2026/09/20 12:46:29 INFO Received uploads request method=POST path=/api/pending_closures5597worker1 # [ 88.442701] nix-grpc-daemon[1104]: time=2026-09-20T12:46:28.750Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5598worker1 # [ 88.444278] nix-grpc-daemon[1104]: time=2026-09-20T12:46:28.752Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5599lb # [ 88.451320] niks3-server[1107]: 2026/09/20 12:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5600lb # [ 88.461423] niks3-server[1107]: 2026/09/20 12:46:29 INFO Signed narinfos id=30 count=15601worker1 # [ 88.489568] nix-grpc-daemon[1104]: time=2026-09-20T12:46:28.797Z level=INFO msg="Uploading 1 narinfos"5602lb # [ 88.467380] niks3-server[1107]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5603lb # [ 88.472720] niks3-server[1107]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5604lb # [ 88.477352] niks3-server[1107]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5605lb # [ 88.486688] niks3-server[1107]: 2026/09/20 12:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5606lb # [ 88.490978] niks3-server[1107]: 2026/09/20 12:46:29 INFO Completed upload id=305607lb # [ 88.494051] niks3-server[1107]: 2026/09/20 12:46:29 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5608worker1 # [ 88.517436] nix-grpc-daemon[1104]: time=2026-09-20T12:46:28.825Z level=INFO msg="Upload complete. (274ms)"5609worker1 # [ 88.528207] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36538 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15610worker1 # [ 88.535763] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:28Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05611worker1 # [ 88.541227] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:28Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15612worker1 # [ 88.550083] nix-daemon[841]: reaped child process 1163, status = succeeded5613worker1 # [ 88.580520] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:365385614worker1 # [ 88.593749] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:365385615worker1 # [ 88.605739] nix-grpc-daemon[1104]: time=2026-09-20T12:46:28.913Z level=INFO msg="All 2 paths already cached"5616client # copying 1 paths to 'lb:50051'...5617client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5618worker1 # [ 88.614214] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:365385619worker1 # [ 88.622931] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5620worker1 # [ 88.783401] nix-daemon[841]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)5621worker1 # [ 88.796412] nix-daemon[841]: reaped child process 1192, status = succeeded5622worker1 # [ 88.956565] nix-daemon[841]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)5623worker1 # [ 88.969074] nix-daemon[841]: reaped child process 1197, status = succeeded5624lb # [ 88.959654] niks3-server[1107]: 2026/09/20 12:46:30 INFO Received uploads request method=POST path=/api/pending_closures5625worker1 # [ 88.991876] nix-grpc-daemon[1104]: time=2026-09-20T12:46:29.299Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5626worker1 # [ 88.993269] nix-grpc-daemon[1104]: time=2026-09-20T12:46:29.299Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5627lb # [ 88.987429] niks3-server[1107]: 2026/09/20 12:46:30 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5628lb # [ 88.995922] niks3-server[1107]: 2026/09/20 12:46:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5629worker1 # [ 89.026122] nix-grpc-daemon[1104]: time=2026-09-20T12:46:29.334Z level=INFO msg="Uploading 1 narinfos"5630lb # [ 89.002115] niks3-server[1107]: 2026/09/20 12:46:30 INFO Signed narinfos id=31 count=15631lb # [ 89.008627] niks3-server[1107]: 2026/09/20 12:46:30 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5632lb # [ 89.017508] niks3-server[1107]: 2026/09/20 12:46:30 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5633worker1 # [ 89.047344] nix-grpc-daemon[1104]: time=2026-09-20T12:46:29.355Z level=INFO msg="Upload complete. (411ms)"5634lb # [ 89.022950] niks3-server[1107]: 2026/09/20 12:46:30 INFO Completed upload id=315635worker1 # [ 89.050109] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36538 duration_s=0 paths=1 nar_bytes_in=7685636lb # [ 89.028515] niks3-server[1107]: 2026/09/20 12:46:30 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5637worker1 # [ 89.055336] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:365385638worker1 # [ 89.062292] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5639worker1 # [ 89.072035] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5640client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5641worker1 # [ 89.074186] nix-daemon[841]: reaped child process 1187, status = succeeded5642client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5643client # farm-top> LOG-farm-top-on-standby5644worker1 # [ 89.696772] nix-daemon[841]: reaped child process 1201, status = succeeded5645worker1 # [ 89.868254] nix-daemon[841]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)5646worker1 # [ 89.883205] nix-daemon[841]: reaped child process 1219, status = succeeded5647lb # [ 89.875933] niks3-server[1107]: 2026/09/20 12:46:31 INFO Received uploads request method=POST path=/api/pending_closures5648worker1 # [ 89.918258] nix-grpc-daemon[1104]: time=2026-09-20T12:46:30.226Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5649worker1 # [ 89.920125] nix-grpc-daemon[1104]: time=2026-09-20T12:46:30.228Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5650lb # [ 89.930813] niks3-server[1107]: 2026/09/20 12:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5651lb # [ 89.936997] niks3-server[1107]: 2026/09/20 12:46:31 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5652lb # [ 89.942788] niks3-server[1107]: 2026/09/20 12:46:31 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5653lb # [ 89.947943] niks3-server[1107]: 2026/09/20 12:46:31 INFO Signed narinfos id=32 count=15654worker1 # [ 89.975401] nix-grpc-daemon[1104]: time=2026-09-20T12:46:30.283Z level=INFO msg="Uploading 1 narinfos"5655lb # [ 89.959566] niks3-server[1107]: 2026/09/20 12:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5656lb # [ 89.964813] niks3-server[1107]: 2026/09/20 12:46:31 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5657lb # [ 89.970163] niks3-server[1107]: 2026/09/20 12:46:31 INFO Completed upload id=325658worker1 # [ 89.994975] nix-grpc-daemon[1104]: time=2026-09-20T12:46:30.302Z level=INFO msg="Upload complete. (314ms)"5659lb # [ 89.976218] niks3-server[1107]: 2026/09/20 12:46:31 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5660worker1 # [ 90.001995] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36538 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15661worker1 # [ 90.017274] nix-daemon[841]: reaped child process 1199, status = succeeded5662client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 5.80 seconds)5663client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5664client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.21 seconds)5665worker2 # [ 90.329596] systemd[1]: Listening on Nix worker-protocol over gRPC.5666worker2 # [ 90.335574] systemd[1]: Starting scheduler-token.service...5667worker2 # [ 90.411863] systemd[1]: scheduler-token.service: Deactivated successfully.5668worker2 # [ 90.412964] systemd[1]: Finished scheduler-token.service.5669worker2 # [ 90.414369] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 76ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5670worker2 # [ 90.419902] systemd[1]: Starting Nix worker-protocol over gRPC...5671worker2 # [ 90.590520] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5672worker2 # [ 90.593101] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5673worker2 # [ 90.597484] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15674worker2 # [ 90.605644] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5675worker2 # [ 90.612600] systemd[1]: Started Nix worker-protocol over gRPC.5676lb: must succeed: curl -sf localhost:9901/clusters5677worker2 # [ 90.624340] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:30Z level=info event=scheduler_connected addr=lb:500515678worker1 # [ 90.624169] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:30Z level=info event=oidc_jwks provider=mock keys=15679worker1 # [ 90.628070] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:365225680worker1 # [ 90.631840] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05681lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5682worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5683worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5684lb: must succeed: curl -sf localhost:9901/clusters5685lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5686lb: must succeed: curl -sf localhost:9901/clusters5687lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5688worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5689worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5690(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.85 seconds)5691subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5692client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5693client # [ 90.955405] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5694client # Running as unit: rs.service; invocation ID: f5df9879dbb2408f94334533d07b50cc5695client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5696client # [ 91.163792] nix[1168]: don't know how to build these paths:5697worker1 # [ 91.162987] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:365385698client # [ 91.164837] nix[1168]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5699worker1 # [ 91.175151] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:31Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05700worker1 # [ 91.180776] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:31Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15701worker1 # [ 91.218861] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:365385702worker1 # [ 91.231373] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:365385703client # [ 91.239241] nix[1168]: copying 1 paths to 'lb:50051'...5704client # [ 91.243067] nix[1168]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5705worker1 # [ 91.245804] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:365385706worker1 # [ 91.254878] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5707worker1 # [ 91.421709] nix-daemon[841]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5708worker1 # [ 91.434312] nix-daemon[841]: reaped child process 1235, status = succeeded5709worker1 # [ 91.585956] nix-daemon[841]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)5710worker1 # [ 91.598497] nix-daemon[841]: reaped child process 1240, status = succeeded5711lb # [ 91.588662] niks3-server[1107]: 2026/09/20 12:46:32 INFO Received uploads request method=POST path=/api/pending_closures5712worker1 # [ 91.621372] nix-grpc-daemon[1104]: time=2026-09-20T12:46:31.929Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5713worker1 # [ 91.623265] nix-grpc-daemon[1104]: time=2026-09-20T12:46:31.931Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5714lb # [ 91.615102] niks3-server[1107]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5715lb # [ 91.626532] niks3-server[1107]: 2026/09/20 12:46:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5716worker1 # [ 91.657126] nix-grpc-daemon[1104]: time=2026-09-20T12:46:31.965Z level=INFO msg="Uploading 1 narinfos"5717lb # [ 91.632789] niks3-server[1107]: 2026/09/20 12:46:32 INFO Signed narinfos id=33 count=15718lb # [ 91.637899] niks3-server[1107]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5719lb # [ 91.648564] niks3-server[1107]: 2026/09/20 12:46:32 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5720lb # [ 91.653781] niks3-server[1107]: 2026/09/20 12:46:32 INFO Completed upload id=335721worker1 # [ 91.678457] nix-grpc-daemon[1104]: time=2026-09-20T12:46:31.986Z level=INFO msg="Upload complete. (409ms)"5722lb # [ 91.656587] niks3-server[1107]: 2026/09/20 12:46:32 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5723worker1 # [ 91.681114] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36538 duration_s=0 paths=1 nar_bytes_in=4165724worker1 # [ 91.683549] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:365385725worker1 # [ 91.691816] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5726worker1 # [ 91.695204] nix-daemon[841]: reaped child process 1230, status = succeeded5727worker1 # [ 91.699836] nix-daemon[841]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5728client # [ 91.696990] nix[1168]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5729lb: must succeed: curl -sf localhost:9901/clusters5730lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5731lb: must succeed: curl -sf localhost:9901/clusters5732lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5733lb: must succeed: curl -sf localhost:9901/clusters5734lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5735lb: must succeed: curl -sf localhost:9901/clusters5736lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5737worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5738worker2 # [ 93.805002] systemd-journald[287]: Received client request to rotate journal, rotating.5739worker2 # [ 93.828850] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/f1bf1b4831d74453bb0bc8eec277fa3e.5740worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5741worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5742worker1 # [ 93.736590] systemd[1]: Stopping Nix worker-protocol over gRPC...5743worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5744??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5745 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735746client: must succeed: sleep 5.05747worker1 # [ 93.922030] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:34Z level=info event=draining5748lb # [ 98.360354] envoy[985]: conn peer=[2001:db8:1::1]:59902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9551 tx=3869 ms=55865749client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5750worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5751worker1 # [ 99.131936] nix-daemon[841]: reaped child process 1244, status = succeeded5752worker1 # [ 99.281984] nix-daemon[841]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5753worker1 # [ 99.294144] nix-daemon[841]: reaped child process 1287, status = succeeded5754lb # [ 99.283912] niks3-server[1107]: 2026/09/20 12:46:40 INFO Received uploads request method=POST path=/api/pending_closures5755worker1 # [ 99.317883] nix-grpc-daemon[1104]: time=2026-09-20T12:46:39.625Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5756worker1 # [ 99.319343] nix-grpc-daemon[1104]: time=2026-09-20T12:46:39.627Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5757lb # [ 99.326758] niks3-server[1107]: 2026/09/20 12:46:40 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5758lb # [ 99.336222] niks3-server[1107]: 2026/09/20 12:46:40 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5759lb # [ 99.345267] niks3-server[1107]: 2026/09/20 12:46:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5760lb # [ 99.352972] niks3-server[1107]: 2026/09/20 12:46:40 INFO Signed narinfos id=34 count=15761worker1 # [ 99.377828] nix-grpc-daemon[1104]: time=2026-09-20T12:46:39.685Z level=INFO msg="Uploading 1 narinfos"5762lb # [ 99.356754] niks3-server[1107]: 2026/09/20 12:46:40 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5763lb # [ 99.366897] niks3-server[1107]: 2026/09/20 12:46:40 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5764lb # [ 99.369583] niks3-server[1107]: 2026/09/20 12:46:40 INFO Completed upload id=345765worker1 # [ 99.394104] nix-grpc-daemon[1104]: time=2026-09-20T12:46:39.702Z level=INFO msg="Upload complete. (278ms)"5766lb # [ 99.370859] niks3-server[1107]: 2026/09/20 12:46:40 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5767worker1 # [ 99.396602] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36538 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15768worker1 # [ 99.404396] nix-daemon[841]: reaped child process 1242, status = succeeded5769client # [ 99.431272] systemd[1]: rs.service: Deactivated successfully.5770client # [ 99.434580] systemd[1]: rs.service: Consumed 122ms CPU time over 8.478s wall clock time, 39.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5771worker1 # [ 99.923582] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=signal_exit5772worker1 # [ 100.146180] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=scheduler_yield5773lb # [ 100.124269] niks3-server[1107]: 2026/09/20 12:46:41 INFO lead: released remote=192.168.1.3:372285774worker1 # [ 100.153902] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=scheduler_restarting addr=lb:500515775worker2 # [ 100.167937] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:40Z level=info event=scheduler_restarting addr=lb:500515776worker1 # [ 100.158617] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:365225777worker1 # [ 100.161902] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=worker_gone addr=192.168.1.3:500515778worker1 # [ 100.164806] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:365225779worker1 # [ 100.167988] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=worker_gone addr=192.168.1.4:500515780worker1 # [ 100.354937] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500515781worker2 # [ 100.367231] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500515782worker1 # [ 100.361576] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5783worker2 # [ 100.372836] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5784worker1 # [ 100.563518] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500515785worker2 # [ 100.574479] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:40Z level=info event=scheduler_connected addr=lb:500515786worker2 # [ 100.580243] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5787worker1 # [ 100.569557] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5788lb # [ 100.581815] niks3-server[1107]: 2026/09/20 12:46:41 INFO lead: acquired remote=192.168.1.4:602105789worker2 # [ 100.620744] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:40Z level=info event=scheduler_take_over5790worker2 # [ 100.781575] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500515791worker1 # [ 100.770766] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500515792worker2 # [ 100.788515] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5793worker1 # [ 100.776728] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5794worker1 # [ 100.977620] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500515795worker2 # [ 100.989811] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500515796worker1 # [ 100.981577] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5797worker2 # [ 100.994327] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5798worker1 # [ 101.150879] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:41Z level=info event=scheduler_restarting5799worker1 # [ 101.172374] nix-daemon[841]: reaped child process 1095, status = succeeded5800worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.15 seconds)5801client: waiting for success: ! systemctl is-active rs5802worker2 # [ 101.195868] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500515803worker2 # [ 101.198959] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5804worker1 # [ 101.194468] nix-grpc-daemon[1034]: ts=2026-09-20T12:46:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5805client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5806client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5807client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5808client: must fail: journalctl -u rs | grep -q 'reconnecting'5809client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5810lb: must succeed: curl -sf localhost:9901/clusters5811lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5812worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5813worker2 # [ 101.399650] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500515814worker2 # [ 101.401725] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5815worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5816worker2 # [ 101.602918] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_connected addr=lb:500515817worker2 # [ 101.607149] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5818worker2 # [ 101.808132] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500515819worker2 # [ 101.811592] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker1 # [ 101.800609] nix-grpc-daemon[1034]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5821worker1 # [ 101.804183] nix-grpc-daemon[1034]: E0000 00:00:1789908402.107473 1077 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5822worker1 # [ 101.808909] nix-grpc-daemon[1034]: E0000 00:00:1789908402.107876 1077 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5823worker2 # [ 102.012696] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500515824worker2 # [ 102.016742] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5825worker2 # [ 102.218634] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:42Z level=info event=scheduler_connected addr=lb:500515826worker2 # [ 102.232132] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:42Z level=info event=oidc_jwks provider=mock keys=15827worker2 # [ 102.235204] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:441545828worker2 # [ 102.239504] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05829lb: must succeed: curl -sf localhost:9901/clusters5830lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5831worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5832worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5833worker1 # [ 102.782616] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5834worker1 # [ 102.786306] systemd[1]: Stopped Nix worker-protocol over gRPC.5835worker1 # [ 102.789302] systemd[1]: nix-grpc-daemon.service: Consumed 1.320s CPU time over 25.029s wall clock time, 33.5M memory peak, 49.5K incoming IP traffic, 91.4K outgoing IP traffic.5836worker1 # [ 102.805442] systemd[1]: Starting Nix worker-protocol over gRPC...5837worker1 # [ 102.954461] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5838worker1 # [ 102.958757] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5839worker1 # [ 102.960832] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15840worker1 # [ 102.966607] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5841worker1 # [ 102.968421] systemd[1]: Started Nix worker-protocol over gRPC.5842worker1 # [ 102.981861] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:43Z level=info event=scheduler_connected addr=lb:500515843worker2 # [ 102.995129] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:441545844worker2 # [ 102.999301] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05845lb: must succeed: curl -sf localhost:9901/clusters5846lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5847worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5848worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5849(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.76 seconds)5850subtest: scheduler restart mid-build: build finishes, no double build5851client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5852client # [ 103.696668] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5853client # Running as unit: mid.service; invocation ID: 8a66b414c2d34321a034e35deb84e8fc5854client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.05 seconds)5855worker1 # [ 103.919540] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:605225856worker1 # [ 103.931898] nix-daemon[841]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)5857client # [ 103.942361] nix[1203]: don't know how to build these paths:5858client # [ 103.943359] nix[1203]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5859worker2 # [ 103.965535] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:44Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05860worker2 # [ 103.967676] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:44Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15861worker2 # [ 104.016715] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418425862worker2 # [ 104.021989] nix-daemon[855]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5863worker2 # [ 104.030225] nix-grpc-daemon[1357]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)5864worker2 # [ 104.054630] nix-grpc-daemon[1357]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 2/5)5865worker2 # [ 104.060913] nix-grpc-daemon[1357]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 3/5)5866worker2 # [ 104.149519] nix-grpc-daemon[1357]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 4/5)5867worker2 # [ 104.166463] nix-grpc-daemon[1357]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5868worker2 # [ 104.184376] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418425869client # [ 104.181195] nix[1203]: copying 1 paths to 'lb:50051'...5870client # [ 104.185165] nix[1203]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5871worker2 # [ 104.199832] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418425872worker2 # [ 104.209174] nix-daemon[855]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5873worker2 # [ 104.227589] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:44Z level=info event=niks3_push_started pid=14785874worker2 # [ 104.414418] nix-grpc-daemon[1478]: time=2026-09-20T12:46:44.771Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5875worker2 # [ 104.579434] nix-daemon[855]: accepted connection from pid 1491, user nix-grpc-daemon (trusted)5876worker2 # [ 104.590829] nix-daemon[855]: reaped child process 1494, status = succeeded5877worker2 # [ 104.742751] nix-daemon[855]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)5878worker2 # [ 104.754544] nix-daemon[855]: reaped child process 1499, status = succeeded5879lb # [ 104.735560] niks3-server[1107]: 2026/09/20 12:46:45 INFO Received uploads request method=POST path=/api/pending_closures5880worker2 # [ 104.780408] nix-grpc-daemon[1478]: time=2026-09-20T12:46:45.137Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5881worker2 # [ 104.783233] nix-grpc-daemon[1478]: time=2026-09-20T12:46:45.139Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5882lb # [ 104.772359] niks3-server[1107]: 2026/09/20 12:46:45 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5883lb # [ 104.784104] niks3-server[1107]: 2026/09/20 12:46:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5884lb # [ 104.790794] niks3-server[1107]: 2026/09/20 12:46:45 INFO Signed narinfos id=35 count=15885worker2 # [ 104.829396] nix-grpc-daemon[1478]: time=2026-09-20T12:46:45.186Z level=INFO msg="Uploading 1 narinfos"5886lb # [ 104.804284] niks3-server[1107]: 2026/09/20 12:46:45 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5887lb # [ 104.811779] niks3-server[1107]: 2026/09/20 12:46:45 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5888lb # [ 104.817903] niks3-server[1107]: 2026/09/20 12:46:45 INFO Completed upload id=355889lb # [ 104.819666] niks3-server[1107]: 2026/09/20 12:46:45 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5890worker2 # [ 104.855093] nix-grpc-daemon[1478]: time=2026-09-20T12:46:45.212Z level=INFO msg="Upload complete. (435ms)"5891worker2 # [ 104.859301] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41842 duration_s=0 paths=1 nar_bytes_in=4245892worker2 # [ 104.866630] nix-grpc-daemon[1357]: ts=2026-09-20T12:46:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418425893worker2 # [ 104.874540] nix-daemon[855]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5894worker2 # [ 104.884123] nix-daemon[855]: reaped child process 1476, status = succeeded5895worker2 # [ 104.888736] nix-daemon[855]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5896client # [ 104.873705] nix[1203]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5897lb: must succeed: curl -sf localhost:9901/clusters5898lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5899lb: must succeed: curl -sf localhost:9901/clusters5900lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5901worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5902worker2 # [ 106.311315] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1357 (nix-grpc-daemon) on client request.5903worker2 # [ 106.313333] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5904lb # [ 106.284203] niks3-server[1107]: 2026/09/20 12:46:47 INFO lead: released remote=192.168.1.4:602105905worker1 # [ 106.315310] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:46Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5906client # [ 106.314063] nix[1203]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5907worker2 # [ 106.331643] systemd[1]: Starting scheduler-token.service...5908client # [ 106.318839] nix[1203]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5909worker2 # [ 106.344092] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5910worker2 # [ 106.345548] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5911worker2 # [ 106.346719] systemd[1]: nix-grpc-daemon.service: Consumed 419ms CPU time over 15.921s wall clock time, 30.9M memory peak, 17.2K incoming IP traffic, 32.1K outgoing IP traffic.5912worker2 # [ 106.357420] nix-daemon[855]: reaped child process 1501, status = succeeded5913worker2 # [ 106.359508] nix-daemon[855]: reaped child process 1471, status = succeeded5914worker2 # [ 106.367252] nix-daemon[1503]: unexpected Nix daemon error: error: interrupted by the user5915worker2 # [ 106.373168] nix-daemon[855]: reaped child process 1503, status = failed with exit code 15916worker2 # [ 106.412075] systemd[1]: scheduler-token.service: Deactivated successfully.5917worker2 # [ 106.413172] systemd[1]: Finished scheduler-token.service.5918worker2 # [ 106.415168] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 80ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5919worker2 # [ 106.417367] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5920worker2 # [ 106.421533] systemd[1]: Starting scheduler-token.service...5921worker2 # [ 106.479600] systemd[1]: scheduler-token.service: Deactivated successfully.5922worker2 # [ 106.480584] systemd[1]: Finished scheduler-token.service.5923worker2 # [ 106.481817] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 57ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5924worker2 # [ 106.487086] systemd[1]: Starting Nix worker-protocol over gRPC...5925worker1 # [ 106.515973] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:46Z level=info event=scheduler_connected addr=lb:500515926worker2 # [ 106.628471] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5927worker2 # [ 106.630272] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5928worker2 # [ 106.633800] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15929worker2 # [ 106.641115] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5930worker2 # [ 106.642788] systemd[1]: Started Nix worker-protocol over gRPC.5931worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.38 seconds)5932client: must succeed: sleep 5.05933lb # [ 106.616395] niks3-server[1107]: 2026/09/20 12:46:47 INFO lead: acquired remote=192.168.1.4:477945934worker2 # [ 106.655321] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=info event=scheduler_take_over5935worker2 # [ 106.660458] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:418565936worker2 # [ 106.662415] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05937worker2 # [ 106.665553] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=info event=scheduler_connected addr=lb:500515938worker2 # [ 106.670571] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=info event=oidc_jwks provider=mock keys=15939worker2 # [ 106.671882] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:418565940worker2 # [ 106.673486] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05941worker2 # [ 106.837987] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05942worker2 # [ 106.842462] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:47Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15943worker1 # [ 106.872103] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605225944worker1 # [ 106.884434] nix-grpc-daemon[1294]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)5945worker1 # [ 106.934216] nix-grpc-daemon[1294]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 110 ms (attempt 2/5)5946worker1 # [ 107.050750] nix-grpc-daemon[1294]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 179 ms (attempt 3/5)5947worker1 # [ 107.235647] nix-grpc-daemon[1294]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 539 ms (attempt 4/5)5948worker1 # [ 107.781866] nix-grpc-daemon[1294]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5949worker1 # [ 107.799165] nix-grpc-daemon[1294]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5950worker1 # [ 107.825329] nix-daemon[841]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)5951worker1 # [ 107.838530] nix-daemon[841]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)5952client # [ 107.836196] nix[1203]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5953lb # [ 108.361238] envoy[985]: conn peer=[2001:db8:1::1]:59918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1324 ms=82485954lb # [ 108.365712] envoy[985]: conn peer=[2001:db8:1::3]:54514 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5966 tx=1377 ms=232645955lb # [ 108.369744] envoy[985]: conn peer=[2001:db8:1::4]:49250 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=9471 tx=674 ms=157065956client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5957client: waiting for success: ! systemctl is-active mid5958worker1 # [ 112.061950] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:52Z level=info event=niks3_push_started pid=13915959worker1 # [ 112.085457] nix-daemon[841]: reaped child process 1367, status = succeeded5960worker1 # [ 112.244672] nix-grpc-daemon[1391]: time=2026-09-20T12:46:52.552Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5961worker1 # [ 112.399619] nix-daemon[841]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)5962worker1 # [ 112.411823] nix-daemon[841]: reaped child process 1407, status = succeeded5963lb # [ 112.405289] niks3-server[1107]: 2026/09/20 12:46:53 INFO Received uploads request method=POST path=/api/pending_closures5964worker1 # [ 112.799139] nix-grpc-daemon[1391]: time=2026-09-20T12:46:53.106Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5965worker1 # [ 112.802573] nix-grpc-daemon[1391]: time=2026-09-20T12:46:53.106Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5966lb # [ 112.848278] niks3-server[1107]: 2026/09/20 12:46:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5967lb # [ 112.856471] niks3-server[1107]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5968lb # [ 112.860698] niks3-server[1107]: 2026/09/20 12:46:54 INFO Signed narinfos id=36 count=15969worker1 # [ 112.886169] nix-grpc-daemon[1391]: time=2026-09-20T12:46:53.193Z level=INFO msg="Uploading 1 narinfos"5970lb # [ 112.877388] niks3-server[1107]: 2026/09/20 12:46:54 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5971lb # [ 112.880357] niks3-server[1107]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5972lb # [ 112.894677] niks3-server[1107]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5973lb # [ 112.907843] niks3-server[1107]: 2026/09/20 12:46:54 INFO Completed upload id=365974worker1 # [ 112.932511] nix-grpc-daemon[1391]: time=2026-09-20T12:46:53.240Z level=INFO msg="Upload complete. (683ms)"5975lb # [ 112.911364] niks3-server[1107]: 2026/09/20 12:46:54 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5976worker1 # [ 112.938314] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60522 duration_s=6 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15977worker1 # [ 112.952920] nix-daemon[841]: reaped child process 1365, status = succeeded5978client # [ 112.966943] systemd[1]: mid.service: Deactivated successfully.5979client # [ 112.969124] systemd[1]: mid.service: Consumed 142ms CPU time over 9.272s wall clock time, 39.3M memory peak, 9.7K incoming IP traffic, 10.8K outgoing IP traffic.5980client: (finished: waiting for success: ! systemctl is-active mid, in 1.10 seconds)5981client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5982client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)5983(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.38 seconds)5984subtest: interrupting the client stops the build on the worker5985client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5986client # [ 113.072224] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5987client # Running as unit: intr.service; invocation ID: 58714df5333a4b4eb38d645d608d9c085988client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5989worker2 # [ 113.287494] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:412845990worker2 # [ 113.293622] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)5991client # [ 113.287442] nix[1238]: don't know how to build these paths:5992client # [ 113.288425] nix[1238]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5993worker2 # [ 113.308077] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:53Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05994worker2 # [ 113.310442] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:53Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=15995worker2 # [ 113.315536] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412845996worker2 # [ 113.322105] nix-grpc-daemon[1545]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)5997worker2 # [ 113.349211] nix-grpc-daemon[1545]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 2/5)5998worker2 # [ 113.407908] nix-grpc-daemon[1545]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 3/5)5999worker2 # [ 113.562981] nix-grpc-daemon[1545]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 752 ms (attempt 4/5)6000worker2 # [ 114.318129] nix-grpc-daemon[1545]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6001worker2 # [ 114.330405] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846002client # [ 114.324503] nix[1238]: copying 1 paths to 'lb:50051'...6003client # [ 114.325352] nix[1238]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6004worker2 # [ 114.338044] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412846005worker2 # [ 114.347601] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6006worker2 # [ 114.360211] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:54Z level=info event=niks3_push_started pid=16216007worker2 # [ 114.510893] nix-grpc-daemon[1621]: time=2026-09-20T12:46:54.867Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6008worker2 # [ 114.674603] nix-daemon[855]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)6009worker2 # [ 114.687329] nix-daemon[855]: reaped child process 1637, status = succeeded6010worker2 # [ 114.855114] nix-daemon[855]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6011worker2 # [ 114.867992] nix-daemon[855]: reaped child process 1642, status = succeeded6012lb # [ 114.843743] niks3-server[1107]: 2026/09/20 12:46:55 INFO Received uploads request method=POST path=/api/pending_closures6013worker2 # [ 114.890560] nix-grpc-daemon[1621]: time=2026-09-20T12:46:55.247Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6014worker2 # [ 114.892800] nix-grpc-daemon[1621]: time=2026-09-20T12:46:55.249Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6015lb # [ 114.882699] niks3-server[1107]: 2026/09/20 12:46:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6016lb # [ 114.887843] niks3-server[1107]: 2026/09/20 12:46:56 INFO Signed narinfos id=37 count=16017worker2 # [ 114.926822] nix-grpc-daemon[1621]: time=2026-09-20T12:46:55.283Z level=INFO msg="Uploading 1 narinfos"6018lb # [ 114.894523] niks3-server[1107]: 2026/09/20 12:46:56 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6019lb # [ 114.907630] niks3-server[1107]: 2026/09/20 12:46:56 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6020lb # [ 114.914373] niks3-server[1107]: 2026/09/20 12:46:56 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6021lb # [ 115.118926] niks3-server[1107]: 2026/09/20 12:46:56 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6022lb # [ 115.129638] niks3-server[1107]: 2026/09/20 12:46:56 INFO Completed upload id=376023worker2 # [ 115.165560] nix-grpc-daemon[1621]: time=2026-09-20T12:46:55.522Z level=INFO msg="Upload complete. (650ms)"6024worker2 # [ 115.171157] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=0 paths=1 nar_bytes_in=4246025worker2 # [ 115.177259] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846026worker2 # [ 115.192135] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6027worker2 # [ 115.199196] nix-daemon[855]: reaped child process 1619, status = succeeded6028client # [ 115.195256] nix[1238]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6029worker2 # [ 115.211362] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6030client: must succeed: systemctl kill -s INT intr6031client # [ 116.564305] systemd[1]: intr.service: Sent signal SIGINT to main process 1238 (nix) on client request.6032client # [ 116.572438] nix[1238]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6033client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6034client # [ 116.579168] nix[1238]: error: interrupted by the user6035client # [ 116.609513] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6036client # [ 116.611584] systemd[1]: intr.service: Failed with result 'exit-code'.6037client # [ 116.613469] systemd[1]: intr.service: Consumed 121ms CPU time over 3.539s wall clock time, 39.1M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.6038worker2 # [ 116.635313] nix-daemon[855]: reaped child process 1644, status = succeeded6039worker2 # [ 116.639327] nix-daemon[1646]: unexpected Nix daemon error: error: interrupted by the user6040worker2 # [ 116.644467] nix-daemon[855]: reaped child process 1646, status = failed with exit code 16041(finished: subtest: interrupting the client stops the build on the worker, in 3.63 seconds)6042subtest: a deploy mid-build drains: the build finishes, the new generation takes over6043client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6044client # [ 116.705530] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6045client # Running as unit: sw.service; invocation ID: 4988788fb56c4ece9b7a8259caa80c8a6046client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6047worker1 # [ 116.912856] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:605226048client # [ 116.916961] nix[1258]: don't know how to build these paths:6049client # [ 116.917939] nix[1258]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6050worker2 # [ 116.940156] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:57Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06051worker2 # [ 116.944355] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:57Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16052worker1 # [ 116.973840] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605226053worker1 # [ 116.987620] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:605226054client # [ 116.995251] nix[1258]: copying 1 paths to 'lb:50051'...6055worker1 # [ 116.999102] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:605226056client # [ 116.997749] nix[1258]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6057worker1 # [ 117.009827] nix-daemon[841]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)6058worker1 # [ 117.185433] nix-daemon[841]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6059worker1 # [ 117.197983] nix-daemon[841]: reaped child process 1438, status = succeeded6060worker1 # [ 117.352587] nix-daemon[841]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)6061worker1 # [ 117.365140] nix-daemon[841]: reaped child process 1443, status = succeeded6062lb # [ 117.353711] niks3-server[1107]: 2026/09/20 12:46:58 INFO Received uploads request method=POST path=/api/pending_closures6063worker1 # [ 117.387951] nix-grpc-daemon[1391]: time=2026-09-20T12:46:57.695Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6064worker1 # [ 117.389557] nix-grpc-daemon[1391]: time=2026-09-20T12:46:57.697Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6065lb # [ 117.389471] niks3-server[1107]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6066lb # [ 117.398373] niks3-server[1107]: 2026/09/20 12:46:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6067worker1 # [ 117.430876] nix-grpc-daemon[1391]: time=2026-09-20T12:46:57.738Z level=INFO msg="Uploading 1 narinfos"6068lb # [ 117.406440] niks3-server[1107]: 2026/09/20 12:46:58 INFO Signed narinfos id=38 count=16069lb # [ 117.410836] niks3-server[1107]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6070lb # [ 117.421542] niks3-server[1107]: 2026/09/20 12:46:58 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6071lb # [ 117.426417] niks3-server[1107]: 2026/09/20 12:46:58 INFO Completed upload id=386072lb # [ 117.428201] niks3-server[1107]: 2026/09/20 12:46:58 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6073worker1 # [ 117.452510] nix-grpc-daemon[1391]: time=2026-09-20T12:46:57.760Z level=INFO msg="Upload complete. (429ms)"6074worker1 # [ 117.456522] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60522 duration_s=0 paths=1 nar_bytes_in=4166075worker1 # [ 117.463278] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605226076worker1 # [ 117.473118] nix-daemon[841]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)6077worker1 # [ 117.480190] nix-daemon[841]: reaped child process 1433, status = succeeded6078worker1 # [ 117.485408] nix-daemon[841]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)6079client # [ 117.482468] nix[1258]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6080lb # [ 118.361278] envoy[985]: conn peer=[2001:db8:1::1]:34590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3865 tx=1678 ms=90386081lb # [ 118.363166] envoy[985]: conn peer=[2001:db8:1::1]:55632 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=757 ms=33196082worker1: must succeed: systemctl reload nix-grpc-daemon.service6083worker1 # [ 119.197149] systemd[1]: Reloading Nix worker-protocol over gRPC...6084worker1 # [ 119.224068] systemd[1]: Reloaded Nix worker-protocol over gRPC.6085worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.07 seconds)6086worker1: must succeed: pgrep -f 'read -t [9]0 x'6087worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6088client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26089worker1 # [ 119.494916] nix-grpc-daemon[1294]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:605226090client # don't know how to build these paths:6091client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6092worker2 # [ 119.514396] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846093worker2 # [ 119.537837] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:59Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06094worker2 # [ 119.542846] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:59Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16095worker2 # [ 119.582647] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846096worker2 # [ 119.595136] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846097client # copying 1 paths to 'lb:50051'...6098worker2 # [ 119.607646] nix-grpc-daemon[1545]: ts=2026-09-20T12:46:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412846099client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6100worker2 # [ 119.635179] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6101worker2 # [ 119.811649] nix-daemon[855]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6102worker2 # [ 119.824204] nix-daemon[855]: reaped child process 1699, status = succeeded6103worker2 # [ 119.978444] nix-daemon[855]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6104worker1 # [ 119.979094] nix-grpc-daemon[1294]: ts=2026-09-20T12:47:00Z level=info event=draining6105worker2 # [ 119.990515] nix-daemon[855]: reaped child process 1704, status = succeeded6106lb # [ 119.970911] niks3-server[1107]: 2026/09/20 12:47:01 INFO Received uploads request method=POST path=/api/pending_closures6107worker2 # [ 120.030077] nix-grpc-daemon[1621]: time=2026-09-20T12:47:00.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6108worker2 # [ 120.032300] nix-grpc-daemon[1621]: time=2026-09-20T12:47:00.389Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6109lb # [ 120.023557] niks3-server[1107]: 2026/09/20 12:47:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6110lb # [ 120.029171] niks3-server[1107]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6111lb # [ 120.036831] niks3-server[1107]: 2026/09/20 12:47:01 INFO Signed narinfos id=39 count=16112worker2 # [ 120.075707] nix-grpc-daemon[1621]: time=2026-09-20T12:47:00.432Z level=INFO msg="Uploading 1 narinfos"6113lb # [ 120.046200] niks3-server[1107]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6114lb # [ 120.054632] niks3-server[1107]: 2026/09/20 12:47:01 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6115lb # [ 120.064423] niks3-server[1107]: 2026/09/20 12:47:01 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6116lb # [ 120.067063] niks3-server[1107]: 2026/09/20 12:47:01 INFO Completed upload id=396117worker2 # [ 120.102884] nix-grpc-daemon[1621]: time=2026-09-20T12:47:00.459Z level=INFO msg="Upload complete. (455ms)"6118worker2 # [ 120.107364] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=0 paths=1 nar_bytes_in=5286119worker2 # [ 120.115307] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846120worker2 # [ 120.127093] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6121worker2 # [ 120.134851] nix-daemon[855]: reaped child process 1694, status = succeeded6122client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6123worker2 # [ 120.143984] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6124client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6125client # farm-a> LOG-farm-a-during-drain6126worker2 # [ 121.371799] nix-daemon[855]: reaped child process 1708, status = succeeded6127worker2 # [ 121.524813] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6128worker2 # [ 121.536965] nix-daemon[855]: reaped child process 1726, status = succeeded6129lb # [ 121.518313] niks3-server[1107]: 2026/09/20 12:47:02 INFO Received uploads request method=POST path=/api/pending_closures6130worker2 # [ 121.566054] nix-grpc-daemon[1621]: time=2026-09-20T12:47:01.922Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6131worker2 # [ 121.568357] nix-grpc-daemon[1621]: time=2026-09-20T12:47:01.924Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6132lb # [ 121.566601] niks3-server[1107]: 2026/09/20 12:47:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6133lb # [ 121.574814] niks3-server[1107]: 2026/09/20 12:47:02 INFO Signed narinfos id=40 count=16134worker2 # [ 121.616301] nix-grpc-daemon[1621]: time=2026-09-20T12:47:01.972Z level=INFO msg="Uploading 1 narinfos"6135lb # [ 121.580634] niks3-server[1107]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6136lb # [ 121.586098] niks3-server[1107]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6137lb # [ 121.594771] niks3-server[1107]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6138lb # [ 121.600145] niks3-server[1107]: 2026/09/20 12:47:02 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6139lb # [ 121.606453] niks3-server[1107]: 2026/09/20 12:47:02 INFO Completed upload id=406140worker2 # [ 121.642296] nix-grpc-daemon[1621]: time=2026-09-20T12:47:01.999Z level=INFO msg="Upload complete. (289ms)"6141lb # [ 121.608230] niks3-server[1107]: 2026/09/20 12:47:02 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6142worker2 # [ 121.650760] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16143worker2 # [ 121.659514] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:02Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06144worker2 # [ 121.664210] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:02Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16145worker2 # [ 121.672847] nix-daemon[855]: reaped child process 1706, status = succeeded6146worker2 # [ 121.709753] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846147worker2 # [ 121.721783] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846148worker2 # [ 121.741735] nix-grpc-daemon[1621]: time=2026-09-20T12:47:02.098Z level=INFO msg="All 1 paths already cached"6149client # copying 1 paths to 'lb:50051'...6150worker2 # [ 121.750131] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412846151client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6152worker2 # [ 121.770223] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6153worker2 # [ 121.960754] nix-daemon[855]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)6154worker2 # [ 121.975614] nix-daemon[855]: reaped child process 1735, status = succeeded6155worker2 # [ 122.138834] nix-daemon[855]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6156worker2 # [ 122.150686] nix-daemon[855]: reaped child process 1740, status = succeeded6157lb # [ 122.131173] niks3-server[1107]: 2026/09/20 12:47:03 INFO Received uploads request method=POST path=/api/pending_closures6158worker2 # [ 122.174659] nix-grpc-daemon[1621]: time=2026-09-20T12:47:02.531Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6159worker2 # [ 122.176775] nix-grpc-daemon[1621]: time=2026-09-20T12:47:02.533Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6160lb # [ 122.161763] niks3-server[1107]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6161lb # [ 122.170977] niks3-server[1107]: 2026/09/20 12:47:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6162worker2 # [ 122.212970] nix-grpc-daemon[1621]: time=2026-09-20T12:47:02.569Z level=INFO msg="Uploading 1 narinfos"6163lb # [ 122.177209] niks3-server[1107]: 2026/09/20 12:47:03 INFO Signed narinfos id=41 count=16164lb # [ 122.181554] niks3-server[1107]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6165lb # [ 122.193428] niks3-server[1107]: 2026/09/20 12:47:03 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6166lb # [ 122.199509] niks3-server[1107]: 2026/09/20 12:47:03 INFO Completed upload id=416167worker2 # [ 122.235148] nix-grpc-daemon[1621]: time=2026-09-20T12:47:02.592Z level=INFO msg="Upload complete. (453ms)"6168worker2 # [ 122.237942] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=0 paths=1 nar_bytes_in=6406169lb # [ 122.203493] niks3-server[1107]: 2026/09/20 12:47:03 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6170worker2 # [ 122.243379] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846171worker2 # [ 122.252754] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6172worker2 # [ 122.257887] nix-daemon[855]: reaped child process 1730, status = succeeded6173worker2 # [ 122.262822] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6174client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6175client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6176worker2 # [ 123.075797] nix-daemon[855]: reaped child process 1744, status = succeeded6177client # farm-b> LOG-farm-b-during-drain6178worker2 # [ 123.219736] nix-daemon[855]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6179worker2 # [ 123.231768] nix-daemon[855]: reaped child process 1762, status = succeeded6180lb # [ 123.215719] niks3-server[1107]: 2026/09/20 12:47:04 INFO Received uploads request method=POST path=/api/pending_closures6181worker2 # [ 123.257522] nix-grpc-daemon[1621]: time=2026-09-20T12:47:03.614Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6182worker2 # [ 123.258984] nix-grpc-daemon[1621]: time=2026-09-20T12:47:03.616Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6183lb # [ 123.249120] niks3-server[1107]: 2026/09/20 12:47:04 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6184lb # [ 123.255898] niks3-server[1107]: 2026/09/20 12:47:04 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6185lb # [ 123.266107] niks3-server[1107]: 2026/09/20 12:47:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6186lb # [ 123.272455] niks3-server[1107]: 2026/09/20 12:47:04 INFO Signed narinfos id=42 count=16187worker2 # [ 123.308369] nix-grpc-daemon[1621]: time=2026-09-20T12:47:03.665Z level=INFO msg="Uploading 1 narinfos"6188lb # [ 123.278162] niks3-server[1107]: 2026/09/20 12:47:04 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6189lb # [ 123.287244] niks3-server[1107]: 2026/09/20 12:47:04 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6190lb # [ 123.291735] niks3-server[1107]: 2026/09/20 12:47:04 INFO Completed upload id=426191lb # [ 123.295256] niks3-server[1107]: 2026/09/20 12:47:04 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6192worker2 # [ 123.329879] nix-grpc-daemon[1621]: time=2026-09-20T12:47:03.686Z level=INFO msg="Upload complete. (260ms)"6193worker2 # [ 123.336146] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16194worker2 # [ 123.349360] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:03Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06195worker2 # [ 123.354112] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:03Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16196worker2 # [ 123.361999] nix-daemon[855]: reaped child process 1742, status = succeeded6197worker2 # [ 123.394148] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846198worker2 # [ 123.406451] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846199worker2 # [ 123.440609] nix-grpc-daemon[1621]: time=2026-09-20T12:47:03.797Z level=INFO msg="All 2 paths already cached"6200client # copying 1 paths to 'lb:50051'...6201client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6202worker2 # [ 123.446483] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412846203worker2 # [ 123.453831] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6204worker2 # [ 123.615675] nix-daemon[855]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)6205worker2 # [ 123.629262] nix-daemon[855]: reaped child process 1771, status = succeeded6206worker2 # [ 123.789040] nix-daemon[855]: accepted connection from pid 1773, user nix-grpc-daemon (trusted)6207worker2 # [ 123.801365] nix-daemon[855]: reaped child process 1776, status = succeeded6208lb # [ 123.781300] niks3-server[1107]: 2026/09/20 12:47:04 INFO Received uploads request method=POST path=/api/pending_closures6209worker2 # [ 124.062170] nix-grpc-daemon[1621]: time=2026-09-20T12:47:04.418Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6210worker2 # [ 124.065516] nix-grpc-daemon[1621]: time=2026-09-20T12:47:04.418Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6211lb # [ 124.048653] niks3-server[1107]: 2026/09/20 12:47:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6212lb # [ 124.055930] niks3-server[1107]: 2026/09/20 12:47:05 INFO Signed narinfos id=43 count=16213worker2 # [ 124.095119] nix-grpc-daemon[1621]: time=2026-09-20T12:47:04.451Z level=INFO msg="Uploading 1 narinfos"6214lb # [ 124.071168] niks3-server[1107]: 2026/09/20 12:47:05 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6215lb # [ 124.190207] niks3-server[1107]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6216lb # [ 124.203605] niks3-server[1107]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6217lb # [ 124.216269] niks3-server[1107]: 2026/09/20 12:47:05 INFO Completed upload id=436218worker2 # [ 124.251899] nix-grpc-daemon[1621]: time=2026-09-20T12:47:04.608Z level=INFO msg="Upload complete. (792ms)"6219lb # [ 124.222419] niks3-server[1107]: 2026/09/20 12:47:05 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6220worker2 # [ 124.257158] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=0 paths=1 nar_bytes_in=7686221worker2 # [ 124.263974] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846222worker2 # [ 124.277163] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6223worker2 # [ 124.288432] nix-daemon[855]: reaped child process 1766, status = succeeded6224client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6225client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6226worker2 # [ 124.298366] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6227client # farm-top> LOG-farm-top-during-drain6228worker2 # [ 125.331325] nix-daemon[855]: reaped child process 1780, status = succeeded6229worker2 # [ 125.460925] nix-daemon[855]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6230worker2 # [ 125.473898] nix-daemon[855]: reaped child process 1798, status = succeeded6231lb # [ 125.449435] niks3-server[1107]: 2026/09/20 12:47:06 INFO Received uploads request method=POST path=/api/pending_closures6232worker2 # [ 125.493508] nix-grpc-daemon[1621]: time=2026-09-20T12:47:05.850Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6233worker2 # [ 125.495083] nix-grpc-daemon[1621]: time=2026-09-20T12:47:05.852Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6234lb # [ 125.485325] niks3-server[1107]: 2026/09/20 12:47:06 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6235lb # [ 125.497355] niks3-server[1107]: 2026/09/20 12:47:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6236worker2 # [ 125.542694] nix-grpc-daemon[1621]: time=2026-09-20T12:47:05.899Z level=INFO msg="Uploading 1 narinfos"6237lb # [ 125.506837] niks3-server[1107]: 2026/09/20 12:47:06 INFO Signed narinfos id=44 count=16238lb # [ 125.512862] niks3-server[1107]: 2026/09/20 12:47:06 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6239lb # [ 125.515476] niks3-server[1107]: 2026/09/20 12:47:06 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6240lb # [ 125.520888] niks3-server[1107]: 2026/09/20 12:47:06 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6241lb # [ 125.524464] niks3-server[1107]: 2026/09/20 12:47:06 INFO Completed upload id=446242worker2 # [ 125.560264] nix-grpc-daemon[1621]: time=2026-09-20T12:47:05.917Z level=INFO msg="Upload complete. (248ms)"6243lb # [ 125.527060] niks3-server[1107]: 2026/09/20 12:47:06 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6244worker2 # [ 125.567248] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16245worker2 # [ 125.578568] nix-daemon[855]: reaped child process 1778, status = succeeded6246client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.33 seconds)6247client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6248client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6249worker1: must succeed: pkill -f 'read -t [9]0 x'6250worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6251client: waiting for success: ! systemctl is-active sw6252worker1 # [ 125.906558] nix-daemon[841]: reaped child process 1447, status = succeeded6253worker1 # [ 126.049829] nix-daemon[841]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)6254worker1 # [ 126.061640] nix-daemon[841]: reaped child process 1491, status = succeeded6255lb # [ 126.052337] niks3-server[1107]: 2026/09/20 12:47:07 INFO Received uploads request method=POST path=/api/pending_closures6256worker1 # [ 126.090913] nix-grpc-daemon[1391]: time=2026-09-20T12:47:06.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6257worker1 # [ 126.092393] nix-grpc-daemon[1391]: time=2026-09-20T12:47:06.400Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6258lb # [ 126.101053] niks3-server[1107]: 2026/09/20 12:47:07 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6259lb # [ 126.107572] niks3-server[1107]: 2026/09/20 12:47:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6260lb # [ 126.111871] niks3-server[1107]: 2026/09/20 12:47:07 INFO Signed narinfos id=45 count=16261worker1 # [ 126.139497] nix-grpc-daemon[1391]: time=2026-09-20T12:47:06.447Z level=INFO msg="Uploading 1 narinfos"6262lb # [ 126.117040] niks3-server[1107]: 2026/09/20 12:47:07 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6263lb # [ 126.123821] niks3-server[1107]: 2026/09/20 12:47:07 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6264lb # [ 126.126794] niks3-server[1107]: 2026/09/20 12:47:07 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6265lb # [ 126.136407] niks3-server[1107]: 2026/09/20 12:47:07 INFO Completed upload id=456266worker1 # [ 126.161146] nix-grpc-daemon[1391]: time=2026-09-20T12:47:06.469Z level=INFO msg="Upload complete. (262ms)"6267lb # [ 126.142856] niks3-server[1107]: 2026/09/20 12:47:07 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6268worker1 # [ 126.166526] nix-grpc-daemon[1294]: ts=2026-09-20T12:47:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60522 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16269worker1 # [ 126.178611] nix-daemon[841]: reaped child process 1445, status = succeeded6270client # [ 126.205445] systemd[1]: sw.service: Deactivated successfully.6271client # [ 126.209514] systemd[1]: sw.service: Consumed 128ms CPU time over 9.502s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.6272client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6273client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6274client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6275worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6276worker1 # [ 126.983986] nix-grpc-daemon[1294]: ts=2026-09-20T12:47:07Z level=info event=signal_exit6277worker2 # [ 128.218690] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:418566278worker2 # [ 128.222336] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:08Z level=info event=worker_gone addr=192.168.1.3:500516279worker1 # [ 128.213424] nix-grpc-daemon[1294]: ts=2026-09-20T12:47:08Z level=info event=scheduler_restarting6280worker1 # [ 128.219548] nix-grpc-daemon[1294]: ts=2026-09-20T12:47:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6281worker1 # [ 128.237350] nix-daemon[841]: reaped child process 1347, status = succeeded6282lb # [ 128.362221] envoy[985]: conn peer=[2001:db8:1::1]:55650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9546 tx=3877 ms=60756283lb # [ 128.365908] envoy[985]: conn peer=[2001:db8:1::1]:55638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2919 tx=1332 ms=92676284lb # [ 128.369566] envoy[985]: conn peer=[2001:db8:1::3]:42790 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1114 tx=425 ms=252306285worker1 # [ 128.826185] nix-grpc-daemon[1294]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6286worker1 # [ 128.829609] nix-grpc-daemon[1294]: E0000 00:00:1789908429.134077 1337 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6287worker1 # [ 128.834855] nix-grpc-daemon[1294]: E0000 00:00:1789908429.142741 1337 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6288worker1 # [ 129.017560] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6289worker1 # [ 129.020165] systemd[1]: nix-grpc-daemon.service: Consumed 636ms CPU time over 26.210s wall clock time, 30.2M memory peak, 21.1K incoming IP traffic, 27.7K outgoing IP traffic.6290worker1 # [ 129.036174] systemd[1]: Starting Nix worker-protocol over gRPC...6291worker1 # [ 129.190417] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6292worker1 # [ 129.192639] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6293worker1 # [ 129.196057] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16294worker1 # [ 129.202880] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6295worker1 # [ 129.208174] systemd[1]: Started Nix worker-protocol over gRPC.6296worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.25 seconds)6297lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6298lb: must succeed: curl -sf localhost:9901/clusters6299worker1 # [ 129.219675] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:09Z level=info event=scheduler_connected addr=lb:500516300worker2 # [ 129.233651] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:418566301worker2 # [ 129.235245] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06302lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6303lb: must succeed: curl -sf localhost:9901/clusters6304lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6305lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.15 seconds)6306lb: must succeed: curl -sf localhost:9901/clusters6307lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6308worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6309worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6310(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.79 seconds)6311subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6312client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16313worker1 # [ 130.690445] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:559546314worker1 # [ 130.702099] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6315worker2 # [ 130.729629] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846316worker2 # [ 130.774791] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:11Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06317worker2 # [ 130.778998] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:11Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16318worker2 # [ 130.783364] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:11Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06319worker2 # [ 130.787188] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:11Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06320worker1 # [ 130.807939] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546321worker1 # [ 130.819968] nix-grpc-daemon[1505]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5)6322worker1 # [ 130.867801] nix-grpc-daemon[1505]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 157 ms (attempt 2/5)6323worker1 # [ 131.030716] nix-grpc-daemon[1505]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 196 ms (attempt 3/5)6324worker1 # [ 131.232238] nix-grpc-daemon[1505]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 562 ms (attempt 4/5)6325worker1 # [ 131.798284] nix-grpc-daemon[1505]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6326worker1 # [ 131.816344] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546327worker1 # [ 131.828818] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546328worker1 # [ 131.840380] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6329worker1 # [ 131.858344] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:12Z level=info event=niks3_push_started pid=15616330worker1 # [ 132.018542] nix-grpc-daemon[1561]: time=2026-09-20T12:47:12.326Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6331worker1 # [ 132.151721] nix-daemon[841]: accepted connection from pid 1574, user nix-grpc-daemon (trusted)6332worker1 # [ 132.165389] nix-daemon[841]: reaped child process 1577, status = succeeded6333worker1 # [ 132.307504] nix-daemon[841]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6334worker1 # [ 132.320588] nix-daemon[841]: reaped child process 1582, status = succeeded6335lb # [ 132.312607] niks3-server[1107]: 2026/09/20 12:47:13 INFO Received uploads request method=POST path=/api/pending_closures6336worker1 # [ 132.344725] nix-grpc-daemon[1561]: time=2026-09-20T12:47:12.652Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6337worker1 # [ 132.347913] nix-grpc-daemon[1561]: time=2026-09-20T12:47:12.654Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6338lb # [ 132.349828] niks3-server[1107]: 2026/09/20 12:47:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6339lb # [ 132.356416] niks3-server[1107]: 2026/09/20 12:47:13 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6340lb # [ 132.361573] niks3-server[1107]: 2026/09/20 12:47:13 INFO Signed narinfos id=46 count=16341worker1 # [ 132.389262] nix-grpc-daemon[1561]: time=2026-09-20T12:47:12.697Z level=INFO msg="Uploading 1 narinfos"6342lb # [ 132.367444] niks3-server[1107]: 2026/09/20 12:47:13 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6343lb # [ 132.377870] niks3-server[1107]: 2026/09/20 12:47:13 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6344worker1 # [ 132.407194] nix-grpc-daemon[1561]: time=2026-09-20T12:47:12.715Z level=INFO msg="Upload complete. (384ms)"6345lb # [ 132.383157] niks3-server[1107]: 2026/09/20 12:47:13 INFO Completed upload id=466346worker1 # [ 132.409184] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=5046347worker1 # [ 132.412766] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546348lb # [ 132.388277] niks3-server[1107]: 2026/09/20 12:47:13 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6349worker1 # [ 132.419879] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6350worker1 # [ 132.422313] nix-daemon[841]: reaped child process 1559, status = succeeded6351worker1 # [ 132.427526] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6352worker1 # [ 133.456401] nix-daemon[841]: reaped child process 1586, status = succeeded6353worker1 # [ 133.613492] nix-daemon[841]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)6354worker1 # [ 133.625233] nix-daemon[841]: reaped child process 1604, status = succeeded6355lb # [ 133.610766] niks3-server[1107]: 2026/09/20 12:47:14 INFO Received uploads request method=POST path=/api/pending_closures6356worker1 # [ 133.643413] nix-grpc-daemon[1561]: time=2026-09-20T12:47:13.951Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6357worker1 # [ 133.645206] nix-grpc-daemon[1561]: time=2026-09-20T12:47:13.953Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6358lb # [ 133.651498] niks3-server[1107]: 2026/09/20 12:47:14 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6359lb # [ 133.658276] niks3-server[1107]: 2026/09/20 12:47:14 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6360lb # [ 133.669537] niks3-server[1107]: 2026/09/20 12:47:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6361worker1 # [ 133.700080] nix-grpc-daemon[1561]: time=2026-09-20T12:47:14.007Z level=INFO msg="Uploading 1 narinfos"6362lb # [ 133.674612] niks3-server[1107]: 2026/09/20 12:47:14 INFO Signed narinfos id=47 count=16363lb # [ 133.680664] niks3-server[1107]: 2026/09/20 12:47:14 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6364lb # [ 133.689784] niks3-server[1107]: 2026/09/20 12:47:14 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6365lb # [ 133.695902] niks3-server[1107]: 2026/09/20 12:47:14 INFO Completed upload id=476366worker1 # [ 133.720535] nix-grpc-daemon[1561]: time=2026-09-20T12:47:14.028Z level=INFO msg="Upload complete. (285ms)"6367lb # [ 133.697127] niks3-server[1107]: 2026/09/20 12:47:14 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6368worker1 # [ 133.725578] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16369worker2 # [ 133.737611] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:14Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16370worker1 # [ 133.732775] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546371worker2 # [ 133.746774] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:14Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06372worker1 # [ 133.743326] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546373worker1 # [ 133.746132] nix-daemon[841]: reaped child process 1584, status = succeeded6374worker1 # [ 133.749873] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546375worker1 # [ 133.753838] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6376worker1 # [ 133.907096] nix-daemon[841]: accepted connection from pid 1608, user nix-grpc-daemon (trusted)6377worker1 # [ 133.919090] nix-daemon[841]: reaped child process 1611, status = succeeded6378worker1 # [ 134.076965] nix-daemon[841]: accepted connection from pid 1613, user nix-grpc-daemon (trusted)6379worker1 # [ 134.088883] nix-daemon[841]: reaped child process 1616, status = succeeded6380lb # [ 134.079351] niks3-server[1107]: 2026/09/20 12:47:15 INFO Received uploads request method=POST path=/api/pending_closures6381worker1 # [ 134.108673] nix-grpc-daemon[1561]: time=2026-09-20T12:47:14.416Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6382worker1 # [ 134.110158] nix-grpc-daemon[1561]: time=2026-09-20T12:47:14.418Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6383lb # [ 134.097786] niks3-server[1107]: 2026/09/20 12:47:15 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6384lb # [ 134.107092] niks3-server[1107]: 2026/09/20 12:47:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6385worker1 # [ 134.133784] nix-grpc-daemon[1561]: time=2026-09-20T12:47:14.441Z level=INFO msg="Uploading 1 narinfos"6386lb # [ 134.109481] niks3-server[1107]: 2026/09/20 12:47:15 INFO Signed narinfos id=48 count=16387lb # [ 134.121213] niks3-server[1107]: 2026/09/20 12:47:15 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6388lb # [ 134.124214] niks3-server[1107]: 2026/09/20 12:47:15 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6389lb # [ 134.127626] niks3-server[1107]: 2026/09/20 12:47:15 INFO Completed upload id=486390worker1 # [ 134.151997] nix-grpc-daemon[1561]: time=2026-09-20T12:47:14.460Z level=INFO msg="Upload complete. (392ms)"6391worker1 # [ 134.153961] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=5046392lb # [ 134.131058] niks3-server[1107]: 2026/09/20 12:47:15 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6393worker1 # [ 134.158350] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546394worker1 # [ 134.163906] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6395worker1 # [ 134.166139] nix-daemon[841]: reaped child process 1606, status = succeeded6396worker1 # [ 134.171360] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6397worker1 # [ 135.399328] nix-daemon[841]: reaped child process 1620, status = succeeded6398worker1 # [ 135.548261] nix-daemon[841]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6399worker1 # [ 135.560325] nix-daemon[841]: reaped child process 1638, status = succeeded6400lb # [ 135.552417] niks3-server[1107]: 2026/09/20 12:47:16 INFO Received uploads request method=POST path=/api/pending_closures6401worker1 # [ 135.584985] nix-grpc-daemon[1561]: time=2026-09-20T12:47:15.892Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6402worker1 # [ 135.586497] nix-grpc-daemon[1561]: time=2026-09-20T12:47:15.894Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6403lb # [ 135.589657] niks3-server[1107]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6404lb # [ 135.600382] niks3-server[1107]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6405lb # [ 135.609114] niks3-server[1107]: 2026/09/20 12:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6406worker1 # [ 135.639349] nix-grpc-daemon[1561]: time=2026-09-20T12:47:15.947Z level=INFO msg="Uploading 1 narinfos"6407lb # [ 135.614712] niks3-server[1107]: 2026/09/20 12:47:16 INFO Signed narinfos id=49 count=16408lb # [ 135.619140] niks3-server[1107]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6409lb # [ 135.629219] niks3-server[1107]: 2026/09/20 12:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6410lb # [ 135.635427] niks3-server[1107]: 2026/09/20 12:47:16 INFO Completed upload id=496411worker1 # [ 135.659942] nix-grpc-daemon[1561]: time=2026-09-20T12:47:15.968Z level=INFO msg="Upload complete. (280ms)"6412lb # [ 135.637098] niks3-server[1107]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6413worker1 # [ 135.663321] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16414worker2 # [ 135.675925] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:16Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16415worker1 # [ 135.666760] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546416worker2 # [ 135.682044] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:16Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06417worker1 # [ 135.673303] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546418worker1 # [ 135.675607] nix-daemon[841]: reaped child process 1618, status = succeeded6419worker1 # [ 135.679039] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546420worker1 # [ 135.683742] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6421worker1 # [ 135.832099] nix-daemon[841]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6422worker1 # [ 135.844260] nix-daemon[841]: reaped child process 1645, status = succeeded6423worker1 # [ 135.974178] nix-daemon[841]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6424worker1 # [ 135.986383] nix-daemon[841]: reaped child process 1650, status = succeeded6425lb # [ 135.972229] niks3-server[1107]: 2026/09/20 12:47:17 INFO Received uploads request method=POST path=/api/pending_closures6426worker1 # [ 136.005882] nix-grpc-daemon[1561]: time=2026-09-20T12:47:16.313Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6427worker1 # [ 136.007266] nix-grpc-daemon[1561]: time=2026-09-20T12:47:16.313Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6428lb # [ 135.999198] niks3-server[1107]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6429lb # [ 136.010493] niks3-server[1107]: 2026/09/20 12:47:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6430lb # [ 136.017158] niks3-server[1107]: 2026/09/20 12:47:17 INFO Signed narinfos id=50 count=16431worker1 # [ 136.044696] nix-grpc-daemon[1561]: time=2026-09-20T12:47:16.352Z level=INFO msg="Uploading 1 narinfos"6432lb # [ 136.021979] niks3-server[1107]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6433lb # [ 136.032930] niks3-server[1107]: 2026/09/20 12:47:17 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6434lb # [ 136.037873] niks3-server[1107]: 2026/09/20 12:47:17 INFO Completed upload id=506435worker1 # [ 136.062209] nix-grpc-daemon[1561]: time=2026-09-20T12:47:16.370Z level=INFO msg="Upload complete. (373ms)"6436worker1 # [ 136.065221] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=5046437lb # [ 136.041993] niks3-server[1107]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6438worker1 # [ 136.070670] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546439worker1 # [ 136.075912] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6440worker1 # [ 136.082798] nix-daemon[841]: reaped child process 1640, status = succeeded6441worker1 # [ 136.086718] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6442worker1 # [ 137.513252] nix-daemon[841]: reaped child process 1654, status = succeeded6443worker1 # [ 137.643959] nix-daemon[841]: accepted connection from pid 1669, user nix-grpc-daemon (trusted)6444worker1 # [ 137.656319] nix-daemon[841]: reaped child process 1672, status = succeeded6445lb # [ 137.642577] niks3-server[1107]: 2026/09/20 12:47:18 INFO Received uploads request method=POST path=/api/pending_closures6446worker1 # [ 137.677373] nix-grpc-daemon[1561]: time=2026-09-20T12:47:17.985Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6447worker1 # [ 137.678859] nix-grpc-daemon[1561]: time=2026-09-20T12:47:17.986Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6448lb # [ 137.685630] niks3-server[1107]: 2026/09/20 12:47:18 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6449lb # [ 137.689736] niks3-server[1107]: 2026/09/20 12:47:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6450lb # [ 137.692277] niks3-server[1107]: 2026/09/20 12:47:18 INFO Signed narinfos id=51 count=16451worker1 # [ 137.718167] nix-grpc-daemon[1561]: time=2026-09-20T12:47:18.026Z level=INFO msg="Uploading 1 narinfos"6452lb # [ 137.698910] niks3-server[1107]: 2026/09/20 12:47:18 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6453lb # [ 137.730546] niks3-server[1107]: 2026/09/20 12:47:18 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6454lb # [ 137.736752] niks3-server[1107]: 2026/09/20 12:47:18 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6455lb # [ 137.742107] niks3-server[1107]: 2026/09/20 12:47:18 INFO Completed upload id=516456worker1 # [ 137.766582] nix-grpc-daemon[1561]: time=2026-09-20T12:47:18.074Z level=INFO msg="Upload complete. (272ms)"6457lb # [ 137.745942] niks3-server[1107]: 2026/09/20 12:47:18 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6458worker1 # [ 137.773106] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16459worker2 # [ 137.787675] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:18Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16460worker1 # [ 137.782404] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546461worker2 # [ 137.797601] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:18Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06462worker1 # [ 137.796868] nix-daemon[841]: reaped child process 1652, status = succeeded6463worker1 # [ 137.822542] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546464worker1 # [ 137.832396] nix-grpc-daemon[1561]: time=2026-09-20T12:47:18.140Z level=INFO msg="All 1 paths already cached"6465worker1 # [ 137.838836] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546466worker1 # [ 137.848347] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6467worker1 # [ 137.996563] nix-daemon[841]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)6468worker1 # [ 138.009706] nix-daemon[841]: reaped child process 1679, status = succeeded6469worker1 # [ 138.163154] nix-daemon[841]: accepted connection from pid 1681, user nix-grpc-daemon (trusted)6470worker1 # [ 138.175646] nix-daemon[841]: reaped child process 1684, status = succeeded6471lb # [ 138.165832] niks3-server[1107]: 2026/09/20 12:47:19 INFO Received uploads request method=POST path=/api/pending_closures6472worker1 # [ 138.197246] nix-grpc-daemon[1561]: time=2026-09-20T12:47:18.505Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6473worker1 # [ 138.198778] nix-grpc-daemon[1561]: time=2026-09-20T12:47:18.506Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6474lb # [ 138.191861] niks3-server[1107]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6475lb # [ 138.203934] niks3-server[1107]: 2026/09/20 12:47:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6476lb # [ 138.210858] niks3-server[1107]: 2026/09/20 12:47:19 INFO Signed narinfos id=52 count=16477worker1 # [ 138.238150] nix-grpc-daemon[1561]: time=2026-09-20T12:47:18.546Z level=INFO msg="Uploading 1 narinfos"6478lb # [ 138.214238] niks3-server[1107]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6479lb # [ 138.226741] niks3-server[1107]: 2026/09/20 12:47:19 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6480lb # [ 138.339636] niks3-server[1107]: 2026/09/20 12:47:19 INFO Completed upload id=526481worker1 # [ 138.367642] nix-grpc-daemon[1561]: time=2026-09-20T12:47:18.675Z level=INFO msg="Upload complete. (508ms)"6482lb # [ 138.348891] niks3-server[1107]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6483worker1 # [ 138.373435] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=6246484worker1 # [ 138.379111] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546485worker1 # [ 138.392129] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6486worker1 # [ 138.400756] nix-daemon[841]: reaped child process 1674, status = succeeded6487worker1 # [ 138.412845] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6488worker1 # [ 139.445672] nix-daemon[841]: reaped child process 1688, status = succeeded6489worker1 # [ 139.582091] nix-daemon[841]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)6490worker1 # [ 139.593777] nix-daemon[841]: reaped child process 1706, status = succeeded6491lb # [ 139.582742] niks3-server[1107]: 2026/09/20 12:47:20 INFO Received uploads request method=POST path=/api/pending_closures6492worker1 # [ 139.615257] nix-grpc-daemon[1561]: time=2026-09-20T12:47:19.923Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6493worker1 # [ 139.617144] nix-grpc-daemon[1561]: time=2026-09-20T12:47:19.924Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6494lb # [ 139.617792] niks3-server[1107]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6495lb # [ 139.629399] niks3-server[1107]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6496lb # [ 139.642055] niks3-server[1107]: 2026/09/20 12:47:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6497worker1 # [ 139.671902] nix-grpc-daemon[1561]: time=2026-09-20T12:47:19.979Z level=INFO msg="Uploading 1 narinfos"6498lb # [ 139.647372] niks3-server[1107]: 2026/09/20 12:47:20 INFO Signed narinfos id=53 count=16499lb # [ 139.650682] niks3-server[1107]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6500lb # [ 139.659620] niks3-server[1107]: 2026/09/20 12:47:20 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6501lb # [ 139.663437] niks3-server[1107]: 2026/09/20 12:47:20 INFO Completed upload id=536502lb # [ 139.664951] niks3-server[1107]: 2026/09/20 12:47:20 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6503worker1 # [ 139.688071] nix-grpc-daemon[1561]: time=2026-09-20T12:47:19.996Z level=INFO msg="Upload complete. (260ms)"6504worker2 # [ 139.706883] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:20Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16505worker1 # [ 139.695292] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16506worker1 # [ 139.702637] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546507worker2 # [ 139.714555] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:20Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06508worker1 # [ 139.711563] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546509worker1 # [ 139.720206] nix-daemon[841]: reaped child process 1686, status = succeeded6510worker1 # [ 139.722448] nix-grpc-daemon[1561]: time=2026-09-20T12:47:20.030Z level=INFO msg="All 1 paths already cached"6511worker1 # [ 139.726845] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546512worker1 # [ 139.730597] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6513worker1 # [ 139.877220] nix-daemon[841]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6514worker1 # [ 139.889778] nix-daemon[841]: reaped child process 1713, status = succeeded6515worker1 # [ 140.050120] nix-daemon[841]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6516worker1 # [ 140.062324] nix-daemon[841]: reaped child process 1718, status = succeeded6517lb # [ 140.052356] niks3-server[1107]: 2026/09/20 12:47:21 INFO Received uploads request method=POST path=/api/pending_closures6518worker1 # [ 140.084401] nix-grpc-daemon[1561]: time=2026-09-20T12:47:20.392Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6519worker1 # [ 140.085961] nix-grpc-daemon[1561]: time=2026-09-20T12:47:20.394Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6520lb # [ 140.082974] niks3-server[1107]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6521lb # [ 140.092326] niks3-server[1107]: 2026/09/20 12:47:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6522lb # [ 140.098876] niks3-server[1107]: 2026/09/20 12:47:21 INFO Signed narinfos id=54 count=16523worker1 # [ 140.126238] nix-grpc-daemon[1561]: time=2026-09-20T12:47:20.434Z level=INFO msg="Uploading 1 narinfos"6524lb # [ 140.102352] niks3-server[1107]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6525lb # [ 140.115277] niks3-server[1107]: 2026/09/20 12:47:21 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6526lb # [ 140.121339] niks3-server[1107]: 2026/09/20 12:47:21 INFO Completed upload id=546527worker1 # [ 140.145605] nix-grpc-daemon[1561]: time=2026-09-20T12:47:20.453Z level=INFO msg="Upload complete. (408ms)"6528lb # [ 140.123970] niks3-server[1107]: 2026/09/20 12:47:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6529worker1 # [ 140.149061] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=6246530worker1 # [ 140.154524] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546531worker1 # [ 140.164667] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6532worker1 # [ 140.168269] nix-daemon[841]: reaped child process 1708, status = succeeded6533worker1 # [ 140.173105] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6534worker1 # [ 141.599509] nix-daemon[841]: reaped child process 1722, status = succeeded6535worker1 # [ 141.757548] nix-daemon[841]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6536worker1 # [ 141.769683] nix-daemon[841]: reaped child process 1740, status = succeeded6537lb # [ 141.762400] niks3-server[1107]: 2026/09/20 12:47:22 INFO Received uploads request method=POST path=/api/pending_closures6538worker1 # [ 141.797036] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.104Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6539worker1 # [ 141.798526] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.106Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6540lb # [ 141.806121] niks3-server[1107]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6541lb # [ 141.818600] niks3-server[1107]: 2026/09/20 12:47:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6542worker1 # [ 141.849708] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.157Z level=INFO msg="Uploading 1 narinfos"6543lb # [ 141.825205] niks3-server[1107]: 2026/09/20 12:47:22 INFO Signed narinfos id=55 count=16544lb # [ 141.827402] niks3-server[1107]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6545lb # [ 141.832068] niks3-server[1107]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6546lb # [ 141.839833] niks3-server[1107]: 2026/09/20 12:47:22 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6547lb # [ 141.842688] niks3-server[1107]: 2026/09/20 12:47:22 INFO Completed upload id=556548worker1 # [ 141.867285] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.175Z level=INFO msg="Upload complete. (286ms)"6549lb # [ 141.845064] niks3-server[1107]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6550worker1 # [ 141.875749] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16551worker2 # [ 141.887922] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:22Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16552worker1 # [ 141.881270] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546553worker2 # [ 141.893615] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:22Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06554worker1 # [ 141.891581] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546555worker1 # [ 141.898134] nix-daemon[841]: reaped child process 1720, status = succeeded6556worker1 # [ 141.900131] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.208Z level=INFO msg="All 1 paths already cached"6557worker1 # [ 141.902997] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546558worker1 # [ 141.908091] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6559worker1 # [ 142.064654] nix-daemon[841]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6560worker1 # [ 142.077596] nix-daemon[841]: reaped child process 1747, status = succeeded6561worker1 # [ 142.234671] nix-daemon[841]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6562worker1 # [ 142.248946] nix-daemon[841]: reaped child process 1752, status = succeeded6563lb # [ 142.240668] niks3-server[1107]: 2026/09/20 12:47:23 INFO Received uploads request method=POST path=/api/pending_closures6564worker1 # [ 142.272819] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6565worker1 # [ 142.274193] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.580Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6566lb # [ 142.267485] niks3-server[1107]: 2026/09/20 12:47:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6567lb # [ 142.271831] niks3-server[1107]: 2026/09/20 12:47:23 INFO Signed narinfos id=56 count=16568worker1 # [ 142.299422] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.607Z level=INFO msg="Uploading 1 narinfos"6569lb # [ 142.280297] niks3-server[1107]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6570lb # [ 142.285141] niks3-server[1107]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6571lb # [ 142.296754] niks3-server[1107]: 2026/09/20 12:47:23 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6572worker1 # [ 142.326177] nix-grpc-daemon[1561]: time=2026-09-20T12:47:22.634Z level=INFO msg="Upload complete. (413ms)"6573lb # [ 142.301753] niks3-server[1107]: 2026/09/20 12:47:23 INFO Completed upload id=566574worker1 # [ 142.329090] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=6246575worker1 # [ 142.331669] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546576lb # [ 142.306585] niks3-server[1107]: 2026/09/20 12:47:23 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6577worker1 # [ 142.336468] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6578worker1 # [ 142.341780] nix-daemon[841]: reaped child process 1742, status = succeeded6579worker1 # [ 142.345959] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6580worker1 # [ 143.772365] nix-daemon[841]: reaped child process 1756, status = succeeded6581worker1 # [ 143.914618] nix-daemon[841]: accepted connection from pid 1771, user nix-grpc-daemon (trusted)6582worker1 # [ 143.926656] nix-daemon[841]: reaped child process 1774, status = succeeded6583lb # [ 143.918139] niks3-server[1107]: 2026/09/20 12:47:25 INFO Received uploads request method=POST path=/api/pending_closures6584worker1 # [ 143.951457] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.259Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6585worker1 # [ 143.952993] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.261Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6586lb # [ 143.953978] niks3-server[1107]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6587lb # [ 143.960998] niks3-server[1107]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6588lb # [ 143.973650] niks3-server[1107]: 2026/09/20 12:47:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6589worker1 # [ 144.003257] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.311Z level=INFO msg="Uploading 1 narinfos"6590lb # [ 143.979910] niks3-server[1107]: 2026/09/20 12:47:25 INFO Signed narinfos id=57 count=16591lb # [ 143.984099] niks3-server[1107]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6592lb # [ 143.994335] niks3-server[1107]: 2026/09/20 12:47:25 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6593lb # [ 144.000317] niks3-server[1107]: 2026/09/20 12:47:25 INFO Completed upload id=576594worker1 # [ 144.024549] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.332Z level=INFO msg="Upload complete. (271ms)"6595lb # [ 144.002470] niks3-server[1107]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6596worker1 # [ 144.027686] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16597worker2 # [ 144.039584] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:24Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16598worker1 # [ 144.035429] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546599worker1 # [ 144.037229] nix-daemon[841]: reaped child process 1754, status = succeeded6600worker2 # [ 144.049806] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:24Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06601worker1 # [ 144.042510] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546602worker1 # [ 144.047803] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.355Z level=INFO msg="All 2 paths already cached"6603worker1 # [ 144.051371] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546604worker1 # [ 144.059674] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6605worker1 # [ 144.203968] nix-daemon[841]: accepted connection from pid 1778, user nix-grpc-daemon (trusted)6606worker1 # [ 144.217084] nix-daemon[841]: reaped child process 1781, status = succeeded6607worker1 # [ 144.374212] nix-daemon[841]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6608worker1 # [ 144.386187] nix-daemon[841]: reaped child process 1786, status = succeeded6609lb # [ 144.373401] niks3-server[1107]: 2026/09/20 12:47:25 INFO Received uploads request method=POST path=/api/pending_closures6610worker1 # [ 144.405542] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.713Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6611worker1 # [ 144.406927] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.713Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6612lb # [ 144.395872] niks3-server[1107]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6613lb # [ 144.408730] niks3-server[1107]: 2026/09/20 12:47:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6614worker1 # [ 144.439724] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.747Z level=INFO msg="Uploading 1 narinfos"6615lb # [ 144.415320] niks3-server[1107]: 2026/09/20 12:47:25 INFO Signed narinfos id=58 count=16616lb # [ 144.419738] niks3-server[1107]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6617lb # [ 144.431066] niks3-server[1107]: 2026/09/20 12:47:25 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6618lb # [ 144.436555] niks3-server[1107]: 2026/09/20 12:47:25 INFO Completed upload id=586619worker1 # [ 144.463236] nix-grpc-daemon[1561]: time=2026-09-20T12:47:24.771Z level=INFO msg="Upload complete. (397ms)"6620lb # [ 144.439821] niks3-server[1107]: 2026/09/20 12:47:25 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6621worker1 # [ 144.465345] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=7526622worker1 # [ 144.470126] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546623worker1 # [ 144.475341] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6624worker1 # [ 144.477623] nix-daemon[841]: reaped child process 1776, status = succeeded6625worker1 # [ 144.482855] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6626worker1 # [ 145.499409] nix-daemon[841]: reaped child process 1790, status = succeeded6627worker1 # [ 145.633124] nix-daemon[841]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)6628worker1 # [ 145.644669] nix-daemon[841]: reaped child process 1808, status = succeeded6629lb # [ 145.634363] niks3-server[1107]: 2026/09/20 12:47:26 INFO Received uploads request method=POST path=/api/pending_closures6630worker1 # [ 145.666861] nix-grpc-daemon[1561]: time=2026-09-20T12:47:25.974Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6631worker1 # [ 145.669061] nix-grpc-daemon[1561]: time=2026-09-20T12:47:25.976Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6632lb # [ 145.672511] niks3-server[1107]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6633lb # [ 145.683651] niks3-server[1107]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6634lb # [ 145.691063] niks3-server[1107]: 2026/09/20 12:47:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6635worker1 # [ 145.720261] nix-grpc-daemon[1561]: time=2026-09-20T12:47:26.028Z level=INFO msg="Uploading 1 narinfos"6636lb # [ 145.695841] niks3-server[1107]: 2026/09/20 12:47:26 INFO Signed narinfos id=59 count=16637lb # [ 145.700490] niks3-server[1107]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6638lb # [ 145.709139] niks3-server[1107]: 2026/09/20 12:47:26 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6639lb # [ 145.713540] niks3-server[1107]: 2026/09/20 12:47:26 INFO Completed upload id=596640worker1 # [ 145.738116] nix-grpc-daemon[1561]: time=2026-09-20T12:47:26.046Z level=WARN msg="Failed to register uploaded object" key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo error="server returned 404: object is not pending upload\n"6641worker1 # [ 145.745839] nix-grpc-daemon[1561]: time=2026-09-20T12:47:26.051Z level=INFO msg="Upload complete. (253ms)"6642worker2 # [ 145.758352] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:26Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16643worker1 # [ 145.750460] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16644worker1 # [ 145.761404] nix-daemon[841]: reaped child process 1788, status = succeeded6645worker1 # [ 145.790772] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546646worker1 # [ 145.802908] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546647worker1 # [ 145.814997] nix-grpc-daemon[1561]: time=2026-09-20T12:47:26.123Z level=INFO msg="All 2 paths already cached"6648worker1 # [ 145.819570] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546649worker1 # [ 145.824438] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6650worker1 # [ 145.958309] nix-daemon[841]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6651worker1 # [ 145.971664] nix-daemon[841]: reaped child process 1815, status = succeeded6652worker1 # [ 146.132987] nix-daemon[841]: accepted connection from pid 1817, user nix-grpc-daemon (trusted)6653worker1 # [ 146.145085] nix-daemon[841]: reaped child process 1820, status = succeeded6654lb # [ 146.135949] niks3-server[1107]: 2026/09/20 12:47:27 INFO Received uploads request method=POST path=/api/pending_closures6655worker1 # [ 146.167819] nix-grpc-daemon[1561]: time=2026-09-20T12:47:26.475Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6656worker1 # [ 146.169190] nix-grpc-daemon[1561]: time=2026-09-20T12:47:26.475Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6657lb # [ 146.158819] niks3-server[1107]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6658lb # [ 146.172558] niks3-server[1107]: 2026/09/20 12:47:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6659worker1 # [ 146.201735] nix-grpc-daemon[1561]: time=2026-09-20T12:47:26.509Z level=INFO msg="Uploading 1 narinfos"6660lb # [ 146.177794] niks3-server[1107]: 2026/09/20 12:47:27 INFO Signed narinfos id=60 count=16661lb # [ 146.182619] niks3-server[1107]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6662lb # [ 146.193509] niks3-server[1107]: 2026/09/20 12:47:27 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6663lb # [ 146.199407] niks3-server[1107]: 2026/09/20 12:47:27 INFO Completed upload id=606664worker1 # [ 146.226068] nix-grpc-daemon[1561]: time=2026-09-20T12:47:26.534Z level=INFO msg="Upload complete. (396ms)"6665lb # [ 146.202141] niks3-server[1107]: 2026/09/20 12:47:27 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6666worker1 # [ 146.228902] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=7526667worker1 # [ 146.233781] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546668worker1 # [ 146.242586] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6669worker1 # [ 146.247500] nix-daemon[841]: reaped child process 1810, status = succeeded6670worker1 # [ 146.252572] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6671worker1 # [ 146.865496] nix-daemon[841]: reaped child process 1824, status = succeeded6672worker1 # [ 146.992828] nix-daemon[841]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6673worker1 # [ 147.005218] nix-daemon[841]: reaped child process 1842, status = succeeded6674lb # [ 146.992749] niks3-server[1107]: 2026/09/20 12:47:28 INFO Received uploads request method=POST path=/api/pending_closures6675worker1 # [ 147.025407] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.333Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6676worker1 # [ 147.027329] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.334Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6677lb # [ 147.027136] niks3-server[1107]: 2026/09/20 12:47:28 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6678lb # [ 147.040958] niks3-server[1107]: 2026/09/20 12:47:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6679worker1 # [ 147.073345] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.381Z level=INFO msg="Uploading 1 narinfos"6680lb # [ 147.048773] niks3-server[1107]: 2026/09/20 12:47:28 INFO Signed narinfos id=61 count=16681lb # [ 147.054265] niks3-server[1107]: 2026/09/20 12:47:28 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6682lb # [ 147.057824] niks3-server[1107]: 2026/09/20 12:47:28 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6683lb # [ 147.066465] niks3-server[1107]: 2026/09/20 12:47:28 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6684lb # [ 147.072276] niks3-server[1107]: 2026/09/20 12:47:28 INFO Completed upload id=616685worker1 # [ 147.096674] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.404Z level=INFO msg="Upload complete. (238ms)"6686lb # [ 147.074862] niks3-server[1107]: 2026/09/20 12:47:28 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6687worker1 # [ 147.102259] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16688worker2 # [ 147.114323] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:27Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16689worker1 # [ 147.113437] nix-daemon[841]: reaped child process 1822, status = succeeded6690worker1 # [ 147.146345] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546691worker1 # [ 147.158510] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546692worker1 # [ 147.170768] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.478Z level=INFO msg="All 2 paths already cached"6693worker1 # [ 147.178199] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559546694worker1 # [ 147.186571] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6695worker1 # [ 147.347982] nix-daemon[841]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6696worker1 # [ 147.361630] nix-daemon[841]: reaped child process 1849, status = succeeded6697worker1 # [ 147.492144] nix-daemon[841]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6698worker1 # [ 147.504155] nix-daemon[841]: reaped child process 1854, status = succeeded6699lb # [ 147.495602] niks3-server[1107]: 2026/09/20 12:47:28 INFO Received uploads request method=POST path=/api/pending_closures6700worker1 # [ 147.527436] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.835Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6701worker1 # [ 147.529147] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.837Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6702lb # [ 147.525248] niks3-server[1107]: 2026/09/20 12:47:28 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6703lb # [ 147.533201] niks3-server[1107]: 2026/09/20 12:47:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6704lb # [ 147.538522] niks3-server[1107]: 2026/09/20 12:47:28 INFO Signed narinfos id=62 count=16705worker1 # [ 147.566227] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.874Z level=INFO msg="Uploading 1 narinfos"6706lb # [ 147.544647] niks3-server[1107]: 2026/09/20 12:47:28 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6707lb # [ 147.556944] niks3-server[1107]: 2026/09/20 12:47:28 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6708worker1 # [ 147.587082] nix-grpc-daemon[1561]: time=2026-09-20T12:47:27.895Z level=INFO msg="Upload complete. (387ms)"6709lb # [ 147.562830] niks3-server[1107]: 2026/09/20 12:47:28 INFO Completed upload id=626710lb # [ 147.566254] niks3-server[1107]: 2026/09/20 12:47:28 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6711worker1 # [ 147.589808] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=0 paths=1 nar_bytes_in=7526712worker1 # [ 147.594796] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559546713worker1 # [ 147.604775] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6714worker1 # [ 147.607968] nix-daemon[841]: reaped child process 1844, status = succeeded6715worker1 # [ 147.613173] nix-daemon[841]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)6716worker1 # [ 148.425770] nix-daemon[841]: reaped child process 1858, status = succeeded6717worker1 # [ 148.571436] nix-daemon[841]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6718worker1 # [ 148.583895] nix-daemon[841]: reaped child process 1876, status = succeeded6719lb # [ 148.575327] niks3-server[1107]: 2026/09/20 12:47:29 INFO Received uploads request method=POST path=/api/pending_closures6720worker1 # [ 148.607457] nix-grpc-daemon[1561]: time=2026-09-20T12:47:28.915Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6721worker1 # [ 148.609234] nix-grpc-daemon[1561]: time=2026-09-20T12:47:28.917Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6722lb # [ 148.617125] niks3-server[1107]: 2026/09/20 12:47:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6723lb # [ 148.624326] niks3-server[1107]: 2026/09/20 12:47:29 INFO Signed narinfos id=63 count=16724worker1 # [ 148.651959] nix-grpc-daemon[1561]: time=2026-09-20T12:47:28.959Z level=INFO msg="Uploading 1 narinfos"6725lb # [ 148.631237] niks3-server[1107]: 2026/09/20 12:47:29 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6726lb # [ 148.635899] niks3-server[1107]: 2026/09/20 12:47:29 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6727lb # [ 148.640101] niks3-server[1107]: 2026/09/20 12:47:29 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6728lb # [ 148.648648] niks3-server[1107]: 2026/09/20 12:47:29 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6729lb # [ 148.651653] niks3-server[1107]: 2026/09/20 12:47:29 INFO Completed upload id=636730worker1 # [ 148.676262] nix-grpc-daemon[1561]: time=2026-09-20T12:47:28.984Z level=INFO msg="Upload complete. (256ms)"6731lb # [ 148.653877] niks3-server[1107]: 2026/09/20 12:47:29 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6732worker1 # [ 148.682321] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55954 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16733worker1 # [ 148.692937] nix-daemon[841]: reaped child process 1856, status = succeeded6734client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.27 seconds)6735worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6736worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6737client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16738worker2 # [ 149.012230] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:412846739worker1 # [ 149.004345] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559546740worker2 # [ 149.035520] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:29Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06741client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.25 seconds)6742(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 18.58 seconds)6743subtest: low disk drains a worker and builds go to the other6744worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6745worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.98 seconds)6746worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6747worker1 # [ 150.211331] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:30Z level=info event=unhealthy reason=min_free6748worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)6749client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26750worker2 # [ 151.371837] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:412846751client # don't know how to build these paths:6752client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6753worker2 # [ 151.380964] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846754worker2 # [ 151.401532] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:31Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06755worker2 # [ 151.406405] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:31Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16756worker2 # [ 151.444877] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846757worker2 # [ 151.456464] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846758client # copying 1 paths to 'lb:50051'...6759client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6760worker2 # [ 151.467633] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412846761worker2 # [ 151.478356] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6762worker2 # [ 151.654061] nix-daemon[855]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)6763worker2 # [ 151.667349] nix-daemon[855]: reaped child process 1826, status = succeeded6764worker2 # [ 151.806683] nix-daemon[855]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)6765worker2 # [ 151.818799] nix-daemon[855]: reaped child process 1831, status = succeeded6766lb # [ 151.795539] niks3-server[1107]: 2026/09/20 12:47:32 INFO Received uploads request method=POST path=/api/pending_closures6767worker2 # [ 151.840979] nix-grpc-daemon[1621]: time=2026-09-20T12:47:32.197Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6768worker2 # [ 151.842358] nix-grpc-daemon[1621]: time=2026-09-20T12:47:32.197Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6769lb # [ 151.821960] niks3-server[1107]: 2026/09/20 12:47:32 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6770lb # [ 151.832612] niks3-server[1107]: 2026/09/20 12:47:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6771lb # [ 151.837761] niks3-server[1107]: 2026/09/20 12:47:32 INFO Signed narinfos id=64 count=16772worker2 # [ 151.877405] nix-grpc-daemon[1621]: time=2026-09-20T12:47:32.234Z level=INFO msg="Uploading 1 narinfos"6773lb # [ 151.843616] niks3-server[1107]: 2026/09/20 12:47:32 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6774lb # [ 151.854230] niks3-server[1107]: 2026/09/20 12:47:33 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6775lb # [ 151.861274] niks3-server[1107]: 2026/09/20 12:47:33 INFO Completed upload id=646776worker2 # [ 151.896579] nix-grpc-daemon[1621]: time=2026-09-20T12:47:32.253Z level=INFO msg="Upload complete. (406ms)"6777lb # [ 151.864224] niks3-server[1107]: 2026/09/20 12:47:33 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6778worker2 # [ 151.901133] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=0 paths=1 nar_bytes_in=5126779worker2 # [ 151.906649] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846780worker2 # [ 151.916442] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6781worker2 # [ 151.919083] nix-daemon[855]: reaped child process 1821, status = succeeded6782worker2 # [ 151.924082] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6783client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6784client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6785client # farm-a> LOG-farm-a-drain6786worker2 # [ 152.948709] nix-daemon[855]: reaped child process 1835, status = succeeded6787worker2 # [ 153.106527] nix-daemon[855]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6788worker2 # [ 153.118517] nix-daemon[855]: reaped child process 1853, status = succeeded6789lb # [ 153.096380] niks3-server[1107]: 2026/09/20 12:47:34 INFO Received uploads request method=POST path=/api/pending_closures6790worker2 # [ 153.138817] nix-grpc-daemon[1621]: time=2026-09-20T12:47:33.495Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6791worker2 # [ 153.140275] nix-grpc-daemon[1621]: time=2026-09-20T12:47:33.497Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6792lb # [ 153.137256] niks3-server[1107]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6793lb # [ 153.147759] niks3-server[1107]: 2026/09/20 12:47:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6794lb # [ 153.154757] niks3-server[1107]: 2026/09/20 12:47:34 INFO Signed narinfos id=65 count=16795worker2 # [ 153.193466] nix-grpc-daemon[1621]: time=2026-09-20T12:47:33.550Z level=INFO msg="Uploading 1 narinfos"6796lb # [ 153.160055] niks3-server[1107]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6797lb # [ 153.162980] niks3-server[1107]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6798lb # [ 153.173746] niks3-server[1107]: 2026/09/20 12:47:34 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6799lb # [ 153.179054] niks3-server[1107]: 2026/09/20 12:47:34 INFO Completed upload id=656800worker2 # [ 153.217190] nix-grpc-daemon[1621]: time=2026-09-20T12:47:33.574Z level=INFO msg="Upload complete. (284ms)"6801lb # [ 153.183130] niks3-server[1107]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6802worker2 # [ 153.222837] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16803worker2 # [ 153.234641] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:33Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06804worker2 # [ 153.237886] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:33Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16805worker2 # [ 153.246460] nix-daemon[855]: reaped child process 1833, status = succeeded6806worker2 # [ 153.277889] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846807worker2 # [ 153.288357] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846808worker2 # [ 153.298937] nix-grpc-daemon[1621]: time=2026-09-20T12:47:33.656Z level=INFO msg="All 1 paths already cached"6809client # copying 1 paths to 'lb:50051'...6810client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6811worker2 # [ 153.306521] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412846812worker2 # [ 153.315232] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6813worker2 # [ 153.482563] nix-daemon[855]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)6814worker2 # [ 153.494795] nix-daemon[855]: reaped child process 1862, status = succeeded6815worker2 # [ 153.657077] nix-daemon[855]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6816worker2 # [ 153.669484] nix-daemon[855]: reaped child process 1867, status = succeeded6817lb # [ 153.650379] niks3-server[1107]: 2026/09/20 12:47:34 INFO Received uploads request method=POST path=/api/pending_closures6818worker2 # [ 153.694530] nix-grpc-daemon[1621]: time=2026-09-20T12:47:34.051Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6819worker2 # [ 153.695927] nix-grpc-daemon[1621]: time=2026-09-20T12:47:34.051Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6820lb # [ 153.676484] niks3-server[1107]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6821lb # [ 153.686331] niks3-server[1107]: 2026/09/20 12:47:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6822worker2 # [ 153.726657] nix-grpc-daemon[1621]: time=2026-09-20T12:47:34.083Z level=INFO msg="Uploading 1 narinfos"6823lb # [ 153.692420] niks3-server[1107]: 2026/09/20 12:47:34 INFO Signed narinfos id=66 count=16824lb # [ 153.697583] niks3-server[1107]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6825lb # [ 153.710101] niks3-server[1107]: 2026/09/20 12:47:34 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6826lb # [ 153.714233] niks3-server[1107]: 2026/09/20 12:47:34 INFO Completed upload id=666827worker2 # [ 153.752413] nix-grpc-daemon[1621]: time=2026-09-20T12:47:34.109Z level=INFO msg="Upload complete. (425ms)"6828lb # [ 153.717417] niks3-server[1107]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6829worker2 # [ 153.755296] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=0 paths=1 nar_bytes_in=6246830worker2 # [ 153.760255] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846831worker2 # [ 153.768260] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6832worker2 # [ 153.774119] nix-daemon[855]: reaped child process 1857, status = succeeded6833worker2 # [ 153.778932] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6834client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6835client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6836client # farm-b> LOG-farm-b-drain6837worker2 # [ 154.792168] nix-daemon[855]: reaped child process 1871, status = succeeded6838worker2 # [ 154.936333] nix-daemon[855]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6839worker2 # [ 154.948251] nix-daemon[855]: reaped child process 1889, status = succeeded6840lb # [ 154.929249] niks3-server[1107]: 2026/09/20 12:47:36 INFO Received uploads request method=POST path=/api/pending_closures6841worker2 # [ 154.974056] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.330Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6842worker2 # [ 154.975510] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.332Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6843lb # [ 154.967237] niks3-server[1107]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6844lb # [ 154.978729] niks3-server[1107]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6845lb # [ 154.988953] niks3-server[1107]: 2026/09/20 12:47:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6846lb # [ 154.993442] niks3-server[1107]: 2026/09/20 12:47:36 INFO Signed narinfos id=67 count=16847worker2 # [ 155.031795] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.388Z level=INFO msg="Uploading 1 narinfos"6848lb # [ 154.999478] niks3-server[1107]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6849lb # [ 155.008359] niks3-server[1107]: 2026/09/20 12:47:36 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6850lb # [ 155.013736] niks3-server[1107]: 2026/09/20 12:47:36 INFO Completed upload id=676851lb # [ 155.016178] niks3-server[1107]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6852worker2 # [ 155.050739] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.407Z level=INFO msg="Upload complete. (265ms)"6853worker2 # [ 155.056427] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16854worker2 # [ 155.068743] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:35Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06855worker2 # [ 155.072150] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:35Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16856worker2 # [ 155.080834] nix-daemon[855]: reaped child process 1869, status = succeeded6857worker2 # [ 155.112187] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846858worker2 # [ 155.124279] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412846859worker2 # [ 155.136310] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.493Z level=INFO msg="All 2 paths already cached"6860client # copying 1 paths to 'lb:50051'...6861client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6862worker2 # [ 155.145780] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412846863worker2 # [ 155.156319] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6864worker2 # [ 155.321310] nix-daemon[855]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6865worker2 # [ 155.333486] nix-daemon[855]: reaped child process 1898, status = succeeded6866worker2 # [ 155.497461] nix-daemon[855]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)6867worker2 # [ 155.509743] nix-daemon[855]: reaped child process 1903, status = succeeded6868lb # [ 155.488663] niks3-server[1107]: 2026/09/20 12:47:36 INFO Received uploads request method=POST path=/api/pending_closures6869worker2 # [ 155.531902] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.889Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6870worker2 # [ 155.533395] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.890Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6871lb # [ 155.516225] niks3-server[1107]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6872lb # [ 155.527959] niks3-server[1107]: 2026/09/20 12:47:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6873lb # [ 155.532130] niks3-server[1107]: 2026/09/20 12:47:36 INFO Signed narinfos id=68 count=16874worker2 # [ 155.570468] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.927Z level=INFO msg="Uploading 1 narinfos"6875lb # [ 155.536622] niks3-server[1107]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6876lb # [ 155.549689] niks3-server[1107]: 2026/09/20 12:47:36 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6877lb # [ 155.556378] niks3-server[1107]: 2026/09/20 12:47:36 INFO Completed upload id=686878worker2 # [ 155.594314] nix-grpc-daemon[1621]: time=2026-09-20T12:47:35.951Z level=INFO msg="Upload complete. (427ms)"6879lb # [ 155.559275] niks3-server[1107]: 2026/09/20 12:47:36 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6880worker2 # [ 155.597458] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=0 paths=1 nar_bytes_in=7526881worker2 # [ 155.603291] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412846882worker2 # [ 155.611340] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6883worker2 # [ 155.618954] nix-daemon[855]: reaped child process 1893, status = succeeded6884client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6885client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6886worker2 # [ 155.624287] nix-daemon[855]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6887client # farm-top> LOG-farm-top-drain6888worker2 # [ 156.645165] nix-daemon[855]: reaped child process 1907, status = succeeded6889worker2 # [ 156.784328] nix-daemon[855]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6890worker2 # [ 156.795362] nix-daemon[855]: reaped child process 1925, status = succeeded6891lb # [ 156.776379] niks3-server[1107]: 2026/09/20 12:47:37 INFO Received uploads request method=POST path=/api/pending_closures6892worker2 # [ 156.820473] nix-grpc-daemon[1621]: time=2026-09-20T12:47:37.177Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6893worker2 # [ 156.822818] nix-grpc-daemon[1621]: time=2026-09-20T12:47:37.179Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6894lb # [ 156.817219] niks3-server[1107]: 2026/09/20 12:47:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6895lb # [ 156.825276] niks3-server[1107]: 2026/09/20 12:47:37 INFO Signed narinfos id=69 count=16896worker2 # [ 156.863741] nix-grpc-daemon[1621]: time=2026-09-20T12:47:37.220Z level=INFO msg="Uploading 1 narinfos"6897lb # [ 156.831146] niks3-server[1107]: 2026/09/20 12:47:37 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6898lb # [ 156.835049] niks3-server[1107]: 2026/09/20 12:47:37 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6899lb # [ 156.839722] niks3-server[1107]: 2026/09/20 12:47:37 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6900lb # [ 156.847335] niks3-server[1107]: 2026/09/20 12:47:37 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6901lb # [ 156.850442] niks3-server[1107]: 2026/09/20 12:47:37 INFO Completed upload id=696902worker2 # [ 156.886203] nix-grpc-daemon[1621]: time=2026-09-20T12:47:37.243Z level=INFO msg="Upload complete. (255ms)"6903lb # [ 156.852699] niks3-server[1107]: 2026/09/20 12:47:37 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6904worker2 # [ 156.892489] nix-grpc-daemon[1545]: ts=2026-09-20T12:47:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41284 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16905worker2 # [ 156.904496] nix-daemon[855]: reaped child process 1905, status = succeeded6906client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.82 seconds)6907client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6908client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.19 seconds)6909worker1: must succeed: rm /nix/.rw-store/fill6910worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.10 seconds)6911worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6912worker1 # [ 158.218656] nix-grpc-daemon[1505]: ts=2026-09-20T12:47:38Z level=info event=healthy reason=min_free6913lb # [ 158.363411] envoy[985]: conn peer=[2001:db8:1::1]:38906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27068 tx=10725 ms=180146914lb # [ 158.368326] envoy[985]: conn peer=[2001:db8:1::1]:44878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=878 tx=481 ms=366915lb # [ 158.371881] envoy[985]: conn peer=[2001:db8:1::1]:44894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9572 tx=3798 ms=55786916worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)6917(finished: subtest: low disk drains a worker and builds go to the other, in 9.38 seconds)6918(finished: run the VM test script, in 159.35 seconds)6919test script finished in 159.59s6920cleanup6921kill QemuMachine (pid 51)6922client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6923kill QemuMachine (pid 49)6924lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6925kill QemuMachine (pid 50)6926worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6927kill QemuMachine (pid 52)6928worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6929(finished: cleanup, in 0.99 seconds)6930additionally exposed symbols:6931 client, lb, worker1, worker2,6932 vlan1,6933 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh