vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #405
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb: QEMU running (pid 50)14lb # Formatting '/build/vm-state-lb/tmp.EclGNE1Brx', fmt=raw size=107374182415lb # mke2fs 1.47.4 (6-Mar-2025)16lb # Discarding device blocks: 0/262144 done17lb # Creating filesystem with 262144 4k blocks and 65536 inodes18lb # Filesystem UUID: 4e83e032-e41b-477b-a30a-79b583d822d519lb # Superblock backups stored on blocks:20lb # 32768, 98304, 163840, 22937621client # Disk image does not exist, creating the virtualisation disk image...22worker1 # Disk image does not exist, creating the virtualisation disk image...23lb # 24worker1: QEMU running (pid 52)25worker1 # Formatting '/build/vm-state-worker1/tmp.H60Kjrkq6D', fmt=raw size=107374182426lb # Allocating group tables: 0/8 done27client # Formatting '/build/vm-state-client/tmp.H1QdGkBOKC', fmt=raw size=107374182428client: QEMU running (pid 49)29worker1 # mke2fs 1.47.4 (6-Mar-2025)30lb # Writing inode tables: 0/8 done31worker1 # Discarding device blocks: 0/262144 done32lb # Creating journal (8192 blocks): done33worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes34lb # Writing superblocks and filesystem accounting information: 0/8 done35worker1 # Filesystem UUID: 0d0a9936-c025-4abe-a05f-e0a14d143f7436worker1 # Superblock backups stored on blocks:37worker1 # 32768, 98304, 163840, 22937638lb # 39client # mke2fs 1.47.4 (6-Mar-2025)40client # Discarding device blocks: 0/262144 done41lb # Virtualisation disk image created.42client # Creating filesystem with 262144 4k blocks and 65536 inodes43worker1 # 44client # Filesystem UUID: 095f374f-28a0-4f22-b10a-1956f8a6a64745worker1 # Allocating group tables: 0/8 done46client # Superblock backups stored on blocks:47worker1 # Writing inode tables: 0/8 done48client # 32768, 98304, 163840, 22937649worker1 # Creating journal (8192 blocks): done50client # 51worker1 # Writing superblocks and filesystem accounting information: 0/8 done52client # Allocating group tables: 0/8 done53worker1 # 54worker2 # Disk image does not exist, creating the virtualisation disk image...55worker2: QEMU running (pid 51)56client # Writing inode tables: 0/8 done57worker1 # Virtualisation disk image created.58client # Creating journal (8192 blocks): done59(finished: start all VMs, in 0.44 seconds)60client # Writing superblocks and filesystem accounting information: 0/8 done61client: waiting for unit network-addresses-eth1.service62client # 63client # Virtualisation disk image created.64client: waiting for the VM to finish booting65worker2 # Formatting '/build/vm-state-worker2/tmp.dXyp6tzw6u', fmt=raw size=107374182466worker2 # mke2fs 1.47.4 (6-Mar-2025)67worker2 # Discarding device blocks: 0/262144 done68worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes69worker2 # Filesystem UUID: 26f2ec62-741f-4e78-aa06-e24d3f7b805770worker2 # Superblock backups stored on blocks:71worker2 # 32768, 98304, 163840, 22937672worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85client # 86client # 87worker1 # 88worker1 # 89lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0090client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0091worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0092lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...93lb # 94lb # 95lb # 96lb # 97client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...98client # 99client # 100client # 101client # 102lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00103worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...104worker1 # 105worker1 # 106worker1 # 107worker1 # 108worker2 # 109worker2 # 110client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00111worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00112worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00113lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114lb # 115lb # 116client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117client # 118client # 119worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...120worker2 # 121worker2 # 122worker2 # 123worker2 # 124worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...125worker1 # 126worker1 # 127worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker2 # 130worker2 # 131lb # Booting from ROM...132worker1 # Booting from ROM...133client # Booting from ROM...134worker2 # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok136lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026137client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026138client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test/init regInfo=/nix/store/32nicp73v9csdj16z0wnrnmm2xg2lsg6-closure-info/registration console=ttyS0,115200n8 console=tty0139lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0140client # [ 0.000000] BIOS-provided physical RAM map:141lb # [ 0.000000] BIOS-provided physical RAM map:142client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable143lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable144worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026145client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved146lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved147client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved148lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved149client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable150client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved151lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable152client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved153lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved154worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hksk4claan751fd0jxjvsclblv80rlg-nixos-system-worker1-test/init regInfo=/nix/store/519l597wjwvy39labl964y6qn8kqizwq-closure-info/registration console=ttyS0,115200n8 console=tty0155client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved156worker1 # [ 0.000000] BIOS-provided physical RAM map:157lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved158client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved159worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable160client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved161lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved162worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved163client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved164lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved165client # [ 0.000000] NX (Execute Disable) protection: active166worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved167client # [ 0.000000] APIC: Static calls initialized168lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved169client # [ 0.000000] SMBIOS 2.8 present.170worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable171lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved172client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014173worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved174lb # [ 0.000000] NX (Execute Disable) protection: active175client # [ 0.000000] DMI: Memory slots populated: 1/1176client # [ 0.000000] Hypervisor detected: KVM177lb # [ 0.000000] APIC: Static calls initialized178worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved179lb # [ 0.000000] SMBIOS 2.8 present.180client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000181worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved182client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00183client # [ 0.000000] kvm-clock: using sched offset of 492937626 cycles184lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014185worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved186lb # [ 0.000000] DMI: Memory slots populated: 1/1187worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved188lb # [ 0.000000] Hypervisor detected: KVM189client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns190lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000191client # [ 0.000006] tsc: Detected 2400.012 MHz processor192worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved193lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00194client # [ 0.000806] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000195lb # [ 0.000000] kvm-clock: using sched offset of 495556153 cycles196worker1 # [ 0.000000] NX (Execute Disable) protection: active197worker1 # [ 0.000000] APIC: Static calls initialized198client # [ 0.000834] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs199worker1 # [ 0.000000] SMBIOS 2.8 present.200client # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT201lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns202client # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f]203worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014204lb # [ 0.000006] tsc: Detected 2400.012 MHz processor205client # [ 0.002754] Using GB pages for direct mapping206worker1 # [ 0.000000] DMI: Memory slots populated: 1/1207client # [ 0.002900] RAMDISK: [mem 0x5e366000-0x5ffcffff]208worker1 # [ 0.000000] Hypervisor detected: KVM209lb # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000210client # [ 0.002911] ACPI: Early table checksum verification disabled211worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000212worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00213client # [ 0.002915] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )214lb # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs215worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026216worker1 # [ 0.000001] kvm-clock: using sched offset of 493418473 cycles217lb # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT218client # [ 0.002920] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)219lb # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]220client # [ 0.002925] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)221worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns222lb # [ 0.002751] Using GB pages for direct mapping223worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor224client # [ 0.002933] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)225lb # [ 0.002858] RAMDISK: [mem 0x5e367000-0x5ffcffff]226worker1 # [ 0.000831] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000227client # [ 0.002935] ACPI: FACS 0x000000005FFE0000 000040228lb # [ 0.002867] ACPI: Early table checksum verification disabled229worker1 # [ 0.000858] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs230worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hj2h3rq6gbw11l2gmb1lylmmrhmr7hvl-nixos-system-worker2-test/init regInfo=/nix/store/iakvcngrvdvx0crqfwcqv5iyh475dgj4-closure-info/registration console=ttyS0,115200n8 console=tty0231client # [ 0.002937] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)232lb # [ 0.002871] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )233worker2 # [ 0.000000] BIOS-provided physical RAM map:234worker1 # [ 0.000861] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT235client # [ 0.002938] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)236worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable237worker1 # [ 0.002779] found SMP MP-table at [mem 0x000f5450-0x000f545f]238lb # [ 0.002876] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)239worker1 # [ 0.002792] Using GB pages for direct mapping240worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved241client # [ 0.002940] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)242worker1 # [ 0.002952] RAMDISK: [mem 0x5e367000-0x5ffcffff]243lb # [ 0.002880] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)244worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved245client # [ 0.002942] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)246worker1 # [ 0.002960] ACPI: Early table checksum verification disabled247worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable248client # [ 0.002943] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]249worker1 # [ 0.002963] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )250lb # [ 0.002888] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)251client # [ 0.002944] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]252worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved253lb # [ 0.002890] ACPI: FACS 0x000000005FFE0000 000040254worker1 # [ 0.002967] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)255client # [ 0.002944] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]256worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved257lb # [ 0.002891] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)258worker1 # [ 0.002972] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)259client # [ 0.002945] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]260worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved261client # [ 0.002945] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]262lb # [ 0.002893] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)263worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved264worker1 # [ 0.002979] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)265worker1 # [ 0.002981] ACPI: FACS 0x000000005FFE0000 000040266client # [ 0.002946] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]267worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved268lb # [ 0.002894] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)269client # [ 0.002946] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]270worker1 # [ 0.002982] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)271worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved272client # [ 0.003187] No NUMA configuration found273worker2 # [ 0.000000] NX (Execute Disable) protection: active274lb # [ 0.002896] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)275worker1 # [ 0.002984] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)276client # [ 0.003187] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]277worker2 # [ 0.000000] APIC: Static calls initialized278worker2 # [ 0.000000] SMBIOS 2.8 present.279lb # [ 0.002897] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]280client # [ 0.003190] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]281worker1 # [ 0.002985] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)282client # [ 0.005664] Zone ranges:283lb # [ 0.002898] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]284client # [ 0.005665] DMA [mem 0x0000000000001000-0x0000000000ffffff]285worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014286worker1 # [ 0.002987] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)287worker2 # [ 0.000000] DMI: Memory slots populated: 1/1288client # [ 0.005666] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]289lb # [ 0.002899] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]290worker2 # [ 0.000000] Hypervisor detected: KVM291worker1 # [ 0.002988] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]292client # [ 0.005668] Normal empty293client # [ 0.005669] Device empty294worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000295client # [ 0.005669] Movable zone start for each node296worker1 # [ 0.002989] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]297lb # [ 0.002899] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]298worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00299client # [ 0.005670] Early memory node ranges300worker1 # [ 0.002990] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]301client # [ 0.005670] node 0: [mem 0x0000000000001000-0x000000000009efff]302lb # [ 0.002900] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]303worker2 # [ 0.000000] kvm-clock: using sched offset of 508861676 cycles304worker1 # [ 0.002990] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]305client # [ 0.005672] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]306lb # [ 0.002900] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]307worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns308client # [ 0.005673] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]309worker1 # [ 0.002991] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]310worker2 # [ 0.000006] tsc: Detected 2400.012 MHz processor311lb # [ 0.002901] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]312client # [ 0.005697] On node 0, zone DMA: 1 pages in unavailable ranges313worker1 # [ 0.002991] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]314worker2 # [ 0.000825] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000315lb # [ 0.003129] No NUMA configuration found316client # [ 0.005992] On node 0, zone DMA: 97 pages in unavailable ranges317worker1 # [ 0.002992] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]318lb # [ 0.003130] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]319worker2 # [ 0.000854] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs320client # [ 0.031799] On node 0, zone DMA32: 40 pages in unavailable ranges321worker1 # [ 0.003215] No NUMA configuration found322client # [ 0.032297] ACPI: PM-Timer IO Port: 0x608323lb # [ 0.003133] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]324worker2 # [ 0.000857] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT325worker1 # [ 0.003216] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]326client # [ 0.032313] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])327lb # [ 0.005413] Zone ranges:328worker2 # [ 0.002774] found SMP MP-table at [mem 0x000f5450-0x000f545f]329worker1 # [ 0.003219] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]330client # [ 0.032353] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23331lb # [ 0.005415] DMA [mem 0x0000000000001000-0x0000000000ffffff]332worker1 # [ 0.005565] Zone ranges:333worker2 # [ 0.002788] Using GB pages for direct mapping334worker2 # [ 0.002882] RAMDISK: [mem 0x5e367000-0x5ffcffff]335client # [ 0.032355] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)336worker1 # [ 0.005566] DMA [mem 0x0000000000001000-0x0000000000ffffff]337lb # [ 0.005416] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]338lb # [ 0.005417] Normal empty339worker2 # [ 0.002892] ACPI: Early table checksum verification disabled340client # [ 0.032357] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)341worker1 # [ 0.005568] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]342lb # [ 0.005418] Device empty343worker1 # [ 0.005569] Normal empty344worker2 # [ 0.002896] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )345client # [ 0.032359] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)346worker1 # [ 0.005570] Device empty347lb # [ 0.005419] Movable zone start for each node348worker1 # [ 0.005571] Movable zone start for each node349lb # [ 0.005420] Early memory node ranges350client # [ 0.032360] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)351worker2 # [ 0.002901] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)352worker1 # [ 0.005571] Early memory node ranges353client # [ 0.032361] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)354lb # [ 0.005420] node 0: [mem 0x0000000000001000-0x000000000009efff]355worker1 # [ 0.005572] node 0: [mem 0x0000000000001000-0x000000000009efff]356worker2 # [ 0.002906] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)357client # [ 0.032363] ACPI: Using ACPI (MADT) for SMP configuration information358lb # [ 0.005421] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]359worker1 # [ 0.005573] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]360client # [ 0.032365] ACPI: HPET id: 0x8086a201 base: 0xfed00000361worker2 # [ 0.002914] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)362client # [ 0.032370] TSC deadline timer available363lb # [ 0.005422] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]364worker1 # [ 0.005574] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]365worker2 # [ 0.002917] ACPI: FACS 0x000000005FFE0000 000040366client # [ 0.032374] CPU topo: Max. logical packages: 1367worker1 # [ 0.005593] On node 0, zone DMA: 1 pages in unavailable ranges368lb # [ 0.005441] On node 0, zone DMA: 1 pages in unavailable ranges369client # [ 0.032375] CPU topo: Max. logical dies: 1370worker2 # [ 0.002918] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)371client # [ 0.032375] CPU topo: Max. dies per package: 1372worker1 # [ 0.005865] On node 0, zone DMA: 97 pages in unavailable ranges373lb # [ 0.005705] On node 0, zone DMA: 97 pages in unavailable ranges374client # [ 0.032381] CPU topo: Max. threads per core: 1375worker2 # [ 0.002920] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)376worker1 # [ 0.032889] On node 0, zone DMA32: 40 pages in unavailable ranges377client # [ 0.032382] CPU topo: Num. cores per package: 1378lb # [ 0.032469] On node 0, zone DMA32: 40 pages in unavailable ranges379worker1 # [ 0.033426] ACPI: PM-Timer IO Port: 0x608380client # [ 0.032382] CPU topo: Num. threads per package: 1381lb # [ 0.032955] ACPI: PM-Timer IO Port: 0x608382worker2 # [ 0.002921] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)383worker1 # [ 0.033443] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])384client # [ 0.032383] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs385lb # [ 0.032969] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])386worker1 # [ 0.033473] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23387worker2 # [ 0.002923] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)388client # [ 0.032410] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()389worker1 # [ 0.033477] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)390lb # [ 0.032997] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23391worker2 # [ 0.002924] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]392client # [ 0.032449] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]393worker1 # [ 0.033479] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)394lb # [ 0.033000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)395worker2 # [ 0.002925] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]396client # [ 0.032450] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]397worker1 # [ 0.033481] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)398worker2 # [ 0.002926] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]399lb # [ 0.033002] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)400client # [ 0.032452] [mem 0x60000000-0xafffffff] available for PCI devices401worker1 # [ 0.033482] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)402client # [ 0.032453] Booting paravirtualized kernel on KVM403worker2 # [ 0.002926] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]404lb # [ 0.033003] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)405worker1 # [ 0.033483] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)406worker2 # [ 0.002927] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]407client # [ 0.032457] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns408lb # [ 0.033004] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)409worker1 # [ 0.033487] ACPI: Using ACPI (MADT) for SMP configuration information410worker2 # [ 0.002927] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]411client # [ 0.036972] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1412worker1 # [ 0.033488] ACPI: HPET id: 0x8086a201 base: 0xfed00000413lb # [ 0.033005] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)414worker1 # [ 0.033494] TSC deadline timer available415worker2 # [ 0.002928] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]416client # [ 0.039115] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152417worker2 # [ 0.003195] No NUMA configuration found418lb # [ 0.033007] ACPI: Using ACPI (MADT) for SMP configuration information419worker1 # [ 0.033500] CPU topo: Max. logical packages: 1420client # [ 0.039164] kvm-guest: PV spinlocks disabled, single CPU421worker1 # [ 0.033500] CPU topo: Max. logical dies: 1422lb # [ 0.033009] ACPI: HPET id: 0x8086a201 base: 0xfed00000423worker2 # [ 0.003196] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]424worker1 # [ 0.033501] CPU topo: Max. dies per package: 1425lb # [ 0.033013] TSC deadline timer available426worker2 # [ 0.003199] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]427worker1 # [ 0.033504] CPU topo: Max. threads per core: 1428worker2 # [ 0.005768] Zone ranges:429lb # [ 0.033018] CPU topo: Max. logical packages: 1430worker1 # [ 0.033505] CPU topo: Num. cores per package: 1431lb # [ 0.033019] CPU topo: Max. logical dies: 1432worker2 # [ 0.005769] DMA [mem 0x0000000000001000-0x0000000000ffffff]433worker1 # [ 0.033505] CPU topo: Num. threads per package: 1434lb # [ 0.033019] CPU topo: Max. dies per package: 1435worker2 # [ 0.005771] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]436worker1 # [ 0.033505] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs437worker2 # [ 0.005772] Normal empty438client # [ 0.039166] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test/init regInfo=/nix/store/32nicp73v9csdj16z0wnrnmm2xg2lsg6-closure-info/registration console=ttyS0,115200n8 console=tty0439lb # [ 0.033023] CPU topo: Max. threads per core: 1440worker2 # [ 0.005773] Device empty441worker1 # [ 0.033530] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()442lb # [ 0.033024] CPU topo: Num. cores per package: 1443worker2 # [ 0.005774] Movable zone start for each node444worker2 # [ 0.005775] Early memory node ranges445lb # [ 0.033024] CPU topo: Num. threads per package: 1446worker1 # [ 0.033568] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]447client # [ 0.039263] Unknown kernel command line parameters "regInfo=/nix/store/32nicp73v9csdj16z0wnrnmm2xg2lsg6-closure-info/registration", will be passed to user space.448worker2 # [ 0.005775] node 0: [mem 0x0000000000001000-0x000000000009efff]449client # [ 0.039277] random: crng init done450lb # [ 0.033025] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs451worker1 # [ 0.033569] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]452worker2 # [ 0.005776] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]453client # [ 0.039278] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes454lb # [ 0.033050] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()455worker1 # [ 0.033571] [mem 0x60000000-0xafffffff] available for PCI devices456worker2 # [ 0.005777] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]457client # [ 0.041360] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)458worker1 # [ 0.033572] Booting paravirtualized kernel on KVM459worker2 # [ 0.005798] On node 0, zone DMA: 1 pages in unavailable ranges460lb # [ 0.033087] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]461client # [ 0.042415] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)462worker2 # [ 0.006071] On node 0, zone DMA: 97 pages in unavailable ranges463client # [ 0.042455] Fallback order for Node 0: 0464worker1 # [ 0.033576] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns465lb # [ 0.033089] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]466worker2 # [ 0.033509] On node 0, zone DMA32: 40 pages in unavailable ranges467client # [ 0.042458] Built 1 zonelists, mobility grouping on. Total pages: 393078468worker1 # [ 0.038077] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1469worker2 # [ 0.034071] ACPI: PM-Timer IO Port: 0x608470client # [ 0.042459] Policy zone: DMA32471lb # [ 0.033090] [mem 0x60000000-0xafffffff] available for PCI devices472worker2 # [ 0.034088] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])473worker1 # [ 0.040378] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152474client # [ 0.045009] mem auto-init: stack:all(zero), heap alloc:on, heap free:off475lb # [ 0.033092] Booting paravirtualized kernel on KVM476worker1 # [ 0.040422] kvm-guest: PV spinlocks disabled, single CPU477client # [ 0.048455] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1478worker2 # [ 0.034125] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23479client # [ 0.051853] allocated 3145728 bytes of page_ext480lb # [ 0.033096] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns481worker2 # [ 0.034128] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)482client # [ 0.062527] ftrace: allocating 48731 entries in 192 pages483lb # [ 0.037594] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1484worker2 # [ 0.034130] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)485client # [ 0.062531] ftrace: allocated 192 pages with 2 groups486lb # [ 0.039851] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152487client # [ 0.063469] Dynamic Preempt: lazy488worker2 # [ 0.034131] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)489client # [ 0.063664] rcu: Preemptible hierarchical RCU implementation.490lb # [ 0.039895] kvm-guest: PV spinlocks disabled, single CPU491worker2 # [ 0.034132] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)492client # [ 0.063665] rcu: RCU event tracing is enabled.493worker1 # [ 0.040424] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hksk4claan751fd0jxjvsclblv80rlg-nixos-system-worker1-test/init regInfo=/nix/store/519l597wjwvy39labl964y6qn8kqizwq-closure-info/registration console=ttyS0,115200n8 console=tty0494worker2 # [ 0.034133] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)495client # [ 0.063666] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.496client # [ 0.063668] Trampoline variant of Tasks RCU enabled.497worker2 # [ 0.034136] ACPI: Using ACPI (MADT) for SMP configuration information498client # [ 0.063668] Rude variant of Tasks RCU enabled.499worker1 # [ 0.040541] Unknown kernel command line parameters "regInfo=/nix/store/519l597wjwvy39labl964y6qn8kqizwq-closure-info/registration", will be passed to user space.500worker2 # [ 0.034137] ACPI: HPET id: 0x8086a201 base: 0xfed00000501worker1 # [ 0.040555] random: crng init done502client # [ 0.063668] Tracing variant of Tasks RCU enabled.503worker2 # [ 0.034141] TSC deadline timer available504lb # [ 0.039897] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0505worker2 # [ 0.034146] CPU topo: Max. logical packages: 1506client # [ 0.063669] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.507worker1 # [ 0.040555] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes508worker2 # [ 0.034147] CPU topo: Max. logical dies: 1509client # [ 0.063670] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1510worker1 # [ 0.042877] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)511worker2 # [ 0.034147] CPU topo: Max. dies per package: 1512lb # [ 0.039994] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.513worker2 # [ 0.034151] CPU topo: Max. threads per core: 1514lb # [ 0.040008] random: crng init done515client # [ 0.063695] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.516worker1 # [ 0.044021] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)517worker2 # [ 0.034152] CPU topo: Num. cores per package: 1518worker1 # [ 0.044064] Fallback order for Node 0: 0519lb # [ 0.040009] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes520worker2 # [ 0.034152] CPU topo: Num. threads per package: 1521client # [ 0.063697] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.522worker1 # [ 0.044068] Built 1 zonelists, mobility grouping on. Total pages: 393078523lb # [ 0.042177] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)524worker2 # [ 0.034152] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs525worker1 # [ 0.044070] Policy zone: DMA32526client # [ 0.063698] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.527worker2 # [ 0.034180] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()528lb # [ 0.043249] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)529worker1 # [ 0.046680] mem auto-init: stack:all(zero), heap alloc:on, heap free:off530client # [ 0.068378] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16531lb # [ 0.043286] Fallback order for Node 0: 0532worker2 # [ 0.034217] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]533client # [ 0.068700] rcu: srcu_init: Setting srcu_struct sizes based on contention.534worker1 # [ 0.050152] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1535lb # [ 0.043289] Built 1 zonelists, mobility grouping on. Total pages: 393078536worker1 # [ 0.053550] allocated 3145728 bytes of page_ext537lb # [ 0.043290] Policy zone: DMA32538worker2 # [ 0.034219] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]539client # [ 0.068711] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns540worker1 # [ 0.064187] ftrace: allocating 48731 entries in 192 pages541lb # [ 0.046014] mem auto-init: stack:all(zero), heap alloc:on, heap free:off542worker2 # [ 0.034221] [mem 0x60000000-0xafffffff] available for PCI devices543worker1 # [ 0.064190] ftrace: allocated 192 pages with 2 groups544worker2 # [ 0.034223] Booting paravirtualized kernel on KVM545worker1 # [ 0.065114] Dynamic Preempt: lazy546lb # [ 0.049557] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1547client # [ 0.068831] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)548client # [ 0.072590] Console: colour VGA+ 80x25549lb # [ 0.053421] allocated 3145728 bytes of page_ext550worker1 # [ 0.065280] rcu: Preemptible hierarchical RCU implementation.551client # [ 0.072594] printk: legacy console [tty0] enabled552worker2 # [ 0.034227] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns553worker1 # [ 0.065281] rcu: RCU event tracing is enabled.554lb # [ 0.064171] ftrace: allocating 48731 entries in 192 pages555client # [ 0.103380] printk: legacy console [ttyS0] enabled556lb # [ 0.064174] ftrace: allocated 192 pages with 2 groups557worker1 # [ 0.065281] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.558lb # [ 0.065068] Dynamic Preempt: lazy559worker2 # [ 0.038874] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1560client # [ 0.200998] ACPI: Core revision 20250807561worker1 # [ 0.065283] Trampoline variant of Tasks RCU enabled.562lb # [ 0.065209] rcu: Preemptible hierarchical RCU implementation.563worker2 # [ 0.041156] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152564worker1 # [ 0.065283] Rude variant of Tasks RCU enabled.565lb # [ 0.065210] rcu: RCU event tracing is enabled.566worker2 # [ 0.041209] kvm-guest: PV spinlocks disabled, single CPU567worker1 # [ 0.065283] Tracing variant of Tasks RCU enabled.568client # [ 0.201902] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns569lb # [ 0.065211] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.570lb # [ 0.065212] Trampoline variant of Tasks RCU enabled.571worker1 # [ 0.065284] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.572client # [ 0.203460] APIC: Switch to symmetric I/O mode setup573lb # [ 0.065213] Rude variant of Tasks RCU enabled.574client # [ 0.204394] x2apic enabled575worker1 # [ 0.065285] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1576lb # [ 0.065213] Tracing variant of Tasks RCU enabled.577client # [ 0.205151] APIC: Switched APIC routing to: physical x2apic578lb # [ 0.065214] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.579worker1 # [ 0.065299] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.580worker2 # [ 0.041211] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hj2h3rq6gbw11l2gmb1lylmmrhmr7hvl-nixos-system-worker2-test/init regInfo=/nix/store/iakvcngrvdvx0crqfwcqv5iyh475dgj4-closure-info/registration console=ttyS0,115200n8 console=tty0581lb # [ 0.065215] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1582worker1 # [ 0.065301] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.583lb # [ 0.065270] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.584client # [ 0.207232] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1585worker1 # [ 0.065302] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.586worker2 # [ 0.041315] Unknown kernel command line parameters "regInfo=/nix/store/iakvcngrvdvx0crqfwcqv5iyh475dgj4-closure-info/registration", will be passed to user space.587lb # [ 0.065272] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.588worker2 # [ 0.041329] random: crng init done589worker1 # [ 0.069902] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16590client # [ 0.208172] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns591worker2 # [ 0.041330] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes592lb # [ 0.065273] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.593worker1 # [ 0.070186] rcu: srcu_init: Setting srcu_struct sizes based on contention.594lb # [ 0.069809] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16595worker2 # [ 0.043568] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)596client # [ 0.209752] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)597worker1 # [ 0.070191] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns598lb # [ 0.070107] rcu: srcu_init: Setting srcu_struct sizes based on contention.599worker2 # [ 0.044675] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)600client # [ 0.210837] x86/cpu: User Mode Instruction Prevention (UMIP) activated601worker2 # [ 0.044722] Fallback order for Node 0: 0602worker1 # [ 0.070303] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)603lb # [ 0.070115] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns604worker1 # [ 0.074329] Console: colour VGA+ 80x25605client # [ 0.211929] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127606worker2 # [ 0.044725] Built 1 zonelists, mobility grouping on. Total pages: 393078607worker2 # [ 0.044726] Policy zone: DMA32608worker1 # [ 0.074333] printk: legacy console [tty0] enabled609lb # [ 0.070227] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)610client # [ 0.213418] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0611worker1 # [ 0.106228] printk: legacy console [ttyS0] enabled612lb # [ 0.073872] Console: colour VGA+ 80x25613worker2 # [ 0.047758] mem auto-init: stack:all(zero), heap alloc:on, heap free:off614lb # [ 0.073876] printk: legacy console [tty0] enabled615worker1 # [ 0.211821] ACPI: Core revision 20250807616worker2 # [ 0.051359] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1617lb # [ 0.104820] printk: legacy console [ttyS0] enabled618worker2 # [ 0.055331] allocated 3145728 bytes of page_ext619client # [ 0.213752] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto620lb # [ 0.217456] ACPI: Core revision 20250807621worker2 # [ 0.066428] ftrace: allocating 48731 entries in 192 pages622worker1 # [ 0.212755] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns623worker2 # [ 0.066431] ftrace: allocated 192 pages with 2 groups624client # [ 0.215748] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl625worker2 # [ 0.067458] Dynamic Preempt: lazy626worker1 # [ 0.214364] APIC: Switch to symmetric I/O mode setup627lb # [ 0.218385] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns628client # [ 0.216748] Transient Scheduler Attacks: Vulnerable: No microcode629worker2 # [ 0.067658] rcu: Preemptible hierarchical RCU implementation.630worker1 # [ 0.215401] x2apic enabled631worker2 # [ 0.067659] rcu: RCU event tracing is enabled.632lb # [ 0.219907] APIC: Switch to symmetric I/O mode setup633client # [ 0.217748] Spectre V2 : Mitigation: Enhanced / Automatic IBRS634lb # [ 0.220857] x2apic enabled635worker2 # [ 0.067659] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.636worker1 # [ 0.216180] APIC: Switched APIC routing to: physical x2apic637client # [ 0.218748] Speculative Return Stack Overflow: Mitigation: Safe RET638worker2 # [ 0.067661] Trampoline variant of Tasks RCU enabled.639lb # [ 0.221604] APIC: Switched APIC routing to: physical x2apic640worker2 # [ 0.067661] Rude variant of Tasks RCU enabled.641worker2 # [ 0.067662] Tracing variant of Tasks RCU enabled.642worker1 # [ 0.218099] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1643worker2 # [ 0.067663] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.644lb # [ 0.223419] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1645worker2 # [ 0.067663] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1646worker1 # [ 0.219158] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns647client # [ 0.219748] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization648lb # [ 0.224426] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns649worker2 # [ 0.067689] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.650client # [ 0.221756] Spectre V2 : Enabling IBPB for BPF651worker1 # [ 0.220896] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)652worker2 # [ 0.067691] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.653lb # [ 0.226058] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)654client # [ 0.222749] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier655worker1 # [ 0.222959] x86/cpu: User Mode Instruction Prevention (UMIP) activated656worker2 # [ 0.067692] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.657client # [ 0.223749] active return thunk: srso_alias_return_thunk658lb # [ 0.227139] x86/cpu: User Mode Instruction Prevention (UMIP) activated659worker1 # [ 0.224028] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127660worker2 # [ 0.072475] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16661lb # [ 0.228186] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127662client # [ 0.224772] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'663worker2 # [ 0.072790] rcu: srcu_init: Setting srcu_struct sizes based on contention.664worker1 # [ 0.224894] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0665lb # [ 0.229055] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0666client # [ 0.225748] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'667worker2 # [ 0.072800] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns668client # [ 0.226748] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'669worker1 # [ 0.225897] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto670lb # [ 0.230058] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto671worker2 # [ 0.072929] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)672client # [ 0.227748] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'673worker2 # [ 0.077025] Console: colour VGA+ 80x25674worker1 # [ 0.226895] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl675worker2 # [ 0.077028] printk: legacy console [tty0] enabled676lb # [ 0.232055] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl677client # [ 0.228748] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'678worker2 # [ 0.109431] printk: legacy console [ttyS0] enabled679worker1 # [ 0.227894] Transient Scheduler Attacks: Vulnerable: No microcode680lb # [ 0.233055] Transient Scheduler Attacks: Vulnerable: No microcode681client # [ 0.229748] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'682worker2 # [ 0.212157] ACPI: Core revision 20250807683worker1 # [ 0.228893] Spectre V2 : Mitigation: Enhanced / Automatic IBRS684lb # [ 0.234054] Spectre V2 : Mitigation: Enhanced / Automatic IBRS685client # [ 0.230748] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'686worker1 # [ 0.229894] Speculative Return Stack Overflow: Mitigation: Safe RET687lb # [ 0.235054] Speculative Return Stack Overflow: Mitigation: Safe RET688worker2 # [ 0.213183] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns689client # [ 0.231748] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256690worker2 # [ 0.214860] APIC: Switch to symmetric I/O mode setup691client # [ 0.232748] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64692worker2 # [ 0.215903] x2apic enabled693client # [ 0.233748] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512694worker2 # [ 0.216683] APIC: Switched APIC routing to: physical x2apic695client # [ 0.234748] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024696lb # [ 0.236054] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization697worker1 # [ 0.230893] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization698client # [ 0.235748] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8699lb # [ 0.238062] Spectre V2 : Enabling IBPB for BPF700worker1 # [ 0.232901] Spectre V2 : Enabling IBPB for BPF701worker2 # [ 0.218655] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1702client # [ 0.236748] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.703lb # [ 0.239055] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier704worker1 # [ 0.234442] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier705lb # [ 0.240055] active return thunk: srso_alias_return_thunk706worker2 # [ 0.219644] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns707worker1 # [ 0.235894] active return thunk: srso_alias_return_thunk708lb # [ 0.241075] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'709worker2 # [ 0.221372] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)710worker1 # [ 0.236917] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'711lb # [ 0.242054] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'712worker2 # [ 0.223408] x86/cpu: User Mode Instruction Prevention (UMIP) activated713worker1 # [ 0.237894] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'714lb # [ 0.243054] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'715worker2 # [ 0.224533] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127716worker1 # [ 0.238894] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'717lb # [ 0.244054] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'718worker2 # [ 0.225369] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0719worker1 # [ 0.239893] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'720lb # [ 0.245054] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'721worker1 # [ 0.240894] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'722lb # [ 0.246054] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'723worker2 # [ 0.226373] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto724worker1 # [ 0.241893] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'725lb # [ 0.247054] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'726worker2 # [ 0.227369] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl727worker1 # [ 0.242893] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'728lb # [ 0.249055] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256729worker2 # [ 0.228369] Transient Scheduler Attacks: Vulnerable: No microcode730lb # [ 0.250054] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64731worker1 # [ 0.244894] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256732worker2 # [ 0.229368] Spectre V2 : Mitigation: Enhanced / Automatic IBRS733lb # [ 0.251004] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512734worker1 # [ 0.245894] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64735worker2 # [ 0.230368] Speculative Return Stack Overflow: Mitigation: Safe RET736lb # [ 0.251806] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024737worker1 # [ 0.246893] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512738lb # [ 0.252812] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8739worker1 # [ 0.247893] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024740worker1 # [ 0.248894] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8741lb # [ 0.253838] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.742worker2 # [ 0.231368] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization743worker1 # [ 0.249893] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.744worker2 # [ 0.233377] Spectre V2 : Enabling IBPB for BPF745worker2 # [ 0.234885] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier746worker2 # [ 0.236369] active return thunk: srso_alias_return_thunk747worker2 # [ 0.237221] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'748worker2 # [ 0.238368] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'749worker2 # [ 0.239368] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'750worker2 # [ 0.240369] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'751worker2 # [ 0.241368] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'752worker2 # [ 0.242368] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'753worker2 # [ 0.243368] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'754worker2 # [ 0.244368] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256755worker2 # [ 0.245368] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64756worker2 # [ 0.246368] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512757worker2 # [ 0.247368] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024758worker2 # [ 0.248368] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8759worker2 # [ 0.249368] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.760client # [ 0.273232] Freeing SMP alternatives memory: 44K761client # [ 0.273752] pid_max: default: 32768 minimum: 301762client # [ 0.274852] LSM: initializing lsm=capability,landlock,yama,bpf,ima763client # [ 0.275863] landlock: Up and running.764client # [ 0.276472] Yama: becoming mindful.765client # [ 0.277383] LSM support for eBPF active766client # [ 0.277883] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767client # [ 0.278790] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)768client # [ 0.281818] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)769client # [ 0.283372] Performance Events: Fam17h+ core perfctr, AMD PMU driver.770client # [ 0.283757] ... version: 2771client # [ 0.284435] ... bit width: 48772client # [ 0.284750] ... generic counters: 6773client # [ 0.285454] ... generic bitmap: 000000000000003f774lb # [ 0.289345] Freeing SMP alternatives memory: 44K775client # [ 0.285750] ... fixed-purpose counters: 0776lb # [ 0.290058] pid_max: default: 32768 minimum: 301777client # [ 0.286425] ... fixed-purpose bitmap: 0000000000000000778client # [ 0.286750] ... value mask: 0000ffffffffffff779lb # [ 0.290917] LSM: initializing lsm=capability,landlock,yama,bpf,ima780worker1 # [ 0.285695] Freeing SMP alternatives memory: 44K781lb # [ 0.292128] landlock: Up and running.782client # [ 0.287623] ... max period: 00007fffffffffff783lb # [ 0.292724] Yama: becoming mindful.784worker1 # [ 0.286458] pid_max: default: 32768 minimum: 301785client # [ 0.288416] ... global_ctrl mask: 000000000000003f786lb # [ 0.293272] LSM support for eBPF active787worker1 # [ 0.286979] LSM: initializing lsm=capability,landlock,yama,bpf,ima788client # [ 0.288853] signal: max sigframe size: 3376789worker1 # [ 0.288008] landlock: Up and running.790client # [ 0.289658] rcu: Hierarchical SRCU implementation.791lb # [ 0.294184] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)792worker1 # [ 0.288894] Yama: becoming mindful.793client # [ 0.290302] rcu: Max phase no-delay instances is 400.794worker1 # [ 0.289924] LSM support for eBPF active795lb # [ 0.295098] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)796worker1 # [ 0.290699] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)797lb # [ 0.297810] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)798worker1 # [ 0.291944] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)799lb # [ 0.298600] Performance Events: Fam17h+ core perfctr, AMD PMU driver.800lb # [ 0.299059] ... version: 2801client # [ 0.295433] smp: Bringing up secondary CPUs ...802worker1 # [ 0.294341] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)803lb # [ 0.299748] ... bit width: 48804client # [ 0.295767] smp: Brought up 1 node, 1 CPU805lb # [ 0.300057] ... generic counters: 6806client # [ 0.296413] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)807worker1 # [ 0.295506] Performance Events: Fam17h+ core perfctr, AMD PMU driver.808lb # [ 0.300768] ... generic bitmap: 000000000000003f809worker1 # [ 0.295903] ... version: 2810lb # [ 0.301067] ... fixed-purpose counters: 0811worker1 # [ 0.296619] ... bit width: 48812lb # [ 0.301761] ... fixed-purpose bitmap: 0000000000000000813worker1 # [ 0.296896] ... generic counters: 6814client # [ 0.296924] Memory: 1454536K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)815lb # [ 0.302056] ... value mask: 0000ffffffffffff816worker1 # [ 0.297586] ... generic bitmap: 000000000000003f817client # [ 0.298049] devtmpfs: initialized818worker1 # [ 0.297896] ... fixed-purpose counters: 0819lb # [ 0.302948] ... max period: 00007fffffffffff820client # [ 0.298909] x86/mm: Memory block size: 128MB821worker1 # [ 0.298609] ... fixed-purpose bitmap: 0000000000000000822lb # [ 0.303738] ... global_ctrl mask: 000000000000003f823worker2 # [ 0.285434] Freeing SMP alternatives memory: 44K824lb # [ 0.304183] signal: max sigframe size: 3376825worker1 # [ 0.298896] ... value mask: 0000ffffffffffff826worker2 # [ 0.286253] pid_max: default: 32768 minimum: 301827client # [ 0.300666] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)828lb # [ 0.304941] rcu: Hierarchical SRCU implementation.829worker1 # [ 0.299811] ... max period: 00007fffffffffff830lb # [ 0.305605] rcu: Max phase no-delay instances is 400.831worker2 # [ 0.287004] LSM: initializing lsm=capability,landlock,yama,bpf,ima832worker1 # [ 0.300556] ... global_ctrl mask: 000000000000003f833client # [ 0.301616] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).834worker2 # [ 0.287491] landlock: Up and running.835worker1 # [ 0.301006] signal: max sigframe size: 3376836client # [ 0.302850] pinctrl core: initialized pinctrl subsystem837worker2 # [ 0.288812] Yama: becoming mindful.838worker1 # [ 0.301822] rcu: Hierarchical SRCU implementation.839client # [ 0.303919] PM: RTC time: 13:32:05, date: 2026-09-20840worker2 # [ 0.289588] LSM support for eBPF active841worker1 # [ 0.302508] rcu: Max phase no-delay instances is 400.842worker2 # [ 0.290505] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)843worker2 # [ 0.291410] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)844lb # [ 0.310660] smp: Bringing up secondary CPUs ...845lb # [ 0.311072] smp: Brought up 1 node, 1 CPU846client # [ 0.307716] NET: Registered PF_NETLINK/PF_ROUTE protocol family847lb # [ 0.311744] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)848client # [ 0.309023] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations849worker1 # [ 0.307547] smp: Bringing up secondary CPUs ...850worker2 # [ 0.294214] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)851worker1 # [ 0.307911] smp: Brought up 1 node, 1 CPU852client # [ 0.309770] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations853lb # [ 0.312217] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)854worker1 # [ 0.308569] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)855worker2 # [ 0.295124] Performance Events: Fam17h+ core perfctr, AMD PMU driver.856lb # [ 0.313325] devtmpfs: initialized857client # [ 0.311051] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations858lb # [ 0.314114] x86/mm: Memory block size: 128MB859client # [ 0.311763] audit: initializing netlink subsys (disabled)860worker2 # [ 0.296167] ... version: 2861worker2 # [ 0.296371] ... bit width: 48862worker1 # [ 0.309026] Memory: 1454580K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)863worker2 # [ 0.297071] ... generic counters: 6864client # [ 0.312845] thermal_sys: Registered thermal governor 'fair_share'865worker1 # [ 0.310158] devtmpfs: initialized866worker2 # [ 0.297385] ... generic bitmap: 000000000000003f867lb # [ 0.315836] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)868client # [ 0.312847] thermal_sys: Registered thermal governor 'bang_bang'869worker2 # [ 0.298218] ... fixed-purpose counters: 0870worker1 # [ 0.310914] x86/mm: Memory block size: 128MB871client # [ 0.313751] thermal_sys: Registered thermal governor 'step_wise'872lb # [ 0.316928] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).873worker2 # [ 0.298857] ... fixed-purpose bitmap: 0000000000000000874worker2 # [ 0.299370] ... value mask: 0000ffffffffffff875client # [ 0.314644] thermal_sys: Registered thermal governor 'user_space'876lb # [ 0.318136] pinctrl core: initialized pinctrl subsystem877worker1 # [ 0.312739] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)878worker2 # [ 0.300247] ... max period: 00007fffffffffff879lb # [ 0.319310] PM: RTC time: 13:32:05, date: 2026-09-20880client # [ 0.315437] audit: type=2000 audit(1789911126.465:1): state=initialized audit_enabled=0 res=1881worker2 # [ 0.301030] ... global_ctrl mask: 000000000000003f882worker1 # [ 0.313755] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).883client # [ 0.316753] thermal_sys: Registered thermal governor 'power_allocator'884worker2 # [ 0.301517] signal: max sigframe size: 3376885client # [ 0.316769] cpuidle: using governor menu886worker1 # [ 0.314975] pinctrl core: initialized pinctrl subsystem887worker2 # [ 0.302348] rcu: Hierarchical SRCU implementation.888worker1 # [ 0.316076] PM: RTC time: 13:32:05, date: 2026-09-20889worker2 # [ 0.302950] rcu: Max phase no-delay instances is 400.890client # [ 0.318989] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5891lb # [ 0.322819] NET: Registered PF_NETLINK/PF_ROUTE protocol family892client # [ 0.319993] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]893lb # [ 0.323575] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations894client # [ 0.320756] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry895worker1 # [ 0.319752] NET: Registered PF_NETLINK/PF_ROUTE protocol family896lb # [ 0.324080] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations897client # [ 0.321769] PCI: Using configuration type 1 for base access898worker1 # [ 0.321120] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations899lb # [ 0.325339] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations900worker2 # [ 0.308459] smp: Bringing up secondary CPUs ...901lb # [ 0.326068] audit: initializing netlink subsys (disabled)902client # [ 0.322994] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.903worker1 # [ 0.321920] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations904worker2 # [ 0.309230] smp: Brought up 1 node, 1 CPU905lb # [ 0.327206] thermal_sys: Registered thermal governor 'fair_share'906worker2 # [ 0.309891] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)907worker1 # [ 0.323177] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations908lb # [ 0.327208] thermal_sys: Registered thermal governor 'bang_bang'909worker1 # [ 0.323907] audit: initializing netlink subsys (disabled)910lb # [ 0.328058] thermal_sys: Registered thermal governor 'step_wise'911lb # [ 0.329004] thermal_sys: Registered thermal governor 'user_space'912worker2 # [ 0.310656] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)913worker1 # [ 0.325113] thermal_sys: Registered thermal governor 'fair_share'914worker2 # [ 0.311700] devtmpfs: initialized915worker1 # [ 0.325116] thermal_sys: Registered thermal governor 'bang_bang'916lb # [ 0.329780] audit: type=2000 audit(1789911126.461:1): state=initialized audit_enabled=0 res=1917client # [ 0.328081] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages918worker2 # [ 0.312521] x86/mm: Memory block size: 128MB919worker1 # [ 0.325897] thermal_sys: Registered thermal governor 'step_wise'920lb # [ 0.331059] thermal_sys: Registered thermal governor 'power_allocator'921client # [ 0.328753] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page922worker1 # [ 0.326827] thermal_sys: Registered thermal governor 'user_space'923lb # [ 0.331074] cpuidle: using governor menu924client # [ 0.329754] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages925worker2 # [ 0.314298] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)926worker1 # [ 0.327604] audit: type=2000 audit(1789911126.470:1): state=initialized audit_enabled=0 res=1927lb # [ 0.333271] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5928worker1 # [ 0.328900] thermal_sys: Registered thermal governor 'power_allocator'929worker2 # [ 0.315250] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).930worker1 # [ 0.328919] cpuidle: using governor menu931lb # [ 0.334291] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]932worker2 # [ 0.316481] pinctrl core: initialized pinctrl subsystem933lb # [ 0.335060] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry934worker1 # [ 0.331134] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5935worker2 # [ 0.317632] PM: RTC time: 13:32:05, date: 2026-09-20936lb # [ 0.336070] PCI: Using configuration type 1 for base access937worker1 # [ 0.332189] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]938lb # [ 0.337164] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.939worker1 # [ 0.332900] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry940client # [ 0.336753] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page941worker1 # [ 0.333909] PCI: Using configuration type 1 for base access942worker2 # [ 0.321198] NET: Registered PF_NETLINK/PF_ROUTE protocol family943worker1 # [ 0.335012] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.944worker2 # [ 0.322650] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations945worker2 # [ 0.323398] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations946worker2 # [ 0.324644] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations947worker2 # [ 0.325386] audit: initializing netlink subsys (disabled)948lb # [ 0.344347] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages949lb # [ 0.345059] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page950worker1 # [ 0.339999] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages951worker2 # [ 0.326584] thermal_sys: Registered thermal governor 'fair_share'952client # [ 0.345119] ACPI: Added _OSI(Module Device)953worker2 # [ 0.326587] thermal_sys: Registered thermal governor 'bang_bang'954worker1 # [ 0.340898] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page955client # [ 0.345753] ACPI: Added _OSI(Processor Device)956worker2 # [ 0.327372] thermal_sys: Registered thermal governor 'step_wise'957client # [ 0.346578] ACPI: Added _OSI(Processor Aggregator Device)958worker2 # [ 0.328308] thermal_sys: Registered thermal governor 'user_space'959worker2 # [ 0.329094] audit: type=2000 audit(1789911126.491:1): state=initialized audit_enabled=0 res=1960lb # [ 0.350058] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages961worker2 # [ 0.330374] thermal_sys: Registered thermal governor 'power_allocator'962worker2 # [ 0.330393] cpuidle: using governor menu963lb # [ 0.351058] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page964worker1 # [ 0.345898] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages965worker1 # [ 0.346896] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page966worker2 # [ 0.332619] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5967worker2 # [ 0.333651] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]968worker2 # [ 0.334376] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry969worker2 # [ 0.335388] PCI: Using configuration type 1 for base access970client # [ 0.356356] ACPI: 1 ACPI AML tables successfully acquired and loaded971worker2 # [ 0.336522] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.972lb # [ 0.359697] ACPI: Added _OSI(Module Device)973lb # [ 0.360058] ACPI: Added _OSI(Processor Device)974lb # [ 0.360818] ACPI: Added _OSI(Processor Aggregator Device)975client # [ 0.360185] ACPI: Interpreter enabled976worker1 # [ 0.357314] ACPI: Added _OSI(Module Device)977client # [ 0.360773] ACPI: PM: (supports S0 S3 S4 S5)978worker1 # [ 0.357898] ACPI: Added _OSI(Processor Device)979client # [ 0.361561] ACPI: Using IOAPIC for interrupt routing980worker1 # [ 0.358664] ACPI: Added _OSI(Processor Aggregator Device)981worker2 # [ 0.345439] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages982client # [ 0.364823] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug983worker2 # [ 0.346373] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page984lb # [ 0.367603] ACPI: 1 ACPI AML tables successfully acquired and loaded985client # [ 0.365753] PCI: Using E820 reservations for host bridge windows986client # [ 0.368923] ACPI: Enabled 2 GPEs in block 00 to 3F987lb # [ 0.371064] ACPI: Interpreter enabled988lb # [ 0.371749] ACPI: PM: (supports S0 S3 S4 S5)989worker2 # [ 0.351372] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages990worker1 # [ 0.367658] ACPI: 1 ACPI AML tables successfully acquired and loaded991lb # [ 0.372058] ACPI: Using IOAPIC for interrupt routing992worker2 # [ 0.352371] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page993lb # [ 0.372923] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug994worker1 # [ 0.371648] ACPI: Interpreter enabled995worker1 # [ 0.371913] ACPI: PM: (supports S0 S3 S4 S5)996worker1 # [ 0.372647] ACPI: Using IOAPIC for interrupt routing997client # [ 0.376109] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])998lb # [ 0.378057] PCI: Using E820 reservations for host bridge windows999worker1 # [ 0.372966] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1000client # [ 0.376761] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1001lb # [ 0.379141] ACPI: Enabled 2 GPEs in block 00 to 3F1002client # [ 0.377850] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1003client # [ 0.378892] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1004worker2 # [ 0.362765] ACPI: Added _OSI(Module Device)1005client # [ 0.380151] PCI host bridge to bus 0000:001006worker2 # [ 0.363373] ACPI: Added _OSI(Processor Device)1007worker1 # [ 0.377896] PCI: Using E820 reservations for host bridge windows1008worker2 # [ 0.364106] ACPI: Added _OSI(Processor Aggregator Device)1009client # [ 0.380758] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1010worker1 # [ 0.378992] ACPI: Enabled 2 GPEs in block 00 to 3F1011client # [ 0.381753] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1012client # [ 0.382753] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1013client # [ 0.383755] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1014lb # [ 0.387964] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1015client # [ 0.384754] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1016lb # [ 0.389711] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1017client # [ 0.385753] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1018client # [ 0.386754] pci_bus 0000:00: root bus resource [bus 00-ff]1019lb # [ 0.391161] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1020worker2 # [ 0.371265] ACPI: 1 ACPI AML tables successfully acquired and loaded1021lb # [ 0.392184] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1022client # [ 0.387942] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1023worker1 # [ 0.387783] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1024lb # [ 0.393421] PCI host bridge to bus 0000:001025worker1 # [ 0.388697] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1026lb # [ 0.394062] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1027client # [ 0.389266] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1028worker2 # [ 0.375164] ACPI: Interpreter enabled1029worker2 # [ 0.375394] ACPI: PM: (supports S0 S3 S4 S5)1030worker1 # [ 0.389977] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1031lb # [ 0.395057] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1032worker2 # [ 0.376133] ACPI: Using IOAPIC for interrupt routing1033worker1 # [ 0.391027] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1034lb # [ 0.396057] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1035worker1 # [ 0.392318] PCI host bridge to bus 0000:001036worker2 # [ 0.376435] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1037lb # [ 0.397057] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1038worker1 # [ 0.392901] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1039lb # [ 0.398057] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1040worker1 # [ 0.393897] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1041lb # [ 0.399057] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1042worker2 # [ 0.379371] PCI: Using E820 reservations for host bridge windows1043worker1 # [ 0.394897] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1044lb # [ 0.400057] pci_bus 0000:00: root bus resource [bus 00-ff]1045worker2 # [ 0.380547] ACPI: Enabled 2 GPEs in block 00 to 3F1046worker1 # [ 0.395897] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1047client # [ 0.391829] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1048lb # [ 0.401044] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1049client # [ 0.392769] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1050worker1 # [ 0.396897] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1051lb # [ 0.402513] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1052client # [ 0.393790] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1053worker1 # [ 0.397897] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1054worker1 # [ 0.398897] pci_bus 0000:00: root bus resource [bus 00-ff]1055client # [ 0.395623] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1056worker1 # [ 0.399931] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1057worker1 # [ 0.401425] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1058client # [ 0.397835] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1059worker2 # [ 0.390066] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1060client # [ 0.400762] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1061worker2 # [ 0.390383] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1062client # [ 0.401675] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1063lb # [ 0.405124] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1064worker2 # [ 0.391485] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1065client # [ 0.402542] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1066lb # [ 0.406142] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1067worker2 # [ 0.392513] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1068client # [ 0.403633] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1069lb # [ 0.407129] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1070worker2 # [ 0.393781] PCI host bridge to bus 0000:001071worker1 # [ 0.403976] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1072client # [ 0.405425] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1073worker2 # [ 0.394377] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1074lb # [ 0.408403] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1075worker1 # [ 0.404909] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1076worker2 # [ 0.395373] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1077worker1 # [ 0.405888] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1078lb # [ 0.409849] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1079client # [ 0.407760] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1080worker2 # [ 0.396372] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1081client # [ 0.408691] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1082worker1 # [ 0.407266] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1083worker2 # [ 0.397374] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1084client # [ 0.409524] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1085lb # [ 0.412077] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1086worker2 # [ 0.398372] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1087worker1 # [ 0.408717] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1088lb # [ 0.412954] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1089lb # [ 0.413802] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1090worker2 # [ 0.399373] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1091client # [ 0.411269] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1092worker2 # [ 0.400386] pci_bus 0000:00: root bus resource [bus 00-ff]1093lb # [ 0.415006] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1094worker1 # [ 0.411920] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1095worker2 # [ 0.401513] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1096worker1 # [ 0.412919] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1097lb # [ 0.416439] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1098client # [ 0.413787] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1099worker2 # [ 0.402863] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1100worker1 # [ 0.413924] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1101client # [ 0.414718] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1102worker1 # [ 0.414905] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1103lb # [ 0.418075] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1104client # [ 0.415559] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1105lb # [ 0.418953] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1106worker1 # [ 0.416529] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1107lb # [ 0.419834] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1108client # [ 0.417416] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1109lb # [ 0.421507] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1110worker1 # [ 0.417916] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1111worker1 # [ 0.418821] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1112worker1 # [ 0.419669] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1113client # [ 0.420801] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1114lb # [ 0.424058] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1115client # [ 0.421758] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1116worker2 # [ 0.405456] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1117lb # [ 0.424942] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1118worker1 # [ 0.421356] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1119client # [ 0.422730] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1120lb # [ 0.425820] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1121worker2 # [ 0.406384] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1122worker2 # [ 0.407395] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1123client # [ 0.424231] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1124lb # [ 0.427538] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125worker1 # [ 0.423928] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1126worker2 # [ 0.408640] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1127worker1 # [ 0.424904] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1128client # [ 0.426787] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1129lb # [ 0.429075] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1130worker1 # [ 0.425922] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1131worker2 # [ 0.410269] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1132client # [ 0.427653] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1133lb # [ 0.429939] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1134lb # [ 0.430815] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1135worker1 # [ 0.427569] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1136client # [ 0.428508] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1137worker2 # [ 0.413399] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1138lb # [ 0.432628] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1139worker2 # [ 0.414266] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1140client # [ 0.430423] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1141worker1 # [ 0.430908] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1142worker2 # [ 0.415104] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1143lb # [ 0.434065] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1144worker2 # [ 0.415390] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1145worker1 # [ 0.431874] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1146client # [ 0.433582] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1147lb # [ 0.434959] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1148client # [ 0.434393] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1149worker1 # [ 0.432921] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1150lb # [ 0.435820] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1151worker2 # [ 0.416971] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1152client # [ 0.434774] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1153worker1 # [ 0.434479] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1154lb # [ 0.436647] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1155client # [ 0.436452] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1156worker2 # [ 0.419379] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1157worker1 # [ 0.435919] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1158worker2 # [ 0.420219] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1159worker1 # [ 0.436813] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1160worker2 # [ 0.421094] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1161lb # [ 0.439504] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1162worker1 # [ 0.437745] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1163client # [ 0.439476] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1164lb # [ 0.440064] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1165client # [ 0.440430] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1166lb # [ 0.441025] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1167worker2 # [ 0.422017] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1168client # [ 0.440774] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1169worker1 # [ 0.439553] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1170client # [ 0.441757] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1171lb # [ 0.442479] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1172worker2 # [ 0.424273] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1173worker1 # [ 0.441912] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1174client # [ 0.443364] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1175worker2 # [ 0.425107] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1176lb # [ 0.444077] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1177worker1 # [ 0.442806] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1178lb # [ 0.444930] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1179worker2 # [ 0.425394] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1180worker1 # [ 0.443675] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1181client # [ 0.444764] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1182lb # [ 0.445790] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1183client # [ 0.445738] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1184lb # [ 0.446063] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1185worker2 # [ 0.426982] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1186worker1 # [ 0.445436] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1187client # [ 0.447210] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1188lb # [ 0.447972] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1189worker1 # [ 0.446933] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1190lb # [ 0.449920] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1191worker2 # [ 0.429388] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1192client # [ 0.449781] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1193worker1 # [ 0.447839] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1194lb # [ 0.450816] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1195worker2 # [ 0.430379] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1196client # [ 0.450635] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1197worker2 # [ 0.431343] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1198worker1 # [ 0.448788] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1199client # [ 0.451469] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1200lb # [ 0.451627] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1201worker1 # [ 0.449881] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1202worker2 # [ 0.432814] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1203client # [ 0.452362] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1204worker1 # [ 0.451284] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1205lb # [ 0.453065] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1206client # [ 0.453771] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1207worker2 # [ 0.435286] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1208lb # [ 0.454066] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1209worker1 # [ 0.452859] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1210worker2 # [ 0.436020] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1211lb # [ 0.455083] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1212worker1 # [ 0.453636] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1213client # [ 0.454621] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1214worker2 # [ 0.436394] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1215client # [ 0.455511] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1216lb # [ 0.456677] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1217worker1 # [ 0.454487] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1218client # [ 0.457247] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1219worker2 # [ 0.438051] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1220lb # [ 0.458066] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1221worker1 # [ 0.455904] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1222lb # [ 0.458970] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1223client # [ 0.458534] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1224worker1 # [ 0.456808] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1225lb # [ 0.459848] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1226worker2 # [ 0.441401] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1227client # [ 0.458988] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1228worker1 # [ 0.457685] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1229worker2 # [ 0.442282] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1230lb # [ 0.461535] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1231client # [ 0.460443] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1232worker2 # [ 0.443119] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1233worker1 # [ 0.459342] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1234lb # [ 0.463014] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1235worker2 # [ 0.443967] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1236worker1 # [ 0.461710] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1237lb # [ 0.463937] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1238worker1 # [ 0.462571] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1239lb # [ 0.465664] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1240worker1 # [ 0.462918] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1241worker2 # [ 0.446622] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1242worker2 # [ 0.448054] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1243worker1 # [ 0.464488] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1244worker2 # [ 0.448393] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1245worker1 # [ 0.465488] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1246worker2 # [ 0.449377] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1247worker1 # [ 0.466107] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1248worker2 # [ 0.450937] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1249worker1 # [ 0.467459] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1250client # [ 0.460961] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1251worker2 # [ 0.452384] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1252worker2 # [ 0.453344] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1253client # [ 0.464689] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1254worker2 # [ 0.454777] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1255lb # [ 0.466299] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1256client # [ 0.465716] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1257lb # [ 0.468650] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1258worker2 # [ 0.458150] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1259client # [ 0.467759] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1260worker2 # [ 0.458986] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1261lb # [ 0.469299] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1262worker2 # [ 0.459393] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1263client # [ 0.468967] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1264lb # [ 0.470660] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1265client # [ 0.470053] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1266worker2 # [ 0.460946] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1267lb # [ 0.471329] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1268worker1 # [ 0.468095] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1269client # [ 0.471033] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1270lb # [ 0.472349] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1271worker2 # [ 0.463191] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1272lb # [ 0.473329] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1273worker1 # [ 0.470938] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1274client # [ 0.472794] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1275worker2 # [ 0.464379] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1276client # [ 0.473627] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1277worker2 # [ 0.465311] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1278lb # [ 0.475096] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1279worker1 # [ 0.472036] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1280lb # [ 0.475951] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1281client # [ 0.474806] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1282worker2 # [ 0.466810] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1283worker1 # [ 0.473903] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1284lb # [ 0.477109] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1285client # [ 0.476779] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1286worker2 # [ 0.468189] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1287worker1 # [ 0.475107] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1288lb # [ 0.478607] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1289worker2 # [ 0.469231] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1290worker1 # [ 0.476186] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1291worker2 # [ 0.471031] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1292worker1 # [ 0.477155] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1293client # [ 0.480745] ACPI: PCI: Interrupt link LNKA configured for IRQ 101294lb # [ 0.482133] ACPI: PCI: Interrupt link LNKA configured for IRQ 101295client # [ 0.481900] ACPI: PCI: Interrupt link LNKB configured for IRQ 101296lb # [ 0.483173] ACPI: PCI: Interrupt link LNKB configured for IRQ 101297worker1 # [ 0.479602] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1298client # [ 0.482859] ACPI: PCI: Interrupt link LNKC configured for IRQ 111299worker1 # [ 0.479902] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1300lb # [ 0.484160] ACPI: PCI: Interrupt link LNKC configured for IRQ 111301client # [ 0.483848] ACPI: PCI: Interrupt link LNKD configured for IRQ 111302lb # [ 0.485159] ACPI: PCI: Interrupt link LNKD configured for IRQ 111303client # [ 0.484847] ACPI: PCI: Interrupt link LNKE configured for IRQ 101304worker1 # [ 0.481187] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1305lb # [ 0.486225] ACPI: PCI: Interrupt link LNKE configured for IRQ 101306client # [ 0.485854] ACPI: PCI: Interrupt link LNKF configured for IRQ 101307worker1 # [ 0.482694] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1308lb # [ 0.487180] ACPI: PCI: Interrupt link LNKF configured for IRQ 101309client # [ 0.486852] ACPI: PCI: Interrupt link LNKG configured for IRQ 111310lb # [ 0.488192] ACPI: PCI: Interrupt link LNKG configured for IRQ 111311client # [ 0.487847] ACPI: PCI: Interrupt link LNKH configured for IRQ 111312client # [ 0.488787] ACPI: PCI: Interrupt link GSIA configured for IRQ 161313lb # [ 0.489162] ACPI: PCI: Interrupt link LNKH configured for IRQ 111314lb # [ 0.490099] ACPI: PCI: Interrupt link GSIA configured for IRQ 161315client # [ 0.489727] ACPI: PCI: Interrupt link GSIB configured for IRQ 171316worker1 # [ 0.486489] ACPI: PCI: Interrupt link LNKA configured for IRQ 101317lb # [ 0.491075] ACPI: PCI: Interrupt link GSIB configured for IRQ 171318client # [ 0.490652] ACPI: PCI: Interrupt link GSIC configured for IRQ 181319worker1 # [ 0.487014] ACPI: PCI: Interrupt link LNKB configured for IRQ 101320lb # [ 0.492039] ACPI: PCI: Interrupt link GSIC configured for IRQ 181321client # [ 0.491506] ACPI: PCI: Interrupt link GSID configured for IRQ 191322worker1 # [ 0.487995] ACPI: PCI: Interrupt link LNKC configured for IRQ 111323client # [ 0.491764] ACPI: PCI: Interrupt link GSIE configured for IRQ 201324lb # [ 0.492805] ACPI: PCI: Interrupt link GSID configured for IRQ 191325worker2 # [ 0.471586] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1326lb # [ 0.493075] ACPI: PCI: Interrupt link GSIE configured for IRQ 201327client # [ 0.492689] ACPI: PCI: Interrupt link GSIF configured for IRQ 211328worker1 # [ 0.488993] ACPI: PCI: Interrupt link LNKD configured for IRQ 111329client # [ 0.493534] ACPI: PCI: Interrupt link GSIG configured for IRQ 221330lb # [ 0.494038] ACPI: PCI: Interrupt link GSIF configured for IRQ 211331worker1 # [ 0.489995] ACPI: PCI: Interrupt link LNKE configured for IRQ 101332client # [ 0.494582] ACPI: PCI: Interrupt link GSIH configured for IRQ 231333worker1 # [ 0.490992] ACPI: PCI: Interrupt link LNKF configured for IRQ 101334worker1 # [ 0.491996] ACPI: PCI: Interrupt link LNKG configured for IRQ 111335worker2 # [ 0.475032] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1336lb # [ 0.494833] ACPI: PCI: Interrupt link GSIG configured for IRQ 221337client # [ 0.496821] iommu: Default domain type: Translated1338worker1 # [ 0.492992] ACPI: PCI: Interrupt link LNKH configured for IRQ 111339lb # [ 0.495079] ACPI: PCI: Interrupt link GSIH configured for IRQ 231340client # [ 0.497592] iommu: DMA domain TLB invalidation policy: lazy mode1341worker2 # [ 0.475679] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1342worker1 # [ 0.493932] ACPI: PCI: Interrupt link GSIA configured for IRQ 161343client # [ 0.498766] ACPI: bus type USB registered1344worker1 # [ 0.494871] ACPI: PCI: Interrupt link GSIB configured for IRQ 171345lb # [ 0.497197] iommu: Default domain type: Translated1346client # [ 0.499501] usbcore: registered new interface driver usbfs1347worker1 # [ 0.495668] ACPI: PCI: Interrupt link GSIC configured for IRQ 181348worker2 # [ 0.477380] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1349lb # [ 0.498031] iommu: DMA domain TLB invalidation policy: lazy mode1350client # [ 0.499770] usbcore: registered new interface driver hub1351worker1 # [ 0.495911] ACPI: PCI: Interrupt link GSID configured for IRQ 191352lb # [ 0.499062] ACPI: bus type USB registered1353worker2 # [ 0.478548] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1354client # [ 0.500620] usbcore: registered new device driver usb1355lb # [ 0.499785] usbcore: registered new interface driver usbfs1356worker2 # [ 0.479687] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1357client # [ 0.502438] NetLabel: Initializing1358lb # [ 0.500077] usbcore: registered new interface driver hub1359worker1 # [ 0.496914] ACPI: PCI: Interrupt link GSIE configured for IRQ 201360client # [ 0.502752] NetLabel: domain hash size = 1281361lb # [ 0.500983] usbcore: registered new device driver usb1362worker1 # [ 0.497914] ACPI: PCI: Interrupt link GSIF configured for IRQ 211363worker2 # [ 0.480633] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1364client # [ 0.503452] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1365worker1 # [ 0.498890] ACPI: PCI: Interrupt link GSIG configured for IRQ 221366client # [ 0.503822] NetLabel: unlabeled traffic allowed by default1367lb # [ 0.502760] NetLabel: Initializing1368worker1 # [ 0.499635] ACPI: PCI: Interrupt link GSIH configured for IRQ 231369client # [ 0.504718] PCI: Using ACPI for IRQ routing1370lb # [ 0.503058] NetLabel: domain hash size = 1281371lb # [ 0.503801] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1372worker2 # [ 0.483400] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1373worker1 # [ 0.501026] iommu: Default domain type: Translated1374lb # [ 0.504095] NetLabel: unlabeled traffic allowed by default1375worker2 # [ 0.484274] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1376worker1 # [ 0.501827] iommu: DMA domain TLB invalidation policy: lazy mode1377lb # [ 0.505044] PCI: Using ACPI for IRQ routing1378worker1 # [ 0.502882] ACPI: bus type USB registered1379worker2 # [ 0.485434] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1380worker1 # [ 0.503434] usbcore: registered new interface driver usbfs1381worker1 # [ 0.503926] usbcore: registered new interface driver hub1382worker2 # [ 0.487292] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1383worker1 # [ 0.504796] usbcore: registered new device driver usb1384worker1 # [ 0.506432] NetLabel: Initializing1385worker1 # [ 0.506897] NetLabel: domain hash size = 1281386worker1 # [ 0.507635] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1387worker1 # [ 0.507926] NetLabel: unlabeled traffic allowed by default1388worker1 # [ 0.508825] PCI: Using ACPI for IRQ routing1389worker2 # [ 0.492534] ACPI: PCI: Interrupt link LNKA configured for IRQ 101390worker2 # [ 0.493482] ACPI: PCI: Interrupt link LNKB configured for IRQ 101391worker2 # [ 0.494472] ACPI: PCI: Interrupt link LNKC configured for IRQ 111392worker2 # [ 0.495472] ACPI: PCI: Interrupt link LNKD configured for IRQ 111393worker2 # [ 0.496469] ACPI: PCI: Interrupt link LNKE configured for IRQ 101394worker2 # [ 0.497469] ACPI: PCI: Interrupt link LNKF configured for IRQ 101395worker2 # [ 0.498471] ACPI: PCI: Interrupt link LNKG configured for IRQ 111396worker2 # [ 0.499480] ACPI: PCI: Interrupt link LNKH configured for IRQ 111397worker2 # [ 0.500409] ACPI: PCI: Interrupt link GSIA configured for IRQ 161398worker2 # [ 0.501388] ACPI: PCI: Interrupt link GSIB configured for IRQ 171399worker2 # [ 0.502387] ACPI: PCI: Interrupt link GSIC configured for IRQ 181400worker2 # [ 0.503388] ACPI: PCI: Interrupt link GSID configured for IRQ 191401worker2 # [ 0.504482] ACPI: PCI: Interrupt link GSIE configured for IRQ 201402worker2 # [ 0.505635] ACPI: PCI: Interrupt link GSIF configured for IRQ 211403worker2 # [ 0.506388] ACPI: PCI: Interrupt link GSIG configured for IRQ 221404worker2 # [ 0.507385] ACPI: PCI: Interrupt link GSIH configured for IRQ 231405worker2 # [ 0.509681] iommu: Default domain type: Translated1406worker2 # [ 0.510379] iommu: DMA domain TLB invalidation policy: lazy mode1407worker2 # [ 0.511683] ACPI: bus type USB registered1408worker2 # [ 0.512438] usbcore: registered new interface driver usbfs1409worker2 # [ 0.513389] usbcore: registered new interface driver hub1410worker2 # [ 0.514312] usbcore: registered new device driver usb1411worker2 # [ 0.516166] NetLabel: Initializing1412worker2 # [ 0.516373] NetLabel: domain hash size = 1281413worker2 # [ 0.517131] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1414worker2 # [ 0.517413] NetLabel: unlabeled traffic allowed by default1415worker2 # [ 0.518360] PCI: Using ACPI for IRQ routing1416client # [ 0.548100] pci 0000:00:01.0: vgaarb: setting as boot VGA device1417client # [ 0.548747] pci 0000:00:01.0: vgaarb: bridge control possible1418client # [ 0.548747] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1419client # [ 0.548757] vgaarb: loaded1420lb # [ 0.549099] pci 0000:00:01.0: vgaarb: setting as boot VGA device1421client # [ 0.549521] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01422lb # [ 0.550053] pci 0000:00:01.0: vgaarb: bridge control possible1423client # [ 0.549754] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1424lb # [ 0.550053] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1425lb # [ 0.550068] vgaarb: loaded1426lb # [ 0.550809] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01427lb # [ 0.551063] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1428client # [ 0.552891] clocksource: Switched to clocksource kvm-clock1429worker1 # [ 0.552265] pci 0000:00:01.0: vgaarb: setting as boot VGA device1430worker1 # [ 0.552892] pci 0000:00:01.0: vgaarb: bridge control possible1431lb # [ 0.554169] clocksource: Switched to clocksource kvm-clock1432client # [ 0.556759] VFS: Disk quotas dquot_6.6.01433worker1 # [ 0.552892] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1434worker1 # [ 0.552900] vgaarb: loaded1435client # [ 0.557459] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1436worker1 # [ 0.553592] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01437client # [ 0.558837] pnp: PnP ACPI init1438worker1 # [ 0.553897] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1439client # [ 0.559676] ACPI: IRQ 4 override to edge(!), high(!)1440lb # [ 0.558192] VFS: Disk quotas dquot_6.6.01441client # [ 0.560666] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1442lb # [ 0.559072] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1443client # [ 0.562229] pnp: PnP ACPI: found 6 devices1444lb # [ 0.560645] pnp: PnP ACPI init1445lb # [ 0.561575] ACPI: IRQ 4 override to edge(!), high(!)1446worker1 # [ 0.557977] clocksource: Switched to clocksource kvm-clock1447lb # [ 0.562546] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1448worker1 # [ 0.559691] VFS: Disk quotas dquot_6.6.01449lb # [ 0.564451] pnp: PnP ACPI: found 6 devices1450worker1 # [ 0.562342] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1451worker1 # [ 0.563696] pnp: PnP ACPI init1452client # [ 0.570047] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1453worker1 # [ 0.564534] ACPI: IRQ 4 override to edge(!), high(!)1454client # [ 0.571456] clocksource: Switched to clocksource acpi_pm1455worker1 # [ 0.565576] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1456client # [ 0.572467] NET: Registered PF_INET protocol family1457worker1 # [ 0.567147] pnp: PnP ACPI: found 6 devices1458client # [ 0.573642] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1459lb # [ 0.572137] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1460lb # [ 0.573843] clocksource: Switched to clocksource acpi_pm1461lb # [ 0.574929] NET: Registered PF_INET protocol family1462lb # [ 0.576281] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1463worker1 # [ 0.575043] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1464worker1 # [ 0.576484] clocksource: Switched to clocksource acpi_pm1465worker1 # [ 0.577471] NET: Registered PF_INET protocol family1466worker1 # [ 0.578655] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1467worker2 # [ 0.561680] pci 0000:00:01.0: vgaarb: setting as boot VGA device1468worker2 # [ 0.562367] pci 0000:00:01.0: vgaarb: bridge control possible1469worker2 # [ 0.562367] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1470worker2 # [ 0.562375] vgaarb: loaded1471worker2 # [ 0.563130] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01472worker2 # [ 0.563372] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1473client # [ 0.590922] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1474client # [ 0.592577] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1475client # [ 0.593864] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1476worker2 # [ 0.566486] clocksource: Switched to clocksource kvm-clock1477client # [ 0.595182] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1478client # [ 0.596374] TCP: Hash tables configured (established 16384 bind 16384)1479lb # [ 0.594273] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1480client # [ 0.597505] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1481lb # [ 0.596147] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1482client # [ 0.598798] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1483worker2 # [ 0.570320] VFS: Disk quotas dquot_6.6.01484client # [ 0.599843] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1485lb # [ 0.597746] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1486worker2 # [ 0.571026] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1487client # [ 0.601088] NET: Registered PF_UNIX/PF_LOCAL protocol family1488worker2 # [ 0.572374] pnp: PnP ACPI init1489lb # [ 0.599345] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1490client # [ 0.601989] NET: Registered PF_XDP protocol family1491worker1 # [ 0.595365] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1492worker2 # [ 0.573185] ACPI: IRQ 4 override to edge(!), high(!)1493client # [ 0.602810] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1494lb # [ 0.600783] TCP: Hash tables configured (established 16384 bind 16384)1495worker1 # [ 0.596868] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1496worker2 # [ 0.574167] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1497client # [ 0.603772] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1498worker2 # [ 0.575671] pnp: PnP ACPI: found 6 devices1499lb # [ 0.602130] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1500worker1 # [ 0.598186] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1501client # [ 0.604737] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1502lb # [ 0.603640] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1503client # [ 0.605787] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1504worker1 # [ 0.599503] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1505client # [ 0.606837] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1506worker1 # [ 0.600763] TCP: Hash tables configured (established 16384 bind 16384)1507lb # [ 0.604926] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1508client # [ 0.607890] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1509worker1 # [ 0.601880] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1510lb # [ 0.606452] NET: Registered PF_UNIX/PF_LOCAL protocol family1511lb # [ 0.607573] NET: Registered PF_XDP protocol family1512worker1 # [ 0.603158] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1513client # [ 0.609645] ACPI: \_SB_.GSIA: Enabled at IRQ 161514lb # [ 0.608522] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1515worker1 # [ 0.604235] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1516lb # [ 0.609593] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1517worker1 # [ 0.605440] NET: Registered PF_UNIX/PF_LOCAL protocol family1518client # [ 0.612567] ACPI: \_SB_.GSIB: Enabled at IRQ 171519worker1 # [ 0.606401] NET: Registered PF_XDP protocol family1520lb # [ 0.610626] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1521worker2 # [ 0.583411] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1522worker1 # [ 0.607240] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1523worker2 # [ 0.584798] clocksource: Switched to clocksource acpi_pm1524lb # [ 0.611787] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1525client # [ 0.614493] ACPI: \_SB_.GSIC: Enabled at IRQ 181526worker1 # [ 0.608252] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1527worker2 # [ 0.585751] NET: Registered PF_INET protocol family1528lb # [ 0.612888] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1529worker1 # [ 0.609249] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1530lb # [ 0.614086] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1531client # [ 0.616414] ACPI: \_SB_.GSID: Enabled at IRQ 191532worker2 # [ 0.586917] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1533worker1 # [ 0.610363] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1534worker1 # [ 0.611448] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1535lb # [ 0.615975] ACPI: \_SB_.GSIA: Enabled at IRQ 161536client # [ 0.618104] PCI: CLS 0 bytes, default 641537worker1 # [ 0.612538] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1538client # [ 0.618973] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1539worker1 # [ 0.614299] ACPI: \_SB_.GSIA: Enabled at IRQ 161540lb # [ 0.618532] ACPI: \_SB_.GSIB: Enabled at IRQ 171541client # [ 0.620694] Trying to unpack rootfs image as initramfs...1542lb # [ 0.620737] ACPI: \_SB_.GSIC: Enabled at IRQ 181543worker1 # [ 0.616651] ACPI: \_SB_.GSIB: Enabled at IRQ 171544lb # [ 0.622774] ACPI: \_SB_.GSID: Enabled at IRQ 191545worker1 # [ 0.619014] ACPI: \_SB_.GSIC: Enabled at IRQ 181546lb # [ 0.624641] PCI: CLS 0 bytes, default 641547worker1 # [ 0.621131] ACPI: \_SB_.GSID: Enabled at IRQ 191548lb # [ 0.625592] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1549worker1 # [ 0.622822] PCI: CLS 0 bytes, default 641550lb # [ 0.627405] Trying to unpack rootfs image as initramfs...1551worker1 # [ 0.623831] Trying to unpack rootfs image as initramfs...1552worker2 # [ 0.604071] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1553worker2 # [ 0.605719] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1554worker2 # [ 0.607066] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1555worker1 # [ 0.630022] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1556worker2 # [ 0.608387] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1557worker2 # [ 0.609695] TCP: Hash tables configured (established 16384 bind 16384)1558worker2 # [ 0.611030] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1559worker2 # [ 0.612382] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1560worker2 # [ 0.613600] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1561worker2 # [ 0.614868] NET: Registered PF_UNIX/PF_LOCAL protocol family1562worker2 # [ 0.615879] NET: Registered PF_XDP protocol family1563worker2 # [ 0.616754] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1564worker2 # [ 0.617794] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1565worker2 # [ 0.618838] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1566worker2 # [ 0.619974] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1567worker2 # [ 0.621118] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1568worker2 # [ 0.622251] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1569worker2 # [ 0.624026] ACPI: \_SB_.GSIA: Enabled at IRQ 161570worker2 # [ 0.626379] ACPI: \_SB_.GSIB: Enabled at IRQ 171571worker2 # [ 0.628327] ACPI: \_SB_.GSIC: Enabled at IRQ 181572worker2 # [ 0.630735] ACPI: \_SB_.GSID: Enabled at IRQ 191573worker2 # [ 0.632406] PCI: CLS 0 bytes, default 641574worker2 # [ 0.633333] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1575worker2 # [ 0.635144] Trying to unpack rootfs image as initramfs...1576client # [ 0.666505] Initialise system trusted keyrings1577client # [ 0.669282] workingset: timestamp_bits=40 max_order=19 bucket_order=01578lb # [ 0.673488] Initialise system trusted keyrings1579lb # [ 0.677019] workingset: timestamp_bits=40 max_order=19 bucket_order=01580worker1 # [ 0.673175] Initialise system trusted keyrings1581worker1 # [ 0.674139] workingset: timestamp_bits=40 max_order=19 bucket_order=01582client # [ 0.691399] Key type asymmetric registered1583client # [ 0.692176] Asymmetric key parser 'x509' registered1584client # [ 0.695073] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1585client # [ 0.698134] io scheduler mq-deadline registered1586lb # [ 0.696462] Key type asymmetric registered1587client # [ 0.698874] io scheduler kyber registered1588lb # [ 0.699972] Asymmetric key parser 'x509' registered1589client # [ 0.703101] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1590lb # [ 0.700892] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1591worker1 # [ 0.698085] Key type asymmetric registered1592client # [ 0.704377] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1593worker1 # [ 0.698832] Asymmetric key parser 'x509' registered1594worker1 # [ 0.699678] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1595lb # [ 0.706098] io scheduler mq-deadline registered1596lb # [ 0.706880] io scheduler kyber registered1597worker1 # [ 0.702876] io scheduler mq-deadline registered1598worker1 # [ 0.703637] io scheduler kyber registered1599worker2 # [ 0.681078] Initialise system trusted keyrings1600client # [ 0.710302] Linux agpgart interface v0.1031601client # [ 0.711113] ACPI: bus type drm_connector registered1602lb # [ 0.711166] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1603worker2 # [ 0.684647] workingset: timestamp_bits=40 max_order=19 bucket_order=01604worker1 # [ 0.707880] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1605lb # [ 0.712493] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1606client # [ 0.715219] usbcore: registered new interface driver usbserial_generic1607worker1 # [ 0.709173] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1608client # [ 0.716297] usbserial: USB Serial support registered for generic1609client # [ 0.719070] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1610lb # [ 0.717804] Linux agpgart interface v0.1031611client # [ 0.719070] Please enable it if your BIOS has the CPPC option.1612worker1 # [ 0.714613] Linux agpgart interface v0.1031613lb # [ 0.718773] ACPI: bus type drm_connector registered1614client # [ 0.721226] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1615worker1 # [ 0.715432] ACPI: bus type drm_connector registered1616lb # [ 0.721476] usbcore: registered new interface driver usbserial_generic1617lb # [ 0.722595] usbserial: USB Serial support registered for generic1618client # [ 0.725302] drop_monitor: Initializing network drop monitor service1619worker1 # [ 0.719888] usbcore: registered new interface driver usbserial_generic1620client # [ 0.726524] NET: Registered PF_INET6 protocol family1621worker1 # [ 0.720963] usbserial: USB Serial support registered for generic1622lb # [ 0.725970] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1623lb # [ 0.725970] Please enable it if your BIOS has the CPPC option.1624client # [ 0.730724] Segment Routing with IPv61625worker1 # [ 0.723730] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1626lb # [ 0.728351] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1627worker1 # [ 0.723730] Please enable it if your BIOS has the CPPC option.1628client # [ 0.731429] In-situ OAM (IOAM) with IPv61629worker1 # [ 0.725953] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1630lb # [ 0.732140] drop_monitor: Initializing network drop monitor service1631client # [ 0.734441] IPI shorthand broadcast: enabled1632worker2 # [ 0.705918] Key type asymmetric registered1633worker1 # [ 0.728898] drop_monitor: Initializing network drop monitor service1634lb # [ 0.733467] NET: Registered PF_INET6 protocol family1635worker2 # [ 0.706656] Asymmetric key parser 'x509' registered1636worker1 # [ 0.730071] NET: Registered PF_INET6 protocol family1637worker2 # [ 0.707523] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1638lb # [ 0.738454] Segment Routing with IPv61639worker1 # [ 0.734849] Segment Routing with IPv61640lb # [ 0.739124] In-situ OAM (IOAM) with IPv61641worker1 # [ 0.735532] In-situ OAM (IOAM) with IPv61642worker2 # [ 0.712803] io scheduler mq-deadline registered1643worker2 # [ 0.713614] io scheduler kyber registered1644client # [ 0.742825] sched_clock: Marking stable (606014988, 136036525)->(812841796, -70790283)1645worker1 # [ 0.738015] IPI shorthand broadcast: enabled1646lb # [ 0.742371] IPI shorthand broadcast: enabled1647client # [ 0.746346] registered taskstats version 11648worker2 # [ 0.717622] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1649client # [ 0.747370] Loading compiled-in X.509 certificates1650worker2 # [ 0.718938] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1651worker1 # [ 0.746288] sched_clock: Marking stable (600015733, 145701277)->(820070800, -74353790)1652lb # [ 0.750817] sched_clock: Marking stable (598015141, 151939190)->(824439487, -74485156)1653worker2 # [ 0.724506] Linux agpgart interface v0.1031654worker2 # [ 0.725318] ACPI: bus type drm_connector registered1655lb # [ 0.753162] registered taskstats version 11656lb # [ 0.754191] Loading compiled-in X.509 certificates1657worker1 # [ 0.750944] registered taskstats version 11658worker1 # [ 0.751937] Loading compiled-in X.509 certificates1659worker2 # [ 0.730163] usbcore: registered new interface driver usbserial_generic1660worker2 # [ 0.731347] usbserial: USB Serial support registered for generic1661worker2 # [ 0.734593] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1662worker2 # [ 0.734593] Please enable it if your BIOS has the CPPC option.1663client # [ 0.766333] Demotion targets for Node 0: null1664worker2 # [ 0.736916] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1665client # [ 0.769139] Key type .fscrypt registered1666worker2 # [ 0.739790] drop_monitor: Initializing network drop monitor service1667client # [ 0.769811] Key type fscrypt-provisioning registered1668worker2 # [ 0.741037] NET: Registered PF_INET6 protocol family1669client # [ 0.770745] ima: No TPM chip found, activating TPM-bypass!1670client # [ 0.773078] ima: Allocated hash algorithm: sha11671client # [ 0.773868] ima: No architecture policies found1672worker2 # [ 0.745183] Segment Routing with IPv61673worker2 # [ 0.747614] In-situ OAM (IOAM) with IPv61674worker1 # [ 0.770734] Demotion targets for Node 0: null1675lb # [ 0.774974] Demotion targets for Node 0: null1676worker1 # [ 0.771630] Key type .fscrypt registered1677worker2 # [ 0.748742] IPI shorthand broadcast: enabled1678lb # [ 0.776026] Key type .fscrypt registered1679client # [ 0.778076] PM: Magic number: 10:378:5341680worker1 # [ 0.772387] Key type fscrypt-provisioning registered1681lb # [ 0.776750] Key type fscrypt-provisioning registered1682worker1 # [ 0.773323] ima: No TPM chip found, activating TPM-bypass!1683lb # [ 0.777742] ima: No TPM chip found, activating TPM-bypass!1684client # [ 0.779827] RAS: Correctable Errors collector initialized.1685lb # [ 0.781965] ima: Allocated hash algorithm: sha11686worker1 # [ 0.777740] ima: Allocated hash algorithm: sha11687lb # [ 0.782788] ima: No architecture policies found1688worker1 # [ 0.778554] ima: No architecture policies found1689worker1 # [ 0.779534] PM: Magic number: 10:378:5341690worker2 # [ 0.756954] sched_clock: Marking stable (613013913, 143567297)->(836289299, -79708089)1691lb # [ 0.784969] PM: Magic number: 10:378:5341692lb # [ 0.786675] RAS: Correctable Errors collector initialized.1693client # [ 0.788919] clk: Disabling unused clocks1694worker1 # [ 0.783250] RAS: Correctable Errors collector initialized.1695worker2 # [ 0.761899] registered taskstats version 11696client # [ 0.791070] PM: genpd: Disabling unused power domains1697worker2 # [ 0.762928] Loading compiled-in X.509 certificates1698lb # [ 0.795850] clk: Disabling unused clocks1699worker1 # [ 0.792071] clk: Disabling unused clocks1700lb # [ 0.798976] PM: genpd: Disabling unused power domains1701worker1 # [ 0.794732] PM: genpd: Disabling unused power domains1702worker2 # [ 0.781612] Demotion targets for Node 0: null1703worker2 # [ 0.782616] Key type .fscrypt registered1704worker2 # [ 0.783326] Key type fscrypt-provisioning registered1705worker2 # [ 0.784331] ima: No TPM chip found, activating TPM-bypass!1706worker2 # [ 0.789603] ima: Allocated hash algorithm: sha11707worker2 # [ 0.790416] ima: No architecture policies found1708worker2 # [ 0.792604] PM: Magic number: 10:378:5341709worker2 # [ 0.794402] RAS: Correctable Errors collector initialized.1710worker2 # [ 0.803356] clk: Disabling unused clocks1711worker2 # [ 0.807609] PM: genpd: Disabling unused power domains1712client # [ 0.977313] Freeing initrd memory: 29096K1713client # [ 0.980742] Freeing unused decrypted memory: 2028K1714worker1 # [ 0.977499] Freeing initrd memory: 29092K1715client # [ 0.983506] Freeing unused kernel image (initmem) memory: 3644K1716client # [ 0.984577] Write protecting the kernel read-only data: 32768k1717client # [ 0.986525] Freeing unused kernel image (text/rodata gap) memory: 1208K1718worker1 # [ 0.980819] Freeing unused decrypted memory: 2028K1719client # [ 0.988056] Freeing unused kernel image (rodata/data gap) memory: 752K1720worker1 # [ 0.983554] Freeing unused kernel image (initmem) memory: 3644K1721lb # [ 0.989188] Freeing initrd memory: 29092K1722worker1 # [ 0.984669] Write protecting the kernel read-only data: 32768k1723worker1 # [ 0.986583] Freeing unused kernel image (text/rodata gap) memory: 1208K1724lb # [ 0.992384] Freeing unused decrypted memory: 2028K1725worker1 # [ 0.988188] Freeing unused kernel image (rodata/data gap) memory: 752K1726lb # [ 0.995034] Freeing unused kernel image (initmem) memory: 3644K1727lb # [ 0.996107] Write protecting the kernel read-only data: 32768k1728lb # [ 0.997991] Freeing unused kernel image (text/rodata gap) memory: 1208K1729lb # [ 0.999478] Freeing unused kernel image (rodata/data gap) memory: 752K1730worker2 # [ 0.984419] Freeing initrd memory: 29092K1731worker2 # [ 0.988029] Freeing unused decrypted memory: 2028K1732worker2 # [ 0.990924] Freeing unused kernel image (initmem) memory: 3644K1733worker2 # [ 0.992085] Write protecting the kernel read-only data: 32768k1734worker2 # [ 0.994124] Freeing unused kernel image (text/rodata gap) memory: 1208K1735worker2 # [ 0.995789] Freeing unused kernel image (rodata/data gap) memory: 752K1736client # [ 1.039785] x86/mm: Checked W+X mappings: passed, no W+X pages found.1737client # [ 1.040903] Run /init as init process1738worker1 # [ 1.040382] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739worker1 # [ 1.041547] Run /init as init process1740client # [ 1.051807] systemd[1]: Inserted module 'autofs4'1741lb # [ 1.051480] x86/mm: Checked W+X mappings: passed, no W+X pages found.1742lb # [ 1.052569] Run /init as init process1743worker1 # [ 1.052534] systemd[1]: Inserted module 'autofs4'1744lb # [ 1.063368] systemd[1]: Inserted module 'autofs4'1745client # [ 1.068921] fuse: init (API version 7.45)1746client # [ 1.075123] ACPI: \_SB_.GSIG: Enabled at IRQ 221747worker1 # [ 1.069600] fuse: init (API version 7.45)1748worker2 # [ 1.047701] x86/mm: Checked W+X mappings: passed, no W+X pages found.1749client # [ 1.077580] ACPI: \_SB_.GSIH: Enabled at IRQ 231750worker2 # [ 1.048875] Run /init as init process1751client # [ 1.080779] ACPI: \_SB_.GSIE: Enabled at IRQ 201752worker1 # [ 1.075858] ACPI: \_SB_.GSIG: Enabled at IRQ 221753lb # [ 1.080476] fuse: init (API version 7.45)1754client # [ 1.082889] ACPI: \_SB_.GSIF: Enabled at IRQ 211755worker1 # [ 1.078355] ACPI: \_SB_.GSIH: Enabled at IRQ 231756worker1 # [ 1.081331] ACPI: \_SB_.GSIE: Enabled at IRQ 201757lb # [ 1.086497] ACPI: \_SB_.GSIG: Enabled at IRQ 221758worker2 # [ 1.060014] systemd[1]: Inserted module 'autofs4'1759worker1 # [ 1.083639] ACPI: \_SB_.GSIF: Enabled at IRQ 211760lb # [ 1.088773] ACPI: \_SB_.GSIH: Enabled at IRQ 231761lb # [ 1.091718] ACPI: \_SB_.GSIE: Enabled at IRQ 201762lb # [ 1.094197] ACPI: \_SB_.GSIF: Enabled at IRQ 211763worker2 # [ 1.077380] fuse: init (API version 7.45)1764client # [ 1.111111] systemd[1]: Successfully made /usr/ read-only.1765worker2 # [ 1.083351] ACPI: \_SB_.GSIG: Enabled at IRQ 221766worker2 # [ 1.085909] ACPI: \_SB_.GSIH: Enabled at IRQ 231767worker2 # [ 1.088958] ACPI: \_SB_.GSIE: Enabled at IRQ 201768worker1 # [ 1.113901] systemd[1]: Successfully made /usr/ read-only.1769worker2 # [ 1.091193] ACPI: \_SB_.GSIF: Enabled at IRQ 211770lb # [ 1.120822] systemd[1]: Successfully made /usr/ read-only.1771worker2 # [ 1.118644] systemd[1]: Successfully made /usr/ read-only.1772client # [ 1.447576] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1773client # [ 1.458766] systemd[1]: Detected virtualization kvm.1774client # [ 1.460690] systemd[1]: Detected architecture x86-64.1775client # [ 1.462732] systemd[1]: Running in initrd.1776client # [ 1.465015] systemd[1]: Initializing machine ID from random generator.1777lb # [ 1.456223] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker1 # [ 1.450661] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1779client # [ 1.467925] systemd[1]: Hostname set to <client>.1780worker1 # [ 1.462842] systemd[1]: Detected virtualization kvm.1781lb # [ 1.466878] systemd[1]: Detected virtualization kvm.1782worker1 # [ 1.464938] systemd[1]: Detected architecture x86-64.1783lb # [ 1.469388] systemd[1]: Detected architecture x86-64.1784worker1 # [ 1.467148] systemd[1]: Running in initrd.1785lb # [ 1.471810] systemd[1]: Running in initrd.1786worker1 # [ 1.469818] systemd[1]: Initializing machine ID from random generator.1787lb # [ 1.474642] systemd[1]: Initializing machine ID from random generator.1788worker1 # [ 1.472885] systemd[1]: Hostname set to <worker1>.1789lb # [ 1.478014] systemd[1]: Hostname set to <lb>.1790worker2 # [ 1.455427] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1791worker2 # [ 1.467452] systemd[1]: Detected virtualization kvm.1792worker2 # [ 1.469653] systemd[1]: Detected architecture x86-64.1793worker2 # [ 1.471762] systemd[1]: Running in initrd.1794worker2 # [ 1.474300] systemd[1]: Initializing machine ID from random generator.1795worker2 # [ 1.477221] systemd[1]: Hostname set to <worker2>.1796worker1 # [ 1.683917] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797client # [ 1.690677] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798lb # [ 1.694644] systemd[1]: bpf-restrict-fs: LSM BPF program attached1799worker2 # [ 1.683280] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker1 # [ 1.726138] systemd[1]: Queued start job for default target Initrd Default Target.1801client # [ 1.733663] systemd[1]: Queued start job for default target Initrd Default Target.1802worker1 # [ 1.730392] systemd[1]: Created slice Slice /system/modprobe.1803client # [ 1.737438] systemd[1]: Created slice Slice /system/modprobe.1804worker1 # [ 1.731576] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1805client # [ 1.738572] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1806worker1 # [ 1.732957] systemd[1]: Expecting device /dev/disk/by-label/nixos...1807client # [ 1.739905] systemd[1]: Expecting device /dev/disk/by-label/nixos...1808worker1 # [ 1.734036] systemd[1]: Reached target Path Units.1809client # [ 1.740952] systemd[1]: Reached target Path Units.1810worker1 # [ 1.734942] systemd[1]: Reached target Slice Units.1811client # [ 1.741844] systemd[1]: Reached target Slice Units.1812worker1 # [ 1.735813] systemd[1]: Reached target Swaps.1813client # [ 1.742711] systemd[1]: Reached target Swaps.1814worker1 # [ 1.736586] systemd[1]: Reached target Timer Units.1815client # [ 1.743476] systemd[1]: Reached target Timer Units.1816worker1 # [ 1.737599] systemd[1]: Listening on D-Bus System Message Bus Socket.1817client # [ 1.744401] systemd[1]: Listening on D-Bus System Message Bus Socket.1818worker1 # [ 1.738832] systemd[1]: Listening on Journal Socket (/dev/log).1819client # [ 1.745557] systemd[1]: Listening on Journal Socket (/dev/log).1820worker1 # [ 1.739969] systemd[1]: Listening on Journal Sockets.1821client # [ 1.746658] systemd[1]: Listening on Journal Sockets.1822worker1 # [ 1.740947] systemd[1]: Listening on udev Control Socket.1823client # [ 1.747614] systemd[1]: Listening on udev Control Socket.1824worker1 # [ 1.741957] systemd[1]: Listening on udev Kernel Socket.1825lb # [ 1.745829] systemd[1]: Queued start job for default target Initrd Default Target.1826client # [ 1.748595] systemd[1]: Listening on udev Kernel Socket.1827worker1 # [ 1.742886] systemd[1]: Reached target Socket Units.1828client # [ 1.749511] systemd[1]: Reached target Socket Units.1829worker1 # [ 1.744903] systemd[1]: Starting Create List of Static Device Nodes...1830client # [ 1.751420] systemd[1]: Starting Create List of Static Device Nodes...1831worker2 # [ 1.722089] systemd[1]: Queued start job for default target Initrd Default Target.1832lb # [ 1.750263] systemd[1]: Created slice Slice /system/modprobe.1833lb # [ 1.751378] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1834lb # [ 1.752727] systemd[1]: Expecting device /dev/disk/by-label/nixos...1835worker2 # [ 1.725979] systemd[1]: Created slice Slice /system/modprobe.1836client # [ 1.755129] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1837lb # [ 1.753780] systemd[1]: Reached target Path Units.1838lb # [ 1.754624] systemd[1]: Reached target Slice Units.1839worker2 # [ 1.727131] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1840worker1 # [ 1.750805] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1841lb # [ 1.755470] systemd[1]: Reached target Swaps.1842worker2 # [ 1.728513] systemd[1]: Expecting device /dev/disk/by-label/nixos...1843lb # [ 1.756273] systemd[1]: Reached target Timer Units.1844worker2 # [ 1.729629] systemd[1]: Reached target Path Units.1845lb # [ 1.757187] systemd[1]: Listening on D-Bus System Message Bus Socket.1846worker2 # [ 1.730470] systemd[1]: Reached target Slice Units.1847lb # [ 1.758330] systemd[1]: Listening on Journal Socket (/dev/log).1848worker2 # [ 1.731322] systemd[1]: Reached target Swaps.1849worker2 # [ 1.732102] systemd[1]: Reached target Timer Units.1850lb # [ 1.759468] systemd[1]: Listening on Journal Sockets.1851client # [ 1.762135] systemd[1]: Starting Load Kernel Module configfs...1852lb # [ 1.760416] systemd[1]: Listening on udev Control Socket.1853worker2 # [ 1.733062] systemd[1]: Listening on D-Bus System Message Bus Socket.1854worker1 # [ 1.756102] systemd[1]: Starting Load Kernel Module configfs...1855lb # [ 1.761359] systemd[1]: Listening on udev Kernel Socket.1856worker2 # [ 1.734254] systemd[1]: Listening on Journal Socket (/dev/log).1857lb # [ 1.762273] systemd[1]: Reached target Socket Units.1858worker2 # [ 1.735372] systemd[1]: Listening on Journal Sockets.1859worker2 # [ 1.736354] systemd[1]: Listening on udev Control Socket.1860lb # [ 1.763929] systemd[1]: Starting Create List of Static Device Nodes...1861worker2 # [ 1.737382] systemd[1]: Listening on udev Kernel Socket.1862worker2 # [ 1.738302] systemd[1]: Reached target Socket Units.1863worker2 # [ 1.740120] systemd[1]: Starting Create List of Static Device Nodes...1864lb # [ 1.767760] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1865worker2 # [ 1.744655] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1866client # [ 1.775128] systemd[1]: Starting Journal Service...1867worker1 # [ 1.770647] systemd[1]: Starting Journal Service...1868lb # [ 1.775134] systemd[1]: Starting Load Kernel Module configfs...1869worker2 # [ 1.752850] systemd[1]: Starting Load Kernel Module configfs...1870client # [ 1.784266] systemd[1]: Starting Load Kernel Modules...1871lb # [ 1.789130] systemd[1]: Starting Journal Service...1872worker1 # [ 1.784820] systemd[1]: Starting Load Kernel Modules...1873worker2 # [ 1.767229] systemd[1]: Starting Journal Service...1874client # [ 1.790140] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1875lb # [ 1.802806] systemd[1]: Starting Load Kernel Modules...1876worker2 # [ 1.779675] systemd[1]: Starting Load Kernel Modules...1877worker1 # [ 1.791834] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1878client # [ 1.812345] netfs: FS-Cache loaded1879worker1 # [ 1.811039] netfs: FS-Cache loaded1880lb # [ 1.803752] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1881client # [ 1.817829] 9pnet: Installing 9P2000 support1882worker1 # [ 1.816204] 9pnet: Installing 9P2000 support1883lb # [ 1.822886] netfs: FS-Cache loaded1884client # [ 1.826413] systemd[1]: Starting Coldplug All udev Devices...1885worker2 # [ 1.786665] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1886lb # [ 1.827588] 9pnet: Installing 9P2000 support1887worker2 # [ 1.803138] netfs: FS-Cache loaded1888worker1 # [ 1.827806] systemd[1]: Starting Coldplug All udev Devices...1889worker2 # [ 1.807843] 9pnet: Installing 9P2000 support1890lb # [ 1.839201] systemd[1]: Starting Coldplug All udev Devices...1891client # [ 1.845180] systemd[1]: Finished Create List of Static Device Nodes.1892worker2 # [ 1.816793] systemd[1]: Starting Coldplug All udev Devices...1893worker1 # [ 1.842866] systemd[1]: Finished Create List of Static Device Nodes.1894client # [ 1.851876] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1895lb # [ 1.852019] systemd[1]: Finished Create List of Static Device Nodes.1896worker1 # [ 1.848492] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897client # [ 1.858123] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1898client # [ 1.859510] systemd-journald[67]: Collecting audit messages is disabled.1899lb # [ 1.857525] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1900worker2 # [ 1.833685] systemd[1]: Finished Create List of Static Device Nodes.1901worker1 # [ 1.857289] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1902client # [ 1.864549] systemd[1]: modprobe@configfs.service: Deactivated successfully.1903lb # [ 1.865058] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1904lb # [ 1.867625] systemd-journald[67]: Collecting audit messages is disabled.1905worker1 # [ 1.863647] systemd[1]: modprobe@configfs.service: Deactivated successfully.1906client # [ 1.871476] systemd[1]: Finished Load Kernel Module configfs.1907worker2 # [ 1.842406] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1908lb # [ 1.870364] systemd[1]: modprobe@configfs.service: Deactivated successfully.1909worker1 # [ 1.868742] systemd-journald[67]: Collecting audit messages is disabled.1910worker1 # [ 1.870784] systemd[1]: Finished Load Kernel Module configfs.1911client # [ 1.875873] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1912lb # [ 1.878315] systemd[1]: Finished Load Kernel Module configfs.1913client # [ 1.879573] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1914worker2 # [ 1.851651] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1915worker1 # [ 1.877327] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1916worker2 # [ 1.856264] systemd-journald[67]: Collecting audit messages is disabled.1917lb # [ 1.884418] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1918worker2 # [ 1.859089] systemd[1]: modprobe@configfs.service: Deactivated successfully.1919worker1 # [ 1.882457] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1920client # [ 1.889300] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1921lb # [ 1.888979] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1922client # [ 1.891398] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1923worker2 # [ 1.866976] systemd[1]: Finished Load Kernel Module configfs.1924lb # [ 1.894040] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1925worker1 # [ 1.890041] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1926worker1 # [ 1.891500] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1927lb # [ 1.898071] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1928worker2 # [ 1.872167] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1929worker2 # [ 1.876852] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1930worker2 # [ 1.883839] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1931worker2 # [ 1.886614] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1932client # [ 1.923650] systemd[1]: Finished Load Kernel Modules.1933lb # [ 1.927516] systemd[1]: Finished Load Kernel Modules.1934worker1 # [ 1.924387] systemd[1]: Finished Load Kernel Modules.1935client # [ 1.932294] systemd[1]: Starting Apply Kernel Variables...1936lb # [ 1.935148] systemd[1]: Starting Apply Kernel Variables...1937worker1 # [ 1.933012] systemd[1]: Starting Apply Kernel Variables...1938client # [ 1.943157] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1939lb # [ 1.948027] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1940worker2 # [ 1.921238] systemd[1]: Finished Load Kernel Modules.1941client # [ 1.954149] systemd[1]: Starting Create Static Device Nodes in /dev...1942lb # [ 1.957187] systemd[1]: Starting Create Static Device Nodes in /dev...1943worker2 # [ 1.930813] systemd[1]: Starting Apply Kernel Variables...1944worker1 # [ 1.954876] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1945worker1 # [ 1.964278] systemd[1]: Starting Create Static Device Nodes in /dev...1946lb # [ 1.970133] systemd[1]: Finished Apply Kernel Variables.1947client # [ 1.973643] systemd[1]: Finished Apply Kernel Variables.1948worker2 # [ 1.947682] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1949client # [ 1.984171] systemd[1]: Started Journal Service.1950lb # [ 1.983167] systemd[1]: Started Journal Service.1951worker2 # [ 1.955982] systemd[1]: Starting Create Static Device Nodes in /dev...1952worker1 # [ 1.979943] systemd[1]: Finished Apply Kernel Variables.1953client # [ 1.846828] systemd-modules-load[68]: Inserted module 'dm_mod'1954client # [ 1.851984] systemd-modules-load[68]: Inserted module 'virtio_balloon'1955lb # [ 1.832237] systemd-modules-load[68]: Inserted module 'dm_mod'1956lb # [ 1.835322] systemd-modules-load[68]: Inserted module 'virtio_balloon'1957lb # [ 1.837722] systemd-modules-load[68]: Inserted module 'virtio_gpu'1958client # [ 1.856089] systemd-modules-load[68]: Inserted module 'virtio_gpu'1959lb # [ 1.842940] systemd[1]: Finished Create Static Device Nodes in /dev.1960worker1 # [ 1.846304] systemd-modules-load[68]: Inserted module 'dm_mod'1961worker1 # [ 1.848314] systemd-modules-load[68]: Inserted module 'virtio_balloon'1962client # [ 1.864218] systemd[1]: Finished Create Static Device Nodes in /dev.1963worker1 # [ 1.994967] systemd[1]: Started Journal Service.1964lb # [ 1.847914] systemd[1]: Reached target Preparation for Local File Systems.1965client # [ 1.865940] systemd[1]: Reached target Preparation for Local File Systems.1966worker1 # [ 1.851383] systemd-modules-load[68]: Inserted module 'virtio_gpu'1967lb # [ 1.851074] systemd[1]: Reached target Local File Systems.1968client # [ 1.869057] systemd[1]: Reached target Local File Systems.1969lb # [ 1.854064] systemd[1]: Starting Create System Files and Directories...1970client # [ 1.872108] systemd[1]: Starting Create System Files and Directories...1971worker2 # [ 1.837251] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.982126] systemd[1]: Started Journal Service.1972worker2 # 1973worker2 # [ 1.839490] systemd-modules-load[68]: Inserted module 'virtio_balloon'1974lb # [ 1.859075] systemd[1]: Starting Rule-based Manager for Device Events and Files...1975worker2 # [ 1.841361] systemd-modules-load[68]: Inserted module 'virtio_gpu'1976client # [ 1.878356] systemd[1]: Starting Rule-based Manager for Device Events and Files...1977worker2 # [ 1.847253] systemd[1]: Finished Apply Kernel Variables.1978worker1 # [ 1.870229] systemd[1]: Finished Create Static Device Nodes in /dev.1979worker1 # [ 1.871904] systemd[1]: Reached target Preparation for Local File Systems.1980worker1 # [ 1.875069] systemd[1]: Reached target Local File Systems.1981worker1 # [ 1.878062] systemd[1]: Starting Create System Files and Directories...1982worker1 # [ 1.883081] systemd[1]: Starting Rule-based Manager for Device Events and Files...1983worker2 # [ 1.864151] systemd[1]: Finished Create Static Device Nodes in /dev.1984worker2 # [ 1.865244] systemd[1]: Reached target Preparation for Local File Systems.1985worker2 # [ 1.866177] systemd[1]: Reached target Local File Systems.1986client # [ 1.905297] systemd[1]: Finished Create System Files and Directories.1987worker2 # [ 1.869071] systemd[1]: Starting Create System Files and Directories...1988lb # [ 1.888660] systemd[1]: Finished Create System Files and Directories.1989worker2 # [ 1.873765] systemd[1]: Starting Rule-based Manager for Device Events and Files...1990client # [ 1.928281] systemd-udevd[83]: Using default interface naming scheme 'v261'.1991lb # [ 1.914308] systemd-udevd[83]: Using default interface naming scheme 'v261'.1992worker1 # [ 1.920654] systemd[1]: Finished Create System Files and Directories.1993worker2 # [ 1.911576] systemd[1]: Finished Create System Files and Directories.1994client # [ 1.955095] systemd[1]: Started Rule-based Manager for Device Events and Files.1995lb # [ 1.938210] systemd[1]: Started Rule-based Manager for Device Events and Files.1996worker1 # [ 1.942838] systemd-udevd[83]: Using default interface naming scheme 'v261'.1997worker2 # [ 1.932460] systemd-udevd[83]: Using default interface naming scheme 'v261'.1998worker1 # [ 1.971110] systemd[1]: Started Rule-based Manager for Device Events and Files.1999worker2 # [ 1.961140] systemd[1]: Started Rule-based Manager for Device Events and Files.2000lb # [ 1.985188] systemd[1]: Finished Coldplug All udev Devices.
loading 2,064 hidden lines…
4065lb # [ 18.685000] envoy[984]: [2026-09-20 13:32:23.772][984][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4066lb # [ 18.687437] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4067lb # [ 18.693863] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4068lb # [ 18.696385] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4069lb # [ 18.698275] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4070lb # [ 18.701530] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4071lb # [ 18.703236] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4072lb # [ 18.708857] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4073lb # [ 18.715190] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4074lb # [ 18.716969] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4075lb # [ 18.719148] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4076lb # [ 18.721422] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4077lb # [ 18.723417] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4078lb # [ 18.726490] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4079lb # [ 18.736142] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4080lb # [ 18.738713] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4081lb # [ 18.740857] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4082lb # [ 18.742603] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4083lb # [ 18.744913] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4084lb # [ 18.746914] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4085lb # [ 18.749356] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4086lb # [ 18.754066] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4087lb # [ 18.755856] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4088lb # [ 18.757930] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4089lb # [ 18.760796] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4090lb # [ 18.768440] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4091lb # [ 18.771255] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4092lb # [ 18.772854] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4093lb # [ 18.775129] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4094lb # [ 18.778257] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4095lb # [ 18.780122] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4096lb # [ 18.782735] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4097lb # [ 18.784608] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4098lb # [ 18.787033] envoy[984]: [2026-09-20 13:32:23.773][984][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4099lb # [ 18.788913] envoy[984]: [2026-09-20 13:32:23.933][984][info][main] [source/server/server.cc:531] HTTP header map info:4100lb # [ 18.790220] envoy[984]: [2026-09-20 13:32:23.967][984][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4101lb # [ 18.803628] envoy[984]: [2026-09-20 13:32:23.967][984][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4102lb # [ 18.804986] envoy[984]: [2026-09-20 13:32:23.967][984][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4103lb # [ 18.812479] envoy[984]: [2026-09-20 13:32:23.967][984][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4104lb # [ 18.848895] envoy[984]: [2026-09-20 13:32:24.146][984][info][main] [source/server/server.cc:968] runtime: {}4105lb # [ 18.855515] envoy[984]: [2026-09-20 13:32:24.153][984][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4106lb # [ 18.857765] envoy[984]: [2026-09-20 13:32:24.155][984][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014107lb # [ 18.860615] envoy[984]: [2026-09-20 13:32:24.158][984][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014108lb # [ 18.863616] envoy[984]: [2026-09-20 13:32:24.161][984][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4109lb # [ 18.865304] envoy[984]: [2026-09-20 13:32:24.163][984][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4110lb # [ 18.866895] envoy[984]: [2026-09-20 13:32:24.163][984][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4111lb # [ 18.924702] envoy[984]: [2026-09-20 13:32:24.222][984][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4112worker1 # [ 18.930460] systemd[1]: Starting Nix worker-protocol over gRPC...4113worker2 # [ 18.908662] systemd[1]: Reached target Network is Online.4114worker2 # [ 18.916718] systemd[1]: Starting scheduler-token.service...4115lb # [ 18.952926] envoy[984]: [2026-09-20 13:32:24.250][984][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4116lb # [ 18.955535] envoy[984]: [2026-09-20 13:32:24.253][984][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4117lb # [ 18.958452] envoy[984]: [2026-09-20 13:32:24.253][984][info][main] [source/server/server.cc:1077] starting main dispatch loop4118worker2 # [ 19.032561] systemd[1]: scheduler-token.service: Deactivated successfully.4119worker2 # [ 19.034177] systemd[1]: Finished scheduler-token.service.4120worker2 # [ 19.035303] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 117ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4121worker2 # [ 19.040712] systemd[1]: Starting Nix worker-protocol over gRPC...4122worker1 # [ 19.145034] nix-grpc-daemon[842]: ts=2026-09-20T13:32:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4123worker1 # [ 19.155148] nix-grpc-daemon[842]: ts=2026-09-20T13:32:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4124worker1 # [ 19.162427] nix-grpc-daemon[842]: ts=2026-09-20T13:32:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14125worker1 # [ 19.206535] nix-grpc-daemon[842]: ts=2026-09-20T13:32:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4126worker1 # [ 19.209444] systemd[1]: Started Nix worker-protocol over gRPC.4127worker2 # [ 19.206250] nix-grpc-daemon[788]: ts=2026-09-20T13:32:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4128worker2 # [ 19.213779] nix-grpc-daemon[788]: ts=2026-09-20T13:32:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4129worker2 # [ 19.218766] nix-grpc-daemon[788]: ts=2026-09-20T13:32:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14130worker1 # [ 19.256501] nix-grpc-daemon[842]: ts=2026-09-20T13:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4131lb # [ 19.257427] envoy[984]: [2026-09-20 13:32:24.555][984][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4132lb # [ 19.261322] envoy[984]: [2026-09-20 13:32:24.555][984][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4133lb # [ 19.265606] envoy[984]: [2026-09-20 13:32:24.555][984][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4134worker2 # [ 19.253415] nix-grpc-daemon[788]: ts=2026-09-20T13:32:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4135lb # [ 19.269818] envoy[984]: [2026-09-20 13:32:24.563][984][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4136worker2 # [ 19.257587] systemd[1]: Started Nix worker-protocol over gRPC.4137worker2 # [ 19.301144] nix-grpc-daemon[788]: ts=2026-09-20T13:32:24Z level=info event=scheduler_connected addr=lb:500514138worker2 # [ 19.311816] nix-grpc-daemon[788]: ts=2026-09-20T13:32:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4139worker1 # [ 19.457423] nix-grpc-daemon[842]: ts=2026-09-20T13:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4140worker2 # [ 19.513567] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 19.515269] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 19.716060] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 19.717771] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 19.858099] nix-grpc-daemon[842]: ts=2026-09-20T13:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4145worker2 # [ 19.918530] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 19.920278] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 20.122088] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 20.125934] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 20.328453] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 20.331740] nix-grpc-daemon[788]: ts=2026-09-20T13:32:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker2 # [ 20.529738] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_connected addr=lb:500514152worker2 # [ 20.533510] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker1 # [ 20.662092] nix-grpc-daemon[842]: ts=2026-09-20T13:32:26Z level=info event=scheduler_connected addr=lb:500514154worker1 # [ 20.666796] nix-grpc-daemon[842]: ts=2026-09-20T13:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker2 # [ 20.734411] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 20.737655] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker1 # [ 20.867981] nix-grpc-daemon[842]: ts=2026-09-20T13:32:26Z level=info event=scheduler_connected addr=lb:500514158worker1 # [ 20.872189] nix-grpc-daemon[842]: ts=2026-09-20T13:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker2 # [ 20.938889] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 20.942873] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 21.072644] nix-grpc-daemon[842]: ts=2026-09-20T13:32:26Z level=info event=scheduler_connected addr=lb:500514162worker1 # [ 21.076722] nix-grpc-daemon[842]: ts=2026-09-20T13:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker2 # [ 21.143812] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_connected addr=lb:500514164worker2 # [ 21.147182] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker1 # [ 21.277154] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514166worker1 # [ 21.279066] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker2 # [ 21.349470] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 21.353172] nix-grpc-daemon[788]: ts=2026-09-20T13:32:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 21.479673] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514170worker1 # [ 21.481558] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker2 # [ 21.550913] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514172worker2 # [ 21.555578] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 21.684100] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 21.687335] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 21.756047] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 21.757861] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 21.884370] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 21.886083] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 21.959305] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 21.963898] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 22.086682] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 22.088449] nix-grpc-daemon[842]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 22.165162] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 22.169693] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 22.289334] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 22.293124] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker2 # [ 22.371385] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_connected addr=lb:500514188worker2 # [ 22.373746] nix-grpc-daemon[788]: ts=2026-09-20T13:32:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker1 # [ 22.493770] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 22.496156] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 22.574289] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514192worker2 # [ 22.576103] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker1 # [ 22.697804] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 22.699612] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker2 # [ 22.776986] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514196worker2 # [ 22.778729] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker1 # [ 22.898623] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514198worker1 # [ 22.901346] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker2 # [ 22.979679] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514200worker2 # [ 22.981510] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker1 # [ 23.101556] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514202worker1 # [ 23.103350] nix-grpc-daemon[842]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker2 # [ 23.182758] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514204worker2 # [ 23.186741] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 23.304410] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514206worker1 # [ 23.308296] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker2 # [ 23.389175] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 23.392367] nix-grpc-daemon[788]: ts=2026-09-20T13:32:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker1 # [ 23.509147] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514210worker1 # [ 23.513158] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker2 # [ 23.590028] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 23.591832] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker1 # [ 23.713811] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514214worker1 # [ 23.715520] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker2 # [ 23.793289] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514216worker2 # [ 23.797867] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker1 # [ 23.916226] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 23.918037] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker2 # [ 23.998638] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514220worker2 # [ 24.000398] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker1 # [ 24.118901] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 24.123126] nix-grpc-daemon[842]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker2 # [ 24.201115] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514224worker2 # [ 24.205409] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker1 # [ 24.323306] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514226worker1 # [ 24.325112] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker2 # [ 24.406524] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_connected addr=lb:500514228worker2 # [ 24.408267] nix-grpc-daemon[788]: ts=2026-09-20T13:32:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker1 # [ 24.526158] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514230worker1 # [ 24.530174] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker2 # [ 24.609458] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514232worker2 # [ 24.611293] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 24.732647] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514234worker1 # [ 24.735801] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker2 # [ 24.812656] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514236worker2 # [ 24.815097] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker1 # [ 24.933319] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514238worker1 # [ 24.935455] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker2 # [ 25.016271] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514240worker2 # [ 25.018962] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 25.136182] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514242worker1 # [ 25.138217] nix-grpc-daemon[842]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker2 # [ 25.219860] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514244worker2 # [ 25.222353] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker1 # [ 25.338907] nix-grpc-daemon[842]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514246worker1 # [ 25.341128] nix-grpc-daemon[842]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker2 # [ 25.424245] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_connected addr=lb:500514248worker2 # [ 25.426619] nix-grpc-daemon[788]: ts=2026-09-20T13:32:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker1 # [ 25.541871] nix-grpc-daemon[842]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514250worker1 # [ 25.545889] nix-grpc-daemon[842]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker2 # [ 25.627878] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514252worker2 # [ 25.630408] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 25.747600] nix-grpc-daemon[842]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514254worker1 # [ 25.751314] nix-grpc-daemon[842]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker2 # [ 25.831502] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514256worker2 # [ 25.833970] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker1 # [ 25.952184] nix-grpc-daemon[842]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514258worker1 # [ 25.956600] nix-grpc-daemon[842]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker2 # [ 26.034757] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514260worker2 # [ 26.036515] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker1 # [ 26.156764] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514262worker1 # [ 26.158530] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker2 # [ 26.237296] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514264worker2 # [ 26.239105] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker1 # [ 26.359729] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514266worker1 # [ 26.363687] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker2 # [ 26.440232] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_connected addr=lb:500514268worker2 # [ 26.441967] nix-grpc-daemon[788]: ts=2026-09-20T13:32:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker1 # [ 26.564507] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514270worker1 # [ 26.568646] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker2 # [ 26.643119] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514272worker2 # [ 26.645124] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker1 # [ 26.769625] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514274worker1 # [ 26.771285] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker2 # [ 26.846034] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514276worker2 # [ 26.847775] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker1 # [ 26.971907] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514278worker1 # [ 26.974963] nix-grpc-daemon[842]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker2 # [ 27.048520] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 27.050275] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 27.176183] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514282worker1 # [ 27.180322] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker2 # [ 27.251267] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514284worker2 # [ 27.253625] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker1 # [ 27.380647] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514286worker1 # [ 27.382478] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker2 # [ 27.455362] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 27.457663] nix-grpc-daemon[788]: ts=2026-09-20T13:32:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 27.583110] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514290worker1 # [ 27.584863] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker2 # [ 27.659778] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514292worker2 # [ 27.661647] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker1 # [ 27.785906] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514294worker1 # [ 27.787604] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker2 # [ 27.862535] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514296worker2 # [ 27.864538] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker1 # [ 27.988664] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514298worker1 # [ 27.992574] nix-grpc-daemon[842]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker2 # [ 28.065332] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514300worker2 # [ 28.067642] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker1 # [ 28.193431] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514302worker1 # [ 28.197361] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker2 # [ 28.268900] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514304worker2 # [ 28.272867] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker1 # [ 28.398371] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 28.402327] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 28.474262] nix-grpc-daemon[788]: ts=2026-09-20T13:32:33Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 28.476145] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 28.602711] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 28.604498] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 28.677023] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 28.678745] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker1 # [ 28.805497] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514314worker1 # [ 28.807168] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4315worker2 # [ 28.879905] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514316worker2 # [ 28.883934] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker1 # [ 29.007816] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514318worker1 # [ 29.009606] nix-grpc-daemon[842]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker2 # [ 29.084765] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514320worker2 # [ 29.086832] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker1 # [ 29.210599] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514322worker1 # [ 29.214952] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker2 # [ 29.287708] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_connected addr=lb:500514324worker2 # [ 29.289870] nix-grpc-daemon[788]: ts=2026-09-20T13:32:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker1 # [ 29.415906] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514326worker1 # [ 29.419665] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker2 # [ 29.491611] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514328worker2 # [ 29.493922] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker1 # [ 29.620762] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514330worker1 # [ 29.624748] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker2 # [ 29.695101] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514332worker2 # [ 29.697628] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker1 # [ 29.826195] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514334worker1 # [ 29.829144] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker2 # [ 29.898544] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514336worker2 # [ 29.900863] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker1 # [ 30.027851] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514338worker1 # [ 30.031099] nix-grpc-daemon[842]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4339worker2 # [ 30.102112] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514340worker2 # [ 30.104593] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker1 # [ 30.231881] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514342worker1 # [ 30.235842] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker2 # [ 30.305614] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_connected addr=lb:500514344worker2 # [ 30.309567] nix-grpc-daemon[788]: ts=2026-09-20T13:32:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4345worker1 # [ 30.436984] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514346worker1 # [ 30.441239] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4347worker2 # [ 30.511092] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514348worker2 # [ 30.513164] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349worker1 # [ 30.642146] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514350worker1 # [ 30.646215] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4351worker2 # [ 30.714579] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514352worker2 # [ 30.716922] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4353worker1 # [ 30.848630] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514354worker1 # [ 30.851743] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4355lb # [ 30.899158] rustfs-bucket-start[1113]: mb s3://farm4356lb # [ 30.906196] systemd[1]: Finished rustfs-bucket.service.4357lb # [ 30.912941] systemd[1]: Starting niks3 server...4358worker2 # [ 30.918118] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514359worker2 # [ 30.920670] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4360worker1 # [ 31.049033] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514361worker1 # [ 31.050907] nix-grpc-daemon[842]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4362worker2 # [ 31.121634] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514363worker2 # [ 31.123962] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4364lb # [ 31.157283] postgres[1129]: [1129] ERROR: relation "goose_db_version" does not exist at character 364365lb # [ 31.158515] postgres[1129]: [1129] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4366lb # [ 31.201800] niks3-server[1123]: 2026/09/20 13:32:36 OK 20241026095416_initial_model.sql (23.55ms)4367lb # [ 31.206239] niks3-server[1123]: 2026/09/20 13:32:36 OK 20251210153512_drop_unused_gin_index.sql (2.49ms)4368lb # [ 31.211583] niks3-server[1123]: 2026/09/20 13:32:36 OK 20251218171726_add_pins.sql (5.06ms)4369lb # [ 31.217534] niks3-server[1123]: 2026/09/20 13:32:36 OK 20260628120000_add_object_size_and_stats.sql (6.07ms)4370lb # [ 31.222181] niks3-server[1123]: 2026/09/20 13:32:36 OK 20260905000000_add_claims.sql (4.65ms)4371lb # [ 31.226759] niks3-server[1123]: 2026/09/20 13:32:36 OK 20260920000000_drop_claims.sql (4.63ms)4372lb # [ 31.228373] niks3-server[1123]: 2026/09/20 13:32:36 goose: successfully migrated database to version: 202609200000004373lb # [ 31.234216] niks3-server[1123]: 2026/09/20 13:32:36 OK 1_commit_pending_closure.sql (5.99ms)4374lb # [ 31.236856] niks3-server[1123]: 2026/09/20 13:32:36 OK 2_object_stats_trigger.sql (1.33ms)4375lb # [ 31.238266] niks3-server[1123]: 2026/09/20 13:32:36 goose: up to current file version: 24376lb # [ 31.244500] niks3-server[1123]: 2026/09/20 13:32:36 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4377worker1 # [ 31.251526] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514378worker1 # [ 31.253462] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4379lb # [ 31.342610] niks3-server[1123]: 2026/09/20 13:32:36 INFO Read proxy enabled — serving cache objects from S34380worker2 # [ 31.324872] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_connected addr=lb:500514381lb # [ 31.345260] niks3-server[1123]: 2026/09/20 13:32:36 INFO Using socket-activated listener address=0.0.0.0:57514382worker2 # [ 31.327263] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4383lb # [ 31.350348] systemd[1]: Started niks3 server.4384lb # [ 31.351152] niks3-server[1123]: 2026/09/20 13:32:36 INFO systemd watchdog enabled interval=15s4385lb # [ 31.352950] systemd[1]: Reached target Multi-User System.4386lb # [ 31.354527] systemd[1]: Startup finished in 904ms (kernel) + 4.790s (initrd) + 25.659s (userspace) = 31.354s.4387lb # [ 31.357783] niks3-server[1123]: 2026/09/20 13:32:36 INFO Starting HTTP server address=0.0.0.0:57514388lb # [ 31.363091] niks3-server[1123]: 2026/09/20 13:32:36 INFO lead: acquired remote=192.168.1.4:407704389worker2 # [ 31.350524] nix-grpc-daemon[788]: ts=2026-09-20T13:32:36Z level=info event=scheduler_take_over4390worker1 # [ 31.454343] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514391worker1 # [ 31.458345] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4392worker2 # [ 31.529527] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514393worker2 # [ 31.531523] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4394worker1 # [ 31.659104] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514395worker1 # [ 31.661414] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4396worker2 # [ 31.730695] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514397worker2 # [ 31.732976] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4398worker1 # [ 31.862783] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514399worker1 # [ 31.865143] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4400worker2 # [ 31.934081] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514401worker2 # [ 31.936509] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4402worker1 # [ 32.065807] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514403worker1 # [ 32.068216] nix-grpc-daemon[842]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4404worker2 # [ 32.137310] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514405worker2 # [ 32.139493] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4406worker1 # [ 32.269060] nix-grpc-daemon[842]: ts=2026-09-20T13:32:38Z level=info event=scheduler_connected addr=lb:500514407worker1 # [ 32.271330] nix-grpc-daemon[842]: ts=2026-09-20T13:32:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4408worker2 # [ 32.340414] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=scheduler_connected addr=lb:500514409worker2 # [ 32.369914] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=oidc_jwks provider=mock keys=14410worker2 # [ 32.372768] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:390224411lb: (finished: waiting for unit niks3.service, in 20.64 seconds)4412worker2 # [ 32.375240] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04413lb: waiting for TCP port 5751 on localhost4414worker1 # [ 32.472149] nix-grpc-daemon[842]: ts=2026-09-20T13:32:38Z level=info event=scheduler_connected addr=lb:500514415worker2 # [ 32.453574] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:390224416lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4417worker2 # [ 32.455641] nix-grpc-daemon[788]: ts=2026-09-20T13:32:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04418lb: (finished: waiting for TCP port 5751 on localhost, in 0.06 seconds)4419lb: waiting for unit envoy.service4420lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4421lb: waiting for TCP port 50051 on localhost4422lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4423lb # [ 32.566946] envoy[984]: conn peer=[::1]:42938 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34424lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4425worker2 # [ 32.640362] systemd[1]: Starting scheduler-token.service...4426worker2 # [ 32.706684] systemd[1]: scheduler-token.service: Deactivated successfully.4427worker2 # [ 32.708416] systemd[1]: Finished scheduler-token.service.4428worker2 # [ 32.709862] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 68ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4429lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4430??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4431 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284432lb: must succeed: curl -sf localhost:9901/clusters4433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4434lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)4435subtest: one node schedules and both builders hold a WorkerSession on it4436lb: must succeed: curl -sf localhost:9901/clusters4437lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4438worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4439worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4440lb: must succeed: curl -sf localhost:9901/clusters4441lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4442lb: must succeed: curl -sf localhost:9901/clusters4443lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4444worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4445worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.03 seconds)4446(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.18 seconds)4447subtest: balancer auth: client cert, bearer token, nothing, unknown CN4448worker1 # [ 33.300687] nix-grpc-daemon[842]: ts=2026-09-20T13:32:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343144449worker1 # [ 33.313303] systemd[1]: Started Nix Daemon.4450worker1 # [ 33.533693] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)4451worker1 # [ 33.581154] nix-grpc-daemon[842]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)4452worker1 # [ 33.618941] nix-grpc-daemon[842]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 2/5)4453worker1 # [ 33.739075] nix-grpc-daemon[842]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 3/5)4454worker1 # [ 33.825466] nix-grpc-daemon[842]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 500 ms (attempt 4/5)4455worker1 # [ 34.338922] nix-grpc-daemon[842]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4456worker2 # [ 34.363182] nix-grpc-daemon[788]: ts=2026-09-20T13:32:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:452524457worker2 # [ 34.371383] systemd[1]: Started Nix Daemon.4458worker2 # [ 34.578152] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4459worker2 # [ 34.606386] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)4460worker2 # [ 34.646099] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5)4461worker2 # [ 34.841843] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 266 ms (attempt 3/5)4462worker2 # [ 35.110360] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 4/5)4463worker2 # [ 35.234340] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4464worker1 # [ 35.279564] nix-grpc-daemon[842]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:343144465worker2 # [ 35.253220] nix-grpc-daemon[788]: ts=2026-09-20T13:32:40Z level=debug event=path_info_miss path=9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test error="error: path '/nix/store/9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test' is required, but there is no substituter that can build it"4466worker1 # [ 35.293727] nix-grpc-daemon[842]: ts=2026-09-20T13:32:41Z level=debug event=path_info_miss path=9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test error="error: path '/nix/store/9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test' is required, but there is no substituter that can build it"4467client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4468client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4469client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4470client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4471worker2 # [ 35.610966] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:452524472worker1 # [ 35.656695] nix-grpc-daemon[842]: ts=2026-09-20T13:32:41Z level=info event=oidc_jwks provider=mock keys=14473worker1 # [ 35.658376] nix-grpc-daemon[842]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:343144474worker1 # [ 35.662571] nix-grpc-daemon[842]: ts=2026-09-20T13:32:41Z level=debug event=path_info_miss path=9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test error="error: path '/nix/store/9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test' is required, but there is no substituter that can build it"4475worker2 # [ 35.645908] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:452524476worker2 # [ 35.654520] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=path_info_miss path=9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test error="error: path '/nix/store/9hk86cyxvc6jyp5g8zgihrck4qjfmf5d-nixos-system-client-test' is required, but there is no substituter that can build it"4477client: must succeed: rm -r /var/lib/nix-grpc-store4478client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.02 seconds)4479worker1 # [ 35.877628] nix-grpc-daemon[842]: ts=2026-09-20T13:32:41Z level=info event=denied method=QueryMissing cn=- role=none4480worker1 # [ 36.068276] nix-grpc-daemon[842]: ts=2026-09-20T13:32:41Z level=info event=denied method=QueryMissing cn=stranger role=none4481(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.10 seconds)4482subtest: DAG build is scheduled across workers and lands in the cache4483client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24484worker2 # [ 36.292785] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:452524485client # don't know how to build these paths:4486client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4487worker2 # [ 36.305929] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524488worker2 # [ 36.335209] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04489worker2 # [ 36.339749] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14490worker2 # [ 36.391397] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524491worker2 # [ 36.413563] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524492client # copying 1 paths to 'lb:50051'...4493worker2 # [ 36.424165] nix-grpc-daemon[788]: ts=2026-09-20T13:32:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452524494client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4495worker2 # [ 36.449918] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4496worker2 # [ 36.576847] nix-grpc-daemon[788]: ts=2026-09-20T13:32:42Z level=info event=niks3_push_started pid=8704497worker2 # [ 36.805919] nix-grpc-daemon[870]: time=2026-09-20T13:32:42.327Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4498worker2 # [ 36.978079] nix-daemon[858]: accepted connection from pid 883, user nix-grpc-daemon (trusted)4499worker2 # [ 36.995423] nix-daemon[858]: reaped child process 886, status = succeeded4500worker2 # [ 37.169556] nix-daemon[858]: accepted connection from pid 888, user nix-grpc-daemon (trusted)4501worker2 # [ 37.185441] nix-daemon[858]: reaped child process 891, status = succeeded4502lb # [ 37.219739] niks3-server[1123]: 2026/09/20 13:32:42 INFO Received uploads request method=POST path=/api/pending_closures4503worker2 # [ 37.217797] nix-grpc-daemon[870]: time=2026-09-20T13:32:42.741Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4504worker2 # [ 37.219887] nix-grpc-daemon[870]: time=2026-09-20T13:32:42.744Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4505lb # [ 37.271149] niks3-server[1123]: 2026/09/20 13:32:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4506lb # [ 37.285162] niks3-server[1123]: 2026/09/20 13:32:42 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4507lb # [ 37.293630] niks3-server[1123]: 2026/09/20 13:32:42 INFO Signed narinfos id=1 count=14508worker2 # [ 37.276972] nix-grpc-daemon[870]: time=2026-09-20T13:32:42.801Z level=INFO msg="Uploading 1 narinfos"4509lb # [ 37.307243] niks3-server[1123]: 2026/09/20 13:32:42 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4510lb # [ 37.309278] niks3-server[1123]: 2026/09/20 13:32:42 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4511lb # [ 37.315812] niks3-server[1123]: 2026/09/20 13:32:42 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4512lb # [ 37.325710] niks3-server[1123]: 2026/09/20 13:32:42 INFO Completed upload id=14513worker2 # [ 37.309194] nix-grpc-daemon[870]: time=2026-09-20T13:32:42.833Z level=INFO msg="Upload complete. (491ms)"4514worker2 # [ 37.314551] nix-grpc-daemon[788]: ts=2026-09-20T13:32:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_in=5044515worker2 # [ 37.324420] nix-grpc-daemon[788]: ts=2026-09-20T13:32:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524516worker2 # [ 37.334123] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4517worker2 # [ 37.343581] nix-daemon[858]: reaped child process 868, status = succeeded4518client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4519worker2 # [ 37.348553] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4520client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4521client # farm-a> LOG-farm-a-t14522worker2 # [ 38.569648] nix-daemon[858]: reaped child process 895, status = succeeded4523worker2 # [ 38.680579] nix-daemon[858]: accepted connection from pid 910, user nix-grpc-daemon (trusted)4524worker2 # [ 38.694607] nix-daemon[858]: reaped child process 914, status = succeeded4525lb # [ 38.720788] niks3-server[1123]: 2026/09/20 13:32:44 INFO Received uploads request method=POST path=/api/pending_closures4526worker2 # [ 38.707772] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.231Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4527worker2 # [ 38.709389] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.233Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4528lb # [ 38.748780] niks3-server[1123]: 2026/09/20 13:32:44 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4529lb # [ 38.751695] niks3-server[1123]: 2026/09/20 13:32:44 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4530lb # [ 38.755530] niks3-server[1123]: 2026/09/20 13:32:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4531lb # [ 38.758848] niks3-server[1123]: 2026/09/20 13:32:44 INFO Signed narinfos id=2 count=14532worker2 # [ 38.740827] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.264Z level=INFO msg="Uploading 1 narinfos"4533lb # [ 38.762408] niks3-server[1123]: 2026/09/20 13:32:44 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4534lb # [ 38.766720] niks3-server[1123]: 2026/09/20 13:32:44 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4535lb # [ 38.773734] niks3-server[1123]: 2026/09/20 13:32:44 INFO Completed upload id=24536worker2 # [ 38.755178] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.279Z level=INFO msg="Upload complete. (198ms)"4537lb # [ 38.775919] niks3-server[1123]: 2026/09/20 13:32:44 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4538worker2 # [ 38.758406] nix-grpc-daemon[788]: ts=2026-09-20T13:32:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14539worker2 # [ 38.771259] nix-daemon[858]: reaped child process 893, status = succeeded4540worker2 # [ 38.773743] nix-grpc-daemon[788]: ts=2026-09-20T13:32:44Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04541worker2 # [ 38.775965] nix-grpc-daemon[788]: ts=2026-09-20T13:32:44Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14542worker2 # [ 38.818415] nix-grpc-daemon[788]: ts=2026-09-20T13:32:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524543worker2 # [ 38.830732] nix-grpc-daemon[788]: ts=2026-09-20T13:32:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524544worker2 # [ 38.844137] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.368Z level=INFO msg="All 1 paths already cached"4545client # copying 1 paths to 'lb:50051'...4546worker2 # [ 38.852835] nix-grpc-daemon[788]: ts=2026-09-20T13:32:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452524547client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4548worker2 # [ 38.873756] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4549lb # [ 38.936361] envoy[984]: conn peer=[2001:db8:1::1]:45900 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=20554550lb # [ 38.940108] envoy[984]: conn peer=[2001:db8:1::1]:45914 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=554551lb # [ 38.943465] envoy[984]: conn peer=[2001:db8:1::1]:45928 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=94552lb # [ 38.946698] envoy[984]: conn peer=[2001:db8:1::1]:45936 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=74553worker2 # [ 39.035406] nix-daemon[858]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4554worker2 # [ 39.050455] nix-daemon[858]: reaped child process 923, status = succeeded4555worker2 # [ 39.179605] nix-daemon[858]: accepted connection from pid 925, user nix-grpc-daemon (trusted)4556worker2 # [ 39.194058] nix-daemon[858]: reaped child process 928, status = succeeded4557lb # [ 39.224971] niks3-server[1123]: 2026/09/20 13:32:44 INFO Received uploads request method=POST path=/api/pending_closures4558worker2 # [ 39.212915] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.737Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4559worker2 # [ 39.214393] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.738Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4560lb # [ 39.245740] niks3-server[1123]: 2026/09/20 13:32:44 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4561lb # [ 39.248492] niks3-server[1123]: 2026/09/20 13:32:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4562lb # [ 39.250501] niks3-server[1123]: 2026/09/20 13:32:44 INFO Signed narinfos id=3 count=14563worker2 # [ 39.231703] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.756Z level=INFO msg="Uploading 1 narinfos"4564lb # [ 39.255066] niks3-server[1123]: 2026/09/20 13:32:44 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4565lb # [ 39.262167] niks3-server[1123]: 2026/09/20 13:32:44 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4566lb # [ 39.267669] niks3-server[1123]: 2026/09/20 13:32:44 INFO Completed upload id=34567worker2 # [ 39.248862] nix-grpc-daemon[870]: time=2026-09-20T13:32:44.773Z level=INFO msg="Upload complete. (357ms)"4568worker2 # [ 39.251049] nix-grpc-daemon[788]: ts=2026-09-20T13:32:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_in=6244569lb # [ 39.270874] niks3-server[1123]: 2026/09/20 13:32:44 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4570worker2 # [ 39.254531] nix-grpc-daemon[788]: ts=2026-09-20T13:32:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524571worker2 # [ 39.260591] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4572worker2 # [ 39.265162] nix-daemon[858]: reaped child process 918, status = succeeded4573client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4574client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4575worker2 # [ 39.270381] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4576worker2 # [ 39.882738] nix-daemon[858]: reaped child process 932, status = succeeded4577client # farm-b> LOG-farm-b-t14578worker2 # [ 40.016194] nix-daemon[858]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4579worker2 # [ 40.030098] nix-daemon[858]: reaped child process 950, status = succeeded4580lb # [ 40.064495] niks3-server[1123]: 2026/09/20 13:32:45 INFO Received uploads request method=POST path=/api/pending_closures4581worker2 # [ 40.057752] nix-grpc-daemon[870]: time=2026-09-20T13:32:45.581Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4582worker2 # [ 40.059900] nix-grpc-daemon[870]: time=2026-09-20T13:32:45.583Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4583lb # [ 40.113231] niks3-server[1123]: 2026/09/20 13:32:45 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4584lb # [ 40.123551] niks3-server[1123]: 2026/09/20 13:32:45 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4585lb # [ 40.128975] niks3-server[1123]: 2026/09/20 13:32:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4586lb # [ 40.131827] niks3-server[1123]: 2026/09/20 13:32:45 INFO Signed narinfos id=4 count=14587worker2 # [ 40.113565] nix-grpc-daemon[870]: time=2026-09-20T13:32:45.637Z level=INFO msg="Uploading 1 narinfos"4588lb # [ 40.140840] niks3-server[1123]: 2026/09/20 13:32:45 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4589lb # [ 40.144636] niks3-server[1123]: 2026/09/20 13:32:45 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4590lb # [ 40.148937] niks3-server[1123]: 2026/09/20 13:32:45 INFO Completed upload id=44591worker2 # [ 40.130281] nix-grpc-daemon[870]: time=2026-09-20T13:32:45.654Z level=INFO msg="Upload complete. (255ms)"4592lb # [ 40.151467] niks3-server[1123]: 2026/09/20 13:32:45 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4593worker2 # [ 40.141267] nix-grpc-daemon[788]: ts=2026-09-20T13:32:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14594worker2 # [ 40.147859] nix-grpc-daemon[788]: ts=2026-09-20T13:32:45Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04595worker2 # [ 40.152286] nix-grpc-daemon[788]: ts=2026-09-20T13:32:45Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14596worker2 # [ 40.161196] nix-daemon[858]: reaped child process 930, status = succeeded4597worker2 # [ 40.191361] nix-grpc-daemon[788]: ts=2026-09-20T13:32:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524598worker2 # [ 40.203273] nix-grpc-daemon[788]: ts=2026-09-20T13:32:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524599worker2 # [ 40.213889] nix-grpc-daemon[870]: time=2026-09-20T13:32:45.738Z level=INFO msg="All 2 paths already cached"4600client # copying 1 paths to 'lb:50051'...4601worker2 # [ 40.220963] nix-grpc-daemon[788]: ts=2026-09-20T13:32:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452524602client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4603worker2 # [ 40.231322] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4604worker2 # [ 40.360508] nix-daemon[858]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4605worker2 # [ 40.375059] nix-daemon[858]: reaped child process 959, status = succeeded4606worker2 # [ 40.495642] nix-daemon[858]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4607worker2 # [ 40.510093] nix-daemon[858]: reaped child process 964, status = succeeded4608lb # [ 40.537442] niks3-server[1123]: 2026/09/20 13:32:45 INFO Received uploads request method=POST path=/api/pending_closures4609worker2 # [ 40.523100] nix-grpc-daemon[870]: time=2026-09-20T13:32:46.047Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4610worker2 # [ 40.524518] nix-grpc-daemon[870]: time=2026-09-20T13:32:46.047Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4611lb # [ 40.550266] niks3-server[1123]: 2026/09/20 13:32:45 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4612lb # [ 40.553844] niks3-server[1123]: 2026/09/20 13:32:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4613lb # [ 40.556709] niks3-server[1123]: 2026/09/20 13:32:45 INFO Signed narinfos id=5 count=14614worker2 # [ 40.538115] nix-grpc-daemon[870]: time=2026-09-20T13:32:46.062Z level=INFO msg="Uploading 1 narinfos"4615lb # [ 40.559431] niks3-server[1123]: 2026/09/20 13:32:45 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4616lb # [ 40.564068] niks3-server[1123]: 2026/09/20 13:32:45 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4617lb # [ 40.568652] niks3-server[1123]: 2026/09/20 13:32:45 INFO Completed upload id=54618worker2 # [ 40.549801] nix-grpc-daemon[870]: time=2026-09-20T13:32:46.074Z level=INFO msg="Upload complete. (311ms)"4619lb # [ 40.570358] niks3-server[1123]: 2026/09/20 13:32:45 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4620worker2 # [ 40.553093] nix-grpc-daemon[788]: ts=2026-09-20T13:32:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_in=7524621worker2 # [ 40.556190] nix-grpc-daemon[788]: ts=2026-09-20T13:32:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524622worker2 # [ 40.562913] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4623worker2 # [ 40.565533] nix-daemon[858]: reaped child process 954, status = succeeded4624client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4625client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4626worker2 # [ 40.571210] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4627client # farm-top> LOG-farm-top-t14628worker2 # [ 41.186577] nix-daemon[858]: reaped child process 968, status = succeeded4629worker2 # [ 41.323120] nix-daemon[858]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4630worker2 # [ 41.339268] nix-daemon[858]: reaped child process 986, status = succeeded4631lb # [ 41.372840] niks3-server[1123]: 2026/09/20 13:32:46 INFO Received uploads request method=POST path=/api/pending_closures4632worker2 # [ 41.363450] nix-grpc-daemon[870]: time=2026-09-20T13:32:46.887Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4633worker2 # [ 41.367236] nix-grpc-daemon[870]: time=2026-09-20T13:32:46.889Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4634lb # [ 41.415722] niks3-server[1123]: 2026/09/20 13:32:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4635worker2 # [ 41.402993] nix-grpc-daemon[870]: time=2026-09-20T13:32:46.927Z level=INFO msg="Uploading 1 narinfos"4636lb # [ 41.421410] niks3-server[1123]: 2026/09/20 13:32:46 INFO Signed narinfos id=6 count=14637lb # [ 41.430115] niks3-server[1123]: 2026/09/20 13:32:46 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4638lb # [ 41.436079] niks3-server[1123]: 2026/09/20 13:32:46 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4639lb # [ 41.443331] niks3-server[1123]: 2026/09/20 13:32:46 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4640lb # [ 41.446719] niks3-server[1123]: 2026/09/20 13:32:46 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4641lb # [ 41.459414] niks3-server[1123]: 2026/09/20 13:32:46 INFO Completed upload id=64642lb # [ 41.460826] niks3-server[1123]: 2026/09/20 13:32:46 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4643worker2 # [ 41.443110] nix-grpc-daemon[870]: time=2026-09-20T13:32:46.965Z level=INFO msg="Upload complete. (260ms)"4644worker2 # [ 41.449291] nix-grpc-daemon[788]: ts=2026-09-20T13:32:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14645worker2 # [ 41.461863] nix-daemon[858]: reaped child process 966, status = succeeded4646client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 5.42 seconds)4647client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4648client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4649client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4650client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4651client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4652client # copying 3 paths...4653client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4654client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4655client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4656client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4657(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.94 seconds)4658subtest: repeat is answered Cached by the scheduler without building4659worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4660worker1 # finding garbage collector roots...4661worker1 # deleting specified paths...4662worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4663worker1 # deleting unused links...4664worker1 # note: hard linking is currently saving 0.0 KiB4665worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.30 seconds)4666worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4667worker2 # finding garbage collector roots...4668worker2 # deleting specified paths...4669worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4670worker2 # deleting unused links...4671worker2 # note: hard linking is currently saving 0.0 KiB4672worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4673client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24674worker2 # [ 42.744053] nix-grpc-daemon[788]: ts=2026-09-20T13:32:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:452524675client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4676client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4677worker2 # [ 42.755952] nix-grpc-daemon[788]: ts=2026-09-20T13:32:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524678worker2 # [ 42.760934] nix-grpc-daemon[870]: time=2026-09-20T13:32:48.284Z level=INFO msg="All 2 paths already cached"4679worker2 # [ 42.767954] nix-grpc-daemon[788]: ts=2026-09-20T13:32:48Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14680client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.23 seconds)4681client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4682client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4683lb: must succeed: curl -sf localhost:9901/clusters4684lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4685lb: must succeed: curl -sf localhost:9901/clusters4686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4687worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4688worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4689(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.12 seconds)4690subtest: inputs already in the cache are not sent to the scheduler4691lb: must succeed: curl -sf localhost:9901/clusters4692lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4693lb: must succeed: curl -sf localhost:9901/clusters4694lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4695worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4696worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4697client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24698client # don't know how to build these paths:4699client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4700worker1 # [ 43.433416] nix-grpc-daemon[842]: ts=2026-09-20T13:32:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343144701worker1 # [ 43.438977] nix-grpc-daemon[842]: ts=2026-09-20T13:32:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343144702worker2 # [ 43.434681] nix-grpc-daemon[788]: ts=2026-09-20T13:32:48Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04703worker2 # [ 43.439484] nix-grpc-daemon[788]: ts=2026-09-20T13:32:48Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14704worker2 # [ 43.446167] nix-grpc-daemon[788]: ts=2026-09-20T13:32:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524705worker2 # [ 43.455213] nix-grpc-daemon[788]: ts=2026-09-20T13:32:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524706worker2 # [ 43.464829] nix-grpc-daemon[870]: time=2026-09-20T13:32:48.989Z level=INFO msg="All 2 paths already cached"4707client # copying 1 paths to 'lb:50051'...4708worker2 # [ 43.468506] nix-grpc-daemon[788]: ts=2026-09-20T13:32:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452524709client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4710worker2 # [ 43.476062] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4711worker2 # [ 43.632470] nix-daemon[858]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)4712worker2 # [ 43.647864] nix-daemon[858]: reaped child process 1019, status = succeeded4713worker2 # [ 43.814061] nix-daemon[858]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4714worker2 # [ 43.828508] nix-daemon[858]: reaped child process 1024, status = succeeded4715lb # [ 43.860666] niks3-server[1123]: 2026/09/20 13:32:49 INFO Received uploads request method=POST path=/api/pending_closures4716worker2 # [ 43.846219] nix-grpc-daemon[870]: time=2026-09-20T13:32:49.370Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4717worker2 # [ 43.847694] nix-grpc-daemon[870]: time=2026-09-20T13:32:49.371Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4718lb # [ 43.876705] niks3-server[1123]: 2026/09/20 13:32:49 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4719lb # [ 43.882918] niks3-server[1123]: 2026/09/20 13:32:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4720worker2 # [ 43.867397] nix-grpc-daemon[870]: time=2026-09-20T13:32:49.391Z level=INFO msg="Uploading 1 narinfos"4721lb # [ 43.886756] niks3-server[1123]: 2026/09/20 13:32:49 INFO Signed narinfos id=7 count=14722lb # [ 43.890789] niks3-server[1123]: 2026/09/20 13:32:49 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4723lb # [ 43.901817] niks3-server[1123]: 2026/09/20 13:32:49 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4724lb # [ 43.904326] niks3-server[1123]: 2026/09/20 13:32:49 INFO Completed upload id=74725worker2 # [ 43.885641] nix-grpc-daemon[870]: time=2026-09-20T13:32:49.409Z level=INFO msg="Upload complete. (404ms)"4726lb # [ 43.906940] niks3-server[1123]: 2026/09/20 13:32:49 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4727worker2 # [ 43.887768] nix-grpc-daemon[788]: ts=2026-09-20T13:32:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_in=7524728worker2 # [ 43.890807] nix-grpc-daemon[788]: ts=2026-09-20T13:32:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524729worker2 # [ 43.897131] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4730worker2 # [ 43.900135] nix-daemon[858]: reaped child process 1014, status = succeeded4731client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4732worker2 # [ 43.905110] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4733client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4734worker2 # [ 45.322222] nix-daemon[858]: reaped child process 1028, status = succeeded4735client # farm-top> LOG-farm-top-t1b4736worker2 # [ 45.461514] nix-daemon[858]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)4737worker2 # [ 45.473348] nix-daemon[858]: reaped child process 1046, status = succeeded4738lb # [ 45.504945] niks3-server[1123]: 2026/09/20 13:32:50 INFO Received uploads request method=POST path=/api/pending_closures4739worker2 # [ 45.490517] nix-grpc-daemon[870]: time=2026-09-20T13:32:51.014Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4740worker2 # [ 45.491995] nix-grpc-daemon[870]: time=2026-09-20T13:32:51.016Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4741lb # [ 45.530582] niks3-server[1123]: 2026/09/20 13:32:50 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4742lb # [ 45.533363] niks3-server[1123]: 2026/09/20 13:32:50 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4743lb # [ 45.538924] niks3-server[1123]: 2026/09/20 13:32:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4744lb # [ 45.541949] niks3-server[1123]: 2026/09/20 13:32:50 INFO Signed narinfos id=8 count=14745worker2 # [ 45.523357] nix-grpc-daemon[870]: time=2026-09-20T13:32:51.047Z level=INFO msg="Uploading 1 narinfos"4746lb # [ 45.543606] niks3-server[1123]: 2026/09/20 13:32:50 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4747lb # [ 45.550524] niks3-server[1123]: 2026/09/20 13:32:50 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4748lb # [ 45.553711] niks3-server[1123]: 2026/09/20 13:32:50 INFO Completed upload id=84749worker2 # [ 45.536159] nix-grpc-daemon[870]: time=2026-09-20T13:32:51.060Z level=INFO msg="Upload complete. (224ms)"4750lb # [ 45.555430] niks3-server[1123]: 2026/09/20 13:32:50 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4751worker2 # [ 45.539436] nix-grpc-daemon[788]: ts=2026-09-20T13:32:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14752worker2 # [ 45.545977] nix-daemon[858]: reaped child process 1026, status = succeeded4753client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.34 seconds)4754client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4755client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4756lb: must succeed: curl -sf localhost:9901/clusters4757lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4758lb: must succeed: curl -sf localhost:9901/clusters4759lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4760worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4761worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4762(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.72 seconds)4763subtest: two clients wanting the same drv build it once4764client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4765client # [ 45.938983] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4766client # Running as unit: dedup1.service; invocation ID: f902d5fbea9640aa9e0ee402a9fa0c134767client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.05 seconds)4768client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4769worker1 # [ 46.111191] nix-grpc-daemon[842]: ts=2026-09-20T13:32:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343144770client # [ 46.130695] nix[887]: don't know how to build these paths:4771client # [ 46.131644] nix[887]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4772worker2 # [ 46.098702] nix-grpc-daemon[788]: ts=2026-09-20T13:32:51Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04773worker1 # [ 46.121249] nix-grpc-daemon[842]: ts=2026-09-20T13:32:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343144774worker2 # [ 46.101076] nix-grpc-daemon[788]: ts=2026-09-20T13:32:51Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14775worker1 # [ 46.130380] nix-grpc-daemon[842]: ts=2026-09-20T13:32:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343144776client # [ 46.153627] nix[887]: copying 1 paths to 'lb:50051'...4777client # [ 46.154437] nix[887]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4778worker1 # [ 46.139204] nix-grpc-daemon[842]: ts=2026-09-20T13:32:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343144779worker1 # [ 46.148734] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)4780worker1 # [ 46.162293] nix-grpc-daemon[842]: ts=2026-09-20T13:32:52Z level=info event=niks3_push_started pid=9264781worker1 # [ 46.377958] nix-grpc-daemon[926]: time=2026-09-20T13:32:52.223Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4782worker1 # [ 46.553902] nix-daemon[901]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4783worker1 # [ 46.568429] nix-daemon[901]: reaped child process 941, status = succeeded4784worker1 # [ 46.722247] nix-daemon[901]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4785worker1 # [ 46.734574] nix-daemon[901]: reaped child process 947, status = succeeded4786lb # [ 46.749639] niks3-server[1123]: 2026/09/20 13:32:52 INFO Received uploads request method=POST path=/api/pending_closures4787worker1 # [ 46.759956] nix-grpc-daemon[926]: time=2026-09-20T13:32:52.608Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4788worker1 # [ 46.762345] nix-grpc-daemon[926]: time=2026-09-20T13:32:52.611Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4789lb # [ 46.789351] niks3-server[1123]: 2026/09/20 13:32:52 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4790lb # [ 46.794517] niks3-server[1123]: 2026/09/20 13:32:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4791worker1 # [ 46.803037] nix-grpc-daemon[926]: time=2026-09-20T13:32:52.651Z level=INFO msg="Uploading 1 narinfos"4792lb # [ 46.801805] niks3-server[1123]: 2026/09/20 13:32:52 INFO Signed narinfos id=9 count=14793lb # [ 46.815549] niks3-server[1123]: 2026/09/20 13:32:52 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4794lb # [ 46.821239] niks3-server[1123]: 2026/09/20 13:32:52 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4795worker1 # [ 46.830176] nix-grpc-daemon[926]: time=2026-09-20T13:32:52.679Z level=INFO msg="Upload complete. (439ms)"4796lb # [ 46.828177] niks3-server[1123]: 2026/09/20 13:32:52 INFO Completed upload id=94797worker1 # [ 46.831927] nix-grpc-daemon[842]: ts=2026-09-20T13:32:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=0 paths=1 nar_bytes_in=4324798lb # [ 46.830666] niks3-server[1123]: 2026/09/20 13:32:52 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4799worker1 # [ 46.838919] nix-grpc-daemon[842]: ts=2026-09-20T13:32:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343144800worker1 # [ 46.845553] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)4801worker1 # [ 46.846859] nix-daemon[901]: reaped child process 924, status = succeeded4802client # [ 46.867000] nix[887]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4803worker1 # [ 46.852384] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)4804client # [ 47.009605] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4805client # Running as unit: dedup2.service; invocation ID: dc85f42b02f3446fbf077239ca2ef10f4806client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4807client # [ 47.254169] nix[902]: don't know how to build these paths:4808worker2 # [ 47.215928] nix-grpc-daemon[788]: ts=2026-09-20T13:32:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:452524809client # [ 47.255305] nix[902]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4810worker2 # [ 47.223450] nix-grpc-daemon[788]: ts=2026-09-20T13:32:52Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04811worker1 # [ 47.250229] nix-grpc-daemon[842]: ts=2026-09-20T13:32:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343144812client # [ 47.271379] nix[902]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4813lb # [ 48.936495] envoy[984]: conn peer=[2001:db8:1::1]:45940 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9438 tx=3802 ms=51954814lb # [ 48.938211] envoy[984]: conn peer=[2001:db8:1::1]:57170 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=478 ms=334815lb # [ 48.939597] envoy[984]: conn peer=[2001:db8:1::1]:57186 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1689 ms=21424816client: must succeed: sleep 24817client: (finished: must succeed: sleep 2, in 2.05 seconds)4818worker1 # [ 51.474251] nix-daemon[901]: reaped child process 951, status = succeeded4819??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4820 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394821client: waiting for success: ! systemctl is-active dedup1 dedup24822??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4823 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394824worker1 # [ 51.623683] nix-daemon[901]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4825worker1 # [ 51.641551] nix-daemon[901]: reaped child process 987, status = succeeded4826lb # [ 51.653323] niks3-server[1123]: 2026/09/20 13:32:56 INFO Received uploads request method=POST path=/api/pending_closures4827worker1 # [ 51.662280] nix-grpc-daemon[926]: time=2026-09-20T13:32:57.510Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4828worker1 # [ 51.664083] nix-grpc-daemon[926]: time=2026-09-20T13:32:57.512Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4829lb # [ 51.690766] niks3-server[1123]: 2026/09/20 13:32:56 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4830lb # [ 51.695712] niks3-server[1123]: 2026/09/20 13:32:56 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4831lb # [ 51.704615] niks3-server[1123]: 2026/09/20 13:32:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4832worker1 # [ 51.707455] nix-grpc-daemon[926]: time=2026-09-20T13:32:57.555Z level=INFO msg="Uploading 1 narinfos"4833lb # [ 51.706181] niks3-server[1123]: 2026/09/20 13:32:57 INFO Signed narinfos id=10 count=14834lb # [ 51.710704] niks3-server[1123]: 2026/09/20 13:32:57 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4835lb # [ 51.716054] niks3-server[1123]: 2026/09/20 13:32:57 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4836lb # [ 51.723135] niks3-server[1123]: 2026/09/20 13:32:57 INFO Completed upload id=104837worker1 # [ 51.725160] nix-grpc-daemon[926]: time=2026-09-20T13:32:57.573Z level=INFO msg="Upload complete. (261ms)"4838lb # [ 51.725238] niks3-server[1123]: 2026/09/20 13:32:57 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4839worker1 # [ 51.729952] nix-grpc-daemon[842]: ts=2026-09-20T13:32:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14840worker1 # [ 51.734386] nix-grpc-daemon[842]: ts=2026-09-20T13:32:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14841worker1 # [ 51.742491] nix-daemon[901]: reaped child process 949, status = succeeded4842client # [ 51.776974] systemd[1]: dedup1.service: Deactivated successfully.4843client # [ 51.778800] systemd[1]: dedup1.service: Consumed 101ms CPU time over 5.839s wall clock time, 39.3M memory peak, 8.3K incoming IP traffic, 8.9K outgoing IP traffic.4844client # [ 51.812164] systemd[1]: dedup2.service: Deactivated successfully.4845client # [ 51.813718] systemd[1]: dedup2.service: Consumed 125ms CPU time over 4.802s wall clock time, 33.2M memory peak, 7.3K incoming IP traffic, 6.4K outgoing IP traffic.4846client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4847client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4848client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4849worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4850worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4851worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4852worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4853worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4854worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4855worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4856worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4857(finished: subtest: two clients wanting the same drv build it once, in 6.99 seconds)4858subtest: build hook: nix-daemon with builders = grpc://lb4859client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4860client # [ 53.096765] systemd[1]: Started Nix Daemon.4861client # [ 53.280740] nix-daemon[943]: accepted connection from pid 940, user root (trusted)4862worker1 # [ 53.468053] nix-grpc-daemon[842]: ts=2026-09-20T13:32:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:343144863worker1 # [ 53.474529] nix-grpc-daemon[842]: ts=2026-09-20T13:32:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343144864worker2 # [ 53.472770] nix-grpc-daemon[788]: ts=2026-09-20T13:32:58Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04865worker2 # [ 53.478564] nix-grpc-daemon[788]: ts=2026-09-20T13:32:58Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14866worker2 # [ 53.519199] nix-grpc-daemon[788]: ts=2026-09-20T13:32:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524867worker2 # [ 53.531988] nix-grpc-daemon[788]: ts=2026-09-20T13:32:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524868worker2 # [ 53.541934] nix-grpc-daemon[788]: ts=2026-09-20T13:32:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452524869worker2 # [ 53.548914] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4870worker2 # [ 53.730179] nix-daemon[858]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)4871worker2 # [ 53.748179] nix-daemon[858]: reaped child process 1089, status = succeeded4872worker2 # [ 53.916177] nix-daemon[858]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)4873worker2 # [ 53.933321] nix-daemon[858]: reaped child process 1094, status = succeeded4874lb # [ 53.968760] niks3-server[1123]: 2026/09/20 13:32:59 INFO Received uploads request method=POST path=/api/pending_closures4875worker2 # [ 53.957822] nix-grpc-daemon[870]: time=2026-09-20T13:32:59.481Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4876worker2 # [ 53.959201] nix-grpc-daemon[870]: time=2026-09-20T13:32:59.481Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4877lb # [ 53.988602] niks3-server[1123]: 2026/09/20 13:32:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4878lb # [ 53.992253] niks3-server[1123]: 2026/09/20 13:32:59 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4879lb # [ 53.995877] niks3-server[1123]: 2026/09/20 13:32:59 INFO Signed narinfos id=11 count=14880worker2 # [ 53.977466] nix-grpc-daemon[870]: time=2026-09-20T13:32:59.501Z level=INFO msg="Uploading 1 narinfos"4881lb # [ 53.998027] niks3-server[1123]: 2026/09/20 13:32:59 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4882lb # [ 54.005274] niks3-server[1123]: 2026/09/20 13:32:59 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4883lb # [ 54.010462] niks3-server[1123]: 2026/09/20 13:32:59 INFO Completed upload id=114884worker2 # [ 53.991841] nix-grpc-daemon[870]: time=2026-09-20T13:32:59.516Z level=INFO msg="Upload complete. (436ms)"4885worker2 # [ 53.994199] nix-grpc-daemon[788]: ts=2026-09-20T13:32:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_in=5124886lb # [ 54.015400] niks3-server[1123]: 2026/09/20 13:32:59 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4887worker2 # [ 53.999289] nix-grpc-daemon[788]: ts=2026-09-20T13:32:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524888worker2 # [ 54.006304] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4889worker2 # [ 54.008790] nix-daemon[858]: reaped child process 1084, status = succeeded4890worker2 # [ 54.014314] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4891worker2 # [ 55.033935] nix-daemon[858]: reaped child process 1098, status = succeeded4892worker2 # [ 55.196080] nix-daemon[858]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4893worker2 # [ 55.212113] nix-daemon[858]: reaped child process 1116, status = succeeded4894lb # [ 55.248609] niks3-server[1123]: 2026/09/20 13:33:00 INFO Received uploads request method=POST path=/api/pending_closures4895worker2 # [ 55.237586] nix-grpc-daemon[870]: time=2026-09-20T13:33:00.761Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4896worker2 # [ 55.239420] nix-grpc-daemon[870]: time=2026-09-20T13:33:00.763Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4897lb # [ 55.273319] niks3-server[1123]: 2026/09/20 13:33:00 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4898lb # [ 55.280182] niks3-server[1123]: 2026/09/20 13:33:00 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4899lb # [ 55.287843] niks3-server[1123]: 2026/09/20 13:33:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4900lb # [ 55.292573] niks3-server[1123]: 2026/09/20 13:33:00 INFO Signed narinfos id=12 count=14901worker2 # [ 55.276066] nix-grpc-daemon[870]: time=2026-09-20T13:33:00.800Z level=INFO msg="Uploading 1 narinfos"4902lb # [ 55.296206] niks3-server[1123]: 2026/09/20 13:33:00 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4903lb # [ 55.301449] niks3-server[1123]: 2026/09/20 13:33:00 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4904lb # [ 55.304602] niks3-server[1123]: 2026/09/20 13:33:00 INFO Completed upload id=124905worker2 # [ 55.287194] nix-grpc-daemon[870]: time=2026-09-20T13:33:00.811Z level=INFO msg="Upload complete. (264ms)"4906lb # [ 55.307256] niks3-server[1123]: 2026/09/20 13:33:00 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4907worker2 # [ 55.290679] nix-grpc-daemon[788]: ts=2026-09-20T13:33:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14908worker2 # [ 55.299499] nix-daemon[858]: reaped child process 1096, status = succeeded4909worker1 # [ 55.320625] nix-grpc-daemon[842]: ts=2026-09-20T13:33:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:343144910worker1 # [ 55.335734] nix-grpc-daemon[842]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4911worker2 # [ 55.353984] nix-grpc-daemon[788]: ts=2026-09-20T13:33:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:452524912worker2 # [ 55.357041] nix-grpc-daemon[788]: ts=2026-09-20T13:33:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_out=1284913worker1 # [ 55.599563] nix-grpc-daemon[842]: ts=2026-09-20T13:33:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:343144914worker1 # [ 55.605094] nix-grpc-daemon[842]: ts=2026-09-20T13:33:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343144915worker1 # [ 55.609687] nix-grpc-daemon[926]: time=2026-09-20T13:33:01.457Z level=INFO msg="All 1 paths already cached"4916worker2 # [ 55.598142] nix-grpc-daemon[788]: ts=2026-09-20T13:33:01Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04917worker2 # [ 55.600565] nix-grpc-daemon[788]: ts=2026-09-20T13:33:01Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14918worker2 # [ 55.602834] nix-grpc-daemon[788]: ts=2026-09-20T13:33:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524919worker2 # [ 55.610191] nix-grpc-daemon[788]: ts=2026-09-20T13:33:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524920worker2 # [ 55.618769] nix-grpc-daemon[870]: time=2026-09-20T13:33:01.143Z level=INFO msg="All 1 paths already cached"4921worker2 # [ 55.623618] nix-grpc-daemon[788]: ts=2026-09-20T13:33:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452524922worker2 # [ 55.630094] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4923worker2 # [ 55.812889] nix-daemon[858]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4924worker2 # [ 55.829386] nix-daemon[858]: reaped child process 1125, status = succeeded4925worker2 # [ 56.001448] nix-daemon[858]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)4926worker2 # [ 56.016925] nix-daemon[858]: reaped child process 1130, status = succeeded4927lb # [ 56.046477] niks3-server[1123]: 2026/09/20 13:33:01 INFO Received uploads request method=POST path=/api/pending_closures4928worker2 # [ 56.076652] nix-grpc-daemon[870]: time=2026-09-20T13:33:01.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4929worker2 # [ 56.078335] nix-grpc-daemon[870]: time=2026-09-20T13:33:01.602Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4930lb # [ 56.109952] niks3-server[1123]: 2026/09/20 13:33:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4931lb # [ 56.112604] niks3-server[1123]: 2026/09/20 13:33:01 INFO Signed narinfos id=13 count=14932worker2 # [ 56.095583] nix-grpc-daemon[870]: time=2026-09-20T13:33:01.619Z level=INFO msg="Uploading 1 narinfos"4933lb # [ 56.115281] niks3-server[1123]: 2026/09/20 13:33:01 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4934lb # [ 56.117368] niks3-server[1123]: 2026/09/20 13:33:01 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4935lb # [ 56.122905] niks3-server[1123]: 2026/09/20 13:33:01 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4936lb # [ 56.127886] niks3-server[1123]: 2026/09/20 13:33:01 INFO Completed upload id=134937worker2 # [ 56.110356] nix-grpc-daemon[870]: time=2026-09-20T13:33:01.634Z level=INFO msg="Upload complete. (473ms)"4938lb # [ 56.129555] niks3-server[1123]: 2026/09/20 13:33:01 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4939worker2 # [ 56.112773] nix-grpc-daemon[788]: ts=2026-09-20T13:33:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_in=6244940worker2 # [ 56.115809] nix-grpc-daemon[788]: ts=2026-09-20T13:33:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524941worker2 # [ 56.125515] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4942worker2 # [ 56.127261] nix-daemon[858]: reaped child process 1120, status = succeeded4943worker2 # [ 56.132589] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4944worker2 # [ 56.745947] nix-daemon[858]: reaped child process 1134, status = succeeded4945worker2 # [ 56.881801] nix-daemon[858]: accepted connection from pid 1149, user nix-grpc-daemon (trusted)4946worker2 # [ 56.894142] nix-daemon[858]: reaped child process 1152, status = succeeded4947lb # [ 56.927970] niks3-server[1123]: 2026/09/20 13:33:02 INFO Received uploads request method=POST path=/api/pending_closures4948worker2 # [ 57.103365] nix-grpc-daemon[870]: time=2026-09-20T13:33:02.626Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4949worker2 # [ 57.112878] nix-grpc-daemon[870]: time=2026-09-20T13:33:02.631Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4950lb # [ 57.157906] niks3-server[1123]: 2026/09/20 13:33:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4951lb # [ 57.165546] niks3-server[1123]: 2026/09/20 13:33:02 INFO Signed narinfos id=14 count=14952worker2 # [ 57.150216] nix-grpc-daemon[870]: time=2026-09-20T13:33:02.673Z level=INFO msg="Uploading 1 narinfos"4953lb # [ 57.170423] niks3-server[1123]: 2026/09/20 13:33:02 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4954lb # [ 57.177451] niks3-server[1123]: 2026/09/20 13:33:02 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4955lb # [ 57.186086] niks3-server[1123]: 2026/09/20 13:33:02 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4956lb # [ 57.189471] niks3-server[1123]: 2026/09/20 13:33:02 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4957lb # [ 57.199443] niks3-server[1123]: 2026/09/20 13:33:02 INFO Completed upload id=144958worker2 # [ 57.180871] nix-grpc-daemon[870]: time=2026-09-20T13:33:02.705Z level=INFO msg="Upload complete. (441ms)"4959lb # [ 57.207152] niks3-server[1123]: 2026/09/20 13:33:02 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4960worker2 # [ 57.188165] nix-grpc-daemon[788]: ts=2026-09-20T13:33:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14961worker2 # [ 57.202361] nix-daemon[858]: reaped child process 1132, status = succeeded4962worker2 # [ 57.230303] nix-grpc-daemon[788]: ts=2026-09-20T13:33:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:452524963worker1 # [ 57.264575] nix-grpc-daemon[842]: ts=2026-09-20T13:33:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:343144964worker1 # [ 57.270122] nix-grpc-daemon[842]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4965worker1 # [ 57.276990] nix-grpc-daemon[842]: ts=2026-09-20T13:33:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=0 paths=1 nar_bytes_out=1764966worker2 # [ 57.496607] nix-grpc-daemon[788]: ts=2026-09-20T13:33:03Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:452524967worker1 # [ 57.521133] nix-grpc-daemon[842]: ts=2026-09-20T13:33:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343144968worker1 # [ 57.530476] nix-grpc-daemon[926]: time=2026-09-20T13:33:03.378Z level=INFO msg="All 2 paths already cached"4969worker2 # [ 57.523230] nix-grpc-daemon[788]: ts=2026-09-20T13:33:03Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04970worker2 # [ 57.526537] nix-grpc-daemon[788]: ts=2026-09-20T13:33:03Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14971worker2 # [ 57.529076] nix-grpc-daemon[788]: ts=2026-09-20T13:33:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524972worker2 # [ 57.537614] nix-grpc-daemon[788]: ts=2026-09-20T13:33:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452524973worker2 # [ 57.545437] nix-grpc-daemon[870]: time=2026-09-20T13:33:03.069Z level=INFO msg="All 2 paths already cached"4974worker2 # [ 57.549986] nix-grpc-daemon[788]: ts=2026-09-20T13:33:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452524975worker2 # [ 57.556093] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4976worker2 # [ 57.711697] nix-daemon[858]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4977worker2 # [ 57.726214] nix-daemon[858]: reaped child process 1161, status = succeeded4978worker2 # [ 57.865738] nix-daemon[858]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4979worker2 # [ 57.880296] nix-daemon[858]: reaped child process 1166, status = succeeded4980lb # [ 57.908056] niks3-server[1123]: 2026/09/20 13:33:03 INFO Received uploads request method=POST path=/api/pending_closures4981worker2 # [ 57.894279] nix-grpc-daemon[870]: time=2026-09-20T13:33:03.418Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4982worker2 # [ 57.895661] nix-grpc-daemon[870]: time=2026-09-20T13:33:03.418Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4983lb # [ 57.922286] niks3-server[1123]: 2026/09/20 13:33:03 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4984lb # [ 57.926584] niks3-server[1123]: 2026/09/20 13:33:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4985lb # [ 57.928748] niks3-server[1123]: 2026/09/20 13:33:03 INFO Signed narinfos id=15 count=14986worker2 # [ 57.911598] nix-grpc-daemon[870]: time=2026-09-20T13:33:03.435Z level=INFO msg="Uploading 1 narinfos"4987lb # [ 57.934352] niks3-server[1123]: 2026/09/20 13:33:03 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4988lb # [ 57.941202] niks3-server[1123]: 2026/09/20 13:33:03 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4989lb # [ 57.943628] niks3-server[1123]: 2026/09/20 13:33:03 INFO Completed upload id=154990worker2 # [ 57.926047] nix-grpc-daemon[870]: time=2026-09-20T13:33:03.450Z level=INFO msg="Upload complete. (363ms)"4991worker2 # [ 57.928393] nix-grpc-daemon[788]: ts=2026-09-20T13:33:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_in=7524992worker2 # [ 57.930469] nix-grpc-daemon[870]: time=2026-09-20T13:33:03.452Z level=WARN msg="Failed to register uploaded object" key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo error="server returned 404: object is not pending upload\n"4993worker2 # [ 57.934125] nix-grpc-daemon[788]: ts=2026-09-20T13:33:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452524994worker2 # [ 57.938305] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4995worker2 # [ 57.946150] nix-daemon[858]: reaped child process 1156, status = succeeded4996worker2 # [ 57.949403] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)4997lb # [ 58.937649] envoy[984]: conn peer=[2001:db8:1::1]:57192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1328 ms=56324998lb # [ 58.941403] envoy[984]: conn peer=[2001:db8:1::1]:57194 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=45554999lb # [ 58.944939] envoy[984]: conn peer=[2001:db8:1::1]:60788 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=265000lb # [ 58.948358] envoy[984]: conn peer=[2001:db8:1::1]:60776 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3384 tx=1758 ms=19315001lb # [ 58.951780] envoy[984]: conn peer=[2001:db8:1::1]:60790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3880 tx=1878 ms=17295002lb # [ 58.955243] envoy[984]: conn peer=[2001:db8:1::1]:60806 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=615003worker2 # [ 59.164364] nix-daemon[858]: reaped child process 1170, status = succeeded5004worker2 # [ 59.302931] nix-daemon[858]: accepted connection from pid 1185, user nix-grpc-daemon (trusted)5005worker2 # [ 59.315088] nix-daemon[858]: reaped child process 1188, status = succeeded5006lb # [ 59.350716] niks3-server[1123]: 2026/09/20 13:33:04 INFO Received uploads request method=POST path=/api/pending_closures5007worker2 # [ 59.342220] nix-grpc-daemon[870]: time=2026-09-20T13:33:04.866Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5008worker2 # [ 59.345077] nix-grpc-daemon[870]: time=2026-09-20T13:33:04.868Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5009lb # [ 59.399562] niks3-server[1123]: 2026/09/20 13:33:04 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5010lb # [ 59.403909] niks3-server[1123]: 2026/09/20 13:33:04 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5011lb # [ 59.411244] niks3-server[1123]: 2026/09/20 13:33:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5012lb # [ 59.417132] niks3-server[1123]: 2026/09/20 13:33:04 INFO Signed narinfos id=16 count=15013worker2 # [ 59.401215] nix-grpc-daemon[870]: time=2026-09-20T13:33:04.925Z level=INFO msg="Uploading 1 narinfos"5014lb # [ 59.419706] niks3-server[1123]: 2026/09/20 13:33:04 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5015lb # [ 59.430491] niks3-server[1123]: 2026/09/20 13:33:04 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5016lb # [ 59.437330] niks3-server[1123]: 2026/09/20 13:33:04 INFO Completed upload id=165017lb # [ 59.439057] niks3-server[1123]: 2026/09/20 13:33:04 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5018worker2 # [ 59.418938] nix-grpc-daemon[870]: time=2026-09-20T13:33:04.943Z level=INFO msg="Upload complete. (262ms)"5019worker2 # [ 59.427243] nix-grpc-daemon[788]: ts=2026-09-20T13:33:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15020worker2 # [ 59.433974] nix-grpc-daemon[788]: ts=2026-09-20T13:33:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:452525021worker2 # [ 59.443130] nix-daemon[858]: reaped child process 1168, status = succeeded5022worker1 # [ 59.470645] nix-grpc-daemon[842]: ts=2026-09-20T13:33:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:343145023worker1 # [ 59.480849] nix-grpc-daemon[842]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5024worker1 # [ 59.493615] nix-grpc-daemon[842]: ts=2026-09-20T13:33:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=0 paths=1 nar_bytes_out=2325025client # [ 59.623610] nix-daemon[943]: reaped child process 945, status = succeeded5026client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.75 seconds)5027client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5028client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5029client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5030client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5031client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5032client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5033(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.85 seconds)5034subtest: an input only one worker has still reaches the builder5035client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5036client # this derivation will be built:5037client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5038client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5039client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.38 seconds)5040client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5041client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds)5042worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5043worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.21 seconds)5044worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5045worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5046client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25047client # [ 60.722536] nix-daemon[943]: accepted connection from pid 1022, user root (trusted)5048client # this derivation will be built:5049client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5050worker1 # [ 60.990888] nix-grpc-daemon[842]: ts=2026-09-20T13:33:06Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:343145051worker1 # [ 60.994755] nix-grpc-daemon[842]: ts=2026-09-20T13:33:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343145052client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5053worker1 # [ 61.110789] nix-daemon[901]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)5054worker1 # [ 61.123089] nix-daemon[901]: reaped child process 1009, status = succeeded5055lb # [ 61.136530] niks3-server[1123]: 2026/09/20 13:33:06 INFO Received uploads request method=POST path=/api/pending_closures5056worker1 # [ 61.147628] nix-grpc-daemon[926]: time=2026-09-20T13:33:06.996Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5057worker1 # [ 61.149634] nix-grpc-daemon[926]: time=2026-09-20T13:33:06.997Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5058lb # [ 61.166471] niks3-server[1123]: 2026/09/20 13:33:06 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5059lb # [ 61.175186] niks3-server[1123]: 2026/09/20 13:33:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5060worker1 # [ 61.182425] nix-grpc-daemon[926]: time=2026-09-20T13:33:07.031Z level=INFO msg="Uploading 1 narinfos"5061lb # [ 61.180593] niks3-server[1123]: 2026/09/20 13:33:06 INFO Signed narinfos id=17 count=15062lb # [ 61.186221] niks3-server[1123]: 2026/09/20 13:33:06 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5063lb # [ 61.196024] niks3-server[1123]: 2026/09/20 13:33:06 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5064lb # [ 61.198834] niks3-server[1123]: 2026/09/20 13:33:06 INFO Completed upload id=175065worker1 # [ 61.200986] nix-grpc-daemon[926]: time=2026-09-20T13:33:07.049Z level=INFO msg="Upload complete. (204ms)"5066lb # [ 61.201490] niks3-server[1123]: 2026/09/20 13:33:06 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5067worker2 # [ 61.197957] nix-grpc-daemon[788]: ts=2026-09-20T13:33:06Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05068worker2 # [ 61.202470] nix-grpc-daemon[788]: ts=2026-09-20T13:33:06Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15069worker1 # [ 61.261709] nix-grpc-daemon[842]: ts=2026-09-20T13:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343145070worker1 # [ 61.273069] nix-grpc-daemon[842]: ts=2026-09-20T13:33:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343145071client # copying 2 paths to 'lb:50051'...5072client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5073worker1 # [ 61.288784] nix-grpc-daemon[842]: ts=2026-09-20T13:33:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343145074client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5075worker1 # [ 61.299666] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)5076worker1 # [ 61.456711] nix-daemon[901]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)5077worker1 # [ 61.469846] nix-daemon[901]: reaped child process 1016, status = succeeded5078worker1 # [ 61.610103] nix-daemon[901]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)5079worker1 # [ 61.622212] nix-daemon[901]: reaped child process 1021, status = succeeded5080lb # [ 61.634958] niks3-server[1123]: 2026/09/20 13:33:06 INFO Received uploads request method=POST path=/api/pending_closures5081lb # [ 61.643621] niks3-server[1123]: 2026/09/20 13:33:06 INFO Received uploads request method=POST path=/api/pending_closures5082worker1 # [ 61.652710] nix-grpc-daemon[926]: time=2026-09-20T13:33:07.501Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5083worker1 # [ 61.654524] nix-grpc-daemon[926]: time=2026-09-20T13:33:07.503Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5084worker1 # [ 61.656107] nix-grpc-daemon[926]: time=2026-09-20T13:33:07.503Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5085lb # [ 61.674142] niks3-server[1123]: 2026/09/20 13:33:06 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5086lb # [ 61.688284] niks3-server[1123]: 2026/09/20 13:33:06 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5087lb # [ 61.692901] niks3-server[1123]: 2026/09/20 13:33:06 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5088lb # [ 61.695584] niks3-server[1123]: 2026/09/20 13:33:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5089lb # [ 61.698241] niks3-server[1123]: 2026/09/20 13:33:06 INFO Signed narinfos id=18 count=15090lb # [ 61.699872] niks3-server[1123]: 2026/09/20 13:33:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5091lb # [ 61.702281] niks3-server[1123]: 2026/09/20 13:33:07 INFO Signed narinfos id=19 count=25092worker1 # [ 61.705490] nix-grpc-daemon[926]: time=2026-09-20T13:33:07.553Z level=INFO msg="Uploading 3 narinfos"5093lb # [ 61.704656] niks3-server[1123]: 2026/09/20 13:33:07 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5094lb # [ 61.718408] niks3-server[1123]: 2026/09/20 13:33:07 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5095lb # [ 61.720992] niks3-server[1123]: 2026/09/20 13:33:07 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5096lb # [ 61.723496] niks3-server[1123]: 2026/09/20 13:33:07 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5097lb # [ 61.725714] niks3-server[1123]: 2026/09/20 13:33:07 INFO Completed upload id=185098lb # [ 61.726962] niks3-server[1123]: 2026/09/20 13:33:07 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5099lb # [ 61.729202] niks3-server[1123]: 2026/09/20 13:33:07 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5100lb # [ 61.731465] niks3-server[1123]: 2026/09/20 13:33:07 INFO Completed upload id=195101worker1 # [ 61.733687] nix-grpc-daemon[926]: time=2026-09-20T13:33:07.582Z level=INFO msg="Upload complete. (417ms)"5102worker1 # [ 61.738933] nix-grpc-daemon[842]: ts=2026-09-20T13:33:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=0 paths=2 nar_bytes_in=9765103worker1 # [ 61.744281] nix-grpc-daemon[842]: ts=2026-09-20T13:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343145104worker1 # [ 61.754107] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)5105worker1 # [ 61.762397] nix-daemon[901]: reaped child process 1011, status = succeeded5106worker1 # [ 61.765353] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)5107client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5108client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5109worker1 # [ 62.780540] nix-daemon[901]: reaped child process 1025, status = succeeded5110worker1 # [ 62.898113] nix-daemon[901]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)5111worker1 # [ 62.910298] nix-daemon[901]: reaped child process 1043, status = succeeded5112lb # [ 62.917871] niks3-server[1123]: 2026/09/20 13:33:08 INFO Received uploads request method=POST path=/api/pending_closures5113worker1 # [ 62.924068] nix-grpc-daemon[926]: time=2026-09-20T13:33:08.772Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5114lb # [ 62.932473] niks3-server[1123]: 2026/09/20 13:33:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5115lb # [ 62.935187] niks3-server[1123]: 2026/09/20 13:33:08 INFO Signed narinfos id=20 count=15116worker1 # [ 62.938883] nix-grpc-daemon[926]: time=2026-09-20T13:33:08.787Z level=INFO msg="Uploading 1 narinfos"5117lb # [ 62.937857] niks3-server[1123]: 2026/09/20 13:33:08 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5118lb # [ 62.939516] niks3-server[1123]: 2026/09/20 13:33:08 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5119lb # [ 62.945675] niks3-server[1123]: 2026/09/20 13:33:08 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5120lb # [ 62.948286] niks3-server[1123]: 2026/09/20 13:33:08 INFO Completed upload id=205121worker1 # [ 62.951517] nix-grpc-daemon[926]: time=2026-09-20T13:33:08.800Z level=INFO msg="Upload complete. (179ms)"5122lb # [ 62.949909] niks3-server[1123]: 2026/09/20 13:33:08 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5123worker1 # [ 62.953984] nix-grpc-daemon[842]: ts=2026-09-20T13:33:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15124worker1 # [ 62.960321] nix-daemon[901]: reaped child process 1023, status = succeeded5125client # copying 1 paths...5126worker2 # [ 62.977393] nix-grpc-daemon[788]: ts=2026-09-20T13:33:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:452525127worker2 # [ 62.986054] nix-grpc-daemon[788]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5128client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5129worker2 # [ 63.017311] nix-grpc-daemon[788]: ts=2026-09-20T13:33:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:452525130worker2 # [ 63.019368] nix-grpc-daemon[788]: ts=2026-09-20T13:33:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_out=1205131client # [ 63.165713] nix-daemon[943]: reaped child process 1025, status = succeeded5132client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.64 seconds)5133client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25134client # [ 63.328103] nix-daemon[943]: accepted connection from pid 1043, user root (trusted)5135client # this derivation will be built:5136client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5137worker2 # [ 63.467490] nix-grpc-daemon[788]: ts=2026-09-20T13:33:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:452525138client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5139worker2 # [ 63.472598] nix-grpc-daemon[788]: ts=2026-09-20T13:33:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452525140worker2 # [ 63.494075] nix-grpc-daemon[788]: ts=2026-09-20T13:33:09Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05141worker2 # [ 63.496461] nix-grpc-daemon[788]: ts=2026-09-20T13:33:09Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15142worker2 # [ 63.499127] nix-grpc-daemon[788]: ts=2026-09-20T13:33:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452525143worker2 # [ 63.505592] nix-grpc-daemon[788]: ts=2026-09-20T13:33:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452525144client # copying 1 paths to 'lb:50051'...5145client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5146worker2 # [ 63.513520] nix-grpc-daemon[788]: ts=2026-09-20T13:33:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452525147worker2 # [ 63.517539] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)5148worker2 # [ 63.526395] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)5149worker2 # [ 63.543761] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 101 ms (attempt 2/5)5150worker2 # [ 63.647293] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 186 ms (attempt 3/5)5151worker2 # [ 63.836145] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 277 ms (attempt 4/5)5152worker2 # [ 64.114640] nix-grpc-daemon[788]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5153worker2 # [ 64.117616] nix-grpc-daemon[788]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5154worker2 # [ 64.308099] nix-daemon[858]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5155worker2 # [ 64.321985] nix-daemon[858]: reaped child process 1207, status = succeeded5156worker2 # [ 64.466532] nix-daemon[858]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5157worker2 # [ 64.480335] nix-daemon[858]: reaped child process 1212, status = succeeded5158lb # [ 64.508174] niks3-server[1123]: 2026/09/20 13:33:09 INFO Received uploads request method=POST path=/api/pending_closures5159worker2 # [ 64.494546] nix-grpc-daemon[870]: time=2026-09-20T13:33:10.018Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5160worker2 # [ 64.496446] nix-grpc-daemon[870]: time=2026-09-20T13:33:10.018Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5161lb # [ 64.523339] niks3-server[1123]: 2026/09/20 13:33:09 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5162lb # [ 64.525412] niks3-server[1123]: 2026/09/20 13:33:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5163lb # [ 64.528639] niks3-server[1123]: 2026/09/20 13:33:09 INFO Signed narinfos id=21 count=15164worker2 # [ 64.512286] nix-grpc-daemon[870]: time=2026-09-20T13:33:10.036Z level=INFO msg="Uploading 1 narinfos"5165lb # [ 64.539429] niks3-server[1123]: 2026/09/20 13:33:09 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5166lb # [ 64.542627] niks3-server[1123]: 2026/09/20 13:33:09 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5167lb # [ 64.546278] niks3-server[1123]: 2026/09/20 13:33:09 INFO Completed upload id=215168worker2 # [ 64.527423] nix-grpc-daemon[870]: time=2026-09-20T13:33:10.051Z level=INFO msg="Upload complete. (394ms)"5169lb # [ 64.547921] niks3-server[1123]: 2026/09/20 13:33:09 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5170worker2 # [ 64.530077] nix-grpc-daemon[788]: ts=2026-09-20T13:33:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=1 paths=1 nar_bytes_in=5605171worker2 # [ 64.533851] nix-grpc-daemon[788]: ts=2026-09-20T13:33:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452525172worker2 # [ 64.538834] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)5173worker2 # [ 64.545556] nix-daemon[858]: reaped child process 1198, status = succeeded5174worker2 # [ 64.547345] nix-grpc-daemon[788]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5175client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5176worker2 # [ 64.556449] nix-daemon[858]: accepted connection from pid 788, user nix-grpc-daemon (trusted)5177client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5178worker2 # [ 65.577776] nix-daemon[858]: reaped child process 1217, status = succeeded5179worker2 # [ 65.708265] nix-daemon[858]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5180worker2 # [ 65.721897] nix-daemon[858]: reaped child process 1235, status = succeeded5181lb # [ 65.749960] niks3-server[1123]: 2026/09/20 13:33:11 INFO Received uploads request method=POST path=/api/pending_closures5182worker2 # [ 65.735474] nix-grpc-daemon[870]: time=2026-09-20T13:33:11.259Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5183lb # [ 65.765399] niks3-server[1123]: 2026/09/20 13:33:11 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5184lb # [ 65.767993] niks3-server[1123]: 2026/09/20 13:33:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5185worker2 # [ 65.755934] nix-grpc-daemon[870]: time=2026-09-20T13:33:11.280Z level=INFO msg="Uploading 1 narinfos"5186lb # [ 65.774653] niks3-server[1123]: 2026/09/20 13:33:11 INFO Signed narinfos id=22 count=15187lb # [ 65.778895] niks3-server[1123]: 2026/09/20 13:33:11 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5188lb # [ 65.789826] niks3-server[1123]: 2026/09/20 13:33:11 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5189worker2 # [ 65.776770] nix-grpc-daemon[870]: time=2026-09-20T13:33:11.301Z level=INFO msg="Upload complete. (212ms)"5190lb # [ 65.795560] niks3-server[1123]: 2026/09/20 13:33:11 INFO Completed upload id=225191lb # [ 65.797536] niks3-server[1123]: 2026/09/20 13:33:11 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5192worker2 # [ 65.779567] nix-grpc-daemon[788]: ts=2026-09-20T13:33:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15193worker2 # [ 65.786705] nix-daemon[858]: reaped child process 1214, status = succeeded5194client # copying 1 paths...5195worker2 # [ 65.824668] nix-grpc-daemon[788]: ts=2026-09-20T13:33:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:452525196client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5197worker2 # [ 65.843974] nix-grpc-daemon[788]: ts=2026-09-20T13:33:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:452525198worker2 # [ 65.848383] nix-grpc-daemon[788]: ts=2026-09-20T13:33:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45252 duration_s=0 paths=1 nar_bytes_out=1205199client # [ 65.987940] nix-daemon[943]: reaped child process 1046, status = succeeded5200client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.82 seconds)5201client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25202client # [ 66.158104] nix-daemon[943]: accepted connection from pid 1063, user root (trusted)5203client # this derivation will be built:5204client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5205worker2 # [ 66.282970] nix-grpc-daemon[788]: ts=2026-09-20T13:33:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:452525206client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5207worker2 # [ 66.287565] nix-grpc-daemon[788]: ts=2026-09-20T13:33:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452525208worker2 # [ 66.292653] nix-grpc-daemon[870]: time=2026-09-20T13:33:11.815Z level=INFO msg="All 1 paths already cached"5209worker2 # [ 66.302160] nix-grpc-daemon[788]: ts=2026-09-20T13:33:11Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05210worker2 # [ 66.306386] nix-grpc-daemon[788]: ts=2026-09-20T13:33:11Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15211worker1 # [ 66.376849] nix-grpc-daemon[842]: ts=2026-09-20T13:33:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343145212worker1 # [ 66.385880] nix-grpc-daemon[842]: ts=2026-09-20T13:33:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:343145213worker1 # [ 66.394497] nix-grpc-daemon[926]: time=2026-09-20T13:33:12.243Z level=INFO msg="All 1 paths already cached"5214client # copying 1 paths to 'lb:50051'...5215worker1 # [ 66.398253] nix-grpc-daemon[842]: ts=2026-09-20T13:33:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:343145216client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5217worker1 # [ 66.402602] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)5218worker1 # [ 66.544168] nix-daemon[901]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5219worker1 # [ 66.557530] nix-daemon[901]: reaped child process 1050, status = succeeded5220worker1 # [ 66.696482] nix-daemon[901]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)5221worker1 # [ 66.709270] nix-daemon[901]: reaped child process 1055, status = succeeded5222lb # [ 66.717982] niks3-server[1123]: 2026/09/20 13:33:12 INFO Received uploads request method=POST path=/api/pending_closures5223worker1 # [ 66.728220] nix-grpc-daemon[926]: time=2026-09-20T13:33:12.577Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5224worker1 # [ 66.729775] nix-grpc-daemon[926]: time=2026-09-20T13:33:12.578Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5225lb # [ 66.742271] niks3-server[1123]: 2026/09/20 13:33:12 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5226lb # [ 66.747190] niks3-server[1123]: 2026/09/20 13:33:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5227lb # [ 66.749459] niks3-server[1123]: 2026/09/20 13:33:12 INFO Signed narinfos id=23 count=15228worker1 # [ 66.752843] nix-grpc-daemon[926]: time=2026-09-20T13:33:12.601Z level=INFO msg="Uploading 1 narinfos"5229lb # [ 66.752267] niks3-server[1123]: 2026/09/20 13:33:12 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5230lb # [ 66.757321] niks3-server[1123]: 2026/09/20 13:33:12 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5231lb # [ 66.760608] niks3-server[1123]: 2026/09/20 13:33:12 INFO Completed upload id=235232worker1 # [ 66.762697] nix-grpc-daemon[926]: time=2026-09-20T13:33:12.611Z level=INFO msg="Upload complete. (353ms)"5233worker1 # [ 66.764604] nix-grpc-daemon[842]: ts=2026-09-20T13:33:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=0 paths=1 nar_bytes_in=5605234lb # [ 66.763191] niks3-server[1123]: 2026/09/20 13:33:12 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5235worker1 # [ 66.767753] nix-grpc-daemon[842]: ts=2026-09-20T13:33:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:343145236worker1 # [ 66.774133] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)5237worker1 # [ 66.776476] nix-daemon[901]: reaped child process 1045, status = succeeded5238worker1 # [ 66.781546] nix-daemon[901]: accepted connection from pid 842, user nix-grpc-daemon (trusted)5239client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5240client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5241worker1 # [ 67.794304] nix-daemon[901]: reaped child process 1059, status = succeeded5242worker1 # [ 67.919600] nix-daemon[901]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)5243worker1 # [ 67.931434] nix-daemon[901]: reaped child process 1077, status = succeeded5244lb # [ 67.946536] niks3-server[1123]: 2026/09/20 13:33:13 INFO Received uploads request method=POST path=/api/pending_closures5245worker1 # [ 67.959636] nix-grpc-daemon[926]: time=2026-09-20T13:33:13.808Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5246lb # [ 67.982658] niks3-server[1123]: 2026/09/20 13:33:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5247lb # [ 67.988383] niks3-server[1123]: 2026/09/20 13:33:13 INFO Signed narinfos id=24 count=15248worker1 # [ 67.993762] nix-grpc-daemon[926]: time=2026-09-20T13:33:13.842Z level=INFO msg="Uploading 1 narinfos"5249lb # [ 67.995565] niks3-server[1123]: 2026/09/20 13:33:13 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5250lb # [ 68.005321] niks3-server[1123]: 2026/09/20 13:33:13 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5251lb # [ 68.013469] niks3-server[1123]: 2026/09/20 13:33:13 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5252lb # [ 68.020102] niks3-server[1123]: 2026/09/20 13:33:13 INFO Completed upload id=245253worker1 # [ 68.022357] nix-grpc-daemon[926]: time=2026-09-20T13:33:13.871Z level=INFO msg="Upload complete. (234ms)"5254lb # [ 68.023464] niks3-server[1123]: 2026/09/20 13:33:13 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5255worker1 # [ 68.026716] nix-grpc-daemon[842]: ts=2026-09-20T13:33:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15256worker1 # [ 68.037925] nix-daemon[901]: reaped child process 1057, status = succeeded5257client # copying 1 paths...5258worker2 # [ 68.053609] nix-grpc-daemon[788]: ts=2026-09-20T13:33:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:452525259worker2 # [ 68.063818] nix-grpc-daemon[788]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5260client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5261worker1 # [ 68.111888] nix-grpc-daemon[842]: ts=2026-09-20T13:33:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:343145262worker1 # [ 68.113640] nix-grpc-daemon[842]: ts=2026-09-20T13:33:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34314 duration_s=0 paths=1 nar_bytes_out=1205263client # [ 68.192959] nix-daemon[943]: reaped child process 1066, status = succeeded5264client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.21 seconds)5265(finished: subtest: an input only one worker has still reaches the builder, in 8.47 seconds)5266subtest: upload whose reference the worker lacks is completed from the cache5267client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5268client # this derivation will be built:5269client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5270client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5271client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.38 seconds)5272client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5273client # this derivation will be built:5274client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5275client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5276lb # [ 68.936465] envoy[984]: conn peer=[2001:db8:1::1]:60818 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2084 ms=20285277lb # [ 68.937935] envoy[984]: conn peer=[2001:db8:1::1]:60834 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=775278lb # [ 68.939331] envoy[984]: conn peer=[2001:db8:1::1]:56388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=605279lb # [ 68.940771] envoy[984]: conn peer=[2001:db8:1::1]:60848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3853 tx=1851 ms=21045280lb # [ 68.942178] envoy[984]: conn peer=[2001:db8:1::1]:56406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575281lb # [ 68.943593] envoy[984]: conn peer=[2001:db8:1::1]:56390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3714 tx=1804 ms=24325282lb # [ 68.945037] envoy[984]: conn peer=[2001:db8:1::1]:56418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3715 tx=1891 ms=18195283lb # [ 68.946487] envoy[984]: conn peer=[2001:db8:1::1]:56422 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=105284client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5285client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5286client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5287worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5288worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.18 seconds)5289worker2 # [ 69.354952] systemd[1]: Stopping Nix worker-protocol over gRPC...5290worker2 # [ 70.280489] nix-grpc-daemon[788]: ts=2026-09-20T13:33:15Z level=info event=signal_exit5291worker2 # [ 70.521512] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_yield5292worker1 # [ 70.545481] nix-grpc-daemon[842]: ts=2026-09-20T13:33:16Z level=info event=scheduler_restarting addr=lb:500515293lb # [ 70.543892] niks3-server[1123]: 2026/09/20 13:33:15 INFO lead: released remote=192.168.1.4:407705294worker2 # [ 70.531697] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_restarting addr=lb:500515295worker2 # [ 70.536423] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:390225296worker2 # [ 70.539906] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=worker_gone addr=192.168.1.4:500515297worker2 # [ 70.542874] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:390225298worker2 # [ 70.546210] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=worker_gone addr=192.168.1.3:500515299worker1 # [ 70.745029] nix-grpc-daemon[842]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500515300worker1 # [ 70.747033] nix-grpc-daemon[842]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5301worker2 # [ 70.732585] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500515302worker2 # [ 70.734421] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303worker1 # [ 70.947486] nix-grpc-daemon[842]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500515304worker1 # [ 70.950354] nix-grpc-daemon[842]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5305worker2 # [ 70.935124] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500515306worker2 # [ 70.936975] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307worker1 # [ 71.151070] nix-grpc-daemon[842]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500515308worker1 # [ 71.155570] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5309worker2 # [ 71.138328] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500515310worker2 # [ 71.142188] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5311worker1 # [ 71.355846] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500515312worker1 # [ 71.357475] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313lb # [ 71.362796] niks3-server[1123]: 2026/09/20 13:33:16 INFO lead: acquired remote=192.168.1.3:452345314worker1 # [ 71.366386] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_take_over5315worker2 # [ 71.344471] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500515316worker2 # [ 71.347535] nix-grpc-daemon[788]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker2 # [ 71.525993] nix-grpc-daemon[788]: ts=2026-09-20T13:33:17Z level=info event=scheduler_restarting5318worker1 # [ 71.559567] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500515319worker1 # [ 71.562651] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5320worker2 # [ 71.561412] nix-daemon[858]: reaped child process 860, status = succeeded5321worker2 # [ 71.563804] nix-grpc-daemon[788]: ts=2026-09-20T13:33:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5322worker1 # [ 71.759796] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500515323worker1 # [ 71.761482] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5324worker1 # [ 71.962357] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500515325worker1 # [ 71.966165] nix-grpc-daemon[842]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5326worker1 # [ 72.166698] nix-grpc-daemon[842]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500515327worker1 # [ 72.170706] nix-grpc-daemon[842]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5328worker1 # [ 72.371295] nix-grpc-daemon[842]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500515329worker1 # [ 72.384351] nix-grpc-daemon[842]: ts=2026-09-20T13:33:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:540085330worker1 # [ 72.388711] nix-grpc-daemon[842]: ts=2026-09-20T13:33:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05331worker2 # [ 72.379302] nix-grpc-daemon[788]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5332worker2 # [ 72.381172] nix-grpc-daemon[788]: E0000 00:00:1789911197.902341 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5333worker2 # [ 72.383552] nix-grpc-daemon[788]: E0000 00:00:1789911197.902975 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5334worker2 # [ 72.902562] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5335worker2 # [ 72.907722] systemd[1]: Stopped Nix worker-protocol over gRPC.5336worker2 # [ 72.910190] systemd[1]: nix-grpc-daemon.service: Consumed 2.661s CPU time over 53.860s wall clock time, 58M memory peak, 93.6K incoming IP traffic, 227K outgoing IP traffic.5337worker2 # [ 72.921416] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5338worker2 # [ 72.925673] systemd[1]: Closed Nix worker-protocol over gRPC.5339worker2 # [ 72.928630] systemd[1]: nix-grpc-daemon.socket: Consumed 313us CPU time over 1min 3.999s wall clock time, 256K memory peak, 90.4K incoming IP traffic, 72.2K outgoing IP traffic.5340lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5341lb: must succeed: curl -sf localhost:9901/clusters5342lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5343lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5344client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25345worker1 # [ 73.224074] nix-grpc-daemon[842]: ts=2026-09-20T13:33:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:343145346client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5347worker1 # [ 73.226438] nix-grpc-daemon[842]: ts=2026-09-20T13:33:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:343145348client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.21 seconds)5349worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5350worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5351worker2 # [ 73.286208] systemd[1]: Listening on Nix worker-protocol over gRPC.5352worker1 # [ 73.355966] systemd[1]: Stopping Nix worker-protocol over gRPC...5353worker2 # [ 73.892793] systemd[1]: Starting scheduler-token.service...5354worker2 # [ 73.988117] systemd[1]: scheduler-token.service: Deactivated successfully.5355worker2 # [ 73.990423] systemd[1]: Finished scheduler-token.service.5356worker2 # [ 73.993539] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 95ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5357worker2 # [ 73.998883] systemd[1]: Starting Nix worker-protocol over gRPC...5358worker2 # [ 74.136459] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5359worker2 # [ 74.138745] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5360worker2 # [ 74.142585] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15361worker2 # [ 74.150106] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5362worker2 # [ 74.153848] systemd[1]: Started Nix worker-protocol over gRPC.5363worker2 # [ 74.170704] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500515364worker1 # [ 74.194561] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540085365worker1 # [ 74.199286] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05366worker1 # [ 74.228897] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=signal_exit5367worker1 # [ 74.486906] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=scheduler_yield5368worker2 # [ 74.469381] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:19Z level=info event=scheduler_restarting addr=lb:500515369lb # [ 74.488442] niks3-server[1123]: 2026/09/20 13:33:19 INFO lead: released remote=192.168.1.3:452345370worker1 # [ 74.494718] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540085371worker1 # [ 74.496655] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=worker_gone addr=192.168.1.4:500515372worker1 # [ 74.498595] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=scheduler_restarting addr=lb:500515373worker1 # [ 74.499750] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540085374worker1 # [ 74.501070] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=worker_gone addr=192.168.1.3:500515375worker2 # [ 74.670684] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500515376worker1 # [ 74.697247] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500515377worker2 # [ 74.674887] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5378worker1 # [ 74.699169] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5379worker2 # [ 74.875790] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500515380worker1 # [ 74.899610] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500515381worker1 # [ 74.901524] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5382worker2 # [ 74.879853] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5383worker2 # [ 75.080953] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500515384worker1 # [ 75.103323] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500515385worker2 # [ 75.084945] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5386worker1 # [ 75.106447] nix-grpc-daemon[842]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker1 # [ 75.304247] nix-grpc-daemon[842]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500515388worker1 # [ 75.305830] nix-grpc-daemon[842]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5389worker2 # [ 75.286262] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500515390worker2 # [ 75.287596] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5391worker1 # [ 75.487469] nix-grpc-daemon[842]: ts=2026-09-20T13:33:21Z level=info event=scheduler_restarting5392worker1 # [ 75.503065] nix-daemon[901]: reaped child process 903, status = succeeded5393worker2 # [ 75.487312] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500515394worker1 # [ 75.511713] nix-grpc-daemon[842]: ts=2026-09-20T13:33:21Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5395worker2 # [ 75.491583] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5396worker2 # [ 75.692225] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500515397worker2 # [ 75.694102] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5398worker2 # [ 75.895475] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500515399worker2 # [ 75.899836] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5400worker2 # [ 76.100964] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500515401worker2 # [ 76.104350] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5402worker2 # [ 76.306705] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500515403worker2 # [ 76.310107] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5404worker1 # [ 76.354894] nix-grpc-daemon[842]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5405worker1 # [ 76.356455] nix-grpc-daemon[842]: E0000 00:00:1789911202.202963 885 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5406worker1 # [ 76.358663] nix-grpc-daemon[842]: E0000 00:00:1789911202.207602 885 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5407worker2 # [ 76.511446] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500515408worker2 # [ 76.515288] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5409worker2 # [ 76.715941] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500515410worker2 # [ 76.717729] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker1 # [ 76.862597] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5412worker1 # [ 76.863741] systemd[1]: Stopped Nix worker-protocol over gRPC.5413worker1 # [ 76.865632] systemd[1]: nix-grpc-daemon.service: Consumed 1.348s CPU time over 57.933s wall clock time, 47.3M memory peak, 55.8K incoming IP traffic, 102.6K outgoing IP traffic.5414worker1 # [ 76.870746] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5415worker1 # [ 76.872619] systemd[1]: Closed Nix worker-protocol over gRPC.5416worker1 # [ 76.874351] systemd[1]: nix-grpc-daemon.socket: Consumed 331us CPU time over 1min 7.857s wall clock time, 256K memory peak, 63.9K incoming IP traffic, 56.7K outgoing IP traffic.5417lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5418lb: must succeed: curl -sf localhost:9901/clusters5419worker2 # [ 76.918955] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500515420worker2 # [ 76.923380] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5422lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5423client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25424worker2 # [ 77.123890] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500515425worker2 # [ 77.125686] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5426worker2 # [ 77.145695] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544065427worker2 # [ 77.151123] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5428worker2 # [ 77.159519] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5)5429worker2 # [ 77.195139] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 2/5)5430worker2 # [ 77.259985] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 3/5)5431worker2 # [ 77.328256] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500515432worker2 # [ 77.331471] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker2 # [ 77.399773] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 429 ms (attempt 4/5)5434worker2 # [ 77.529860] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500515435worker2 # [ 77.533742] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5436worker2 # [ 77.734883] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500515437worker2 # [ 77.738762] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5438worker2 # [ 77.835100] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5439client # copying 2 paths...5440client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5441worker2 # [ 77.856135] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544065442client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5443worker2 # [ 77.869162] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5444worker2 # [ 77.888967] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=niks3_push_started pid=13185445worker2 # [ 77.939391] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500515446worker2 # [ 77.941278] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5447worker2 # [ 78.068044] nix-grpc-daemon[1318]: time=2026-09-20T13:33:23.590Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5448worker2 # [ 78.143782] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500515449worker2 # [ 78.145686] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450worker2 # [ 78.192360] nix-daemon[858]: accepted connection from pid 1331, user nix-grpc-daemon (trusted)5451worker2 # [ 78.204938] nix-daemon[858]: reaped child process 1334, status = succeeded5452lb # [ 78.234775] niks3-server[1123]: 2026/09/20 13:33:23 INFO Received uploads request method=POST path=/api/pending_closures5453lb # [ 78.241477] niks3-server[1123]: 2026/09/20 13:33:23 INFO Received uploads request method=POST path=/api/pending_closures5454worker2 # [ 78.230124] nix-grpc-daemon[1318]: time=2026-09-20T13:33:23.754Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5455worker2 # [ 78.231607] nix-grpc-daemon[1318]: time=2026-09-20T13:33:23.755Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5456worker2 # [ 78.233555] nix-grpc-daemon[1318]: time=2026-09-20T13:33:23.757Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5457lb # [ 78.288701] niks3-server[1123]: 2026/09/20 13:33:23 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5458lb # [ 78.298944] niks3-server[1123]: 2026/09/20 13:33:23 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5459lb # [ 78.307820] niks3-server[1123]: 2026/09/20 13:33:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5460lb # [ 78.312510] niks3-server[1123]: 2026/09/20 13:33:23 INFO Signed narinfos id=26 count=25461lb # [ 78.314497] niks3-server[1123]: 2026/09/20 13:33:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5462lb # [ 78.318902] niks3-server[1123]: 2026/09/20 13:33:23 INFO Signed narinfos id=25 count=15463worker2 # [ 78.300228] nix-grpc-daemon[1318]: time=2026-09-20T13:33:23.823Z level=INFO msg="Uploading 3 narinfos"5464lb # [ 78.323062] niks3-server[1123]: 2026/09/20 13:33:23 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5465lb # [ 78.338267] niks3-server[1123]: 2026/09/20 13:33:23 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5466lb # [ 78.346820] niks3-server[1123]: 2026/09/20 13:33:23 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5467lb # [ 78.352313] niks3-server[1123]: 2026/09/20 13:33:23 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5468lb # [ 78.355148] niks3-server[1123]: 2026/09/20 13:33:23 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5469lb # [ 78.357825] niks3-server[1123]: 2026/09/20 13:33:23 INFO Completed upload id=255470lb # [ 78.359384] niks3-server[1123]: 2026/09/20 13:33:23 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5471lb # [ 78.363319] niks3-server[1123]: 2026/09/20 13:33:23 INFO Completed upload id=265472worker2 # [ 78.344733] nix-grpc-daemon[1318]: time=2026-09-20T13:33:23.868Z level=INFO msg="Upload complete. (277ms)"5473worker2 # [ 78.347891] nix-grpc-daemon[1318]: time=2026-09-20T13:33:23.870Z level=WARN msg="Failed to register uploaded object" key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo error="server returned 404: object is not pending upload\n"5474worker2 # [ 78.351386] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500515475worker2 # [ 78.352579] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=0 paths=2 nar_bytes_in=3125476worker2 # [ 78.354448] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5477worker2 # [ 78.359286] nix-daemon[858]: reaped child process 1316, status = succeeded5478client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.44 seconds)5479worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5480worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.01 seconds)5481worker1 # [ 78.462647] systemd[1]: Listening on Nix worker-protocol over gRPC.5482worker1 # [ 78.470500] systemd[1]: Starting Nix worker-protocol over gRPC...5483worker2 # [ 78.549185] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500515484worker2 # [ 78.553160] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5485worker1 # [ 78.622844] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5486worker1 # [ 78.624623] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5487worker1 # [ 78.628404] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15488worker1 # [ 78.634887] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5489worker1 # [ 78.637577] systemd[1]: Started Nix worker-protocol over gRPC.5490lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5491lb: must succeed: curl -sf localhost:9901/clusters5492worker1 # [ 78.646537] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=scheduler_take_over5493lb # [ 78.645822] niks3-server[1123]: 2026/09/20 13:33:23 INFO lead: acquired remote=192.168.1.3:533945494worker1 # [ 78.657450] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500515495worker1 # [ 78.659085] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5496lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5497worker2 # [ 78.754342] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500515498worker2 # [ 78.758240] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker1 # [ 78.863408] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500515500worker1 # [ 78.866442] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5501lb # [ 78.937911] envoy[984]: conn peer=[2001:db8:1::4]:42960 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=58859 tx=3240 ms=522645502lb # [ 78.941947] envoy[984]: conn peer=[2001:db8:1::1]:37184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=95503lb # [ 78.945548] envoy[984]: conn peer=[2001:db8:1::3]:43384 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15365 tx=2911 ms=554215504lb # [ 78.948105] envoy[984]: conn peer=[2001:db8:1::1]:37186 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1007 tx=305 ms=12115505worker2 # [ 78.958853] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500515506worker1 # [ 78.990098] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=oidc_jwks provider=mock keys=15507worker1 # [ 78.993505] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:432585508worker1 # [ 78.997445] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05509worker1 # [ 79.064358] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500515510worker1 # [ 79.068439] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:432585511worker1 # [ 79.072114] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05512lb: must succeed: curl -sf localhost:9901/clusters5513lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5514lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.12 seconds)5515(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.58 seconds)5516subtest: scheduler down: the other node takes the lock and keeps it5517lb: must succeed: curl -sf localhost:9901/clusters5518lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5519lb: must succeed: curl -sf localhost:9901/clusters5520lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5521lb: must succeed: curl -sf localhost:9901/clusters5522lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5523lb: must succeed: curl -sf localhost:9901/clusters5524lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5525worker1 # [ 79.978507] systemd[1]: Stopping Nix worker-protocol over gRPC...5526worker1 # [ 80.639119] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=signal_exit5527worker1 # [ 80.845108] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=scheduler_yield5528worker1 # [ 80.849178] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=scheduler_restarting addr=lb:500515529lb # [ 80.848124] niks3-server[1123]: 2026/09/20 13:33:26 INFO lead: released remote=192.168.1.3:533945530worker1 # [ 80.851073] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:432585531worker1 # [ 80.852517] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=worker_gone addr=192.168.1.3:500515532worker1 # [ 80.854177] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:432585533worker1 # [ 80.855613] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=worker_gone addr=192.168.1.4:500515534worker2 # [ 80.833773] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:26Z level=info event=scheduler_restarting addr=lb:500515535worker1 # [ 81.050422] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500515536worker1 # [ 81.054341] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5537worker2 # [ 81.034480] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5538worker1 # [ 81.254692] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500515539worker1 # [ 81.258859] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5540worker2 # [ 81.435750] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:26Z level=info event=scheduler_connected addr=lb:500515541worker1 # [ 81.460237] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500515542worker2 # [ 81.440335] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5543worker1 # [ 81.463910] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5544worker1 # [ 81.661391] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500515545worker2 # [ 81.641868] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500515546worker1 # [ 81.665904] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5547worker2 # [ 81.646354] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5548worker1 # [ 81.845811] nix-grpc-daemon[1100]: ts=2026-09-20T13:33:27Z level=info event=scheduler_restarting5549worker1 # [ 81.868189] nix-grpc-daemon[1100]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5550worker2 # [ 81.848753] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500515551worker1 # [ 81.869793] nix-grpc-daemon[1100]: E0000 00:00:1789911207.717076 1143 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5552worker2 # [ 81.851918] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5553worker1 # [ 81.872676] nix-grpc-daemon[1100]: E0000 00:00:1789911207.717138 1143 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5554worker2 # [ 82.049364] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500515555worker2 # [ 82.053213] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5556worker2 # [ 82.253712] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500515557worker2 # [ 82.255529] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5558worker2 # [ 82.456469] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_connected addr=lb:500515559worker2 # [ 82.458219] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5560worker1 # [ 82.644509] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5561worker1 # [ 82.646655] systemd[1]: Stopped Nix worker-protocol over gRPC.5562worker1 # [ 82.647542] systemd[1]: nix-grpc-daemon.service: Consumed 119ms CPU time over 4.174s wall clock time, 10.3M memory peak, 9.7K incoming IP traffic, 9.1K outgoing IP traffic.5563worker1 # [ 82.654502] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5564worker1 # [ 82.658149] systemd[1]: Closed Nix worker-protocol over gRPC.5565worker1 # [ 82.660534] systemd[1]: nix-grpc-daemon.socket: Consumed 516us CPU time over 4.201s wall clock time, 256K memory peak, 13.9K incoming IP traffic, 10.5K outgoing IP traffic.5566worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5567worker2 # [ 82.659525] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500515568worker2 # [ 82.664229] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5569worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5570worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5571worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5572worker2 # [ 82.864661] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500515573worker2 # [ 82.866638] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5574worker2 # [ 83.067529] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500515575worker2 # [ 83.069879] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5576worker2 # [ 83.270308] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500515577worker2 # [ 83.271995] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5578worker2 # [ 83.472664] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_connected addr=lb:500515579worker2 # [ 83.474545] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5580worker2 # [ 83.675252] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500515581worker2 # [ 83.676984] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5582worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5583worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5584worker2 # [ 83.878162] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500515585worker2 # [ 83.879753] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5586worker2 # [ 84.080637] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500515587worker2 # [ 84.082495] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5588worker2 # [ 84.164699] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=scheduler_take_over5589lb # [ 84.184152] niks3-server[1123]: 2026/09/20 13:33:29 INFO lead: acquired remote=192.168.1.4:502805590worker2 # [ 84.283582] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=scheduler_connected addr=lb:500515591worker2 # [ 84.300144] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=oidc_jwks provider=mock keys=15592worker2 # [ 84.304063] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:434545593worker2 # [ 84.307918] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05594worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5595worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5596client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25597worker2 # [ 85.065873] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544065598client # don't know how to build these paths:5599client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5600worker2 # [ 85.073963] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544065601worker2 # [ 85.091147] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:30Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05602worker2 # [ 85.096375] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:30Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15603worker2 # [ 85.133881] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544065604worker2 # [ 85.144645] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544065605client # copying 1 paths to 'lb:50051'...5606client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5607worker2 # [ 85.155218] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544065608worker2 # [ 85.166893] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5609worker2 # [ 85.343707] nix-daemon[858]: accepted connection from pid 1358, user nix-grpc-daemon (trusted)5610worker2 # [ 85.356582] nix-daemon[858]: reaped child process 1361, status = succeeded5611worker2 # [ 85.508718] nix-daemon[858]: accepted connection from pid 1363, user nix-grpc-daemon (trusted)5612worker2 # [ 85.521132] nix-daemon[858]: reaped child process 1366, status = succeeded5613lb # [ 85.554606] niks3-server[1123]: 2026/09/20 13:33:30 INFO Received uploads request method=POST path=/api/pending_closures5614worker2 # [ 85.544689] nix-grpc-daemon[1318]: time=2026-09-20T13:33:31.068Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5615worker2 # [ 85.546222] nix-grpc-daemon[1318]: time=2026-09-20T13:33:31.068Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5616lb # [ 85.589412] niks3-server[1123]: 2026/09/20 13:33:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5617lb # [ 85.598209] niks3-server[1123]: 2026/09/20 13:33:30 INFO Signed narinfos id=27 count=15618worker2 # [ 85.582981] nix-grpc-daemon[1318]: time=2026-09-20T13:33:31.107Z level=INFO msg="Uploading 1 narinfos"5619lb # [ 85.605056] niks3-server[1123]: 2026/09/20 13:33:30 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5620lb # [ 85.609234] niks3-server[1123]: 2026/09/20 13:33:30 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5621lb # [ 85.621130] niks3-server[1123]: 2026/09/20 13:33:30 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5622worker2 # [ 85.607262] nix-grpc-daemon[1318]: time=2026-09-20T13:33:31.131Z level=INFO msg="Upload complete. (429ms)"5623lb # [ 85.625958] niks3-server[1123]: 2026/09/20 13:33:30 INFO Completed upload id=275624worker2 # [ 85.610073] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=0 paths=1 nar_bytes_in=5205625worker2 # [ 85.612492] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544065626lb # [ 85.632365] niks3-server[1123]: 2026/09/20 13:33:30 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5627worker2 # [ 85.617936] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5628worker2 # [ 85.622483] nix-daemon[858]: reaped child process 1356, status = succeeded5629worker2 # [ 85.627452] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5630client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5631client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5632client # farm-a> LOG-farm-a-on-standby5633worker2 # [ 86.648824] nix-daemon[858]: reaped child process 1370, status = succeeded5634worker2 # [ 86.809199] nix-daemon[858]: accepted connection from pid 1385, user nix-grpc-daemon (trusted)5635worker2 # [ 86.820888] nix-daemon[858]: reaped child process 1388, status = succeeded5636lb # [ 86.855664] niks3-server[1123]: 2026/09/20 13:33:32 INFO Received uploads request method=POST path=/api/pending_closures5637worker2 # [ 86.842688] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.366Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5638worker2 # [ 86.846066] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.368Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5639lb # [ 86.874373] niks3-server[1123]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5640lb # [ 86.877796] niks3-server[1123]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5641lb # [ 86.881865] niks3-server[1123]: 2026/09/20 13:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5642lb # [ 86.883661] niks3-server[1123]: 2026/09/20 13:33:32 INFO Signed narinfos id=28 count=15643worker2 # [ 86.865171] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.389Z level=INFO msg="Uploading 1 narinfos"5644lb # [ 86.888838] niks3-server[1123]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5645lb # [ 86.891815] niks3-server[1123]: 2026/09/20 13:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5646lb # [ 86.895143] niks3-server[1123]: 2026/09/20 13:33:32 INFO Completed upload id=285647worker2 # [ 86.876447] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.400Z level=INFO msg="Upload complete. (242ms)"5648lb # [ 86.896735] niks3-server[1123]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5649worker2 # [ 86.878945] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15650worker2 # [ 86.886813] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:32Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05651worker2 # [ 86.889562] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:32Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=15652worker2 # [ 86.893492] nix-daemon[858]: reaped child process 1368, status = succeeded5653worker2 # [ 86.930360] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544065654worker2 # [ 86.938149] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544065655worker2 # [ 86.947917] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.472Z level=INFO msg="All 1 paths already cached"5656client # copying 1 paths to 'lb:50051'...5657client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5658worker2 # [ 86.954570] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544065659worker2 # [ 86.963133] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5660worker2 # [ 87.122843] nix-daemon[858]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5661worker2 # [ 87.136162] nix-daemon[858]: reaped child process 1397, status = succeeded5662worker2 # [ 87.298466] nix-daemon[858]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5663worker2 # [ 87.311664] nix-daemon[858]: reaped child process 1402, status = succeeded5664lb # [ 87.344144] niks3-server[1123]: 2026/09/20 13:33:32 INFO Received uploads request method=POST path=/api/pending_closures5665worker2 # [ 87.334240] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.858Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5666worker2 # [ 87.336348] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.860Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5667lb # [ 87.368559] niks3-server[1123]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5668lb # [ 87.379454] niks3-server[1123]: 2026/09/20 13:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5669lb # [ 87.386513] niks3-server[1123]: 2026/09/20 13:33:32 INFO Signed narinfos id=29 count=15670worker2 # [ 87.371238] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.895Z level=INFO msg="Uploading 1 narinfos"5671lb # [ 87.390284] niks3-server[1123]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5672lb # [ 87.401974] niks3-server[1123]: 2026/09/20 13:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5673worker2 # [ 87.389441] nix-grpc-daemon[1318]: time=2026-09-20T13:33:32.913Z level=INFO msg="Upload complete. (420ms)"5674lb # [ 87.408236] niks3-server[1123]: 2026/09/20 13:33:32 INFO Completed upload id=295675lb # [ 87.412382] niks3-server[1123]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5676worker2 # [ 87.391588] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=0 paths=1 nar_bytes_in=6405677worker2 # [ 87.397278] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544065678worker2 # [ 87.403908] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5679worker2 # [ 87.409438] nix-daemon[858]: reaped child process 1392, status = succeeded5680worker2 # [ 87.414469] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5681client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5682client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5683client # farm-b> LOG-farm-b-on-standby5684worker2 # [ 88.630171] nix-daemon[858]: reaped child process 1406, status = succeeded5685worker2 # [ 88.777647] nix-daemon[858]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5686worker2 # [ 88.790561] nix-daemon[858]: reaped child process 1424, status = succeeded5687lb # [ 88.826515] niks3-server[1123]: 2026/09/20 13:33:34 INFO Received uploads request method=POST path=/api/pending_closures5688worker2 # [ 88.816247] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.340Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5689worker2 # [ 88.817719] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.342Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5690lb # [ 88.850516] niks3-server[1123]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5691lb # [ 88.855646] niks3-server[1123]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5692lb # [ 88.857215] niks3-server[1123]: 2026/09/20 13:33:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5693lb # [ 88.859185] niks3-server[1123]: 2026/09/20 13:33:34 INFO Signed narinfos id=30 count=15694worker2 # [ 88.841977] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.366Z level=INFO msg="Uploading 1 narinfos"5695lb # [ 88.862982] niks3-server[1123]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5696lb # [ 88.867701] niks3-server[1123]: 2026/09/20 13:33:34 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5697lb # [ 88.871859] niks3-server[1123]: 2026/09/20 13:33:34 INFO Completed upload id=305698worker2 # [ 88.853720] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.377Z level=INFO msg="Upload complete. (231ms)"5699lb # [ 88.873967] niks3-server[1123]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5700worker2 # [ 88.856157] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15701worker2 # [ 88.864537] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:34Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05702worker2 # [ 88.866586] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:34Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=15703worker2 # [ 88.868782] nix-daemon[858]: reaped child process 1404, status = succeeded5704worker2 # [ 88.909169] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544065705lb # [ 88.937120] envoy[984]: conn peer=[2001:db8:1::3]:37308 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1746 tx=428 ms=32195706worker2 # [ 88.918478] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544065707worker2 # [ 88.927520] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.451Z level=INFO msg="All 2 paths already cached"5708client # copying 1 paths to 'lb:50051'...5709client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5710worker2 # [ 88.934544] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544065711worker2 # [ 88.943102] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5712worker2 # [ 89.089079] nix-daemon[858]: accepted connection from pid 1430, user nix-grpc-daemon (trusted)5713worker2 # [ 89.101966] nix-daemon[858]: reaped child process 1433, status = succeeded5714worker2 # [ 89.231320] nix-daemon[858]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5715worker2 # [ 89.244686] nix-daemon[858]: reaped child process 1438, status = succeeded5716lb # [ 89.274917] niks3-server[1123]: 2026/09/20 13:33:34 INFO Received uploads request method=POST path=/api/pending_closures5717worker2 # [ 89.259852] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.784Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5718worker2 # [ 89.261358] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.785Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5719lb # [ 89.292354] niks3-server[1123]: 2026/09/20 13:33:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5720lb # [ 89.294313] niks3-server[1123]: 2026/09/20 13:33:34 INFO Signed narinfos id=31 count=15721worker2 # [ 89.277505] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.801Z level=INFO msg="Uploading 1 narinfos"5722lb # [ 89.298890] niks3-server[1123]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5723lb # [ 89.304121] niks3-server[1123]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5724lb # [ 89.312935] niks3-server[1123]: 2026/09/20 13:33:34 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5725lb # [ 89.317653] niks3-server[1123]: 2026/09/20 13:33:34 INFO Completed upload id=315726worker2 # [ 89.300867] nix-grpc-daemon[1318]: time=2026-09-20T13:33:34.825Z level=INFO msg="Upload complete. (349ms)"5727lb # [ 89.320940] niks3-server[1123]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5728worker2 # [ 89.302885] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=0 paths=1 nar_bytes_in=7685729worker2 # [ 89.306069] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544065730worker2 # [ 89.310576] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5731worker2 # [ 89.315803] nix-daemon[858]: reaped child process 1428, status = succeeded5732client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5733client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5734worker2 # [ 89.321099] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5735client # farm-top> LOG-farm-top-on-standby5736worker2 # [ 89.944448] nix-daemon[858]: reaped child process 1442, status = succeeded5737worker2 # [ 90.085492] nix-daemon[858]: accepted connection from pid 1457, user nix-grpc-daemon (trusted)5738worker2 # [ 90.097767] nix-daemon[858]: reaped child process 1460, status = succeeded5739lb # [ 90.130854] niks3-server[1123]: 2026/09/20 13:33:35 INFO Received uploads request method=POST path=/api/pending_closures5740worker2 # [ 90.116803] nix-grpc-daemon[1318]: time=2026-09-20T13:33:35.640Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5741worker2 # [ 90.118390] nix-grpc-daemon[1318]: time=2026-09-20T13:33:35.642Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5742lb # [ 90.162455] niks3-server[1123]: 2026/09/20 13:33:35 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5743lb # [ 90.174702] niks3-server[1123]: 2026/09/20 13:33:35 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5744lb # [ 90.184120] niks3-server[1123]: 2026/09/20 13:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5745lb # [ 90.189665] niks3-server[1123]: 2026/09/20 13:33:35 INFO Signed narinfos id=32 count=15746worker2 # [ 90.173391] nix-grpc-daemon[1318]: time=2026-09-20T13:33:35.697Z level=INFO msg="Uploading 1 narinfos"5747lb # [ 90.194224] niks3-server[1123]: 2026/09/20 13:33:35 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5748lb # [ 90.205611] niks3-server[1123]: 2026/09/20 13:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5749lb # [ 90.209910] niks3-server[1123]: 2026/09/20 13:33:35 INFO Completed upload id=325750lb # [ 90.212417] niks3-server[1123]: 2026/09/20 13:33:35 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5751worker2 # [ 90.193075] nix-grpc-daemon[1318]: time=2026-09-20T13:33:35.717Z level=INFO msg="Upload complete. (266ms)"5752worker2 # [ 90.200786] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15753worker2 # [ 90.212967] nix-daemon[858]: reaped child process 1440, status = succeeded5754client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 5.38 seconds)5755client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5756client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5757worker1 # [ 90.521623] systemd[1]: Listening on Nix worker-protocol over gRPC.5758worker1 # [ 90.529118] systemd[1]: Starting Nix worker-protocol over gRPC...5759worker1 # [ 90.668442] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5760worker1 # [ 90.670186] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5761worker1 # [ 90.673667] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15762worker1 # [ 90.682453] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5763worker1 # [ 90.686152] systemd[1]: Started Nix worker-protocol over gRPC.5764worker1 # [ 90.697701] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:36Z level=info event=scheduler_connected addr=lb:500515765lb: must succeed: curl -sf localhost:9901/clusters5766worker2 # [ 90.678180] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:434545767worker2 # [ 90.681959] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05768lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5769worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5770worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5771lb: must succeed: curl -sf localhost:9901/clusters5772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5773lb: must succeed: curl -sf localhost:9901/clusters5774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5775worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5776worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5777(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.17 seconds)5778subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5779client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5780client # [ 91.010784] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5781client # Running as unit: rs.service; invocation ID: 75efcb295420477ba6d19db9bc70db475782client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5783worker1 # [ 91.203623] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:346585784worker1 # [ 91.212890] nix-daemon[901]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5785client # [ 91.242993] nix[1164]: don't know how to build these paths:5786client # [ 91.243874] nix[1164]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5787worker2 # [ 91.219538] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:36Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05788worker2 # [ 91.223446] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:36Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=15789worker2 # [ 91.261942] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544065790worker2 # [ 91.275202] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544065791client # [ 91.320177] nix[1164]: copying 1 paths to 'lb:50051'...5792worker2 # [ 91.284475] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544065793client # [ 91.322769] nix[1164]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5794worker2 # [ 91.297219] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5795worker2 # [ 91.466232] nix-daemon[858]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5796worker2 # [ 91.479480] nix-daemon[858]: reaped child process 1476, status = succeeded5797worker2 # [ 91.641207] nix-daemon[858]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)5798worker2 # [ 91.653338] nix-daemon[858]: reaped child process 1481, status = succeeded5799lb # [ 91.685763] niks3-server[1123]: 2026/09/20 13:33:36 INFO Received uploads request method=POST path=/api/pending_closures5800worker2 # [ 91.672930] nix-grpc-daemon[1318]: time=2026-09-20T13:33:37.196Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5801worker2 # [ 91.674358] nix-grpc-daemon[1318]: time=2026-09-20T13:33:37.196Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5802lb # [ 91.702611] niks3-server[1123]: 2026/09/20 13:33:37 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5803lb # [ 91.704338] niks3-server[1123]: 2026/09/20 13:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5804lb # [ 91.706394] niks3-server[1123]: 2026/09/20 13:33:37 INFO Signed narinfos id=33 count=15805worker2 # [ 91.688993] nix-grpc-daemon[1318]: time=2026-09-20T13:33:37.213Z level=INFO msg="Uploading 1 narinfos"5806lb # [ 91.709886] niks3-server[1123]: 2026/09/20 13:33:37 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5807lb # [ 91.714947] niks3-server[1123]: 2026/09/20 13:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5808lb # [ 91.718260] niks3-server[1123]: 2026/09/20 13:33:37 INFO Completed upload id=335809worker2 # [ 91.700785] nix-grpc-daemon[1318]: time=2026-09-20T13:33:37.225Z level=INFO msg="Upload complete. (391ms)"5810lb # [ 91.719829] niks3-server[1123]: 2026/09/20 13:33:37 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5811worker2 # [ 91.702816] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=0 paths=1 nar_bytes_in=4165812worker2 # [ 91.705748] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544065813worker2 # [ 91.711543] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5814worker2 # [ 91.714458] nix-daemon[858]: reaped child process 1471, status = succeeded5815client # [ 91.755294] nix[1164]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5816worker2 # [ 91.719313] nix-daemon[858]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5817lb: must succeed: curl -sf localhost:9901/clusters5818lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5819lb: must succeed: curl -sf localhost:9901/clusters5820lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5821lb: must succeed: curl -sf localhost:9901/clusters5822lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5823lb: must succeed: curl -sf localhost:9901/clusters5824lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5825worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5826worker1 # [ 93.893076] systemd-journald[287]: Received client request to rotate journal, rotating.5827worker1 # [ 93.914445] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/3ea75aa9261142578d8a26ac77473da4.5828worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5829worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5830worker2 # [ 93.798390] systemd[1]: Stopping Nix worker-protocol over gRPC...5831worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5832??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5833 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735834client: must succeed: sleep 5.05835worker2 # [ 94.156346] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:39Z level=info event=draining5836client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5837lb # [ 98.938987] envoy[984]: conn peer=[2001:db8:1::1]:57988 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9560 tx=3824 ms=51465838worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5839worker2 # [ 99.143303] nix-daemon[858]: reaped child process 1485, status = succeeded5840worker2 # [ 99.287169] nix-daemon[858]: accepted connection from pid 1526, user nix-grpc-daemon (trusted)5841worker2 # [ 99.299580] nix-daemon[858]: reaped child process 1529, status = succeeded5842lb # [ 99.331539] niks3-server[1123]: 2026/09/20 13:33:44 INFO Received uploads request method=POST path=/api/pending_closures5843worker2 # [ 99.323522] nix-grpc-daemon[1318]: time=2026-09-20T13:33:44.847Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5844worker2 # [ 99.327238] nix-grpc-daemon[1318]: time=2026-09-20T13:33:44.849Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5845lb # [ 99.382563] niks3-server[1123]: 2026/09/20 13:33:44 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5846lb # [ 99.389858] niks3-server[1123]: 2026/09/20 13:33:44 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5847lb # [ 99.395813] niks3-server[1123]: 2026/09/20 13:33:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5848lb # [ 99.399604] niks3-server[1123]: 2026/09/20 13:33:44 INFO Signed narinfos id=34 count=15849worker2 # [ 99.385765] nix-grpc-daemon[1318]: time=2026-09-20T13:33:44.909Z level=INFO msg="Uploading 1 narinfos"5850lb # [ 99.410521] niks3-server[1123]: 2026/09/20 13:33:44 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5851lb # [ 99.413386] niks3-server[1123]: 2026/09/20 13:33:44 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5852lb # [ 99.418753] niks3-server[1123]: 2026/09/20 13:33:44 INFO Completed upload id=345853worker2 # [ 99.400172] nix-grpc-daemon[1318]: time=2026-09-20T13:33:44.924Z level=INFO msg="Upload complete. (266ms)"5854lb # [ 99.420814] niks3-server[1123]: 2026/09/20 13:33:44 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5855worker2 # [ 99.403340] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54406 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15856worker2 # [ 99.411722] nix-daemon[858]: reaped child process 1483, status = succeeded5857client # [ 99.472653] systemd[1]: rs.service: Deactivated successfully.5858client # [ 99.474285] systemd[1]: rs.service: Consumed 111ms CPU time over 8.463s wall clock time, 39.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5859worker2 # [ 100.160302] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=signal_exit5860worker2 # [ 100.186738] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=scheduler_yield5861lb # [ 100.208534] niks3-server[1123]: 2026/09/20 13:33:45 INFO lead: released remote=192.168.1.4:502805862worker1 # [ 100.212356] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_restarting addr=lb:500515863worker2 # [ 100.190313] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=scheduler_restarting addr=lb:500515864worker2 # [ 100.194459] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:434545865worker2 # [ 100.197788] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=worker_gone addr=192.168.1.4:500515866worker2 # [ 100.200887] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:434545867worker2 # [ 100.204236] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=worker_gone addr=192.168.1.3:500515868worker1 # [ 100.412485] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500515869worker2 # [ 100.391886] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=scheduler_connected addr=lb:500515870worker1 # [ 100.414461] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5871worker2 # [ 100.396239] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5872worker1 # [ 100.615570] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500515873worker2 # [ 100.598700] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500515874worker1 # [ 100.619703] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5875worker2 # [ 100.601824] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5876lb # [ 100.690050] niks3-server[1123]: 2026/09/20 13:33:45 INFO lead: acquired remote=192.168.1.3:571865877worker1 # [ 100.692113] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_take_over5878worker2 # [ 100.799811] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500515879worker1 # [ 100.820479] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500515880worker2 # [ 100.803652] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5881worker1 # [ 100.824533] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5882worker2 # [ 101.004554] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500515883worker1 # [ 101.025434] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_connected addr=lb:500515884worker2 # [ 101.007880] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5885worker1 # [ 101.029430] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5886worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.13 seconds)5887client: waiting for success: ! systemctl is-active rs5888worker2 # [ 101.188073] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:46Z level=info event=scheduler_restarting5889worker1 # [ 101.231415] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500515890worker1 # [ 101.232785] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5891worker2 # [ 101.215669] nix-daemon[858]: reaped child process 1311, status = succeeded5892worker2 # [ 101.224199] nix-grpc-daemon[1262]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5893client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5894client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5895worker2 # [ 101.227522] nix-grpc-daemon[1262]: E0000 00:00:1789911226.736770 1305 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5896worker2 # [ 101.232196] nix-grpc-daemon[1262]: E0000 00:00:1789911226.738310 1305 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5897worker2 # [ 101.236870] nix-grpc-daemon[1262]: ts=2026-09-20T13:33:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5898client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5899client: must fail: journalctl -u rs | grep -q 'reconnecting'5900client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5901lb: must succeed: curl -sf localhost:9901/clusters5902lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5903worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5904worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5905worker1 # [ 101.432108] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500515906worker1 # [ 101.434049] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5907worker1 # [ 101.634915] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500515908worker1 # [ 101.636956] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5909worker1 # [ 101.838233] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=scheduler_connected addr=lb:500515910worker1 # [ 101.846141] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:345385911worker1 # [ 101.850447] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05912lb: must succeed: curl -sf localhost:9901/clusters5913lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5914worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5915worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5916worker2 # [ 102.807483] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5917worker2 # [ 102.809496] systemd[1]: Stopped Nix worker-protocol over gRPC.5918worker2 # [ 102.810519] systemd[1]: nix-grpc-daemon.service: Consumed 1.403s CPU time over 28.808s wall clock time, 33.3M memory peak, 62.5K incoming IP traffic, 157.3K outgoing IP traffic.5919worker2 # [ 102.818714] systemd[1]: Starting scheduler-token.service...5920worker2 # [ 102.874919] systemd[1]: scheduler-token.service: Deactivated successfully.5921worker2 # [ 102.876555] systemd[1]: Finished scheduler-token.service.5922worker2 # [ 102.877869] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 57ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5923worker2 # [ 102.882712] systemd[1]: Starting Nix worker-protocol over gRPC...5924worker2 # [ 103.030350] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5925worker2 # [ 103.032239] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5926worker2 # [ 103.036147] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15927worker2 # [ 103.042916] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5928worker2 # [ 103.045845] systemd[1]: Started Nix worker-protocol over gRPC.5929worker2 # [ 103.057535] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:48Z level=info event=scheduler_connected addr=lb:500515930worker1 # [ 103.085543] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:48Z level=info event=oidc_jwks provider=mock keys=15931worker1 # [ 103.087327] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:345385932worker1 # [ 103.088922] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05933lb: must succeed: curl -sf localhost:9901/clusters5934lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5935worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5936worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5937(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.73 seconds)5938subtest: scheduler restart mid-build: build finishes, no double build5939client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5940client # [ 103.723044] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5941client # Running as unit: mid.service; invocation ID: 93bf6f45107645aa9531fce9504af8ee5942client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.04 seconds)5943worker1 # [ 103.909489] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:346585944client # [ 103.929434] nix[1199]: don't know how to build these paths:5945client # [ 103.930566] nix[1199]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5946worker1 # [ 103.920912] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:49Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05947worker1 # [ 103.923206] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:49Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15948worker1 # [ 103.967249] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346585949worker1 # [ 103.976563] nix-grpc-daemon[1164]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)5950worker1 # [ 104.000236] nix-grpc-daemon[1164]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5)5951worker1 # [ 104.179986] nix-grpc-daemon[1164]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 266 ms (attempt 3/5)5952worker1 # [ 104.446696] nix-grpc-daemon[1164]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 4/5)5953worker1 # [ 104.785333] nix-grpc-daemon[1164]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5954worker1 # [ 104.805135] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346585955client # [ 104.831985] nix[1199]: copying 1 paths to 'lb:50051'...5956client # [ 104.836869] nix[1199]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5957worker1 # [ 104.821146] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346585958worker1 # [ 104.831689] nix-daemon[901]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5959worker1 # [ 104.863626] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:50Z level=info event=niks3_push_started pid=12885960worker1 # [ 105.036829] nix-grpc-daemon[1288]: time=2026-09-20T13:33:50.884Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5961worker1 # [ 105.194154] nix-daemon[901]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5962worker1 # [ 105.210079] nix-daemon[901]: reaped child process 1305, status = succeeded5963worker1 # [ 105.370238] nix-daemon[901]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5964worker1 # [ 105.384590] nix-daemon[901]: reaped child process 1310, status = succeeded5965lb # [ 105.396422] niks3-server[1123]: 2026/09/20 13:33:50 INFO Received uploads request method=POST path=/api/pending_closures5966worker1 # [ 105.404172] nix-grpc-daemon[1288]: time=2026-09-20T13:33:51.253Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5967worker1 # [ 105.405827] nix-grpc-daemon[1288]: time=2026-09-20T13:33:51.254Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5968lb # [ 105.418671] niks3-server[1123]: 2026/09/20 13:33:50 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5969lb # [ 105.421529] niks3-server[1123]: 2026/09/20 13:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5970lb # [ 105.423505] niks3-server[1123]: 2026/09/20 13:33:50 INFO Signed narinfos id=35 count=15971worker1 # [ 105.426755] nix-grpc-daemon[1288]: time=2026-09-20T13:33:51.275Z level=INFO msg="Uploading 1 narinfos"5972lb # [ 105.427432] niks3-server[1123]: 2026/09/20 13:33:50 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5973lb # [ 105.432401] niks3-server[1123]: 2026/09/20 13:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5974lb # [ 105.436204] niks3-server[1123]: 2026/09/20 13:33:50 INFO Completed upload id=355975worker1 # [ 105.438328] nix-grpc-daemon[1288]: time=2026-09-20T13:33:51.287Z level=INFO msg="Upload complete. (402ms)"5976worker1 # [ 105.442276] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34658 duration_s=0 paths=1 nar_bytes_in=4245977worker1 # [ 105.444739] nix-grpc-daemon[1164]: ts=2026-09-20T13:33:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346585978worker1 # [ 105.446662] nix-grpc-daemon[1288]: time=2026-09-20T13:33:51.295Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"5979worker1 # [ 105.454801] nix-daemon[901]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5980worker1 # [ 105.457166] nix-daemon[901]: reaped child process 1285, status = succeeded5981client # [ 105.477391] nix[1199]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5982worker1 # [ 105.461872] nix-daemon[901]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5983lb: must succeed: curl -sf localhost:9901/clusters5984lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5985lb: must succeed: curl -sf localhost:9901/clusters5986lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5987worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5988worker1 # [ 107.379311] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1164 (nix-grpc-daemon) on client request.5989worker1 # [ 107.381864] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5990lb # [ 107.386814] niks3-server[1123]: 2026/09/20 13:33:52 INFO lead: released remote=192.168.1.3:571865991worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5992client: must succeed: sleep 5.05993worker2 # [ 107.376300] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5994client # [ 107.413827] nix[1199]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5995client # [ 107.419568] nix[1199]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5996worker1 # [ 107.410610] nix-daemon[901]: reaped child process 1312, status = succeeded5997worker1 # [ 107.414413] nix-daemon[901]: reaped child process 1220, status = succeeded5998worker1 # [ 107.415486] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5999worker1 # [ 107.417161] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6000worker1 # [ 107.419167] systemd[1]: nix-grpc-daemon.service: Consumed 453ms CPU time over 16.884s wall clock time, 31.1M memory peak, 17.1K incoming IP traffic, 25K outgoing IP traffic.6001worker1 # [ 107.427249] nix-daemon[1314]: unexpected Nix daemon error: error: interrupted by the user6002worker1 # [ 107.430976] nix-daemon[901]: reaped child process 1314, status = failed with exit code 16003worker2 # [ 107.576862] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500516004worker1 # [ 107.601314] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6005worker1 # [ 107.610672] systemd[1]: Starting Nix worker-protocol over gRPC...6006worker1 # [ 107.769391] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6007worker1 # [ 107.771242] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6008worker1 # [ 107.775141] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16009worker1 # [ 107.781523] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6010worker1 # [ 107.783198] systemd[1]: Started Nix worker-protocol over gRPC.6011lb # [ 107.792836] niks3-server[1123]: 2026/09/20 13:33:53 INFO lead: acquired remote=192.168.1.3:545526012worker1 # [ 107.799170] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=scheduler_take_over6013worker1 # [ 107.807645] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500516014worker1 # [ 107.809290] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6015worker1 # [ 107.811880] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=oidc_jwks provider=mock keys=16016worker1 # [ 107.813195] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:598866017worker1 # [ 107.814803] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06018client # [ 107.920790] nix[1199]: scheduler at lb:50051: no healthy upstream, reconnecting6019worker1 # [ 108.010363] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=scheduler_connected addr=lb:500516020worker1 # [ 108.014337] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6021worker1 # [ 108.215071] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=scheduler_connected addr=lb:500516022worker1 # [ 108.218984] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6023worker1 # [ 108.419619] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=scheduler_connected addr=lb:500516024worker1 # [ 108.423610] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6025worker1 # [ 108.624403] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=scheduler_connected addr=lb:500516026worker1 # [ 108.628228] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6027worker1 # [ 108.829527] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=scheduler_connected addr=lb:500516028worker1 # [ 108.833455] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:598866029worker1 # [ 108.837519] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06030worker1 # [ 108.912752] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06031worker1 # [ 108.918143] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:54Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16032lb # [ 108.937939] envoy[984]: conn peer=[2001:db8:1::1]:57990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1326 ms=82346033lb # [ 108.939503] envoy[984]: conn peer=[2001:db8:1::4]:46516 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=36544 tx=2039 ms=270466034lb # [ 108.940943] envoy[984]: conn peer=[2001:db8:1::3]:37326 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2187 tx=567 ms=167016035worker2 # [ 108.943492] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344626036worker2 # [ 108.955279] nix-daemon[858]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6037worker2 # [ 108.974617] nix-grpc-daemon[1539]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)6038worker2 # [ 108.994773] nix-grpc-daemon[1539]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)6039worker2 # [ 109.036868] nix-grpc-daemon[1539]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 400 ms (attempt 3/5)6040worker2 # [ 109.442905] nix-grpc-daemon[1539]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 4/5)6041worker2 # [ 109.568312] nix-grpc-daemon[1539]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6042worker2 # [ 109.585156] nix-grpc-daemon[1539]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6043worker2 # [ 109.648760] nix-daemon[858]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6044client # [ 109.701428] nix[1199]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6045worker2 # [ 109.665141] nix-daemon[858]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6046client: (finished: must succeed: sleep 5.0, in 5.06 seconds)6047client: waiting for success: ! systemctl is-active mid6048worker2 # [ 112.682895] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:58Z level=info event=niks3_push_started pid=16406049worker2 # [ 112.695831] nix-daemon[858]: reaped child process 1616, status = succeeded6050worker2 # [ 112.859910] nix-grpc-daemon[1640]: time=2026-09-20T13:33:58.383Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6051worker2 # [ 113.020317] nix-daemon[858]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)6052worker2 # [ 113.032295] nix-daemon[858]: reaped child process 1656, status = succeeded6053lb # [ 113.065131] niks3-server[1123]: 2026/09/20 13:33:58 INFO Received uploads request method=POST path=/api/pending_closures6054worker2 # [ 113.049928] nix-grpc-daemon[1640]: time=2026-09-20T13:33:58.574Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6055worker2 # [ 113.051496] nix-grpc-daemon[1640]: time=2026-09-20T13:33:58.575Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6056lb # [ 113.107139] niks3-server[1123]: 2026/09/20 13:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6057lb # [ 113.125751] niks3-server[1123]: 2026/09/20 13:33:58 INFO Signed narinfos id=36 count=16058worker2 # [ 113.107333] nix-grpc-daemon[1640]: time=2026-09-20T13:33:58.631Z level=INFO msg="Uploading 1 narinfos"6059lb # [ 113.138457] niks3-server[1123]: 2026/09/20 13:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6060lb # [ 113.221849] niks3-server[1123]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6061lb # [ 113.232977] niks3-server[1123]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6062lb # [ 113.238161] niks3-server[1123]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6063lb # [ 113.246754] niks3-server[1123]: 2026/09/20 13:33:58 INFO Completed upload id=366064worker2 # [ 113.228592] nix-grpc-daemon[1640]: time=2026-09-20T13:33:58.752Z level=INFO msg="Upload complete. (364ms)"6065lb # [ 113.251281] niks3-server[1123]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6066worker2 # [ 113.236159] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34462 duration_s=4 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16067worker2 # [ 113.255266] nix-daemon[858]: reaped child process 1614, status = succeeded6068client # [ 113.317196] systemd[1]: mid.service: Deactivated successfully.6069client # [ 113.320866] systemd[1]: mid.service: Consumed 138ms CPU time over 9.596s wall clock time, 39.1M memory peak, 9.9K incoming IP traffic, 10.9K outgoing IP traffic.6070client: (finished: waiting for success: ! systemctl is-active mid, in 1.10 seconds)6071client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6072client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6073(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.13 seconds)6074subtest: interrupting the client stops the build on the worker6075client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6076client # [ 113.847452] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6077client # Running as unit: intr.service; invocation ID: e25b74fe0e0e4e8ca685f4f37d2fbcb76078client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.03 seconds)6079client # [ 114.038543] nix[1234]: don't know how to build these paths:6080worker2 # [ 114.000983] nix-grpc-daemon[1539]: ts=2026-09-20T13:33:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:344626081client # [ 114.039435] nix[1234]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6082worker1 # [ 114.034480] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:59Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06083worker1 # [ 114.038737] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:59Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16084worker1 # [ 114.085170] nix-grpc-daemon[1350]: ts=2026-09-20T13:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346085worker1 # [ 114.097092] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6086worker1 # [ 114.115624] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5)6087worker1 # [ 114.212959] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 142 ms (attempt 2/5)6088worker1 # [ 114.362333] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 215 ms (attempt 3/5)6089worker1 # [ 114.584138] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 542 ms (attempt 4/5)6090worker1 # [ 115.132087] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6091worker1 # [ 115.149809] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346092client # [ 115.177834] nix[1234]: copying 1 paths to 'lb:50051'...6093client # [ 115.182107] nix[1234]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6094worker1 # [ 115.165959] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346095worker1 # [ 115.178993] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6096worker1 # [ 115.249901] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:01Z level=info event=niks3_push_started pid=14266097worker1 # [ 115.474337] nix-grpc-daemon[1426]: time=2026-09-20T13:34:01.322Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6098worker1 # [ 115.628070] nix-daemon[901]: accepted connection from pid 1439, user nix-grpc-daemon (trusted)6099worker1 # [ 115.639422] nix-daemon[901]: reaped child process 1442, status = succeeded6100worker1 # [ 115.804203] nix-daemon[901]: accepted connection from pid 1444, user nix-grpc-daemon (trusted)6101worker1 # [ 115.816528] nix-daemon[901]: reaped child process 1447, status = succeeded6102lb # [ 115.832614] niks3-server[1123]: 2026/09/20 13:34:01 INFO Received uploads request method=POST path=/api/pending_closures6103worker1 # [ 115.851240] nix-grpc-daemon[1426]: time=2026-09-20T13:34:01.700Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6104worker1 # [ 115.854651] nix-grpc-daemon[1426]: time=2026-09-20T13:34:01.701Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6105lb # [ 115.879402] niks3-server[1123]: 2026/09/20 13:34:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6106lb # [ 115.884446] niks3-server[1123]: 2026/09/20 13:34:01 INFO Signed narinfos id=37 count=16107worker1 # [ 115.889106] nix-grpc-daemon[1426]: time=2026-09-20T13:34:01.737Z level=INFO msg="Uploading 1 narinfos"6108lb # [ 115.895686] niks3-server[1123]: 2026/09/20 13:34:01 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6109lb # [ 116.174460] niks3-server[1123]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6110lb # [ 116.197239] niks3-server[1123]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6111worker1 # [ 116.214755] nix-grpc-daemon[1426]: time=2026-09-20T13:34:02.063Z level=INFO msg="Upload complete. (737ms)"6112lb # [ 116.212573] niks3-server[1123]: 2026/09/20 13:34:01 INFO Completed upload id=376113worker1 # [ 116.218326] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 paths=1 nar_bytes_in=4246114worker1 # [ 116.223661] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346115lb # [ 116.227211] niks3-server[1123]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6116worker1 # [ 116.239960] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6117worker1 # [ 116.243182] nix-daemon[901]: reaped child process 1424, status = succeeded6118client # [ 116.266628] nix[1234]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6119worker1 # [ 116.252486] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6120client: must succeed: systemctl kill -s INT intr6121client # [ 118.528331] systemd[1]: intr.service: Sent signal SIGINT to main process 1234 (nix) on client request.6122client # [ 118.531480] nix[1234]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6123client: (finished: must succeed: systemctl kill -s INT intr, in 0.03 seconds)6124client # [ 118.535630] nix[1234]: error: interrupted by the user6125client # [ 118.562974] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6126client # [ 118.564508] systemd[1]: intr.service: Failed with result 'exit-code'.6127client # [ 118.566726] systemd[1]: intr.service: Consumed 102ms CPU time over 4.718s wall clock time, 39.2M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.6128worker1 # [ 118.673364] nix-daemon[901]: reaped child process 1453, status = succeeded6129worker1 # [ 118.675407] nix-daemon[1455]: unexpected Nix daemon error: error: interrupted by the user6130worker1 # [ 118.679685] nix-daemon[901]: reaped child process 1455, status = failed with exit code 16131lb # [ 118.938182] envoy[984]: conn peer=[2001:db8:1::1]:56488 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4020 tx=1755 ms=93716132lb # [ 118.942212] envoy[984]: conn peer=[2001:db8:1::1]:36610 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=755 ms=45056133(finished: subtest: interrupting the client stops the build on the worker, in 5.97 seconds)6134subtest: a deploy mid-build drains: the build finishes, the new generation takes over6135client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6136client # [ 119.829973] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6137client # Running as unit: sw.service; invocation ID: bc944c0a93c24f77bd989ff92c467aee6138client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.05 seconds)6139client # [ 120.036130] nix[1254]: don't know how to build these paths:6140client # [ 120.037643] nix[1254]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6141worker1 # [ 120.020309] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429346142worker1 # [ 120.032068] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:05Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06143worker1 # [ 120.036440] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:05Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16144worker2 # [ 120.056373] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344626145worker2 # [ 120.065988] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344626146client # [ 120.111242] nix[1254]: copying 1 paths to 'lb:50051'...6147worker2 # [ 120.075300] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344626148client # [ 120.113429] nix[1254]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6149worker2 # [ 120.087683] nix-daemon[858]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6150worker2 # [ 120.255664] nix-daemon[858]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6151worker2 # [ 120.269133] nix-daemon[858]: reaped child process 1695, status = succeeded6152worker2 # [ 120.437648] nix-daemon[858]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6153worker2 # [ 120.450735] nix-daemon[858]: reaped child process 1700, status = succeeded6154lb # [ 120.484124] niks3-server[1123]: 2026/09/20 13:34:05 INFO Received uploads request method=POST path=/api/pending_closures6155worker2 # [ 120.469974] nix-grpc-daemon[1640]: time=2026-09-20T13:34:05.994Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6156worker2 # [ 120.471985] nix-grpc-daemon[1640]: time=2026-09-20T13:34:05.995Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6157lb # [ 120.500736] niks3-server[1123]: 2026/09/20 13:34:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6158lb # [ 120.502658] niks3-server[1123]: 2026/09/20 13:34:05 INFO Signed narinfos id=38 count=16159worker2 # [ 120.484258] nix-grpc-daemon[1640]: time=2026-09-20T13:34:06.008Z level=INFO msg="Uploading 1 narinfos"6160lb # [ 120.508854] niks3-server[1123]: 2026/09/20 13:34:05 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6161lb # [ 120.722921] niks3-server[1123]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6162lb # [ 120.736315] niks3-server[1123]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6163lb # [ 120.743468] niks3-server[1123]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6164lb # [ 120.756843] niks3-server[1123]: 2026/09/20 13:34:06 INFO Completed upload id=386165worker2 # [ 120.738491] nix-grpc-daemon[1640]: time=2026-09-20T13:34:06.262Z level=INFO msg="Upload complete. (639ms)"6166worker2 # [ 120.743140] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34462 duration_s=0 paths=1 nar_bytes_in=4166167worker2 # [ 120.748216] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344626168worker2 # [ 120.760134] nix-daemon[858]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6169worker2 # [ 120.772282] nix-daemon[858]: reaped child process 1690, status = succeeded6170client # [ 120.814442] nix[1254]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6171worker2 # [ 120.779550] nix-daemon[858]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6172worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26173worker2 # Checking switch inhibitors... done6174worker2 # [ 122.884037] nixos[1735]: switching to system configuration /nix/store/08fjjr0ri2w3dv65p591rq8il7pbnsmc-nixos-system-worker2-test6175worker2 # [ 122.885790] systemd[1]: Stopped target Remote File Systems.6176worker2 # activating the configuration...6177worker2 # [ 122.888805] systemd[1]: Stopped target Local File Systems.6178worker2 # setting up /etc...6179worker2 # [ 123.530653] systemd[1]: Reload requested from client PID 1735 ('.switch-to-conf') (unit backdoor.service)...6180worker2 # [ 123.532226] systemd[1]: Reloading...6181worker2 # [ 124.201381] systemd[1]: Reloading finished in 664 ms.6182worker2 # restarting sysinit-reactivation.target6183worker2 # [ 124.256083] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6184worker2 # [ 124.381236] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6185worker2 # [ 124.384481] systemd[1]: Reached target Reactivate sysinit units.6186worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6187worker2 # [ 124.389281] systemd[1]: Reloading D-Bus System Message Bus...6188worker2 # [ 124.390594] dbus-broker-launch[535]: Caught SIGHUP, trigger reload.6189worker2 # [ 124.394678] systemd[1]: Reloading Nix worker-protocol over gRPC...6190worker2 # [ 124.423730] dbus-broker-launch[535]: Looking up NSS user entry for 'systemd-timesync'...6191worker2 # [ 124.426374] dbus-broker-launch[535]: NSS returned no entry for 'systemd-timesync'6192worker2 # [ 124.427691] dbus-broker-launch[535]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6193worker2 # [ 124.438597] systemd[1]: Reloaded Nix worker-protocol over gRPC.6194worker2 # [ 124.460395] systemd[1]: Reloaded D-Bus System Message Bus.6195worker2 # [ 124.463930] systemd[1]: Reached target Remote File Systems.6196worker2 # [ 124.466830] systemd[1]: Reached target Local File Systems.6197worker2 # [ 124.469134] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6198worker2 # [ 124.473158] systemd[1]: Starting Create SUID/SGID Wrappers...6199worker2 # [ 124.473974] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6200worker2 # [ 124.476191] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676201worker2 # [ 124.481040] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6202worker2 # [ 124.482500] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6203worker2 # [ 124.484270] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6204worker2 # [ 124.485589] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6205worker2 # [ 124.487304] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6206worker2 # [ 124.907087] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6207worker2 # [ 124.910174] systemd[1]: Finished Create SUID/SGID Wrappers.6208worker2 # [ 124.913694] systemd[1]: Starting linger-users.service...6209worker2 # [ 124.957842] systemd[1]: linger-users.service: Deactivated successfully.6210worker2 # [ 124.959178] systemd[1]: Finished linger-users.service.6211worker2 # [ 125.051227] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:10Z level=info event=draining6212worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6213worker2 # [ 125.214650] nixos[1735]: finished switching to system configuration /nix/store/08fjjr0ri2w3dv65p591rq8il7pbnsmc-nixos-system-worker2-test6214worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.93 seconds)6215worker2: must succeed: pgrep -f 'read -t [9]0 x'6216worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6217client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26218worker1 # [ 125.513884] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429346219client # don't know how to build these paths:6220client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6221worker1 # [ 125.521733] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346222worker1 # [ 125.537869] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:11Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06223worker1 # [ 125.542936] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:11Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16224worker1 # [ 125.583290] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346225worker1 # [ 125.592158] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346226client # copying 1 paths to 'lb:50051'...6227worker1 # [ 125.598488] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346228client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6229worker1 # [ 125.616423] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6230worker1 # [ 125.780094] nix-daemon[901]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)6231worker1 # [ 125.793150] nix-daemon[901]: reaped child process 1512, status = succeeded6232worker1 # [ 125.963392] nix-daemon[901]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6233worker1 # [ 125.975264] nix-daemon[901]: reaped child process 1517, status = succeeded6234lb # [ 125.990708] niks3-server[1123]: 2026/09/20 13:34:11 INFO Received uploads request method=POST path=/api/pending_closures6235worker1 # [ 126.103744] nix-grpc-daemon[1426]: time=2026-09-20T13:34:11.951Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6236worker1 # [ 126.107753] nix-grpc-daemon[1426]: time=2026-09-20T13:34:11.951Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6237lb # [ 126.133535] niks3-server[1123]: 2026/09/20 13:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6238lb # [ 126.139643] niks3-server[1123]: 2026/09/20 13:34:11 INFO Signed narinfos id=39 count=16239worker1 # [ 126.145296] nix-grpc-daemon[1426]: time=2026-09-20T13:34:11.993Z level=INFO msg="Uploading 1 narinfos"6240lb # [ 126.142877] niks3-server[1123]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6241lb # [ 126.157478] niks3-server[1123]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6242lb # [ 126.164070] niks3-server[1123]: 2026/09/20 13:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6243lb # [ 126.174379] niks3-server[1123]: 2026/09/20 13:34:11 INFO Completed upload id=396244lb # [ 126.177136] niks3-server[1123]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6245worker1 # [ 126.178959] nix-grpc-daemon[1426]: time=2026-09-20T13:34:12.027Z level=INFO msg="Upload complete. (557ms)"6246worker1 # [ 126.184151] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=5286247worker1 # [ 126.189851] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346248worker1 # [ 126.203142] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6249worker1 # [ 126.211948] nix-daemon[901]: reaped child process 1507, status = succeeded6250client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6251client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6252worker1 # [ 126.225352] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6253client # farm-a> LOG-farm-a-during-drain6254worker1 # [ 127.657916] nix-daemon[901]: reaped child process 1521, status = succeeded6255worker1 # [ 127.797319] nix-daemon[901]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6256worker1 # [ 127.809496] nix-daemon[901]: reaped child process 1539, status = succeeded6257lb # [ 127.820367] niks3-server[1123]: 2026/09/20 13:34:13 INFO Received uploads request method=POST path=/api/pending_closures6258worker1 # [ 127.830307] nix-grpc-daemon[1426]: time=2026-09-20T13:34:13.678Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6259worker1 # [ 127.831900] nix-grpc-daemon[1426]: time=2026-09-20T13:34:13.680Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6260lb # [ 127.860198] niks3-server[1123]: 2026/09/20 13:34:13 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6261lb # [ 127.867956] niks3-server[1123]: 2026/09/20 13:34:13 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6262lb # [ 127.878642] niks3-server[1123]: 2026/09/20 13:34:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6263lb # [ 127.884380] niks3-server[1123]: 2026/09/20 13:34:13 INFO Signed narinfos id=40 count=16264worker1 # [ 127.886928] nix-grpc-daemon[1426]: time=2026-09-20T13:34:13.735Z level=INFO msg="Uploading 1 narinfos"6265lb # [ 127.887914] niks3-server[1123]: 2026/09/20 13:34:13 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6266lb # [ 127.897802] niks3-server[1123]: 2026/09/20 13:34:13 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6267lb # [ 127.902955] niks3-server[1123]: 2026/09/20 13:34:13 INFO Completed upload id=406268worker1 # [ 127.905240] nix-grpc-daemon[1426]: time=2026-09-20T13:34:13.754Z level=INFO msg="Upload complete. (268ms)"6269lb # [ 127.905320] niks3-server[1123]: 2026/09/20 13:34:13 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6270worker1 # [ 127.910794] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16271worker1 # [ 127.920457] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:13Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06272worker1 # [ 127.924461] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:13Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16273worker1 # [ 127.931986] nix-daemon[901]: reaped child process 1519, status = succeeded6274worker1 # [ 127.965289] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346275worker1 # [ 127.982077] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346276worker1 # [ 127.993912] nix-grpc-daemon[1426]: time=2026-09-20T13:34:13.842Z level=INFO msg="All 1 paths already cached"6277client # copying 1 paths to 'lb:50051'...6278worker1 # [ 128.001372] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346279client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6280worker1 # [ 128.020134] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6281worker1 # [ 128.199196] nix-daemon[901]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6282worker1 # [ 128.211563] nix-daemon[901]: reaped child process 1548, status = succeeded6283worker1 # [ 128.375077] nix-daemon[901]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6284worker1 # [ 128.387332] nix-daemon[901]: reaped child process 1553, status = succeeded6285lb # [ 128.401509] niks3-server[1123]: 2026/09/20 13:34:13 INFO Received uploads request method=POST path=/api/pending_closures6286worker1 # [ 128.412495] nix-grpc-daemon[1426]: time=2026-09-20T13:34:14.261Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6287worker1 # [ 128.413908] nix-grpc-daemon[1426]: time=2026-09-20T13:34:14.261Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6288lb # [ 128.425797] niks3-server[1123]: 2026/09/20 13:34:13 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6289lb # [ 128.437899] niks3-server[1123]: 2026/09/20 13:34:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6290worker1 # [ 128.445125] nix-grpc-daemon[1426]: time=2026-09-20T13:34:14.293Z level=INFO msg="Uploading 1 narinfos"6291lb # [ 128.443069] niks3-server[1123]: 2026/09/20 13:34:13 INFO Signed narinfos id=41 count=16292lb # [ 128.447985] niks3-server[1123]: 2026/09/20 13:34:13 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6293lb # [ 128.460971] niks3-server[1123]: 2026/09/20 13:34:13 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6294worker1 # [ 128.470309] nix-grpc-daemon[1426]: time=2026-09-20T13:34:14.319Z level=INFO msg="Upload complete. (440ms)"6295lb # [ 128.468205] niks3-server[1123]: 2026/09/20 13:34:13 INFO Completed upload id=416296worker1 # [ 128.473080] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=6406297lb # [ 128.472809] niks3-server[1123]: 2026/09/20 13:34:13 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6298worker1 # [ 128.479095] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346299worker1 # [ 128.480809] nix-daemon[901]: reaped child process 1543, status = succeeded6300worker1 # [ 128.484069] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6301worker1 # [ 128.490823] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6302client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6303client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6304client # farm-b> LOG-farm-b-during-drain6305worker1 # [ 129.911637] nix-daemon[901]: reaped child process 1557, status = succeeded6306worker1 # [ 130.055473] nix-daemon[901]: accepted connection from pid 1572, user nix-grpc-daemon (trusted)6307worker1 # [ 130.068585] nix-daemon[901]: reaped child process 1575, status = succeeded6308lb # [ 130.083556] niks3-server[1123]: 2026/09/20 13:34:15 INFO Received uploads request method=POST path=/api/pending_closures6309worker1 # [ 130.097419] nix-grpc-daemon[1426]: time=2026-09-20T13:34:15.946Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6310worker1 # [ 130.099556] nix-grpc-daemon[1426]: time=2026-09-20T13:34:15.947Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6311lb # [ 130.124724] niks3-server[1123]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6312lb # [ 130.135220] niks3-server[1123]: 2026/09/20 13:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6313lb # [ 130.138959] niks3-server[1123]: 2026/09/20 13:34:15 INFO Signed narinfos id=42 count=16314worker1 # [ 130.143671] nix-grpc-daemon[1426]: time=2026-09-20T13:34:15.992Z level=INFO msg="Uploading 1 narinfos"6315lb # [ 130.143503] niks3-server[1123]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6316lb # [ 130.154190] niks3-server[1123]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6317lb # [ 130.156781] niks3-server[1123]: 2026/09/20 13:34:15 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6318lb # [ 130.160893] niks3-server[1123]: 2026/09/20 13:34:15 INFO Completed upload id=426319worker1 # [ 130.163182] nix-grpc-daemon[1426]: time=2026-09-20T13:34:16.011Z level=INFO msg="Upload complete. (265ms)"6320lb # [ 130.162442] niks3-server[1123]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6321worker1 # [ 130.169688] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16322worker1 # [ 130.179387] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:16Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06323worker1 # [ 130.184225] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:16Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16324worker1 # [ 130.192301] nix-daemon[901]: reaped child process 1555, status = succeeded6325worker1 # [ 130.228272] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346326worker1 # [ 130.240694] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346327worker1 # [ 130.253503] nix-grpc-daemon[1426]: time=2026-09-20T13:34:16.102Z level=INFO msg="All 2 paths already cached"6328client # copying 1 paths to 'lb:50051'...6329worker1 # [ 130.262164] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346330client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6331worker1 # [ 130.281355] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6332worker1 # [ 130.465270] nix-daemon[901]: accepted connection from pid 1581, user nix-grpc-daemon (trusted)6333worker1 # [ 130.477481] nix-daemon[901]: reaped child process 1584, status = succeeded6334worker1 # [ 130.638104] nix-daemon[901]: accepted connection from pid 1586, user nix-grpc-daemon (trusted)6335worker1 # [ 130.650174] nix-daemon[901]: reaped child process 1589, status = succeeded6336lb # [ 130.664180] niks3-server[1123]: 2026/09/20 13:34:15 INFO Received uploads request method=POST path=/api/pending_closures6337worker1 # [ 130.674401] nix-grpc-daemon[1426]: time=2026-09-20T13:34:16.523Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6338worker1 # [ 130.675787] nix-grpc-daemon[1426]: time=2026-09-20T13:34:16.523Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6339lb # [ 130.692308] niks3-server[1123]: 2026/09/20 13:34:15 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6340lb # [ 130.700656] niks3-server[1123]: 2026/09/20 13:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6341lb # [ 130.705115] niks3-server[1123]: 2026/09/20 13:34:16 INFO Signed narinfos id=43 count=16342worker1 # [ 130.710648] nix-grpc-daemon[1426]: time=2026-09-20T13:34:16.559Z level=INFO msg="Uploading 1 narinfos"6343lb # [ 130.711649] niks3-server[1123]: 2026/09/20 13:34:16 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6344lb # [ 130.723695] niks3-server[1123]: 2026/09/20 13:34:16 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6345worker1 # [ 130.729626] nix-grpc-daemon[1426]: time=2026-09-20T13:34:16.578Z level=INFO msg="Upload complete. (435ms)"6346lb # [ 130.727784] niks3-server[1123]: 2026/09/20 13:34:16 INFO Completed upload id=436347worker1 # [ 130.733066] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=7686348lb # [ 130.733152] niks3-server[1123]: 2026/09/20 13:34:16 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6349worker1 # [ 130.735622] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346350worker1 # [ 130.741599] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6351worker1 # [ 130.744601] nix-daemon[901]: reaped child process 1579, status = succeeded6352worker1 # [ 130.749794] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6353client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6354client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6355worker1 # [ 131.563232] nix-daemon[901]: reaped child process 1593, status = succeeded6356client # farm-top> LOG-farm-top-during-drain6357worker1 # [ 131.684825] nix-daemon[901]: accepted connection from pid 1608, user nix-grpc-daemon (trusted)6358worker1 # [ 131.696773] nix-daemon[901]: reaped child process 1611, status = succeeded6359lb # [ 131.711373] niks3-server[1123]: 2026/09/20 13:34:17 INFO Received uploads request method=POST path=/api/pending_closures6360worker1 # [ 131.721738] nix-grpc-daemon[1426]: time=2026-09-20T13:34:17.570Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6361worker1 # [ 131.724060] nix-grpc-daemon[1426]: time=2026-09-20T13:34:17.572Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6362lb # [ 131.750059] niks3-server[1123]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6363lb # [ 131.755712] niks3-server[1123]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6364lb # [ 131.767698] niks3-server[1123]: 2026/09/20 13:34:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6365worker1 # [ 131.775266] nix-grpc-daemon[1426]: time=2026-09-20T13:34:17.623Z level=INFO msg="Uploading 1 narinfos"6366lb # [ 131.772979] niks3-server[1123]: 2026/09/20 13:34:17 INFO Signed narinfos id=44 count=16367lb # [ 131.776503] niks3-server[1123]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6368lb # [ 131.785656] niks3-server[1123]: 2026/09/20 13:34:17 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6369lb # [ 131.788044] niks3-server[1123]: 2026/09/20 13:34:17 INFO Completed upload id=446370worker1 # [ 131.790329] nix-grpc-daemon[1426]: time=2026-09-20T13:34:17.639Z level=INFO msg="Upload complete. (234ms)"6371lb # [ 131.790397] niks3-server[1123]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6372worker1 # [ 131.797055] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16373worker1 # [ 131.808166] nix-daemon[901]: reaped child process 1591, status = succeeded6374client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.54 seconds)6375client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6376client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.21 seconds)6377worker2: must succeed: pkill -f 'read -t [9]0 x'6378worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6379client: waiting for success: ! systemctl is-active sw6380worker2 # [ 132.222656] nix-daemon[858]: reaped child process 1704, status = succeeded6381worker2 # [ 132.375460] nix-daemon[858]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6382worker2 # [ 132.387182] nix-daemon[858]: reaped child process 1878, status = succeeded6383lb # [ 132.419971] niks3-server[1123]: 2026/09/20 13:34:17 INFO Received uploads request method=POST path=/api/pending_closures6384worker2 # [ 132.409783] nix-grpc-daemon[1640]: time=2026-09-20T13:34:17.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6385worker2 # [ 132.411300] nix-grpc-daemon[1640]: time=2026-09-20T13:34:17.935Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6386lb # [ 132.458687] niks3-server[1123]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6387lb # [ 132.466273] niks3-server[1123]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6388lb # [ 132.475868] niks3-server[1123]: 2026/09/20 13:34:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6389lb # [ 132.478852] niks3-server[1123]: 2026/09/20 13:34:17 INFO Signed narinfos id=45 count=16390worker2 # [ 132.460570] nix-grpc-daemon[1640]: time=2026-09-20T13:34:17.984Z level=INFO msg="Uploading 1 narinfos"6391lb # [ 132.480783] niks3-server[1123]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6392lb # [ 132.487190] niks3-server[1123]: 2026/09/20 13:34:17 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6393lb # [ 132.489699] niks3-server[1123]: 2026/09/20 13:34:17 INFO Completed upload id=456394worker2 # [ 132.472154] nix-grpc-daemon[1640]: time=2026-09-20T13:34:17.996Z level=INFO msg="Upload complete. (268ms)"6395lb # [ 132.491877] niks3-server[1123]: 2026/09/20 13:34:17 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6396worker2 # [ 132.474716] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34462 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16397worker2 # [ 132.482064] nix-daemon[858]: reaped child process 1702, status = succeeded6398client # [ 132.549838] systemd[1]: sw.service: Deactivated successfully.6399client # [ 132.553686] systemd[1]: sw.service: Consumed 113ms CPU time over 12.722s wall clock time, 39.1M memory peak, 8.3K incoming IP traffic, 8.9K outgoing IP traffic.6400worker2 # [ 133.053885] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:18Z level=info event=signal_exit6401client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6402client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6403client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6404worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6405worker2 # [ 133.278180] systemd[1]: Starting scheduler-token.service...6406worker2 # [ 133.343454] systemd[1]: scheduler-token.service: Deactivated successfully.6407worker2 # [ 133.344696] systemd[1]: Finished scheduler-token.service.6408worker2 # [ 133.346474] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 65ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6409worker1 # [ 134.312269] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:598866410worker1 # [ 134.316550] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:20Z level=info event=worker_gone addr=192.168.1.4:500516411worker2 # [ 134.294640] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:19Z level=info event=scheduler_restarting6412worker2 # [ 134.303279] nix-grpc-daemon[1539]: ts=2026-09-20T13:34:19Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6413worker2 # [ 134.322159] nix-daemon[858]: reaped child process 1608, status = succeeded6414worker2 # [ 134.868218] nix-grpc-daemon[1539]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6415worker2 # [ 134.871620] nix-grpc-daemon[1539]: E0000 00:00:1789911260.392203 1582 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6416worker2 # [ 134.876340] nix-grpc-daemon[1539]: E0000 00:00:1789911260.392425 1582 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6417worker2 # [ 135.100590] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6418worker2 # [ 135.103516] systemd[1]: nix-grpc-daemon.service: Consumed 647ms CPU time over 32.216s wall clock time, 30.4M memory peak, 21.8K incoming IP traffic, 29.4K outgoing IP traffic.6419worker2 # [ 135.121273] systemd[1]: Starting Nix worker-protocol over gRPC...6420worker2 # [ 135.292867] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6421worker2 # [ 135.295207] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6422worker2 # [ 135.298798] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16423worker2 # [ 135.305857] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6424worker2 # [ 135.308599] systemd[1]: Started Nix worker-protocol over gRPC.6425worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.11 seconds)6426lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6427lb: must succeed: curl -sf localhost:9901/clusters6428worker2 # [ 135.323596] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:20Z level=info event=scheduler_connected addr=lb:500516429worker1 # [ 135.347679] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:598866430worker1 # [ 135.351955] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06431lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6432lb: must succeed: curl -sf localhost:9901/clusters6433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6434lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6435lb: must succeed: curl -sf localhost:9901/clusters6436lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6437worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6438worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6439(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.75 seconds)6440subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6441client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16442worker2 # [ 136.721352] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372866443worker2 # [ 136.733113] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6444worker2 # [ 136.746630] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372866445worker2 # [ 136.757710] nix-grpc-daemon[1895]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)6446worker2 # [ 136.851371] nix-grpc-daemon[1895]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 2/5)6447worker2 # [ 136.872574] nix-grpc-daemon[1895]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 3/5)6448worker2 # [ 136.880775] nix-grpc-daemon[1895]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 380 ms (attempt 4/5)6449worker2 # [ 137.266769] nix-grpc-daemon[1895]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6450worker1 # [ 137.338157] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06451worker1 # [ 137.342384] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16452worker1 # [ 137.346714] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06453worker1 # [ 137.350511] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06454worker1 # [ 137.381726] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346455worker1 # [ 137.394966] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346456worker1 # [ 137.406725] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346457worker1 # [ 137.415951] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6458worker1 # [ 137.585335] nix-daemon[901]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6459worker1 # [ 137.601194] nix-daemon[901]: reaped child process 1623, status = succeeded6460worker1 # [ 137.756347] nix-daemon[901]: accepted connection from pid 1625, user nix-grpc-daemon (trusted)6461worker1 # [ 137.768268] nix-daemon[901]: reaped child process 1628, status = succeeded6462lb # [ 137.782645] niks3-server[1123]: 2026/09/20 13:34:23 INFO Received uploads request method=POST path=/api/pending_closures6463worker1 # [ 137.792642] nix-grpc-daemon[1426]: time=2026-09-20T13:34:23.641Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6464worker1 # [ 137.794326] nix-grpc-daemon[1426]: time=2026-09-20T13:34:23.643Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6465lb # [ 137.810140] niks3-server[1123]: 2026/09/20 13:34:23 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6466lb # [ 137.823658] niks3-server[1123]: 2026/09/20 13:34:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6467lb # [ 137.827719] niks3-server[1123]: 2026/09/20 13:34:23 INFO Signed narinfos id=46 count=16468worker1 # [ 137.832395] nix-grpc-daemon[1426]: time=2026-09-20T13:34:23.681Z level=INFO msg="Uploading 1 narinfos"6469lb # [ 137.837436] niks3-server[1123]: 2026/09/20 13:34:23 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6470lb # [ 137.841255] niks3-server[1123]: 2026/09/20 13:34:23 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6471lb # [ 137.844233] niks3-server[1123]: 2026/09/20 13:34:23 INFO Completed upload id=466472worker1 # [ 137.847609] nix-grpc-daemon[1426]: time=2026-09-20T13:34:23.696Z level=INFO msg="Upload complete. (418ms)"6473lb # [ 137.846640] niks3-server[1123]: 2026/09/20 13:34:23 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6474worker1 # [ 137.849506] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=5046475worker1 # [ 137.853468] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346476worker1 # [ 137.859979] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6477worker1 # [ 137.862337] nix-daemon[901]: reaped child process 1618, status = succeeded6478worker1 # [ 137.867618] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6479lb # [ 138.939405] envoy[984]: conn peer=[2001:db8:1::1]:46964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9598 tx=3849 ms=63016480lb # [ 138.940962] envoy[984]: conn peer=[2001:db8:1::1]:36624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2932 tx=1310 ms=124926481lb # [ 138.942408] envoy[984]: conn peer=[2001:db8:1::4]:45408 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2366 tx=448 ms=312386482worker1 # [ 139.088442] nix-daemon[901]: reaped child process 1632, status = succeeded6483worker1 # [ 139.231187] nix-daemon[901]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6484worker1 # [ 139.243632] nix-daemon[901]: reaped child process 1650, status = succeeded6485lb # [ 139.258282] niks3-server[1123]: 2026/09/20 13:34:24 INFO Received uploads request method=POST path=/api/pending_closures6486worker1 # [ 139.268235] nix-grpc-daemon[1426]: time=2026-09-20T13:34:25.116Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6487worker1 # [ 139.269746] nix-grpc-daemon[1426]: time=2026-09-20T13:34:25.118Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6488lb # [ 139.295636] niks3-server[1123]: 2026/09/20 13:34:24 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6489lb # [ 139.306419] niks3-server[1123]: 2026/09/20 13:34:24 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6490lb # [ 139.312199] niks3-server[1123]: 2026/09/20 13:34:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6491lb # [ 139.320054] niks3-server[1123]: 2026/09/20 13:34:24 INFO Signed narinfos id=47 count=16492worker1 # [ 139.322306] nix-grpc-daemon[1426]: time=2026-09-20T13:34:25.171Z level=INFO msg="Uploading 1 narinfos"6493lb # [ 139.325529] niks3-server[1123]: 2026/09/20 13:34:24 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6494lb # [ 139.333936] niks3-server[1123]: 2026/09/20 13:34:24 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6495lb # [ 139.339123] niks3-server[1123]: 2026/09/20 13:34:24 INFO Completed upload id=476496worker1 # [ 139.341424] nix-grpc-daemon[1426]: time=2026-09-20T13:34:25.190Z level=INFO msg="Upload complete. (266ms)"6497lb # [ 139.342720] niks3-server[1123]: 2026/09/20 13:34:24 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6498worker1 # [ 139.352601] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16499worker1 # [ 139.357725] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:25Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16500worker1 # [ 139.362229] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346501worker1 # [ 139.365885] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:25Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06502worker1 # [ 139.374380] nix-daemon[901]: reaped child process 1630, status = succeeded6503worker1 # [ 139.376522] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346504worker1 # [ 139.383377] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346505worker1 # [ 139.387244] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6506worker1 # [ 139.539081] nix-daemon[901]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6507worker1 # [ 139.553291] nix-daemon[901]: reaped child process 1659, status = succeeded6508worker1 # [ 139.714583] nix-daemon[901]: accepted connection from pid 1661, user nix-grpc-daemon (trusted)6509worker1 # [ 139.727340] nix-daemon[901]: reaped child process 1664, status = succeeded6510lb # [ 139.740317] niks3-server[1123]: 2026/09/20 13:34:25 INFO Received uploads request method=POST path=/api/pending_closures6511worker1 # [ 140.072699] nix-grpc-daemon[1426]: time=2026-09-20T13:34:25.920Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6512worker1 # [ 140.076119] nix-grpc-daemon[1426]: time=2026-09-20T13:34:25.920Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6513lb # [ 140.092573] niks3-server[1123]: 2026/09/20 13:34:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6514lb # [ 140.098569] niks3-server[1123]: 2026/09/20 13:34:25 INFO Signed narinfos id=48 count=16515worker1 # [ 140.105697] nix-grpc-daemon[1426]: time=2026-09-20T13:34:25.954Z level=INFO msg="Uploading 1 narinfos"6516lb # [ 140.108471] niks3-server[1123]: 2026/09/20 13:34:25 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6517lb # [ 140.119418] niks3-server[1123]: 2026/09/20 13:34:25 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6518lb # [ 140.130080] niks3-server[1123]: 2026/09/20 13:34:25 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6519lb # [ 140.145638] niks3-server[1123]: 2026/09/20 13:34:25 INFO Completed upload id=486520worker1 # [ 140.147629] nix-grpc-daemon[1426]: time=2026-09-20T13:34:25.996Z level=INFO msg="Upload complete. (754ms)"6521worker1 # [ 140.152287] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=5046522worker1 # [ 140.158329] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346523lb # [ 140.159778] niks3-server[1123]: 2026/09/20 13:34:25 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6524worker1 # [ 140.173498] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6525worker1 # [ 140.183763] nix-daemon[901]: reaped child process 1654, status = succeeded6526worker1 # [ 140.192375] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6527worker1 # [ 141.424426] nix-daemon[901]: reaped child process 1668, status = succeeded6528worker1 # [ 141.579245] nix-daemon[901]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6529worker1 # [ 141.591558] nix-daemon[901]: reaped child process 1686, status = succeeded6530lb # [ 141.602864] niks3-server[1123]: 2026/09/20 13:34:26 INFO Received uploads request method=POST path=/api/pending_closures6531worker1 # [ 141.613691] nix-grpc-daemon[1426]: time=2026-09-20T13:34:27.462Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6532worker1 # [ 141.615217] nix-grpc-daemon[1426]: time=2026-09-20T13:34:27.464Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6533lb # [ 141.643119] niks3-server[1123]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6534lb # [ 141.651728] niks3-server[1123]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6535lb # [ 141.662541] niks3-server[1123]: 2026/09/20 13:34:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6536lb # [ 141.668145] niks3-server[1123]: 2026/09/20 13:34:26 INFO Signed narinfos id=49 count=16537worker1 # [ 141.670026] nix-grpc-daemon[1426]: time=2026-09-20T13:34:27.518Z level=INFO msg="Uploading 1 narinfos"6538lb # [ 141.673122] niks3-server[1123]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6539lb # [ 141.683717] niks3-server[1123]: 2026/09/20 13:34:26 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6540lb # [ 141.686845] niks3-server[1123]: 2026/09/20 13:34:26 INFO Completed upload id=496541lb # [ 141.688537] niks3-server[1123]: 2026/09/20 13:34:26 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6542worker1 # [ 141.690495] nix-grpc-daemon[1426]: time=2026-09-20T13:34:27.539Z level=INFO msg="Upload complete. (285ms)"6543worker1 # [ 141.701490] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16544worker1 # [ 141.707673] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:27Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16545worker1 # [ 141.711585] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346546worker1 # [ 141.714969] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:27Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06547worker1 # [ 141.724306] nix-daemon[901]: reaped child process 1666, status = succeeded6548worker1 # [ 141.725608] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346549worker1 # [ 141.732440] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346550worker1 # [ 141.736210] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6551worker1 # [ 141.867178] nix-daemon[901]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6552worker1 # [ 141.880651] nix-daemon[901]: reaped child process 1695, status = succeeded6553worker1 # [ 142.012112] nix-daemon[901]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6554worker1 # [ 142.023889] nix-daemon[901]: reaped child process 1700, status = succeeded6555lb # [ 142.032720] niks3-server[1123]: 2026/09/20 13:34:27 INFO Received uploads request method=POST path=/api/pending_closures6556worker1 # [ 142.043181] nix-grpc-daemon[1426]: time=2026-09-20T13:34:27.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6557worker1 # [ 142.044574] nix-grpc-daemon[1426]: time=2026-09-20T13:34:27.891Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6558lb # [ 142.059372] niks3-server[1123]: 2026/09/20 13:34:27 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6559lb # [ 142.070310] niks3-server[1123]: 2026/09/20 13:34:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6560lb # [ 142.075840] niks3-server[1123]: 2026/09/20 13:34:27 INFO Signed narinfos id=50 count=16561worker1 # [ 142.081132] nix-grpc-daemon[1426]: time=2026-09-20T13:34:27.929Z level=INFO msg="Uploading 1 narinfos"6562lb # [ 142.083572] niks3-server[1123]: 2026/09/20 13:34:27 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6563lb # [ 142.092631] niks3-server[1123]: 2026/09/20 13:34:27 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6564lb # [ 142.100172] niks3-server[1123]: 2026/09/20 13:34:27 INFO Completed upload id=506565worker1 # [ 142.102322] nix-grpc-daemon[1426]: time=2026-09-20T13:34:27.951Z level=INFO msg="Upload complete. (360ms)"6566worker1 # [ 142.105278] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=5046567lb # [ 142.104190] niks3-server[1123]: 2026/09/20 13:34:27 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6568worker1 # [ 142.110771] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346569worker1 # [ 142.118547] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6570worker1 # [ 142.123573] nix-daemon[901]: reaped child process 1690, status = succeeded6571worker1 # [ 142.127526] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6572worker1 # [ 142.740629] nix-daemon[901]: reaped child process 1704, status = succeeded6573worker1 # [ 142.881280] nix-daemon[901]: accepted connection from pid 1719, user nix-grpc-daemon (trusted)6574worker1 # [ 142.891979] nix-daemon[901]: reaped child process 1722, status = succeeded6575lb # [ 142.905586] niks3-server[1123]: 2026/09/20 13:34:28 INFO Received uploads request method=POST path=/api/pending_closures6576worker1 # [ 142.916352] nix-grpc-daemon[1426]: time=2026-09-20T13:34:28.765Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6577worker1 # [ 142.917868] nix-grpc-daemon[1426]: time=2026-09-20T13:34:28.766Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6578lb # [ 142.946523] niks3-server[1123]: 2026/09/20 13:34:28 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6579lb # [ 142.952807] niks3-server[1123]: 2026/09/20 13:34:28 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6580lb # [ 142.964289] niks3-server[1123]: 2026/09/20 13:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6581lb # [ 142.971203] niks3-server[1123]: 2026/09/20 13:34:28 INFO Signed narinfos id=51 count=16582worker1 # [ 142.975964] nix-grpc-daemon[1426]: time=2026-09-20T13:34:28.824Z level=INFO msg="Uploading 1 narinfos"6583lb # [ 142.975413] niks3-server[1123]: 2026/09/20 13:34:28 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6584lb # [ 142.984745] niks3-server[1123]: 2026/09/20 13:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6585lb # [ 142.988120] niks3-server[1123]: 2026/09/20 13:34:28 INFO Completed upload id=516586worker1 # [ 142.990432] nix-grpc-daemon[1426]: time=2026-09-20T13:34:28.839Z level=INFO msg="Upload complete. (257ms)"6587lb # [ 142.990724] niks3-server[1123]: 2026/09/20 13:34:28 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6588worker1 # [ 143.001242] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16589worker1 # [ 143.008269] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:28Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16590worker1 # [ 143.011763] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346591worker1 # [ 143.014860] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:28Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06592worker1 # [ 143.024979] nix-daemon[901]: reaped child process 1702, status = succeeded6593worker1 # [ 143.026136] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346594worker1 # [ 143.033900] nix-grpc-daemon[1426]: time=2026-09-20T13:34:28.882Z level=INFO msg="All 1 paths already cached"6595worker1 # [ 143.038125] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346596worker1 # [ 143.041703] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6597worker1 # [ 143.188200] nix-daemon[901]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)6598worker1 # [ 143.200188] nix-daemon[901]: reaped child process 1731, status = succeeded6599worker1 # [ 143.352157] nix-daemon[901]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6600worker1 # [ 143.364139] nix-daemon[901]: reaped child process 1736, status = succeeded6601lb # [ 143.377575] niks3-server[1123]: 2026/09/20 13:34:28 INFO Received uploads request method=POST path=/api/pending_closures6602worker1 # [ 143.388932] nix-grpc-daemon[1426]: time=2026-09-20T13:34:29.237Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6603worker1 # [ 143.390380] nix-grpc-daemon[1426]: time=2026-09-20T13:34:29.237Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6604lb # [ 143.404716] niks3-server[1123]: 2026/09/20 13:34:28 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6605lb # [ 143.415654] niks3-server[1123]: 2026/09/20 13:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6606lb # [ 143.420927] niks3-server[1123]: 2026/09/20 13:34:28 INFO Signed narinfos id=52 count=16607worker1 # [ 143.426129] nix-grpc-daemon[1426]: time=2026-09-20T13:34:29.274Z level=INFO msg="Uploading 1 narinfos"6608lb # [ 143.428219] niks3-server[1123]: 2026/09/20 13:34:28 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6609lb # [ 143.438222] niks3-server[1123]: 2026/09/20 13:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6610lb # [ 143.443890] niks3-server[1123]: 2026/09/20 13:34:28 INFO Completed upload id=526611worker1 # [ 143.445597] nix-grpc-daemon[1426]: time=2026-09-20T13:34:29.294Z level=INFO msg="Upload complete. (398ms)"6612worker1 # [ 143.450125] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=6246613lb # [ 143.449261] niks3-server[1123]: 2026/09/20 13:34:28 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6614worker1 # [ 143.455593] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346615worker1 # [ 143.464726] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6616worker1 # [ 143.466972] nix-daemon[901]: reaped child process 1726, status = succeeded6617worker1 # [ 143.472362] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6618worker1 # [ 144.486371] nix-daemon[901]: reaped child process 1740, status = succeeded6619worker1 # [ 144.633360] nix-daemon[901]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)6620worker1 # [ 144.645681] nix-daemon[901]: reaped child process 1758, status = succeeded6621lb # [ 144.661219] niks3-server[1123]: 2026/09/20 13:34:29 INFO Received uploads request method=POST path=/api/pending_closures6622worker1 # [ 144.672412] nix-grpc-daemon[1426]: time=2026-09-20T13:34:30.521Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6623worker1 # [ 144.673933] nix-grpc-daemon[1426]: time=2026-09-20T13:34:30.522Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6624lb # [ 144.702639] niks3-server[1123]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6625lb # [ 144.707442] niks3-server[1123]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6626lb # [ 144.717326] niks3-server[1123]: 2026/09/20 13:34:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6627worker1 # [ 144.725126] nix-grpc-daemon[1426]: time=2026-09-20T13:34:30.573Z level=INFO msg="Uploading 1 narinfos"6628lb # [ 144.724220] niks3-server[1123]: 2026/09/20 13:34:30 INFO Signed narinfos id=53 count=16629lb # [ 144.729198] niks3-server[1123]: 2026/09/20 13:34:30 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6630lb # [ 144.731696] niks3-server[1123]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6631lb # [ 144.735288] niks3-server[1123]: 2026/09/20 13:34:30 INFO Completed upload id=536632lb # [ 144.736472] niks3-server[1123]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6633worker1 # [ 144.737587] nix-grpc-daemon[1426]: time=2026-09-20T13:34:30.586Z level=INFO msg="Upload complete. (258ms)"6634worker1 # [ 144.744397] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16635worker1 # [ 144.748830] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:30Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16636worker1 # [ 144.757087] nix-daemon[901]: reaped child process 1738, status = succeeded6637worker1 # [ 144.785792] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346638worker1 # [ 144.794098] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:30Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06639worker1 # [ 144.804685] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346640worker1 # [ 144.814982] nix-grpc-daemon[1426]: time=2026-09-20T13:34:30.663Z level=INFO msg="All 1 paths already cached"6641worker1 # [ 144.820402] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346642worker1 # [ 144.828952] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6643worker1 # [ 144.997852] nix-daemon[901]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6644worker1 # [ 145.010959] nix-daemon[901]: reaped child process 1767, status = succeeded6645worker1 # [ 145.176596] nix-daemon[901]: accepted connection from pid 1769, user nix-grpc-daemon (trusted)6646worker1 # [ 145.188684] nix-daemon[901]: reaped child process 1772, status = succeeded6647lb # [ 145.202495] niks3-server[1123]: 2026/09/20 13:34:30 INFO Received uploads request method=POST path=/api/pending_closures6648worker1 # [ 145.212224] nix-grpc-daemon[1426]: time=2026-09-20T13:34:31.061Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6649worker1 # [ 145.214084] nix-grpc-daemon[1426]: time=2026-09-20T13:34:31.062Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6650lb # [ 145.230080] niks3-server[1123]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6651lb # [ 145.241941] niks3-server[1123]: 2026/09/20 13:34:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6652worker1 # [ 145.249092] nix-grpc-daemon[1426]: time=2026-09-20T13:34:31.097Z level=INFO msg="Uploading 1 narinfos"6653lb # [ 145.248158] niks3-server[1123]: 2026/09/20 13:34:30 INFO Signed narinfos id=54 count=16654lb # [ 145.253150] niks3-server[1123]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6655lb # [ 145.265056] niks3-server[1123]: 2026/09/20 13:34:30 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6656worker1 # [ 145.272839] nix-grpc-daemon[1426]: time=2026-09-20T13:34:31.121Z level=INFO msg="Upload complete. (435ms)"6657lb # [ 145.270959] niks3-server[1123]: 2026/09/20 13:34:30 INFO Completed upload id=546658lb # [ 145.274375] niks3-server[1123]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6659worker1 # [ 145.275601] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=6246660worker1 # [ 145.280860] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346661worker1 # [ 145.291253] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6662worker1 # [ 145.295694] nix-daemon[901]: reaped child process 1762, status = succeeded6663worker1 # [ 145.300850] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6664worker1 # [ 146.527661] nix-daemon[901]: reaped child process 1776, status = succeeded6665worker1 # [ 146.668333] nix-daemon[901]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6666worker1 # [ 146.680308] nix-daemon[901]: reaped child process 1794, status = succeeded6667lb # [ 146.690307] niks3-server[1123]: 2026/09/20 13:34:31 INFO Received uploads request method=POST path=/api/pending_closures6668worker1 # [ 146.700924] nix-grpc-daemon[1426]: time=2026-09-20T13:34:32.549Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6669worker1 # [ 146.702725] nix-grpc-daemon[1426]: time=2026-09-20T13:34:32.551Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6670lb # [ 146.729257] niks3-server[1123]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6671lb # [ 146.741330] niks3-server[1123]: 2026/09/20 13:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6672lb # [ 146.746792] niks3-server[1123]: 2026/09/20 13:34:32 INFO Signed narinfos id=55 count=16673worker1 # [ 146.752705] nix-grpc-daemon[1426]: time=2026-09-20T13:34:32.601Z level=INFO msg="Uploading 1 narinfos"6674lb # [ 146.752842] niks3-server[1123]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6675lb # [ 146.756881] niks3-server[1123]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6676lb # [ 146.769765] niks3-server[1123]: 2026/09/20 13:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6677lb # [ 146.772902] niks3-server[1123]: 2026/09/20 13:34:32 INFO Completed upload id=556678lb # [ 146.774699] niks3-server[1123]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6679worker1 # [ 146.776752] nix-grpc-daemon[1426]: time=2026-09-20T13:34:32.625Z level=INFO msg="Upload complete. (268ms)"6680worker1 # [ 146.788853] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16681worker1 # [ 146.795246] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:32Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16682worker1 # [ 146.799936] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346683worker1 # [ 146.803663] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:32Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06684worker1 # [ 146.814543] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346685worker1 # [ 146.816860] nix-daemon[901]: reaped child process 1774, status = succeeded6686worker1 # [ 146.823643] nix-grpc-daemon[1426]: time=2026-09-20T13:34:32.672Z level=INFO msg="All 1 paths already cached"6687worker1 # [ 146.827945] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346688worker1 # [ 146.831922] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6689worker1 # [ 146.986532] nix-daemon[901]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6690worker1 # [ 146.998887] nix-daemon[901]: reaped child process 1803, status = succeeded6691worker1 # [ 147.153970] nix-daemon[901]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)6692worker1 # [ 147.165956] nix-daemon[901]: reaped child process 1808, status = succeeded6693lb # [ 147.178286] niks3-server[1123]: 2026/09/20 13:34:32 INFO Received uploads request method=POST path=/api/pending_closures6694worker1 # [ 147.187966] nix-grpc-daemon[1426]: time=2026-09-20T13:34:33.036Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6695worker1 # [ 147.189663] nix-grpc-daemon[1426]: time=2026-09-20T13:34:33.038Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6696lb # [ 147.207699] niks3-server[1123]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6697lb # [ 147.219189] niks3-server[1123]: 2026/09/20 13:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6698lb # [ 147.224741] niks3-server[1123]: 2026/09/20 13:34:32 INFO Signed narinfos id=56 count=16699worker1 # [ 147.229848] nix-grpc-daemon[1426]: time=2026-09-20T13:34:33.078Z level=INFO msg="Uploading 1 narinfos"6700lb # [ 147.231372] niks3-server[1123]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6701lb # [ 147.243280] niks3-server[1123]: 2026/09/20 13:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6702worker1 # [ 147.252306] nix-grpc-daemon[1426]: time=2026-09-20T13:34:33.101Z level=INFO msg="Upload complete. (414ms)"6703lb # [ 147.250213] niks3-server[1123]: 2026/09/20 13:34:32 INFO Completed upload id=566704worker1 # [ 147.255088] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=6246705lb # [ 147.253813] niks3-server[1123]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6706worker1 # [ 147.260396] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346707worker1 # [ 147.264552] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6708worker1 # [ 147.266789] nix-daemon[901]: reaped child process 1798, status = succeeded6709worker1 # [ 147.271905] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6710worker1 # [ 148.701599] nix-daemon[901]: reaped child process 1812, status = succeeded6711worker1 # [ 148.833588] nix-daemon[901]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)6712worker1 # [ 148.845879] nix-daemon[901]: reaped child process 1830, status = succeeded6713lb # [ 148.853673] niks3-server[1123]: 2026/09/20 13:34:34 INFO Received uploads request method=POST path=/api/pending_closures6714worker1 # [ 148.860680] nix-grpc-daemon[1426]: time=2026-09-20T13:34:34.709Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6715worker1 # [ 148.862279] nix-grpc-daemon[1426]: time=2026-09-20T13:34:34.711Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6716lb # [ 148.874769] niks3-server[1123]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6717lb # [ 148.880571] niks3-server[1123]: 2026/09/20 13:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6718lb # [ 148.884192] niks3-server[1123]: 2026/09/20 13:34:34 INFO Signed narinfos id=57 count=16719worker1 # [ 148.886467] nix-grpc-daemon[1426]: time=2026-09-20T13:34:34.735Z level=INFO msg="Uploading 1 narinfos"6720lb # [ 148.886301] niks3-server[1123]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6721lb # [ 148.890125] niks3-server[1123]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6722lb # [ 148.894707] niks3-server[1123]: 2026/09/20 13:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6723lb # [ 148.897637] niks3-server[1123]: 2026/09/20 13:34:34 INFO Completed upload id=576724worker1 # [ 148.899528] nix-grpc-daemon[1426]: time=2026-09-20T13:34:34.748Z level=INFO msg="Upload complete. (218ms)"6725lb # [ 148.900237] niks3-server[1123]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6726worker1 # [ 148.901753] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16727worker1 # [ 148.906327] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:34Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16728worker1 # [ 148.909467] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:34Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06729worker1 # [ 148.914329] nix-daemon[901]: reaped child process 1810, status = succeeded6730worker1 # [ 148.948425] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346731worker1 # [ 148.957408] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346732worker1 # [ 148.966130] nix-grpc-daemon[1426]: time=2026-09-20T13:34:34.814Z level=INFO msg="All 2 paths already cached"6733worker1 # [ 148.971903] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346734worker1 # [ 148.980128] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6735worker1 # [ 149.136262] nix-daemon[901]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6736worker1 # [ 149.149201] nix-daemon[901]: reaped child process 1839, status = succeeded6737worker1 # [ 149.313548] nix-daemon[901]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)6738worker1 # [ 149.325662] nix-daemon[901]: reaped child process 1844, status = succeeded6739lb # [ 149.339538] niks3-server[1123]: 2026/09/20 13:34:34 INFO Received uploads request method=POST path=/api/pending_closures6740worker1 # [ 149.345917] nix-grpc-daemon[1426]: time=2026-09-20T13:34:35.194Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6741worker1 # [ 149.347311] nix-grpc-daemon[1426]: time=2026-09-20T13:34:35.194Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6742lb # [ 149.351232] niks3-server[1123]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6743lb # [ 149.354995] niks3-server[1123]: 2026/09/20 13:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6744lb # [ 149.356881] niks3-server[1123]: 2026/09/20 13:34:34 INFO Signed narinfos id=58 count=16745worker1 # [ 149.359231] nix-grpc-daemon[1426]: time=2026-09-20T13:34:35.208Z level=INFO msg="Uploading 1 narinfos"6746lb # [ 149.360834] niks3-server[1123]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6747lb # [ 149.364928] niks3-server[1123]: 2026/09/20 13:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6748lb # [ 149.367516] niks3-server[1123]: 2026/09/20 13:34:34 INFO Completed upload id=586749worker1 # [ 149.369655] nix-grpc-daemon[1426]: time=2026-09-20T13:34:35.218Z level=INFO msg="Upload complete. (383ms)"6750lb # [ 149.370040] niks3-server[1123]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6751worker1 # [ 149.373068] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=7526752worker1 # [ 149.375605] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346753worker1 # [ 149.381093] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6754worker1 # [ 149.385558] nix-daemon[901]: reaped child process 1834, status = succeeded6755worker1 # [ 149.391890] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6756worker1 # [ 150.607828] nix-daemon[901]: reaped child process 1848, status = succeeded6757worker1 # [ 150.747556] nix-daemon[901]: accepted connection from pid 1863, user nix-grpc-daemon (trusted)6758worker1 # [ 150.759765] nix-daemon[901]: reaped child process 1866, status = succeeded6759lb # [ 150.774492] niks3-server[1123]: 2026/09/20 13:34:36 INFO Received uploads request method=POST path=/api/pending_closures6760worker1 # [ 150.784535] nix-grpc-daemon[1426]: time=2026-09-20T13:34:36.633Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6761worker1 # [ 150.786032] nix-grpc-daemon[1426]: time=2026-09-20T13:34:36.634Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6762lb # [ 150.804573] niks3-server[1123]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6763lb # [ 150.816221] niks3-server[1123]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6764lb # [ 150.827892] niks3-server[1123]: 2026/09/20 13:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6765lb # [ 150.831840] niks3-server[1123]: 2026/09/20 13:34:36 INFO Signed narinfos id=59 count=16766worker1 # [ 150.836619] nix-grpc-daemon[1426]: time=2026-09-20T13:34:36.685Z level=INFO msg="Uploading 1 narinfos"6767lb # [ 150.837467] niks3-server[1123]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6768lb # [ 150.846574] niks3-server[1123]: 2026/09/20 13:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6769lb # [ 150.848956] niks3-server[1123]: 2026/09/20 13:34:36 INFO Completed upload id=596770worker1 # [ 150.852222] nix-grpc-daemon[1426]: time=2026-09-20T13:34:36.701Z level=INFO msg="Upload complete. (253ms)"6771lb # [ 150.851143] niks3-server[1123]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6772worker1 # [ 150.854576] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16773worker1 # [ 150.858405] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:36Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16774worker1 # [ 150.863952] nix-daemon[901]: reaped child process 1846, status = succeeded6775worker1 # [ 150.901208] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346776worker1 # [ 150.909907] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346777worker1 # [ 150.918605] nix-grpc-daemon[1426]: time=2026-09-20T13:34:36.767Z level=INFO msg="All 2 paths already cached"6778worker1 # [ 150.923208] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346779worker1 # [ 150.928381] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6780worker1 # [ 151.080487] nix-daemon[901]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6781worker1 # [ 151.092927] nix-daemon[901]: reaped child process 1873, status = succeeded6782worker1 # [ 151.248526] nix-daemon[901]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6783worker1 # [ 151.260508] nix-daemon[901]: reaped child process 1878, status = succeeded6784lb # [ 151.274096] niks3-server[1123]: 2026/09/20 13:34:36 INFO Received uploads request method=POST path=/api/pending_closures6785worker1 # [ 151.284273] nix-grpc-daemon[1426]: time=2026-09-20T13:34:37.132Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6786worker1 # [ 151.285655] nix-grpc-daemon[1426]: time=2026-09-20T13:34:37.132Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6787lb # [ 151.297098] niks3-server[1123]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6788lb # [ 151.309798] niks3-server[1123]: 2026/09/20 13:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6789worker1 # [ 151.316583] nix-grpc-daemon[1426]: time=2026-09-20T13:34:37.165Z level=INFO msg="Uploading 1 narinfos"6790lb # [ 151.314383] niks3-server[1123]: 2026/09/20 13:34:36 INFO Signed narinfos id=60 count=16791lb # [ 151.321148] niks3-server[1123]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6792lb # [ 151.332054] niks3-server[1123]: 2026/09/20 13:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6793worker1 # [ 151.339541] nix-grpc-daemon[1426]: time=2026-09-20T13:34:37.188Z level=INFO msg="Upload complete. (405ms)"6794lb # [ 151.337633] niks3-server[1123]: 2026/09/20 13:34:36 INFO Completed upload id=606795lb # [ 151.341266] niks3-server[1123]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6796worker1 # [ 151.343192] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=7526797worker1 # [ 151.347879] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346798worker1 # [ 151.356472] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6799worker1 # [ 151.359136] nix-daemon[901]: reaped child process 1868, status = succeeded6800worker1 # [ 151.365376] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6801worker1 # [ 152.790153] nix-daemon[901]: reaped child process 1882, status = succeeded6802worker1 # [ 152.921460] nix-daemon[901]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6803worker1 # [ 152.933388] nix-daemon[901]: reaped child process 1900, status = succeeded6804lb # [ 152.941731] niks3-server[1123]: 2026/09/20 13:34:38 INFO Received uploads request method=POST path=/api/pending_closures6805worker1 # [ 152.949244] nix-grpc-daemon[1426]: time=2026-09-20T13:34:38.797Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6806worker1 # [ 152.950702] nix-grpc-daemon[1426]: time=2026-09-20T13:34:38.799Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6807lb # [ 152.964163] niks3-server[1123]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6808lb # [ 152.968179] niks3-server[1123]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6809lb # [ 152.975047] niks3-server[1123]: 2026/09/20 13:34:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6810lb # [ 152.978689] niks3-server[1123]: 2026/09/20 13:34:38 INFO Signed narinfos id=61 count=16811worker1 # [ 152.981039] nix-grpc-daemon[1426]: time=2026-09-20T13:34:38.829Z level=INFO msg="Uploading 1 narinfos"6812lb # [ 152.980600] niks3-server[1123]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6813lb # [ 152.985148] niks3-server[1123]: 2026/09/20 13:34:38 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6814lb # [ 152.988487] niks3-server[1123]: 2026/09/20 13:34:38 INFO Completed upload id=616815worker1 # [ 152.990660] nix-grpc-daemon[1426]: time=2026-09-20T13:34:38.839Z level=INFO msg="Upload complete. (216ms)"6816lb # [ 152.990082] niks3-server[1123]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6817worker1 # [ 152.992951] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16818worker1 # [ 152.996682] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:38Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16819worker1 # [ 153.002128] nix-daemon[901]: reaped child process 1880, status = succeeded6820worker1 # [ 153.039538] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346821worker1 # [ 153.050422] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429346822worker1 # [ 153.061708] nix-grpc-daemon[1426]: time=2026-09-20T13:34:38.910Z level=INFO msg="All 2 paths already cached"6823worker1 # [ 153.067613] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429346824worker1 # [ 153.073995] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6825worker1 # [ 153.205438] nix-daemon[901]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)6826worker1 # [ 153.217791] nix-daemon[901]: reaped child process 1907, status = succeeded6827worker1 # [ 153.373169] nix-daemon[901]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)6828worker1 # [ 153.385150] nix-daemon[901]: reaped child process 1912, status = succeeded6829lb # [ 153.399034] niks3-server[1123]: 2026/09/20 13:34:38 INFO Received uploads request method=POST path=/api/pending_closures6830worker1 # [ 153.405534] nix-grpc-daemon[1426]: time=2026-09-20T13:34:39.254Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6831worker1 # [ 153.407054] nix-grpc-daemon[1426]: time=2026-09-20T13:34:39.255Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6832lb # [ 153.415906] niks3-server[1123]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6833lb # [ 153.421254] niks3-server[1123]: 2026/09/20 13:34:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6834lb # [ 153.424646] niks3-server[1123]: 2026/09/20 13:34:38 INFO Signed narinfos id=62 count=16835worker1 # [ 153.428490] nix-grpc-daemon[1426]: time=2026-09-20T13:34:39.277Z level=INFO msg="Uploading 1 narinfos"6836lb # [ 153.426444] niks3-server[1123]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6837lb # [ 153.433988] niks3-server[1123]: 2026/09/20 13:34:38 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6838lb # [ 153.437689] niks3-server[1123]: 2026/09/20 13:34:38 INFO Completed upload id=626839worker1 # [ 153.439794] nix-grpc-daemon[1426]: time=2026-09-20T13:34:39.288Z level=INFO msg="Upload complete. (354ms)"6840worker1 # [ 153.441780] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 paths=1 nar_bytes_in=7526841lb # [ 153.441501] niks3-server[1123]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6842worker1 # [ 153.446674] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429346843worker1 # [ 153.452091] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6844worker1 # [ 153.454440] nix-daemon[901]: reaped child process 1902, status = succeeded6845worker1 # [ 153.459886] nix-daemon[901]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)6846worker1 # [ 154.084494] nix-daemon[901]: reaped child process 1916, status = succeeded6847worker1 # [ 154.226235] nix-daemon[901]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6848worker1 # [ 154.238095] nix-daemon[901]: reaped child process 1934, status = succeeded6849lb # [ 154.248548] niks3-server[1123]: 2026/09/20 13:34:39 INFO Received uploads request method=POST path=/api/pending_closures6850worker1 # [ 154.261030] nix-grpc-daemon[1426]: time=2026-09-20T13:34:40.109Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6851worker1 # [ 154.262537] nix-grpc-daemon[1426]: time=2026-09-20T13:34:40.111Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6852lb # [ 154.294145] niks3-server[1123]: 2026/09/20 13:34:39 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6853lb # [ 154.310454] niks3-server[1123]: 2026/09/20 13:34:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6854worker1 # [ 154.316281] nix-grpc-daemon[1426]: time=2026-09-20T13:34:40.165Z level=INFO msg="Uploading 1 narinfos"6855lb # [ 154.314187] niks3-server[1123]: 2026/09/20 13:34:39 INFO Signed narinfos id=63 count=16856lb # [ 154.318435] niks3-server[1123]: 2026/09/20 13:34:39 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6857lb # [ 154.325356] niks3-server[1123]: 2026/09/20 13:34:39 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6858lb # [ 154.332095] niks3-server[1123]: 2026/09/20 13:34:39 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6859lb # [ 154.336958] niks3-server[1123]: 2026/09/20 13:34:39 INFO Completed upload id=636860lb # [ 154.338701] niks3-server[1123]: 2026/09/20 13:34:39 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6861worker1 # [ 154.339482] nix-grpc-daemon[1426]: time=2026-09-20T13:34:40.188Z level=INFO msg="Upload complete. (272ms)"6862worker1 # [ 154.346940] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42934 duration_s=0 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16863worker1 # [ 154.359737] nix-daemon[901]: reaped child process 1914, status = succeeded6864client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.87 seconds)6865worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6866worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6867client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16868worker1 # [ 154.671589] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429346869worker2 # [ 154.655669] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372866870worker1 # [ 154.698688] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:40Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06871client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.27 seconds)6872(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 18.20 seconds)6873subtest: low disk drains a worker and builds go to the other6874worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6875worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.98 seconds)6876worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6877worker1 # [ 155.802661] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:41Z level=info event=unhealthy reason=min_free6878worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6879client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26880client # don't know how to build these paths:6881client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6882worker1 # [ 157.030990] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429346883worker2 # [ 157.016363] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372866884worker1 # [ 157.058849] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:42Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06885worker1 # [ 157.062980] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:42Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16886worker2 # [ 157.082319] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372866887worker2 # [ 157.095277] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372866888client # copying 1 paths to 'lb:50051'...6889client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6890worker2 # [ 157.108382] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372866891worker2 # [ 157.120123] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6892worker2 # [ 157.141901] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:42Z level=info event=niks3_push_started pid=19516893worker2 # [ 157.316126] nix-grpc-daemon[1951]: time=2026-09-20T13:34:42.839Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6894worker2 # [ 157.461677] nix-daemon[858]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6895worker2 # [ 157.474371] nix-daemon[858]: reaped child process 1967, status = succeeded6896worker2 # [ 157.624717] nix-daemon[858]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)6897worker2 # [ 157.636956] nix-daemon[858]: reaped child process 1972, status = succeeded6898lb # [ 157.671790] niks3-server[1123]: 2026/09/20 13:34:42 INFO Received uploads request method=POST path=/api/pending_closures6899worker2 # [ 157.662823] nix-grpc-daemon[1951]: time=2026-09-20T13:34:43.186Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6900worker2 # [ 157.664226] nix-grpc-daemon[1951]: time=2026-09-20T13:34:43.186Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6901lb # [ 157.709736] niks3-server[1123]: 2026/09/20 13:34:43 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6902lb # [ 157.718427] niks3-server[1123]: 2026/09/20 13:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6903lb # [ 157.724797] niks3-server[1123]: 2026/09/20 13:34:43 INFO Signed narinfos id=64 count=16904worker2 # [ 157.709145] nix-grpc-daemon[1951]: time=2026-09-20T13:34:43.233Z level=INFO msg="Uploading 1 narinfos"6905lb # [ 157.733603] niks3-server[1123]: 2026/09/20 13:34:43 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6906lb # [ 157.743255] niks3-server[1123]: 2026/09/20 13:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6907lb # [ 157.750284] niks3-server[1123]: 2026/09/20 13:34:43 INFO Completed upload id=646908worker2 # [ 157.731643] nix-grpc-daemon[1951]: time=2026-09-20T13:34:43.255Z level=INFO msg="Upload complete. (411ms)"6909lb # [ 157.752658] niks3-server[1123]: 2026/09/20 13:34:43 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6910worker2 # [ 157.735498] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37286 duration_s=0 paths=1 nar_bytes_in=5126911worker2 # [ 157.741420] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372866912worker2 # [ 157.749867] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6913worker2 # [ 157.758168] nix-daemon[858]: reaped child process 1949, status = succeeded6914client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6915worker2 # [ 157.762239] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6916client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6917client # farm-a> LOG-farm-a-drain6918worker2 # [ 158.787224] nix-daemon[858]: reaped child process 1976, status = succeeded6919lb # [ 158.940175] envoy[984]: conn peer=[2001:db8:1::1]:54924 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27145 tx=10749 ms=176266920lb # [ 158.943900] envoy[984]: conn peer=[2001:db8:1::1]:50800 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=878 tx=481 ms=456921worker2 # [ 158.944080] nix-daemon[858]: accepted connection from pid 1991, user nix-grpc-daemon (trusted)6922worker2 # [ 158.956582] nix-daemon[858]: reaped child process 1994, status = succeeded6923lb # [ 158.991895] niks3-server[1123]: 2026/09/20 13:34:44 INFO Received uploads request method=POST path=/api/pending_closures6924worker2 # [ 158.983343] nix-grpc-daemon[1951]: time=2026-09-20T13:34:44.506Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6925worker2 # [ 158.984728] nix-grpc-daemon[1951]: time=2026-09-20T13:34:44.506Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6926lb # [ 159.023562] niks3-server[1123]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6927lb # [ 159.039931] niks3-server[1123]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6928lb # [ 159.050166] niks3-server[1123]: 2026/09/20 13:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6929worker2 # [ 159.036084] nix-grpc-daemon[1951]: time=2026-09-20T13:34:44.559Z level=INFO msg="Uploading 1 narinfos"6930lb # [ 159.054532] niks3-server[1123]: 2026/09/20 13:34:44 INFO Signed narinfos id=65 count=16931lb # [ 159.059744] niks3-server[1123]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6932lb # [ 159.067063] niks3-server[1123]: 2026/09/20 13:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6933lb # [ 159.071224] niks3-server[1123]: 2026/09/20 13:34:44 INFO Completed upload id=656934worker2 # [ 159.052523] nix-grpc-daemon[1951]: time=2026-09-20T13:34:44.576Z level=INFO msg="Upload complete. (284ms)"6935lb # [ 159.072789] niks3-server[1123]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6936worker2 # [ 159.058173] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37286 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16937worker1 # [ 159.083776] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:44Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06938worker2 # [ 159.069868] nix-daemon[858]: reaped child process 1974, status = succeeded6939worker1 # [ 159.088271] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:44Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16940worker2 # [ 159.110840] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372866941worker2 # [ 159.123485] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372866942worker2 # [ 159.136074] nix-grpc-daemon[1951]: time=2026-09-20T13:34:44.660Z level=INFO msg="All 1 paths already cached"6943client # copying 1 paths to 'lb:50051'...6944client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6945worker2 # [ 159.143836] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372866946worker2 # [ 159.152722] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6947worker2 # [ 159.325243] nix-daemon[858]: accepted connection from pid 1998, user nix-grpc-daemon (trusted)6948worker2 # [ 159.338519] nix-daemon[858]: reaped child process 2001, status = succeeded6949worker2 # [ 159.493031] nix-daemon[858]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)6950worker2 # [ 159.505685] nix-daemon[858]: reaped child process 2006, status = succeeded6951lb # [ 159.540158] niks3-server[1123]: 2026/09/20 13:34:44 INFO Received uploads request method=POST path=/api/pending_closures6952worker2 # [ 159.532610] nix-grpc-daemon[1951]: time=2026-09-20T13:34:45.056Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6953worker2 # [ 159.534684] nix-grpc-daemon[1951]: time=2026-09-20T13:34:45.058Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6954lb # [ 159.571659] niks3-server[1123]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6955lb # [ 159.579732] niks3-server[1123]: 2026/09/20 13:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6956lb # [ 159.581778] niks3-server[1123]: 2026/09/20 13:34:44 INFO Signed narinfos id=66 count=16957worker2 # [ 159.563127] nix-grpc-daemon[1951]: time=2026-09-20T13:34:45.087Z level=INFO msg="Uploading 1 narinfos"6958lb # [ 159.584741] niks3-server[1123]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6959lb # [ 159.589129] niks3-server[1123]: 2026/09/20 13:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6960lb # [ 159.592338] niks3-server[1123]: 2026/09/20 13:34:44 INFO Completed upload id=666961worker2 # [ 159.573591] nix-grpc-daemon[1951]: time=2026-09-20T13:34:45.097Z level=INFO msg="Upload complete. (407ms)"6962lb # [ 159.593803] niks3-server[1123]: 2026/09/20 13:34:44 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6963worker2 # [ 159.575628] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37286 duration_s=0 paths=1 nar_bytes_in=6246964worker2 # [ 159.579420] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372866965worker2 # [ 159.585901] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6966worker2 # [ 159.588211] nix-daemon[858]: reaped child process 1996, status = succeeded6967worker2 # [ 159.593615] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6968client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6969client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6970client # farm-b> LOG-farm-b-drain6971worker2 # [ 160.420522] nix-daemon[858]: reaped child process 2010, status = succeeded6972worker2 # [ 160.570399] nix-daemon[858]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)6973worker2 # [ 160.582914] nix-daemon[858]: reaped child process 2028, status = succeeded6974lb # [ 160.618803] niks3-server[1123]: 2026/09/20 13:34:45 INFO Received uploads request method=POST path=/api/pending_closures6975worker2 # [ 160.607723] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.131Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6976worker2 # [ 160.609695] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.133Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6977lb # [ 160.653990] niks3-server[1123]: 2026/09/20 13:34:45 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6978lb # [ 160.659076] niks3-server[1123]: 2026/09/20 13:34:45 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6979lb # [ 160.671354] niks3-server[1123]: 2026/09/20 13:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6980lb # [ 160.675515] niks3-server[1123]: 2026/09/20 13:34:45 INFO Signed narinfos id=67 count=16981worker2 # [ 160.659853] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.183Z level=INFO msg="Uploading 1 narinfos"6982lb # [ 160.686748] niks3-server[1123]: 2026/09/20 13:34:45 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6983lb # [ 160.693661] niks3-server[1123]: 2026/09/20 13:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6984worker2 # [ 160.679975] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.204Z level=INFO msg="Upload complete. (279ms)"6985lb # [ 160.698482] niks3-server[1123]: 2026/09/20 13:34:45 INFO Completed upload id=676986lb # [ 160.701548] niks3-server[1123]: 2026/09/20 13:34:45 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6987worker2 # [ 160.683333] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37286 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16988worker2 # [ 160.689372] nix-daemon[858]: reaped child process 2008, status = succeeded6989worker1 # [ 160.711311] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:46Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06990worker1 # [ 160.715434] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:46Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16991worker2 # [ 160.734096] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372866992worker2 # [ 160.746533] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372866993worker2 # [ 160.764175] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.288Z level=INFO msg="All 2 paths already cached"6994client # copying 1 paths to 'lb:50051'...6995client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6996worker2 # [ 160.772660] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372866997worker2 # [ 160.780891] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6998worker2 # [ 160.931617] nix-daemon[858]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)6999worker2 # [ 160.944348] nix-daemon[858]: reaped child process 2035, status = succeeded7000worker2 # [ 161.098631] nix-daemon[858]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)7001worker2 # [ 161.110976] nix-daemon[858]: reaped child process 2040, status = succeeded7002lb # [ 161.145407] niks3-server[1123]: 2026/09/20 13:34:46 INFO Received uploads request method=POST path=/api/pending_closures7003worker2 # [ 161.132118] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.656Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7004worker2 # [ 161.134081] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.658Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7005lb # [ 161.162065] niks3-server[1123]: 2026/09/20 13:34:46 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7006lb # [ 161.165946] niks3-server[1123]: 2026/09/20 13:34:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7007lb # [ 161.168047] niks3-server[1123]: 2026/09/20 13:34:46 INFO Signed narinfos id=68 count=17008worker2 # [ 161.150834] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.674Z level=INFO msg="Uploading 1 narinfos"7009lb # [ 161.170375] niks3-server[1123]: 2026/09/20 13:34:46 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7010lb # [ 161.175057] niks3-server[1123]: 2026/09/20 13:34:46 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7011lb # [ 161.178776] niks3-server[1123]: 2026/09/20 13:34:46 INFO Completed upload id=687012worker2 # [ 161.160136] nix-grpc-daemon[1951]: time=2026-09-20T13:34:46.684Z level=INFO msg="Upload complete. (365ms)"7013lb # [ 161.180320] niks3-server[1123]: 2026/09/20 13:34:46 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7014worker2 # [ 161.161975] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37286 duration_s=0 paths=1 nar_bytes_in=7527015worker2 # [ 161.165918] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372867016worker2 # [ 161.171858] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)7017worker2 # [ 161.174199] nix-daemon[858]: reaped child process 2030, status = succeeded7018worker2 # [ 161.179783] nix-daemon[858]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)7019client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7020client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7021client # farm-top> LOG-farm-top-drain7022worker2 # [ 162.795446] nix-daemon[858]: reaped child process 2044, status = succeeded7023worker2 # [ 162.929617] nix-daemon[858]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)7024worker2 # [ 162.942525] nix-daemon[858]: reaped child process 2062, status = succeeded7025lb # [ 162.976408] niks3-server[1123]: 2026/09/20 13:34:48 INFO Received uploads request method=POST path=/api/pending_closures7026worker2 # [ 162.965460] nix-grpc-daemon[1951]: time=2026-09-20T13:34:48.489Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7027worker2 # [ 162.967352] nix-grpc-daemon[1951]: time=2026-09-20T13:34:48.491Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7028lb # [ 163.013163] niks3-server[1123]: 2026/09/20 13:34:48 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7029lb # [ 163.018597] niks3-server[1123]: 2026/09/20 13:34:48 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7030lb # [ 163.032336] niks3-server[1123]: 2026/09/20 13:34:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7031worker2 # [ 163.017147] nix-grpc-daemon[1951]: time=2026-09-20T13:34:48.541Z level=INFO msg="Uploading 1 narinfos"7032lb # [ 163.036120] niks3-server[1123]: 2026/09/20 13:34:48 INFO Signed narinfos id=69 count=17033lb # [ 163.041591] niks3-server[1123]: 2026/09/20 13:34:48 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7034lb # [ 163.050113] niks3-server[1123]: 2026/09/20 13:34:48 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7035lb # [ 163.055181] niks3-server[1123]: 2026/09/20 13:34:48 INFO Completed upload id=697036worker2 # [ 163.038291] nix-grpc-daemon[1951]: time=2026-09-20T13:34:48.562Z level=INFO msg="Upload complete. (250ms)"7037lb # [ 163.058429] niks3-server[1123]: 2026/09/20 13:34:48 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7038worker2 # [ 163.044384] nix-grpc-daemon[1895]: ts=2026-09-20T13:34:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37286 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=17039worker2 # [ 163.056821] nix-daemon[858]: reaped child process 2042, status = succeeded7040client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.29 seconds)7041client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7042client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.18 seconds)7043worker1: must succeed: rm /nix/.rw-store/fill7044worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)7045worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7046worker1 # [ 163.809940] nix-grpc-daemon[1350]: ts=2026-09-20T13:34:49Z level=info event=healthy reason=min_free7047worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)7048(finished: subtest: low disk drains a worker and builds go to the other, in 9.87 seconds)7049(finished: run the VM test script, in 165.72 seconds)7050test script finished in 165.98s7051cleanup7052kill QemuMachine (pid 49)7053client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7054kill QemuMachine (pid 50)7055lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7056kill QemuMachine (pid 52)7057worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7058kill QemuMachine (pid 51)7059worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7060(finished: cleanup, in 1.11 seconds)7061additionally exposed symbols:7062 client, lb, worker1, worker2,7063 vlan1,7064 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh