vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #392
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.01 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Disk image does not exist, creating the virtualisation disk image...14worker2: QEMU running (pid 51)15worker1: QEMU running (pid 49)16worker2 # Formatting '/build/vm-state-worker2/tmp.MscljpRps9', fmt=raw size=107374182417worker1 # Formatting '/build/vm-state-worker1/tmp.asaXROMU0N', fmt=raw size=107374182418lb # Disk image does not exist, creating the virtualisation disk image...19worker2 # mke2fs 1.47.4 (6-Mar-2025)20client: QEMU running (pid 52)21lb: QEMU running (pid 50)22client # Disk image does not exist, creating the virtualisation disk image...23worker1 # mke2fs 1.47.4 (6-Mar-2025)24worker2 # Discarding device blocks: 0/262144 done25lb # Formatting '/build/vm-state-lb/tmp.uGjGeNs5Ks', fmt=raw size=107374182426client # Formatting '/build/vm-state-client/tmp.uJu5sPStHs', fmt=raw size=107374182427client # mke2fs 1.47.4 (6-Mar-2025)28worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes29lb # mke2fs 1.47.4 (6-Mar-2025)30lb # Discarding device blocks: 0/262144 done31client # Discarding device blocks: 0/262144 done32worker1 # Discarding device blocks: 0/262144 done33worker2 # Filesystem UUID: 5fd6ac4b-0f1e-4014-96b8-3efb70ce294f34client # Creating filesystem with 262144 4k blocks and 65536 inodes35worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes36worker2 # Superblock backups stored on blocks:37client # Filesystem UUID: 224d6ce9-fd00-40c1-bc06-3685d99f74c738worker1 # Filesystem UUID: aa9f7eca-7407-4473-9e5a-cc243630f32b39worker2 # 32768, 98304, 163840, 22937640client # Superblock backups stored on blocks:41(finished: start all VMs, in 0.43 seconds)42lb # Creating filesystem with 262144 4k blocks and 65536 inodes43client # 32768, 98304, 163840, 22937644worker1 # Superblock backups stored on blocks:45worker2 # 46client: waiting for unit network-addresses-eth1.service47worker1 # 32768, 98304, 163840, 22937648worker2 # Allocating group tables: 0/8 done49client: waiting for the VM to finish booting50worker1 # 51worker2 # Writing inode tables: 0/8 done52lb # Filesystem UUID: 6a2479e4-ad18-428d-84c1-8aa1abf4502753client # 54worker1 # Allocating group tables: 0/8 done55worker2 # Creating journal (8192 blocks): done56lb # Superblock backups stored on blocks:57client # Allocating group tables: 0/8 done58worker1 # Writing inode tables: 0/8 done59worker2 # Writing superblocks and filesystem accounting information: 0/8 done60lb # 32768, 98304, 163840, 22937661client # Writing inode tables: 0/8 done62worker1 # Creating journal (8192 blocks): done63lb # 64worker2 # 65client # Creating journal (8192 blocks): done66worker1 # Writing superblocks and filesystem accounting information: 0/8 done67lb # Allocating group tables: 0/8 done68worker2 # Virtualisation disk image created.69client # Writing superblocks and filesystem accounting information: 0/8 done70worker1 # 71client # 72worker1 # Virtualisation disk image created.73client # Virtualisation disk image created.74lb # Writing inode tables: 0/8 done75lb # Creating journal (8192 blocks): done76lb # Writing superblocks and filesystem accounting information: 0/8 done77lb # 78lb # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker1 # 86worker1 # 87worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0088worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90worker1 # 91worker1 # 92worker1 # 93worker1 # 94worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...95worker2 # 96worker2 # 97worker2 # 98worker2 # 99worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00100worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00101lb # 102lb # 103lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00104worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105worker2 # 106worker2 # 107worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...108worker1 # 109worker1 # 110client # 111client # 112client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00113lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...114lb # 115lb # 116lb # 117lb # 118lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00119client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...120client # 121client # 122client # 123client # 124client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00125client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126client # 127lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...128lb # 129lb # 130client # 131worker2 # Booting from ROM...132worker1 # Booting from ROM...133worker1 # Probing EDD (edd=off to disable)... ok134worker2 # Probing EDD (edd=off to disable)... ok135lb # Booting from ROM...136client # Booting from ROM...137lb # Probing EDD (edd=off to disable)... ok138client # Probing EDD (edd=off to disable)... ok139worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026140worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026141worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8kpqmrhi2ysjv2lzzbs4kwkqigrnq7rp-nixos-system-worker1-test/init regInfo=/nix/store/73z9971ny8gkwdzknxyrbhzswk2v5jr4-closure-info/registration console=ttyS0,115200n8 console=tty0142worker1 # [ 0.000000] BIOS-provided physical RAM map:143worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/740kgpv5n4wafyn7r0lx6yn52fgir5zj-nixos-system-worker2-test/init regInfo=/nix/store/3ha7iid6n7y7pck1mk7866h4f1x97961-closure-info/registration console=ttyS0,115200n8 console=tty0144worker2 # [ 0.000000] BIOS-provided physical RAM map:145worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable146worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable147worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved148worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved149worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved150worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved151worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable152worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable153worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved154client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026155worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved156worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved157worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved158worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved159worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved160worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved161worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved162worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved163client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test/init regInfo=/nix/store/b7672wjba7g7xvs0xbisc17mbs2br5bd-closure-info/registration console=ttyS0,115200n8 console=tty0164worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved165client # [ 0.000000] BIOS-provided physical RAM map:166worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved167worker2 # [ 0.000000] NX (Execute Disable) protection: active168client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable169worker2 # [ 0.000000] APIC: Static calls initialized170worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved171worker2 # [ 0.000000] SMBIOS 2.8 present.172client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved173worker1 # [ 0.000000] NX (Execute Disable) protection: active174client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved175worker1 # [ 0.000000] APIC: Static calls initialized176worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014177worker1 # [ 0.000000] SMBIOS 2.8 present.178worker2 # [ 0.000000] DMI: Memory slots populated: 1/1179client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable180lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026181worker2 # [ 0.000000] Hypervisor detected: KVM182client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved183worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000184worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014185worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00186client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved187worker1 # [ 0.000000] DMI: Memory slots populated: 1/1188worker2 # [ 0.000001] kvm-clock: using sched offset of 581679884 cycles189worker1 # [ 0.000000] Hypervisor detected: KVM190client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved191worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000192client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved193worker2 # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns194worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00195lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0196worker2 # [ 0.000007] tsc: Detected 2400.012 MHz processor197client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved198worker1 # [ 0.000000] kvm-clock: using sched offset of 579398696 cycles199lb # [ 0.000000] BIOS-provided physical RAM map:200client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved201worker2 # [ 0.000925] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000202lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable203client # [ 0.000000] NX (Execute Disable) protection: active204worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns205client # [ 0.000000] APIC: Static calls initialized206worker2 # [ 0.000956] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs207worker1 # [ 0.000006] tsc: Detected 2400.012 MHz processor208lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved209client # [ 0.000000] SMBIOS 2.8 present.210worker1 # [ 0.000928] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000211worker2 # [ 0.000960] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT212lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved213client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014214worker2 # [ 0.003103] found SMP MP-table at [mem 0x000f5450-0x000f545f]215worker1 # [ 0.000960] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs216client # [ 0.000000] DMI: Memory slots populated: 1/1217lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable218client # [ 0.000000] Hypervisor detected: KVM219worker2 # [ 0.003120] Using GB pages for direct mapping220worker1 # [ 0.000963] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT221client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000222lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved223worker2 # [ 0.003202] RAMDISK: [mem 0x5e367000-0x5ffcffff]224worker1 # [ 0.003115] found SMP MP-table at [mem 0x000f5450-0x000f545f]225client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00226worker2 # [ 0.003212] ACPI: Early table checksum verification disabled227worker1 # [ 0.003131] Using GB pages for direct mapping228lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved229client # [ 0.000000] kvm-clock: using sched offset of 575635433 cycles230worker1 # [ 0.003230] RAMDISK: [mem 0x5e367000-0x5ffcffff]231worker2 # [ 0.003216] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )232lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved233worker1 # [ 0.003240] ACPI: Early table checksum verification disabled234client # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns235lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved236worker1 # [ 0.003245] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )237worker2 # [ 0.003221] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238client # [ 0.000006] tsc: Detected 2400.012 MHz processor239lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved240worker1 # [ 0.003250] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)241client # [ 0.000921] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000242worker2 # [ 0.003226] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)243lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved244lb # [ 0.000000] NX (Execute Disable) protection: active245worker2 # [ 0.003234] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)246worker1 # [ 0.003255] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)247lb # [ 0.000000] APIC: Static calls initialized248client # [ 0.000952] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs249worker1 # [ 0.003263] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)250lb # [ 0.000000] SMBIOS 2.8 present.251worker2 # [ 0.003237] ACPI: FACS 0x000000005FFE0000 000040252client # [ 0.000955] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT253worker1 # [ 0.003266] ACPI: FACS 0x000000005FFE0000 000040254client # [ 0.003099] found SMP MP-table at [mem 0x000f5450-0x000f545f]255worker2 # [ 0.003238] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)256client # [ 0.003114] Using GB pages for direct mapping257lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014258client # [ 0.003255] RAMDISK: [mem 0x5e366000-0x5ffcffff]259worker1 # [ 0.003267] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)260lb # [ 0.000000] DMI: Memory slots populated: 1/1261worker2 # [ 0.003240] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)262lb # [ 0.000000] Hypervisor detected: KVM263client # [ 0.003264] ACPI: Early table checksum verification disabled264worker1 # [ 0.003269] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)265lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000266worker2 # [ 0.003242] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)267client # [ 0.003268] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )268lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00269worker1 # [ 0.003271] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)270worker2 # [ 0.003244] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)271lb # [ 0.000001] kvm-clock: using sched offset of 585452475 cycles272client # [ 0.003272] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)273worker1 # [ 0.003273] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)274worker2 # [ 0.003245] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]275client # [ 0.003278] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)276lb # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns277worker1 # [ 0.003274] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]278worker2 # [ 0.003246] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]279lb # [ 0.000006] tsc: Detected 2400.012 MHz processor280client # [ 0.003286] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)281worker2 # [ 0.003247] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]282worker1 # [ 0.003275] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]283lb # [ 0.000911] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000284client # [ 0.003288] ACPI: FACS 0x000000005FFE0000 000040285worker2 # [ 0.003247] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]286worker1 # [ 0.003276] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]287lb # [ 0.000944] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs288client # [ 0.003290] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)289worker2 # [ 0.003248] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]290worker1 # [ 0.003276] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]291lb # [ 0.000947] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT292client # [ 0.003291] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)293worker2 # [ 0.003249] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]294worker1 # [ 0.003277] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]295lb # [ 0.003060] found SMP MP-table at [mem 0x000f5450-0x000f545f]296lb # [ 0.003075] Using GB pages for direct mapping297worker2 # [ 0.003249] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]298client # [ 0.003293] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)299worker1 # [ 0.003278] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]300lb # [ 0.003247] RAMDISK: [mem 0x5e367000-0x5ffcffff]301worker2 # [ 0.003511] No NUMA configuration found302worker1 # [ 0.003278] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]303client # [ 0.003295] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)304lb # [ 0.003256] ACPI: Early table checksum verification disabled305worker2 # [ 0.003512] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]306worker1 # [ 0.003533] No NUMA configuration found307lb # [ 0.003260] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )308client # [ 0.003296] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]309worker2 # [ 0.003515] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]310worker1 # [ 0.003534] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]311worker2 # [ 0.005830] Zone ranges:312client # [ 0.003298] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]313lb # [ 0.003265] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)314worker1 # [ 0.003537] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]315worker2 # [ 0.005831] DMA [mem 0x0000000000001000-0x0000000000ffffff]316worker1 # [ 0.005916] Zone ranges:317client # [ 0.003298] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]318worker2 # [ 0.005833] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]319lb # [ 0.003270] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)320worker1 # [ 0.005918] DMA [mem 0x0000000000001000-0x0000000000ffffff]321worker2 # [ 0.005834] Normal empty322client # [ 0.003299] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]323worker2 # [ 0.005844] Device empty324lb # [ 0.003278] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)325worker1 # [ 0.005921] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]326worker2 # [ 0.005845] Movable zone start for each node327worker1 # [ 0.005923] Normal empty328client # [ 0.003299] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]329lb # [ 0.003281] ACPI: FACS 0x000000005FFE0000 000040330worker2 # [ 0.005846] Early memory node ranges331worker1 # [ 0.005924] Device empty332client # [ 0.003300] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]333worker2 # [ 0.005846] node 0: [mem 0x0000000000001000-0x000000000009efff]334worker1 # [ 0.005925] Movable zone start for each node335lb # [ 0.003282] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)336worker1 # [ 0.005926] Early memory node ranges337worker2 # [ 0.005848] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]338client # [ 0.003300] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]339worker1 # [ 0.005926] node 0: [mem 0x0000000000001000-0x000000000009efff]340client # [ 0.003578] No NUMA configuration found341lb # [ 0.003284] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)342worker2 # [ 0.005849] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]343worker1 # [ 0.005928] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]344client # [ 0.003578] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]345lb # [ 0.003286] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)346worker2 # [ 0.005872] On node 0, zone DMA: 1 pages in unavailable ranges347client # [ 0.003581] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]348worker1 # [ 0.005929] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]349worker2 # [ 0.006146] On node 0, zone DMA: 97 pages in unavailable ranges350client # [ 0.006155] Zone ranges:351lb # [ 0.003288] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)352worker1 # [ 0.005952] On node 0, zone DMA: 1 pages in unavailable ranges353worker2 # [ 0.036478] On node 0, zone DMA32: 40 pages in unavailable ranges354client # [ 0.006156] DMA [mem 0x0000000000001000-0x0000000000ffffff]355lb # [ 0.003289] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]356worker2 # [ 0.037009] ACPI: PM-Timer IO Port: 0x608357worker1 # [ 0.006226] On node 0, zone DMA: 97 pages in unavailable ranges358client # [ 0.006158] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]359client # [ 0.006159] Normal empty360worker2 # [ 0.037024] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])361worker1 # [ 0.036457] On node 0, zone DMA32: 40 pages in unavailable ranges362lb # [ 0.003290] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]363client # [ 0.006160] Device empty364worker1 # [ 0.036986] ACPI: PM-Timer IO Port: 0x608365worker2 # [ 0.037052] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23366client # [ 0.006161] Movable zone start for each node367lb # [ 0.003291] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]368worker1 # [ 0.037001] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])369client # [ 0.006162] Early memory node ranges370worker2 # [ 0.037055] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)371lb # [ 0.003291] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]372client # [ 0.006162] node 0: [mem 0x0000000000001000-0x000000000009efff]373worker1 # [ 0.037034] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23374worker2 # [ 0.037057] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)375lb # [ 0.003292] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]376client # [ 0.006163] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]377worker1 # [ 0.037037] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)378worker2 # [ 0.037059] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)379lb # [ 0.003292] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]380client # [ 0.006165] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]381worker1 # [ 0.037038] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)382worker2 # [ 0.037060] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)383client # [ 0.006187] On node 0, zone DMA: 1 pages in unavailable ranges384lb # [ 0.003293] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]385worker1 # [ 0.037040] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)386lb # [ 0.003546] No NUMA configuration found387worker2 # [ 0.037061] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)388client # [ 0.006482] On node 0, zone DMA: 97 pages in unavailable ranges389worker1 # [ 0.037041] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)390lb # [ 0.003547] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]391worker2 # [ 0.037064] ACPI: Using ACPI (MADT) for SMP configuration information392client # [ 0.036269] On node 0, zone DMA32: 40 pages in unavailable ranges393worker1 # [ 0.037042] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)394worker2 # [ 0.037065] ACPI: HPET id: 0x8086a201 base: 0xfed00000395client # [ 0.036808] ACPI: PM-Timer IO Port: 0x608396lb # [ 0.003550] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]397lb # [ 0.006027] Zone ranges:398worker2 # [ 0.037070] TSC deadline timer available399client # [ 0.036822] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])400worker1 # [ 0.037044] ACPI: Using ACPI (MADT) for SMP configuration information401worker2 # [ 0.037075] CPU topo: Max. logical packages: 1402lb # [ 0.006029] DMA [mem 0x0000000000001000-0x0000000000ffffff]403worker1 # [ 0.037046] ACPI: HPET id: 0x8086a201 base: 0xfed00000404client # [ 0.036858] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23405worker2 # [ 0.037075] CPU topo: Max. logical dies: 1406lb # [ 0.006030] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]407worker1 # [ 0.037051] TSC deadline timer available408lb # [ 0.006032] Normal empty409worker2 # [ 0.037076] CPU topo: Max. dies per package: 1410client # [ 0.036861] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)411worker1 # [ 0.037056] CPU topo: Max. logical packages: 1412lb # [ 0.006033] Device empty413worker2 # [ 0.037080] CPU topo: Max. threads per core: 1414worker1 # [ 0.037056] CPU topo: Max. logical dies: 1415lb # [ 0.006033] Movable zone start for each node416client # [ 0.036863] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)417worker2 # [ 0.037081] CPU topo: Num. cores per package: 1418lb # [ 0.006034] Early memory node ranges419worker1 # [ 0.037057] CPU topo: Max. dies per package: 1420client # [ 0.036864] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)421worker2 # [ 0.037081] CPU topo: Num. threads per package: 1422worker1 # [ 0.037061] CPU topo: Max. threads per core: 1423lb # [ 0.006035] node 0: [mem 0x0000000000001000-0x000000000009efff]424worker2 # [ 0.037081] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs425client # [ 0.036865] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)426worker1 # [ 0.037062] CPU topo: Num. cores per package: 1427lb # [ 0.006036] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]428worker1 # [ 0.037062] CPU topo: Num. threads per package: 1429worker2 # [ 0.037106] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()430client # [ 0.036866] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)431lb # [ 0.006037] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]432worker1 # [ 0.037062] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs433client # [ 0.036869] ACPI: Using ACPI (MADT) for SMP configuration information434worker2 # [ 0.037144] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]435lb # [ 0.006060] On node 0, zone DMA: 1 pages in unavailable ranges436client # [ 0.036870] ACPI: HPET id: 0x8086a201 base: 0xfed00000437worker1 # [ 0.037087] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()438lb # [ 0.006362] On node 0, zone DMA: 97 pages in unavailable ranges439worker2 # [ 0.037146] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]440client # [ 0.036875] TSC deadline timer available441client # [ 0.036880] CPU topo: Max. logical packages: 1442worker1 # [ 0.037127] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]443worker2 # [ 0.037147] [mem 0x60000000-0xafffffff] available for PCI devices444lb # [ 0.036000] On node 0, zone DMA32: 40 pages in unavailable ranges445client # [ 0.036881] CPU topo: Max. logical dies: 1446lb # [ 0.036538] ACPI: PM-Timer IO Port: 0x608447worker2 # [ 0.037150] Booting paravirtualized kernel on KVM448worker1 # [ 0.037129] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]449client # [ 0.036881] CPU topo: Max. dies per package: 1450lb # [ 0.036553] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])451client # [ 0.036885] CPU topo: Max. threads per core: 1452worker1 # [ 0.037130] [mem 0x60000000-0xafffffff] available for PCI devices453worker2 # [ 0.037153] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns454lb # [ 0.036588] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23455client # [ 0.036886] CPU topo: Num. cores per package: 1456worker1 # [ 0.037132] Booting paravirtualized kernel on KVM457worker2 # [ 0.042255] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1458client # [ 0.036886] CPU topo: Num. threads per package: 1459lb # [ 0.036590] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)460client # [ 0.036887] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs461worker2 # [ 0.044743] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152462lb # [ 0.036592] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)463worker1 # [ 0.037136] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns464worker2 # [ 0.044794] kvm-guest: PV spinlocks disabled, single CPU465lb # [ 0.036593] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)466client # [ 0.036912] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()467worker1 # [ 0.042193] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1468lb # [ 0.036595] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)469client # [ 0.036951] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]470worker1 # [ 0.044581] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152471lb # [ 0.036595] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)472client # [ 0.036953] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]473worker1 # [ 0.044642] kvm-guest: PV spinlocks disabled, single CPU474lb # [ 0.036598] ACPI: Using ACPI (MADT) for SMP configuration information475client # [ 0.036954] [mem 0x60000000-0xafffffff] available for PCI devices476lb # [ 0.036599] ACPI: HPET id: 0x8086a201 base: 0xfed00000477client # [ 0.036956] Booting paravirtualized kernel on KVM478lb # [ 0.036605] TSC deadline timer available479worker2 # [ 0.044795] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/740kgpv5n4wafyn7r0lx6yn52fgir5zj-nixos-system-worker2-test/init regInfo=/nix/store/3ha7iid6n7y7pck1mk7866h4f1x97961-closure-info/registration console=ttyS0,115200n8 console=tty0480lb # [ 0.036610] CPU topo: Max. logical packages: 1481client # [ 0.036960] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns482lb # [ 0.036611] CPU topo: Max. logical dies: 1483lb # [ 0.036611] CPU topo: Max. dies per package: 1484client # [ 0.042005] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1485worker2 # [ 0.044913] Unknown kernel command line parameters "regInfo=/nix/store/3ha7iid6n7y7pck1mk7866h4f1x97961-closure-info/registration", will be passed to user space.486lb # [ 0.036615] CPU topo: Max. threads per core: 1487worker2 # [ 0.044928] random: crng init done488client # [ 0.044349] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152489worker1 # [ 0.044644] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8kpqmrhi2ysjv2lzzbs4kwkqigrnq7rp-nixos-system-worker1-test/init regInfo=/nix/store/73z9971ny8gkwdzknxyrbhzswk2v5jr4-closure-info/registration console=ttyS0,115200n8 console=tty0490lb # [ 0.036616] CPU topo: Num. cores per package: 1491worker2 # [ 0.044929] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes492client # [ 0.044398] kvm-guest: PV spinlocks disabled, single CPU493lb # [ 0.036616] CPU topo: Num. threads per package: 1494worker2 # [ 0.047319] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)495lb # [ 0.036617] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs496worker1 # [ 0.044751] Unknown kernel command line parameters "regInfo=/nix/store/73z9971ny8gkwdzknxyrbhzswk2v5jr4-closure-info/registration", will be passed to user space.497worker1 # [ 0.044766] random: crng init done498lb # [ 0.036642] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()499worker2 # [ 0.048446] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)500worker2 # [ 0.048487] Fallback order for Node 0: 0501worker1 # [ 0.044767] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes502lb # [ 0.036680] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]503worker2 # [ 0.048491] Built 1 zonelists, mobility grouping on. Total pages: 393078504worker2 # [ 0.048492] Policy zone: DMA32505worker1 # [ 0.047128] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)506lb # [ 0.036682] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]507client # [ 0.044400] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test/init regInfo=/nix/store/b7672wjba7g7xvs0xbisc17mbs2br5bd-closure-info/registration console=ttyS0,115200n8 console=tty0508worker2 # [ 0.051141] mem auto-init: stack:all(zero), heap alloc:on, heap free:off509lb # [ 0.036684] [mem 0x60000000-0xafffffff] available for PCI devices510worker1 # [ 0.048443] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)511lb # [ 0.036686] Booting paravirtualized kernel on KVM512worker1 # [ 0.048483] Fallback order for Node 0: 0513worker2 # [ 0.054811] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1514client # [ 0.044507] Unknown kernel command line parameters "regInfo=/nix/store/b7672wjba7g7xvs0xbisc17mbs2br5bd-closure-info/registration", will be passed to user space.515worker2 # [ 0.058456] allocated 3145728 bytes of page_ext516client # [ 0.044532] random: crng init done517worker1 # [ 0.048487] Built 1 zonelists, mobility grouping on. Total pages: 393078518lb # [ 0.036689] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns519worker1 # [ 0.048488] Policy zone: DMA32520worker2 # [ 0.069521] ftrace: allocating 48731 entries in 192 pages521client # [ 0.044533] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes522worker2 # [ 0.069525] ftrace: allocated 192 pages with 2 groups523lb # [ 0.041711] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1524worker1 # [ 0.051115] mem auto-init: stack:all(zero), heap alloc:on, heap free:off525worker2 # [ 0.070558] Dynamic Preempt: lazy526client # [ 0.046827] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)527lb # [ 0.043974] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152528worker2 # [ 0.070750] rcu: Preemptible hierarchical RCU implementation.529worker1 # [ 0.054756] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1530worker2 # [ 0.070750] rcu: RCU event tracing is enabled.531lb # [ 0.044027] kvm-guest: PV spinlocks disabled, single CPU532worker1 # [ 0.058277] allocated 3145728 bytes of page_ext533client # [ 0.048105] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)534client # [ 0.048145] Fallback order for Node 0: 0535worker2 # [ 0.070751] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.536worker1 # [ 0.069283] ftrace: allocating 48731 entries in 192 pages537worker2 # [ 0.070753] Trampoline variant of Tasks RCU enabled.538client # [ 0.048149] Built 1 zonelists, mobility grouping on. Total pages: 393078539worker1 # [ 0.069286] ftrace: allocated 192 pages with 2 groups540client # [ 0.048150] Policy zone: DMA32541worker2 # [ 0.070754] Rude variant of Tasks RCU enabled.542worker1 # [ 0.070290] Dynamic Preempt: lazy543worker2 # [ 0.070754] Tracing variant of Tasks RCU enabled.544client # [ 0.050793] mem auto-init: stack:all(zero), heap alloc:on, heap free:off545worker1 # [ 0.070430] rcu: Preemptible hierarchical RCU implementation.546worker1 # [ 0.070431] rcu: RCU event tracing is enabled.547client # [ 0.054425] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1548worker2 # [ 0.070755] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.549lb # [ 0.044028] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0550client # [ 0.058009] allocated 3145728 bytes of page_ext551worker1 # [ 0.070432] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.552worker2 # [ 0.070756] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1553client # [ 0.069092] ftrace: allocating 48731 entries in 192 pages554worker1 # [ 0.070433] Trampoline variant of Tasks RCU enabled.555client # [ 0.069097] ftrace: allocated 192 pages with 2 groups556worker2 # [ 0.070775] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.557worker1 # [ 0.070434] Rude variant of Tasks RCU enabled.558lb # [ 0.044134] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.559client # [ 0.070116] Dynamic Preempt: lazy560lb # [ 0.044148] random: crng init done561worker1 # [ 0.070434] Tracing variant of Tasks RCU enabled.562client # [ 0.070295] rcu: Preemptible hierarchical RCU implementation.563worker2 # [ 0.070777] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.564lb # [ 0.044149] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes565client # [ 0.070296] rcu: RCU event tracing is enabled.566worker1 # [ 0.070435] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.567worker2 # [ 0.070778] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.568client # [ 0.070296] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.569lb # [ 0.046415] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)570worker1 # [ 0.070436] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1571worker2 # [ 0.075640] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16572client # [ 0.070298] Trampoline variant of Tasks RCU enabled.573lb # [ 0.047543] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)574client # [ 0.070299] Rude variant of Tasks RCU enabled.575worker2 # [ 0.075948] rcu: srcu_init: Setting srcu_struct sizes based on contention.576worker1 # [ 0.070453] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.577lb # [ 0.047582] Fallback order for Node 0: 0578client # [ 0.070299] Tracing variant of Tasks RCU enabled.579lb # [ 0.047586] Built 1 zonelists, mobility grouping on. Total pages: 393078580worker2 # [ 0.075954] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns581worker1 # [ 0.070455] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.582lb # [ 0.047587] Policy zone: DMA32583client # [ 0.070300] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.584lb # [ 0.050367] mem auto-init: stack:all(zero), heap alloc:on, heap free:off585client # [ 0.070301] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1586worker2 # [ 0.076075] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)587worker1 # [ 0.070456] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.588worker2 # [ 0.080076] Console: colour VGA+ 80x25589lb # [ 0.053913] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1590client # [ 0.070329] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.591worker1 # [ 0.075534] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16592worker2 # [ 0.080079] printk: legacy console [tty0] enabled593lb # [ 0.057401] allocated 3145728 bytes of page_ext594worker2 # [ 0.113548] printk: legacy console [ttyS0] enabled595lb # [ 0.068530] ftrace: allocating 48731 entries in 192 pages596worker1 # [ 0.075854] rcu: srcu_init: Setting srcu_struct sizes based on contention.597client # [ 0.070331] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.598lb # [ 0.068534] ftrace: allocated 192 pages with 2 groups599worker2 # [ 0.224459] ACPI: Core revision 20250807600lb # [ 0.069501] Dynamic Preempt: lazy601client # [ 0.070332] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.602worker1 # [ 0.075863] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns603lb # [ 0.069635] rcu: Preemptible hierarchical RCU implementation.604client # [ 0.075474] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16605lb # [ 0.069636] rcu: RCU event tracing is enabled.606worker2 # [ 0.225468] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns607worker1 # [ 0.075983] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)608client # [ 0.075796] rcu: srcu_init: Setting srcu_struct sizes based on contention.609lb # [ 0.069636] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.610worker1 # [ 0.080071] Console: colour VGA+ 80x25611worker2 # [ 0.227119] APIC: Switch to symmetric I/O mode setup612lb # [ 0.069638] Trampoline variant of Tasks RCU enabled.613worker1 # [ 0.080075] printk: legacy console [tty0] enabled614client # [ 0.075806] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns615worker2 # [ 0.228168] x2apic enabled616lb # [ 0.069638] Rude variant of Tasks RCU enabled.617worker1 # [ 0.113762] printk: legacy console [ttyS0] enabled618lb # [ 0.069638] Tracing variant of Tasks RCU enabled.619worker2 # [ 0.228975] APIC: Switched APIC routing to: physical x2apic620client # [ 0.075931] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)621worker1 # [ 0.232102] ACPI: Core revision 20250807622client # [ 0.080035] Console: colour VGA+ 80x25623lb # [ 0.069639] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.624client # [ 0.080039] printk: legacy console [tty0] enabled625lb # [ 0.069640] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1626client # [ 0.113673] printk: legacy console [ttyS0] enabled627worker2 # [ 0.230955] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1628worker1 # [ 0.233110] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns629client # [ 0.222507] ACPI: Core revision 20250807630lb # [ 0.069650] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.631worker1 # [ 0.235023] APIC: Switch to symmetric I/O mode setup632worker2 # [ 0.232000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns633lb # [ 0.069652] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.634worker1 # [ 0.236151] x2apic enabled635client # [ 0.223483] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns636lb # [ 0.069653] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.637worker2 # [ 0.233795] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)638worker1 # [ 0.236999] APIC: Switched APIC routing to: physical x2apic639client # [ 0.225239] APIC: Switch to symmetric I/O mode setup640lb # [ 0.074573] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16641client # [ 0.226304] x2apic enabled642worker2 # [ 0.235888] x86/cpu: User Mode Instruction Prevention (UMIP) activated643lb # [ 0.074862] rcu: srcu_init: Setting srcu_struct sizes based on contention.644client # [ 0.227122] APIC: Switched APIC routing to: physical x2apic645worker1 # [ 0.239150] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1646worker2 # [ 0.236920] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127647lb # [ 0.074870] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns648worker2 # [ 0.237793] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0649lb # [ 0.074985] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)650client # [ 0.229129] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1651worker1 # [ 0.240314] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns652lb # [ 0.078973] Console: colour VGA+ 80x25653lb # [ 0.078977] printk: legacy console [tty0] enabled654worker2 # [ 0.238796] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto655lb # [ 0.112600] printk: legacy console [ttyS0] enabled656worker1 # [ 0.242308] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)657client # [ 0.230169] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns658lb # [ 0.222425] ACPI: Core revision 20250807659worker2 # [ 0.240792] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl660worker1 # [ 0.243399] x86/cpu: User Mode Instruction Prevention (UMIP) activated661client # [ 0.231955] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)662worker2 # [ 0.241792] Transient Scheduler Attacks: Vulnerable: No microcode663lb # [ 0.223387] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns664worker1 # [ 0.245436] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127665client # [ 0.234053] x86/cpu: User Mode Instruction Prevention (UMIP) activated666worker2 # [ 0.242792] Spectre V2 : Mitigation: Enhanced / Automatic IBRS667lb # [ 0.225097] APIC: Switch to symmetric I/O mode setup668worker1 # [ 0.246305] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0669client # [ 0.235095] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127670worker2 # [ 0.243792] Speculative Return Stack Overflow: Mitigation: Safe RET671lb # [ 0.226154] x2apic enabled672client # [ 0.235952] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0673lb # [ 0.226958] APIC: Switched APIC routing to: physical x2apic674worker1 # [ 0.247309] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto675client # [ 0.236957] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto676worker1 # [ 0.249305] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl677lb # [ 0.229063] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1678worker2 # [ 0.244792] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization679client # [ 0.238952] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl680worker1 # [ 0.251305] Transient Scheduler Attacks: Vulnerable: No microcode681worker2 # [ 0.246800] Spectre V2 : Enabling IBPB for BPF682lb # [ 0.230188] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns683client # [ 0.239952] Transient Scheduler Attacks: Vulnerable: No microcode684worker1 # [ 0.252304] Spectre V2 : Mitigation: Enhanced / Automatic IBRS685client # [ 0.240952] Spectre V2 : Mitigation: Enhanced / Automatic IBRS686worker2 # [ 0.247792] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier687lb # [ 0.231953] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)688worker1 # [ 0.253305] Speculative Return Stack Overflow: Mitigation: Safe RET689client # [ 0.241952] Speculative Return Stack Overflow: Mitigation: Safe RET690worker2 # [ 0.248793] active return thunk: srso_alias_return_thunk691lb # [ 0.234044] x86/cpu: User Mode Instruction Prevention (UMIP) activated692worker2 # [ 0.250507] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'693lb # [ 0.235081] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127694worker2 # [ 0.251792] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'695lb # [ 0.235949] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0696worker2 # [ 0.252791] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'697worker1 # [ 0.254304] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization698client # [ 0.242952] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization699lb # [ 0.236953] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto700worker1 # [ 0.256313] Spectre V2 : Enabling IBPB for BPF701worker2 # [ 0.253792] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'702client # [ 0.244960] Spectre V2 : Enabling IBPB for BPF703worker2 # [ 0.254792] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'704lb # [ 0.238949] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl705worker1 # [ 0.257306] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier706client # [ 0.245953] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier707worker2 # [ 0.255791] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'708lb # [ 0.240949] Transient Scheduler Attacks: Vulnerable: No microcode709worker1 # [ 0.258306] active return thunk: srso_alias_return_thunk710client # [ 0.246953] active return thunk: srso_alias_return_thunk711lb # [ 0.241949] Spectre V2 : Mitigation: Enhanced / Automatic IBRS712worker2 # [ 0.256791] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'713worker1 # [ 0.260050] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'714client # [ 0.247955] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'715lb # [ 0.242949] Speculative Return Stack Overflow: Mitigation: Safe RET716worker2 # [ 0.258792] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256717worker1 # [ 0.261305] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'718client # [ 0.248952] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'719worker2 # [ 0.259792] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64720worker1 # [ 0.262304] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'721client # [ 0.249951] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'722worker2 # [ 0.260791] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512723worker1 # [ 0.263304] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'724client # [ 0.250952] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'725worker2 # [ 0.261792] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024726worker1 # [ 0.264305] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'727client # [ 0.251951] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'728lb # [ 0.243949] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization729worker2 # [ 0.262792] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8730lb # [ 0.245956] Spectre V2 : Enabling IBPB for BPF731client # [ 0.252952] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'732worker1 # [ 0.266305] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'733worker2 # [ 0.263792] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.734lb # [ 0.246950] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier735client # [ 0.254808] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'736worker1 # [ 0.267305] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'737lb # [ 0.247950] active return thunk: srso_alias_return_thunk738client # [ 0.255952] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256739worker1 # [ 0.268305] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256740client # [ 0.256951] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64741lb # [ 0.248893] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'742worker1 # [ 0.270166] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64743client # [ 0.257952] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512744lb # [ 0.249949] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'745worker1 # [ 0.271304] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512746client # [ 0.258952] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024747lb # [ 0.250949] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'748worker1 # [ 0.272304] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024749client # [ 0.259952] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8750lb # [ 0.251949] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'751worker1 # [ 0.273304] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8752lb # [ 0.252949] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'753client # [ 0.260951] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.754worker1 # [ 0.274304] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.755lb # [ 0.253949] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'756lb # [ 0.254948] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'757lb # [ 0.256949] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256758lb # [ 0.257949] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64759lb # [ 0.258949] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512760lb # [ 0.259948] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024761lb # [ 0.260949] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8762lb # [ 0.261949] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.763worker2 # [ 0.302055] Freeing SMP alternatives memory: 44K764worker2 # [ 0.302797] pid_max: default: 32768 minimum: 301765worker2 # [ 0.303884] LSM: initializing lsm=capability,landlock,yama,bpf,ima766worker2 # [ 0.304906] landlock: Up and running.767worker2 # [ 0.305590] Yama: becoming mindful.768worker2 # [ 0.306495] LSM support for eBPF active769worker2 # [ 0.307401] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)770worker2 # [ 0.308829] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)771client # [ 0.300755] Freeing SMP alternatives memory: 44K772worker1 # [ 0.313056] Freeing SMP alternatives memory: 44K773client # [ 0.301576] pid_max: default: 32768 minimum: 301774worker1 # [ 0.313963] pid_max: default: 32768 minimum: 301775client # [ 0.302048] LSM: initializing lsm=capability,landlock,yama,bpf,ima776worker2 # [ 0.311501] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)777worker1 # [ 0.314445] LSM: initializing lsm=capability,landlock,yama,bpf,ima778client # [ 0.303060] landlock: Up and running.779client # [ 0.304436] Yama: becoming mindful.780worker1 # [ 0.315428] landlock: Up and running.781worker2 # [ 0.312377] Performance Events: Fam17h+ core perfctr, AMD PMU driver.782worker1 # [ 0.316791] Yama: becoming mindful.783client # [ 0.305209] LSM support for eBPF active784worker2 # [ 0.312797] ... version: 2785worker1 # [ 0.317547] LSM support for eBPF active786worker2 # [ 0.313548] ... bit width: 48787client # [ 0.306077] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)788worker2 # [ 0.313795] ... generic counters: 6789worker1 # [ 0.318412] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)790worker2 # [ 0.314579] ... generic bitmap: 000000000000003f791client # [ 0.306990] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)792worker2 # [ 0.315542] ... fixed-purpose counters: 0793worker1 # [ 0.319344] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)794lb # [ 0.300436] Freeing SMP alternatives memory: 44K795worker2 # [ 0.315794] ... fixed-purpose bitmap: 0000000000000000796lb # [ 0.300953] pid_max: default: 32768 minimum: 301797worker2 # [ 0.316753] ... value mask: 0000ffffffffffff798client # [ 0.310110] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)799lb # [ 0.302051] LSM: initializing lsm=capability,landlock,yama,bpf,ima800worker2 # [ 0.317538] ... max period: 00007fffffffffff801worker1 # [ 0.322654] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)802lb # [ 0.303069] landlock: Up and running.803client # [ 0.311568] Performance Events: Fam17h+ core perfctr, AMD PMU driver.804worker2 # [ 0.317794] ... global_ctrl mask: 000000000000003f805lb # [ 0.303950] Yama: becoming mindful.806client # [ 0.311962] ... version: 2807worker1 # [ 0.323943] Performance Events: Fam17h+ core perfctr, AMD PMU driver.808worker2 # [ 0.318903] signal: max sigframe size: 3376809lb # [ 0.305057] LSM support for eBPF active810client # [ 0.312740] ... bit width: 48811worker1 # [ 0.324315] ... version: 2812worker2 # [ 0.319751] rcu: Hierarchical SRCU implementation.813client # [ 0.312954] ... generic counters: 6814worker1 # [ 0.325085] ... bit width: 48815worker2 # [ 0.320417] rcu: Max phase no-delay instances is 400.816lb # [ 0.305866] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)817client # [ 0.313708] ... generic bitmap: 000000000000003f818worker1 # [ 0.325308] ... generic counters: 6819client # [ 0.313954] ... fixed-purpose counters: 0820worker1 # [ 0.326106] ... generic bitmap: 000000000000003f821lb # [ 0.306986] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)822worker1 # [ 0.327069] ... fixed-purpose counters: 0823client # [ 0.314739] ... fixed-purpose bitmap: 0000000000000000824worker1 # [ 0.327307] ... fixed-purpose bitmap: 0000000000000000825client # [ 0.315787] ... value mask: 0000ffffffffffff826lb # [ 0.309668] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)827client # [ 0.316720] ... max period: 00007fffffffffff828worker1 # [ 0.328307] ... value mask: 0000ffffffffffff829client # [ 0.316955] ... global_ctrl mask: 000000000000003f830worker1 # [ 0.329307] ... max period: 00007fffffffffff831worker2 # [ 0.325882] smp: Bringing up secondary CPUs ...832lb # [ 0.310554] Performance Events: Fam17h+ core perfctr, AMD PMU driver.833client # [ 0.318059] signal: max sigframe size: 3376834worker1 # [ 0.330307] ... global_ctrl mask: 000000000000003f835worker2 # [ 0.326695] smp: Brought up 1 node, 1 CPU836lb # [ 0.310955] ... version: 2837client # [ 0.318970] rcu: Hierarchical SRCU implementation.838worker1 # [ 0.331436] signal: max sigframe size: 3376839lb # [ 0.311710] ... bit width: 48840worker2 # [ 0.327353] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)841client # [ 0.319834] rcu: Max phase no-delay instances is 400.842lb # [ 0.311952] ... generic counters: 6843worker1 # [ 0.332381] rcu: Hierarchical SRCU implementation.844lb # [ 0.312750] ... generic bitmap: 000000000000003f845worker1 # [ 0.333286] rcu: Max phase no-delay instances is 400.846lb # [ 0.313762] ... fixed-purpose counters: 0847worker2 # [ 0.328026] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)848lb # [ 0.314580] ... fixed-purpose bitmap: 0000000000000000849worker2 # [ 0.329059] devtmpfs: initialized850lb # [ 0.314952] ... value mask: 0000ffffffffffff851worker2 # [ 0.329966] x86/mm: Memory block size: 128MB852lb # [ 0.315939] ... max period: 00007fffffffffff853client # [ 0.325790] smp: Bringing up secondary CPUs ...854lb # [ 0.316725] ... global_ctrl mask: 000000000000003f855worker2 # [ 0.331966] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)856client # [ 0.326614] smp: Brought up 1 node, 1 CPU857lb # [ 0.317842] signal: max sigframe size: 3376858worker1 # [ 0.339152] smp: Bringing up secondary CPUs ...859client # [ 0.326957] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)860lb # [ 0.318632] rcu: Hierarchical SRCU implementation.861worker1 # [ 0.339977] smp: Brought up 1 node, 1 CPU862worker2 # [ 0.332827] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).863lb # [ 0.318956] rcu: Max phase no-delay instances is 400.864worker2 # [ 0.333895] pinctrl core: initialized pinctrl subsystem865worker1 # [ 0.340308] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)866client # [ 0.328131] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)867worker2 # [ 0.335060] PM: RTC time: 13:25:51, date: 2026-09-20868client # [ 0.329229] devtmpfs: initialized869worker1 # [ 0.341489] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)870client # [ 0.330066] x86/mm: Memory block size: 128MB871worker1 # [ 0.342580] devtmpfs: initialized872worker1 # [ 0.343555] x86/mm: Memory block size: 128MB873worker2 # [ 0.338658] NET: Registered PF_NETLINK/PF_ROUTE protocol family874client # [ 0.332041] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)875lb # [ 0.324983] smp: Bringing up secondary CPUs ...876lb # [ 0.325853] smp: Brought up 1 node, 1 CPU877worker2 # [ 0.340148] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations878client # [ 0.332981] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).879worker1 # [ 0.345447] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)880lb # [ 0.326498] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)881client # [ 0.334050] pinctrl core: initialized pinctrl subsystem882worker2 # [ 0.340814] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations883worker1 # [ 0.346337] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).884client # [ 0.335310] PM: RTC time: 13:25:51, date: 2026-09-20885worker1 # [ 0.347403] pinctrl core: initialized pinctrl subsystem886worker2 # [ 0.342077] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations887lb # [ 0.327158] Memory: 1454616K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)888worker2 # [ 0.342807] audit: initializing netlink subsys (disabled)889lb # [ 0.328239] devtmpfs: initialized890worker1 # [ 0.348630] PM: RTC time: 13:25:51, date: 2026-09-20891lb # [ 0.329186] x86/mm: Memory block size: 128MB892worker2 # [ 0.344094] thermal_sys: Registered thermal governor 'fair_share'893worker2 # [ 0.344097] thermal_sys: Registered thermal governor 'bang_bang'894client # [ 0.338942] NET: Registered PF_NETLINK/PF_ROUTE protocol family895worker2 # [ 0.344797] thermal_sys: Registered thermal governor 'step_wise'896lb # [ 0.331165] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)897worker1 # [ 0.352182] NET: Registered PF_NETLINK/PF_ROUTE protocol family898client # [ 0.340316] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations899worker2 # [ 0.345799] audit: type=2000 audit(1789910751.785:1): state=initialized audit_enabled=0 res=1900lb # [ 0.331986] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).901client # [ 0.340976] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations902worker2 # [ 0.347796] thermal_sys: Registered thermal governor 'user_space'903worker1 # [ 0.353780] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations904lb # [ 0.333050] pinctrl core: initialized pinctrl subsystem905worker2 # [ 0.347798] thermal_sys: Registered thermal governor 'power_allocator'906client # [ 0.342247] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations907lb # [ 0.334322] PM: RTC time: 13:25:51, date: 2026-09-20908worker1 # [ 0.354332] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations909worker2 # [ 0.348817] cpuidle: using governor menu910client # [ 0.342971] audit: initializing netlink subsys (disabled)911worker1 # [ 0.355595] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations912worker2 # [ 0.351302] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5913client # [ 0.344343] thermal_sys: Registered thermal governor 'fair_share'914worker1 # [ 0.356324] audit: initializing netlink subsys (disabled)915client # [ 0.344346] thermal_sys: Registered thermal governor 'bang_bang'916lb # [ 0.337820] NET: Registered PF_NETLINK/PF_ROUTE protocol family917worker2 # [ 0.352065] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]918client # [ 0.344956] thermal_sys: Registered thermal governor 'step_wise'919worker1 # [ 0.357679] thermal_sys: Registered thermal governor 'fair_share'920worker2 # [ 0.352799] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry921worker1 # [ 0.357681] thermal_sys: Registered thermal governor 'bang_bang'922lb # [ 0.339293] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations923client # [ 0.345959] audit: type=2000 audit(1789910751.796:1): state=initialized audit_enabled=0 res=1924worker2 # [ 0.353810] PCI: Using configuration type 1 for base access925client # [ 0.347956] thermal_sys: Registered thermal governor 'user_space'926lb # [ 0.339974] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations927worker1 # [ 0.358312] audit: type=2000 audit(1789910751.787:1): state=initialized audit_enabled=0 res=1928worker2 # [ 0.354995] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.929client # [ 0.347958] thermal_sys: Registered thermal governor 'power_allocator'930worker1 # [ 0.360310] thermal_sys: Registered thermal governor 'step_wise'931lb # [ 0.341238] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations932client # [ 0.348998] cpuidle: using governor menu933worker1 # [ 0.360311] thermal_sys: Registered thermal governor 'user_space'934lb # [ 0.341966] audit: initializing netlink subsys (disabled)935client # [ 0.351340] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5936worker1 # [ 0.361307] thermal_sys: Registered thermal governor 'power_allocator'937lb # [ 0.343276] thermal_sys: Registered thermal governor 'fair_share'938worker1 # [ 0.362330] cpuidle: using governor menu939lb # [ 0.343279] thermal_sys: Registered thermal governor 'bang_bang'940client # [ 0.352257] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]941worker1 # [ 0.364746] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5942lb # [ 0.343953] thermal_sys: Registered thermal governor 'step_wise'943client # [ 0.352961] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry944lb # [ 0.344956] audit: type=2000 audit(1789910751.803:1): state=initialized audit_enabled=0 res=1945worker1 # [ 0.365572] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]946client # [ 0.353995] PCI: Using configuration type 1 for base access947lb # [ 0.346954] thermal_sys: Registered thermal governor 'user_space'948worker1 # [ 0.366311] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry949lb # [ 0.346955] thermal_sys: Registered thermal governor 'power_allocator'950worker2 # [ 0.364024] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages951worker1 # [ 0.367324] PCI: Using configuration type 1 for base access952client # [ 0.355325] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.953lb # [ 0.347975] cpuidle: using governor menu954worker2 # [ 0.364797] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page955worker1 # [ 0.368529] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.956lb # [ 0.350306] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5957lb # [ 0.351228] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]958lb # [ 0.351956] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry959lb # [ 0.352967] PCI: Using configuration type 1 for base access960client # [ 0.360420] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages961worker2 # [ 0.369796] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages962worker1 # [ 0.373446] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages963worker2 # [ 0.370794] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page964client # [ 0.360958] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page965lb # [ 0.354183] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.966worker1 # [ 0.374309] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page967client # [ 0.361959] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages968worker1 # [ 0.379309] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages969worker1 # [ 0.380307] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page970client # [ 0.368959] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page971lb # [ 0.363161] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages972worker2 # [ 0.381255] ACPI: Added _OSI(Module Device)973lb # [ 0.363954] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page974worker2 # [ 0.381797] ACPI: Added _OSI(Processor Device)975worker2 # [ 0.382598] ACPI: Added _OSI(Processor Aggregator Device)976worker1 # [ 0.388585] ACPI: Added _OSI(Module Device)977lb # [ 0.368953] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages978worker1 # [ 0.389309] ACPI: Added _OSI(Processor Device)979lb # [ 0.369952] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page980worker1 # [ 0.390145] ACPI: Added _OSI(Processor Aggregator Device)981client # [ 0.378622] ACPI: Added _OSI(Module Device)982client # [ 0.378959] ACPI: Added _OSI(Processor Device)983worker2 # [ 0.391655] ACPI: 1 ACPI AML tables successfully acquired and loaded984worker2 # [ 0.395691] ACPI: Interpreter enabled985client # [ 0.383958] ACPI: Added _OSI(Processor Aggregator Device)986worker2 # [ 0.396287] ACPI: PM: (supports S0 S3 S4 S5)987worker2 # [ 0.396796] ACPI: Using IOAPIC for interrupt routing988lb # [ 0.380446] ACPI: Added _OSI(Module Device)989worker2 # [ 0.397753] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug990lb # [ 0.380954] ACPI: Added _OSI(Processor Device)991lb # [ 0.381780] ACPI: Added _OSI(Processor Aggregator Device)992worker1 # [ 0.402727] ACPI: 1 ACPI AML tables successfully acquired and loaded993worker2 # [ 0.402795] PCI: Using E820 reservations for host bridge windows994worker1 # [ 0.406773] ACPI: Interpreter enabled995worker2 # [ 0.403952] ACPI: Enabled 2 GPEs in block 00 to 3F996worker1 # [ 0.407326] ACPI: PM: (supports S0 S3 S4 S5)997worker1 # [ 0.408130] ACPI: Using IOAPIC for interrupt routing998client # [ 0.396697] ACPI: 1 ACPI AML tables successfully acquired and loaded999lb # [ 0.392091] ACPI: 1 ACPI AML tables successfully acquired and loaded1000worker1 # [ 0.411388] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1001worker1 # [ 0.412308] PCI: Using E820 reservations for host bridge windows1002worker2 # [ 0.412599] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1003lb # [ 0.396272] ACPI: Interpreter enabled1004worker1 # [ 0.415469] ACPI: Enabled 2 GPEs in block 00 to 3F1005lb # [ 0.396966] ACPI: PM: (supports S0 S3 S4 S5)1006worker2 # [ 0.413644] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1007lb # [ 0.397735] ACPI: Using IOAPIC for interrupt routing1008client # [ 0.404434] ACPI: Interpreter enabled1009client # [ 0.404974] ACPI: PM: (supports S0 S3 S4 S5)1010worker2 # [ 0.414890] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1011client # [ 0.405758] ACPI: Using IOAPIC for interrupt routing1012worker2 # [ 0.415931] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1013worker2 # [ 0.417188] PCI host bridge to bus 0000:001014lb # [ 0.401035] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1015worker2 # [ 0.417800] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1016lb # [ 0.401953] PCI: Using E820 reservations for host bridge windows1017worker2 # [ 0.418795] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1018client # [ 0.409006] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1019client # [ 0.409956] PCI: Using E820 reservations for host bridge windows1020worker2 # [ 0.419795] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1021client # [ 0.411113] ACPI: Enabled 2 GPEs in block 00 to 3F1022lb # [ 0.405120] ACPI: Enabled 2 GPEs in block 00 to 3F1023worker2 # [ 0.420802] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1024worker1 # [ 0.424652] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1025worker2 # [ 0.421795] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1026worker2 # [ 0.422824] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1027worker1 # [ 0.426892] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1028worker2 # [ 0.423795] pci_bus 0000:00: root bus resource [bus 00-ff]1029worker1 # [ 0.427397] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1030worker2 # [ 0.424897] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1031worker1 # [ 0.428444] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1032lb # [ 0.412237] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1033worker2 # [ 0.426317] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1034worker1 # [ 0.429818] PCI host bridge to bus 0000:001035lb # [ 0.412959] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1036worker1 # [ 0.430314] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1037client # [ 0.420328] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1038lb # [ 0.414040] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1039worker1 # [ 0.431308] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1040lb # [ 0.415106] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1041client # [ 0.421866] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1042worker1 # [ 0.432308] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1043lb # [ 0.416347] PCI host bridge to bus 0000:001044client # [ 0.423059] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1045worker1 # [ 0.433308] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1046lb # [ 0.416958] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1047client # [ 0.424093] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1048worker1 # [ 0.434307] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1049lb # [ 0.417952] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1050client # [ 0.425356] PCI host bridge to bus 0000:001051worker2 # [ 0.428898] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1052worker1 # [ 0.435308] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1053lb # [ 0.418952] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1054worker2 # [ 0.429808] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1055client # [ 0.425959] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1056worker1 # [ 0.436309] pci_bus 0000:00: root bus resource [bus 00-ff]1057lb # [ 0.419961] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1058worker2 # [ 0.430836] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1059client # [ 0.426955] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1060worker1 # [ 0.437409] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1061client # [ 0.427955] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1062lb # [ 0.420952] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1063worker2 # [ 0.431996] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1064client # [ 0.428955] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1065lb # [ 0.421989] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1066worker1 # [ 0.438847] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1067worker2 # [ 0.433617] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1068client # [ 0.429955] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1069lb # [ 0.422955] pci_bus 0000:00: root bus resource [bus 00-ff]1070client # [ 0.430955] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1071lb # [ 0.424129] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1072client # [ 0.431956] pci_bus 0000:00: root bus resource [bus 00-ff]1073worker2 # [ 0.436828] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1074client # [ 0.433044] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1075lb # [ 0.425491] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1076worker2 # [ 0.437795] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1077worker2 # [ 0.438818] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1078client # [ 0.434505] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1079worker2 # [ 0.439801] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1080worker1 # [ 0.441498] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1081worker2 # [ 0.441458] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1082worker1 # [ 0.442324] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1083worker1 # [ 0.443334] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1084worker2 # [ 0.443825] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1085worker1 # [ 0.444546] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1086lb # [ 0.428037] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1087lb # [ 0.428971] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1088worker2 # [ 0.444786] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1089client # [ 0.437050] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1090worker1 # [ 0.446217] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1091worker2 # [ 0.445836] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1092lb # [ 0.429986] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1093client # [ 0.437969] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1094client # [ 0.438979] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1095lb # [ 0.431279] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1096worker1 # [ 0.449354] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1097client # [ 0.440236] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1098worker2 # [ 0.447493] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1099worker1 # [ 0.450316] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1100lb # [ 0.432902] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1101client # [ 0.441853] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1102worker1 # [ 0.451333] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1103worker2 # [ 0.450833] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1104worker1 # [ 0.452315] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1105worker2 # [ 0.451802] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1106lb # [ 0.435990] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1107worker2 # [ 0.452820] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1108client # [ 0.444992] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1109lb # [ 0.436964] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1110worker1 # [ 0.454040] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1111client # [ 0.445933] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1112lb # [ 0.437985] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1113worker2 # [ 0.454480] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1114client # [ 0.446785] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1115worker1 # [ 0.457349] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1116lb # [ 0.438965] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1117client # [ 0.447954] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1118worker1 # [ 0.458315] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1119lb # [ 0.440651] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1120worker1 # [ 0.459334] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1121client # [ 0.449619] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1122worker2 # [ 0.458214] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1123worker2 # [ 0.459549] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1124worker1 # [ 0.461472] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125lb # [ 0.442988] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1126client # [ 0.451988] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1127worker2 # [ 0.459819] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1128client # [ 0.452964] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1129lb # [ 0.443963] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1130worker1 # [ 0.463317] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1131client # [ 0.453980] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1132worker2 # [ 0.461452] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1133lb # [ 0.444986] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1134worker1 # [ 0.464299] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1135worker2 # [ 0.463803] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1136client # [ 0.455667] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1137lb # [ 0.446685] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1138worker2 # [ 0.464751] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1139worker1 # [ 0.465332] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1140client # [ 0.457996] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1141worker2 # [ 0.465639] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1142lb # [ 0.448979] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1143client # [ 0.458963] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144lb # [ 0.449963] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1145worker2 # [ 0.467476] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1146worker1 # [ 0.467019] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1147client # [ 0.459979] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1148lb # [ 0.450984] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1149client # [ 0.461592] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150worker2 # [ 0.469829] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1151worker1 # [ 0.470348] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1152lb # [ 0.452668] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1153worker2 # [ 0.470802] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1154worker1 # [ 0.471315] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1155worker2 # [ 0.471820] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1156worker1 # [ 0.472331] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1157client # [ 0.465181] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1158worker2 # [ 0.473526] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1159lb # [ 0.454986] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1160client # [ 0.465991] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1161worker1 # [ 0.474008] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1162lb # [ 0.455980] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1163client # [ 0.466988] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1164lb # [ 0.457003] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1165worker2 # [ 0.476803] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1166worker2 # [ 0.477802] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1167worker1 # [ 0.478203] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1168lb # [ 0.458666] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1169worker2 # [ 0.478833] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1170worker1 # [ 0.479082] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1171worker2 # [ 0.479801] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1172client # [ 0.469586] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1173worker1 # [ 0.480179] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1174worker2 # [ 0.481441] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1175worker1 # [ 0.482026] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1176lb # [ 0.462855] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1177worker2 # [ 0.482835] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1178lb # [ 0.463796] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1179client # [ 0.473025] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1180worker1 # [ 0.485337] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1181worker2 # [ 0.483819] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1182lb # [ 0.464914] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1183client # [ 0.473965] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1184client # [ 0.474983] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1185worker1 # [ 0.486300] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1186worker2 # [ 0.485426] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1187lb # [ 0.466691] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1188worker1 # [ 0.488126] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1189client # [ 0.476727] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1190worker2 # [ 0.487815] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1191worker1 # [ 0.490024] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1192worker2 # [ 0.488774] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1193worker2 # [ 0.489672] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1194worker1 # [ 0.492352] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1195lb # [ 0.470707] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1196worker2 # [ 0.491434] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1197worker1 # [ 0.493316] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1198lb # [ 0.471813] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1199client # [ 0.479991] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1200worker1 # [ 0.494334] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1201lb # [ 0.472904] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1202client # [ 0.480976] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1203worker2 # [ 0.494802] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1204worker1 # [ 0.495314] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1205worker2 # [ 0.495776] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1206client # [ 0.482000] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1207lb # [ 0.474736] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1208worker1 # [ 0.496974] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1209worker2 # [ 0.496608] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1210lb # [ 0.477705] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1211client # [ 0.484087] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1212worker1 # [ 0.499322] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1213worker2 # [ 0.498403] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1214worker1 # [ 0.500333] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1215lb # [ 0.478787] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1216worker2 # [ 0.499461] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1217lb # [ 0.479985] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1218worker1 # [ 0.501958] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1219worker2 # [ 0.500042] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1220lb # [ 0.480965] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1221client # [ 0.486991] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1222worker2 # [ 0.501828] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1223client # [ 0.487972] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1224lb # [ 0.482686] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1225worker1 # [ 0.505192] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1226client # [ 0.488989] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1227worker1 # [ 0.506071] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1228client # [ 0.489968] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1229lb # [ 0.484991] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1230worker1 # [ 0.507170] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1231lb # [ 0.485984] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1232client # [ 0.491861] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1233worker1 # [ 0.508995] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1234lb # [ 0.487645] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1235worker1 # [ 0.511343] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1236client # [ 0.494993] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1237worker1 # [ 0.512315] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1238lb # [ 0.489982] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1239client # [ 0.495988] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1240lb # [ 0.490962] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1241worker1 # [ 0.513332] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1242lb # [ 0.491984] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1243client # [ 0.497828] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1244worker1 # [ 0.514986] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1245worker2 # [ 0.503064] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1246lb # [ 0.493665] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1247worker1 # [ 0.517030] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1248worker2 # [ 0.505461] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1249worker1 # [ 0.517548] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1250worker2 # [ 0.506104] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1251lb # [ 0.496986] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1252client # [ 0.501035] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1253worker1 # [ 0.519007] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1254worker2 # [ 0.507430] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1255lb # [ 0.497962] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1256client # [ 0.501970] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1257lb # [ 0.498985] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1258client # [ 0.502995] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1259worker2 # [ 0.508111] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1260worker2 # [ 0.509115] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1261lb # [ 0.500648] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1262client # [ 0.504856] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1263worker2 # [ 0.510105] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1264lb # [ 0.502747] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1265worker2 # [ 0.511783] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1266lb # [ 0.504048] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1267worker2 # [ 0.512514] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1268client # [ 0.507996] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1269lb # [ 0.505648] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1270client # [ 0.508979] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1271worker2 # [ 0.513176] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1272client # [ 0.510000] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1273worker2 # [ 0.514391] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1274worker1 # [ 0.519535] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1275client # [ 0.512101] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1276worker2 # [ 0.518175] ACPI: PCI: Interrupt link LNKA configured for IRQ 101277worker1 # [ 0.522954] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1278worker2 # [ 0.518930] ACPI: PCI: Interrupt link LNKB configured for IRQ 101279client # [ 0.514024] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1280worker2 # [ 0.519905] ACPI: PCI: Interrupt link LNKC configured for IRQ 111281worker1 # [ 0.523626] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1282worker2 # [ 0.520905] ACPI: PCI: Interrupt link LNKD configured for IRQ 111283client # [ 0.515451] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1284worker1 # [ 0.525316] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1285worker2 # [ 0.521916] ACPI: PCI: Interrupt link LNKE configured for IRQ 101286worker2 # [ 0.522901] ACPI: PCI: Interrupt link LNKF configured for IRQ 101287worker1 # [ 0.526604] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1288client # [ 0.518004] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1289worker2 # [ 0.523904] ACPI: PCI: Interrupt link LNKG configured for IRQ 111290lb # [ 0.506206] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1291worker1 # [ 0.527635] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1292worker2 # [ 0.524973] ACPI: PCI: Interrupt link LNKH configured for IRQ 111293lb # [ 0.508726] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1294worker2 # [ 0.526042] ACPI: PCI: Interrupt link GSIA configured for IRQ 161295worker1 # [ 0.528600] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1296worker2 # [ 0.526814] ACPI: PCI: Interrupt link GSIB configured for IRQ 171297lb # [ 0.510093] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1298worker2 # [ 0.527816] ACPI: PCI: Interrupt link GSIC configured for IRQ 181299worker2 # [ 0.528810] ACPI: PCI: Interrupt link GSID configured for IRQ 191300worker1 # [ 0.530341] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1301lb # [ 0.511961] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1302worker1 # [ 0.531315] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1303worker2 # [ 0.529811] ACPI: PCI: Interrupt link GSIE configured for IRQ 201304lb # [ 0.513295] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1305worker2 # [ 0.530808] ACPI: PCI: Interrupt link GSIF configured for IRQ 211306worker1 # [ 0.532708] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1307worker2 # [ 0.531808] ACPI: PCI: Interrupt link GSIG configured for IRQ 221308lb # [ 0.514273] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1309worker2 # [ 0.532808] ACPI: PCI: Interrupt link GSIH configured for IRQ 231310worker1 # [ 0.534405] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1311lb # [ 0.515263] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1312worker2 # [ 0.535016] iommu: Default domain type: Translated1313worker2 # [ 0.535803] iommu: DMA domain TLB invalidation policy: lazy mode1314lb # [ 0.517008] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1315worker2 # [ 0.537070] ACPI: bus type USB registered1316lb # [ 0.517959] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1317worker2 # [ 0.537853] usbcore: registered new interface driver usbfs1318client # [ 0.519570] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1319worker2 # [ 0.538808] usbcore: registered new interface driver hub1320worker1 # [ 0.539227] ACPI: PCI: Interrupt link LNKA configured for IRQ 101321lb # [ 0.519764] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1322worker2 # [ 0.539807] usbcore: registered new device driver usb1323worker1 # [ 0.540241] ACPI: PCI: Interrupt link LNKB configured for IRQ 101324client # [ 0.521994] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1325lb # [ 0.521603] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1326worker1 # [ 0.541309] ACPI: PCI: Interrupt link LNKC configured for IRQ 111327worker2 # [ 0.541720] NetLabel: Initializing1328worker1 # [ 0.542417] ACPI: PCI: Interrupt link LNKD configured for IRQ 111329worker2 # [ 0.542422] NetLabel: domain hash size = 1281330client # [ 0.523490] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1331worker1 # [ 0.543421] ACPI: PCI: Interrupt link LNKE configured for IRQ 101332worker2 # [ 0.542799] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1333client # [ 0.524891] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1334worker2 # [ 0.543868] NetLabel: unlabeled traffic allowed by default1335worker1 # [ 0.544450] ACPI: PCI: Interrupt link LNKF configured for IRQ 101336lb # [ 0.525648] ACPI: PCI: Interrupt link LNKA configured for IRQ 101337worker2 # [ 0.544824] PCI: Using ACPI for IRQ routing1338worker1 # [ 0.545425] ACPI: PCI: Interrupt link LNKG configured for IRQ 111339lb # [ 0.526075] ACPI: PCI: Interrupt link LNKB configured for IRQ 101340client # [ 0.526417] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1341worker1 # [ 0.546432] ACPI: PCI: Interrupt link LNKH configured for IRQ 111342lb # [ 0.527063] ACPI: PCI: Interrupt link LNKC configured for IRQ 111343worker1 # [ 0.547348] ACPI: PCI: Interrupt link GSIA configured for IRQ 161344client # [ 0.527291] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1345lb # [ 0.528063] ACPI: PCI: Interrupt link LNKD configured for IRQ 111346worker1 # [ 0.548324] ACPI: PCI: Interrupt link GSIB configured for IRQ 171347lb # [ 0.529059] ACPI: PCI: Interrupt link LNKE configured for IRQ 101348client # [ 0.528297] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1349worker1 # [ 0.549321] ACPI: PCI: Interrupt link GSIC configured for IRQ 181350lb # [ 0.530091] ACPI: PCI: Interrupt link LNKF configured for IRQ 101351worker1 # [ 0.550323] ACPI: PCI: Interrupt link GSID configured for IRQ 191352lb # [ 0.531061] ACPI: PCI: Interrupt link LNKG configured for IRQ 111353client # [ 0.529992] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1354worker1 # [ 0.551321] ACPI: PCI: Interrupt link GSIE configured for IRQ 201355lb # [ 0.532058] ACPI: PCI: Interrupt link LNKH configured for IRQ 111356client # [ 0.530941] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1357worker1 # [ 0.552321] ACPI: PCI: Interrupt link GSIF configured for IRQ 211358lb # [ 0.532994] ACPI: PCI: Interrupt link GSIA configured for IRQ 161359worker1 # [ 0.553323] ACPI: PCI: Interrupt link GSIG configured for IRQ 221360lb # [ 0.533966] ACPI: PCI: Interrupt link GSIB configured for IRQ 171361client # [ 0.532167] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1362worker1 # [ 0.554323] ACPI: PCI: Interrupt link GSIH configured for IRQ 231363lb # [ 0.534969] ACPI: PCI: Interrupt link GSIC configured for IRQ 181364lb # [ 0.535968] ACPI: PCI: Interrupt link GSID configured for IRQ 191365worker1 # [ 0.556480] iommu: Default domain type: Translated1366client # [ 0.534002] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1367lb # [ 0.536967] ACPI: PCI: Interrupt link GSIE configured for IRQ 201368worker1 # [ 0.557315] iommu: DMA domain TLB invalidation policy: lazy mode1369lb # [ 0.537974] ACPI: PCI: Interrupt link GSIF configured for IRQ 211370worker1 # [ 0.558588] ACPI: bus type USB registered1371lb # [ 0.538969] ACPI: PCI: Interrupt link GSIG configured for IRQ 221372worker1 # [ 0.559372] usbcore: registered new interface driver usbfs1373worker1 # [ 0.560327] usbcore: registered new interface driver hub1374lb # [ 0.539968] ACPI: PCI: Interrupt link GSIH configured for IRQ 231375worker1 # [ 0.561292] usbcore: registered new device driver usb1376client # [ 0.538473] ACPI: PCI: Interrupt link LNKA configured for IRQ 101377lb # [ 0.542113] iommu: Default domain type: Translated1378client # [ 0.539086] ACPI: PCI: Interrupt link LNKB configured for IRQ 101379worker1 # [ 0.562951] NetLabel: Initializing1380lb # [ 0.542960] iommu: DMA domain TLB invalidation policy: lazy mode1381worker1 # [ 0.563309] NetLabel: domain hash size = 1281382client # [ 0.540069] ACPI: PCI: Interrupt link LNKC configured for IRQ 111383lb # [ 0.544222] ACPI: bus type USB registered1384worker1 # [ 0.564116] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1385client # [ 0.541073] ACPI: PCI: Interrupt link LNKD configured for IRQ 111386lb # [ 0.545015] usbcore: registered new interface driver usbfs1387worker1 # [ 0.565101] NetLabel: unlabeled traffic allowed by default1388client # [ 0.542082] ACPI: PCI: Interrupt link LNKE configured for IRQ 101389worker1 # [ 0.566073] PCI: Using ACPI for IRQ routing1390lb # [ 0.545967] usbcore: registered new interface driver hub1391lb # [ 0.546976] usbcore: registered new device driver usb1392client # [ 0.543065] ACPI: PCI: Interrupt link LNKF configured for IRQ 101393client # [ 0.544117] ACPI: PCI: Interrupt link LNKG configured for IRQ 111394lb # [ 0.549078] NetLabel: Initializing1395lb # [ 0.549744] NetLabel: domain hash size = 1281396client # [ 0.545074] ACPI: PCI: Interrupt link LNKH configured for IRQ 111397lb # [ 0.550569] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1398client # [ 0.546003] ACPI: PCI: Interrupt link GSIA configured for IRQ 161399lb # [ 0.551024] NetLabel: unlabeled traffic allowed by default1400client # [ 0.546973] ACPI: PCI: Interrupt link GSIB configured for IRQ 171401lb # [ 0.551965] PCI: Using ACPI for IRQ routing1402client # [ 0.547973] ACPI: PCI: Interrupt link GSIC configured for IRQ 181403client # [ 0.548975] ACPI: PCI: Interrupt link GSID configured for IRQ 191404client # [ 0.549968] ACPI: PCI: Interrupt link GSIE configured for IRQ 201405client # [ 0.550970] ACPI: PCI: Interrupt link GSIF configured for IRQ 211406client # [ 0.551971] ACPI: PCI: Interrupt link GSIG configured for IRQ 221407client # [ 0.552969] ACPI: PCI: Interrupt link GSIH configured for IRQ 231408client # [ 0.555457] iommu: Default domain type: Translated1409client # [ 0.555964] iommu: DMA domain TLB invalidation policy: lazy mode1410client # [ 0.557319] ACPI: bus type USB registered1411client # [ 0.558033] usbcore: registered new interface driver usbfs1412client # [ 0.558973] usbcore: registered new interface driver hub1413client # [ 0.559947] usbcore: registered new device driver usb1414client # [ 0.562075] NetLabel: Initializing1415client # [ 0.562787] NetLabel: domain hash size = 1281416client # [ 0.563568] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1417client # [ 0.564028] NetLabel: unlabeled traffic allowed by default1418client # [ 0.564973] PCI: Using ACPI for IRQ routing1419worker2 # [ 0.604639] pci 0000:00:01.0: vgaarb: setting as boot VGA device1420worker2 # [ 0.604790] pci 0000:00:01.0: vgaarb: bridge control possible1421worker2 # [ 0.604790] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1422worker2 # [ 0.604800] vgaarb: loaded1423worker2 # [ 0.605634] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01424worker2 # [ 0.606466] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1425worker2 # [ 0.610897] clocksource: Switched to clocksource kvm-clock1426worker2 # [ 0.612775] VFS: Disk quotas dquot_6.6.01427worker1 # [ 0.617634] pci 0000:00:01.0: vgaarb: setting as boot VGA device1428worker2 # [ 0.615628] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1429worker1 # [ 0.618303] pci 0000:00:01.0: vgaarb: bridge control possible1430worker2 # [ 0.617161] pnp: PnP ACPI init1431worker1 # [ 0.618303] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1432worker2 # [ 0.618075] ACPI: IRQ 4 override to edge(!), high(!)1433worker1 # [ 0.618315] vgaarb: loaded1434worker2 # [ 0.619199] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1435lb # [ 0.601907] pci 0000:00:01.0: vgaarb: setting as boot VGA device1436worker1 # [ 0.619214] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01437lb # [ 0.601948] pci 0000:00:01.0: vgaarb: bridge control possible1438worker1 # [ 0.620062] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1439worker2 # [ 0.621043] pnp: PnP ACPI: found 6 devices1440lb # [ 0.601948] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1441lb # [ 0.601960] vgaarb: loaded1442lb # [ 0.602789] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01443lb # [ 0.603608] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1444worker1 # [ 0.625500] clocksource: Switched to clocksource kvm-clock1445worker2 # [ 0.630101] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1446lb # [ 0.608083] clocksource: Switched to clocksource kvm-clock1447worker1 # [ 0.629497] VFS: Disk quotas dquot_6.6.01448worker2 # [ 0.632121] clocksource: Switched to clocksource acpi_pm1449worker1 # [ 0.630499] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1450worker2 # [ 0.633519] NET: Registered PF_INET protocol family1451worker1 # [ 0.632734] pnp: PnP ACPI init1452lb # [ 0.612813] VFS: Disk quotas dquot_6.6.01453worker1 # [ 0.633769] ACPI: IRQ 4 override to edge(!), high(!)1454worker2 # [ 0.635191] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1455lb # [ 0.613673] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1456worker1 # [ 0.634966] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1457lb # [ 0.615389] pnp: PnP ACPI init1458worker1 # [ 0.636732] pnp: PnP ACPI: found 6 devices1459lb # [ 0.616434] ACPI: IRQ 4 override to edge(!), high(!)1460client # [ 0.615765] pci 0000:00:01.0: vgaarb: setting as boot VGA device1461client # [ 0.615950] pci 0000:00:01.0: vgaarb: bridge control possible1462lb # [ 0.617664] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1463client # [ 0.615950] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1464lb # [ 0.619584] pnp: PnP ACPI: found 6 devices1465client # [ 0.615966] vgaarb: loaded1466client # [ 0.616841] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01467client # [ 0.617637] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1468worker1 # [ 0.644875] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1469worker1 # [ 0.646490] clocksource: Switched to clocksource acpi_pm1470client # [ 0.621116] clocksource: Switched to clocksource kvm-clock1471worker1 # [ 0.647618] NET: Registered PF_INET protocol family1472worker1 # [ 0.648971] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1473client # [ 0.623081] VFS: Disk quotas dquot_6.6.01474lb # [ 0.628177] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1475lb # [ 0.629995] clocksource: Switched to clocksource acpi_pm1476lb # [ 0.631167] NET: Registered PF_INET protocol family1477client # [ 0.626476] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1478lb # [ 0.632566] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1479client # [ 0.628320] pnp: PnP ACPI init1480client # [ 0.629320] ACPI: IRQ 4 override to edge(!), high(!)1481client # [ 0.630491] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1482worker2 # [ 0.658714] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1483client # [ 0.632302] pnp: PnP ACPI: found 6 devices1484worker2 # [ 0.660913] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1485worker2 # [ 0.662587] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1486worker2 # [ 0.664474] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1487worker2 # [ 0.666217] TCP: Hash tables configured (established 16384 bind 16384)1488worker2 # [ 0.667829] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1489client # [ 0.640622] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1490worker2 # [ 0.669573] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1491client # [ 0.642278] clocksource: Switched to clocksource acpi_pm1492worker1 # [ 0.668659] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1493worker2 # [ 0.671089] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1494client # [ 0.643411] NET: Registered PF_INET protocol family1495worker1 # [ 0.670294] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1496worker2 # [ 0.672797] NET: Registered PF_UNIX/PF_LOCAL protocol family1497client # [ 0.644764] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1498worker1 # [ 0.671805] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1499worker2 # [ 0.674237] NET: Registered PF_XDP protocol family1500worker1 # [ 0.673295] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1501worker2 # [ 0.675484] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1502worker1 # [ 0.674731] TCP: Hash tables configured (established 16384 bind 16384)1503worker2 # [ 0.676866] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1504lb # [ 0.653538] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1505worker1 # [ 0.676004] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1506worker2 # [ 0.678228] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1507lb # [ 0.655536] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1508worker1 # [ 0.677436] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1509worker2 # [ 0.679880] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1510lb # [ 0.657251] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1511worker1 # [ 0.678699] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1512worker2 # [ 0.681417] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1513worker1 # [ 0.680100] NET: Registered PF_UNIX/PF_LOCAL protocol family1514lb # [ 0.659013] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1515worker1 # [ 0.681231] NET: Registered PF_XDP protocol family1516worker2 # [ 0.682998] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1517lb # [ 0.660610] TCP: Hash tables configured (established 16384 bind 16384)1518worker1 # [ 0.682186] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1519worker1 # [ 0.683317] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1520worker2 # [ 0.685351] ACPI: \_SB_.GSIA: Enabled at IRQ 161521lb # [ 0.662140] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1522worker1 # [ 0.684606] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1523lb # [ 0.663834] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1524worker1 # [ 0.685959] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1525worker2 # [ 0.688431] ACPI: \_SB_.GSIB: Enabled at IRQ 171526lb # [ 0.665307] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1527worker1 # [ 0.687380] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1528lb # [ 0.666990] NET: Registered PF_UNIX/PF_LOCAL protocol family1529worker1 # [ 0.688749] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1530lb # [ 0.668327] NET: Registered PF_XDP protocol family1531worker2 # [ 0.691295] ACPI: \_SB_.GSIC: Enabled at IRQ 181532lb # [ 0.669462] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1533worker1 # [ 0.690749] ACPI: \_SB_.GSIA: Enabled at IRQ 161534lb # [ 0.670739] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1535worker2 # [ 0.694343] ACPI: \_SB_.GSID: Enabled at IRQ 191536client # [ 0.665459] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1537worker1 # [ 0.693247] ACPI: \_SB_.GSIB: Enabled at IRQ 171538lb # [ 0.671929] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1539client # [ 0.667298] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1540lb # [ 0.673324] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1541worker2 # [ 0.697448] PCI: CLS 0 bytes, default 641542lb # [ 0.674609] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1543client # [ 0.668838] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1544worker1 # [ 0.696846] ACPI: \_SB_.GSIC: Enabled at IRQ 181545lb # [ 0.675864] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1546worker2 # [ 0.699034] Trying to unpack rootfs image as initramfs...1547client # [ 0.670439] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1548client # [ 0.671864] TCP: Hash tables configured (established 16384 bind 16384)1549lb # [ 0.677944] ACPI: \_SB_.GSIA: Enabled at IRQ 161550worker1 # [ 0.699292] ACPI: \_SB_.GSID: Enabled at IRQ 191551client # [ 0.673305] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1552lb # [ 0.680451] ACPI: \_SB_.GSIB: Enabled at IRQ 171553worker1 # [ 0.701592] PCI: CLS 0 bytes, default 641554client # [ 0.674810] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1555client # [ 0.676068] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1556worker1 # [ 0.702711] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1557lb # [ 0.682928] ACPI: \_SB_.GSIC: Enabled at IRQ 181558client # [ 0.677544] NET: Registered PF_UNIX/PF_LOCAL protocol family1559worker2 # [ 0.705351] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1560worker1 # [ 0.704633] Trying to unpack rootfs image as initramfs...1561client # [ 0.678790] NET: Registered PF_XDP protocol family1562lb # [ 0.685475] ACPI: \_SB_.GSID: Enabled at IRQ 191563client # [ 0.679806] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1564client # [ 0.680955] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1565lb # [ 0.687355] PCI: CLS 0 bytes, default 641566client # [ 0.682084] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1567lb # [ 0.688334] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1568client # [ 0.683342] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1569lb # [ 0.690247] Trying to unpack rootfs image as initramfs...1570client # [ 0.684590] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1571client # [ 0.685854] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1572client # [ 0.688013] ACPI: \_SB_.GSIA: Enabled at IRQ 161573client # [ 0.690768] ACPI: \_SB_.GSIB: Enabled at IRQ 171574client # [ 0.693073] ACPI: \_SB_.GSIC: Enabled at IRQ 181575client # [ 0.695345] ACPI: \_SB_.GSID: Enabled at IRQ 191576client # [ 0.697269] PCI: CLS 0 bytes, default 641577client # [ 0.698383] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1578client # [ 0.700434] Trying to unpack rootfs image as initramfs...1579worker1 # [ 0.760976] Initialise system trusted keyrings1580lb # [ 0.742662] Initialise system trusted keyrings1581worker1 # [ 0.764532] workingset: timestamp_bits=40 max_order=19 bucket_order=01582worker2 # [ 0.767789] Initialise system trusted keyrings1583lb # [ 0.746166] workingset: timestamp_bits=40 max_order=19 bucket_order=01584worker2 # [ 0.771091] workingset: timestamp_bits=40 max_order=19 bucket_order=01585client # [ 0.754306] Initialise system trusted keyrings1586client # [ 0.757254] workingset: timestamp_bits=40 max_order=19 bucket_order=01587worker1 # [ 0.789628] Key type asymmetric registered1588lb # [ 0.770732] Key type asymmetric registered1589lb # [ 0.771533] Asymmetric key parser 'x509' registered1590worker1 # [ 0.792514] Asymmetric key parser 'x509' registered1591worker1 # [ 0.793784] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1592lb # [ 0.774188] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1593lb # [ 0.777159] io scheduler mq-deadline registered1594lb # [ 0.777977] io scheduler kyber registered1595worker1 # [ 0.799702] io scheduler mq-deadline registered1596worker1 # [ 0.800927] io scheduler kyber registered1597lb # [ 0.780976] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1598worker2 # [ 0.804902] Key type asymmetric registered1599lb # [ 0.784441] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1600worker2 # [ 0.807886] Asymmetric key parser 'x509' registered1601worker1 # [ 0.805783] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1602worker2 # [ 0.809194] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1603client # [ 0.782246] Key type asymmetric registered1604client # [ 0.783076] Asymmetric key parser 'x509' registered1605worker1 # [ 0.809757] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1606lb # [ 0.790605] Linux agpgart interface v0.1031607lb # [ 0.791568] ACPI: bus type drm_connector registered1608worker2 # [ 0.815043] io scheduler mq-deadline registered1609client # [ 0.786279] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1610worker2 # [ 0.816091] io scheduler kyber registered1611client # [ 0.787885] io scheduler mq-deadline registered1612lb # [ 0.794639] usbcore: registered new interface driver usbserial_generic1613client # [ 0.790224] io scheduler kyber registered1614lb # [ 0.795796] usbserial: USB Serial support registered for generic1615worker1 # [ 0.816958] Linux agpgart interface v0.1031616worker1 # [ 0.818080] ACPI: bus type drm_connector registered1617client # [ 0.793469] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1618worker2 # [ 0.821931] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1619lb # [ 0.799116] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1620lb # [ 0.799116] Please enable it if your BIOS has the CPPC option.1621client # [ 0.794956] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1622worker2 # [ 0.823624] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1623lb # [ 0.801444] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1624worker1 # [ 0.823299] usbcore: registered new interface driver usbserial_generic1625worker1 # [ 0.824923] usbserial: USB Serial support registered for generic1626lb # [ 0.805356] drop_monitor: Initializing network drop monitor service1627client # [ 0.800407] Linux agpgart interface v0.1031628lb # [ 0.806721] NET: Registered PF_INET6 protocol family1629client # [ 0.801362] ACPI: bus type drm_connector registered1630worker2 # [ 0.829956] Linux agpgart interface v0.1031631worker1 # [ 0.829521] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1632worker2 # [ 0.832969] ACPI: bus type drm_connector registered1633worker1 # [ 0.829521] Please enable it if your BIOS has the CPPC option.1634lb # [ 0.811887] Segment Routing with IPv61635client # [ 0.806419] usbcore: registered new interface driver usbserial_generic1636worker2 # [ 0.835020] usbcore: registered new interface driver usbserial_generic1637worker1 # [ 0.832758] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1638lb # [ 0.812752] In-situ OAM (IOAM) with IPv61639client # [ 0.807634] usbserial: USB Serial support registered for generic1640worker2 # [ 0.836373] usbserial: USB Serial support registered for generic1641lb # [ 0.816494] IPI shorthand broadcast: enabled1642client # [ 0.811192] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1643client # [ 0.811192] Please enable it if your BIOS has the CPPC option.1644client # [ 0.813625] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1645worker2 # [ 0.841882] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1646worker1 # [ 0.840806] drop_monitor: Initializing network drop monitor service1647worker2 # [ 0.841882] Please enable it if your BIOS has the CPPC option.1648worker1 # [ 0.842677] NET: Registered PF_INET6 protocol family1649worker2 # [ 0.844903] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1650client # [ 0.817440] drop_monitor: Initializing network drop monitor service1651client # [ 0.818832] NET: Registered PF_INET6 protocol family1652lb # [ 0.825433] sched_clock: Marking stable (672015373, 153078855)->(908563807, -83469579)1653worker1 # [ 0.847105] Segment Routing with IPv61654worker2 # [ 0.850170] drop_monitor: Initializing network drop monitor service1655client # [ 0.823944] Segment Routing with IPv61656worker1 # [ 0.850538] In-situ OAM (IOAM) with IPv61657client # [ 0.824772] In-situ OAM (IOAM) with IPv61658lb # [ 0.830491] registered taskstats version 11659worker1 # [ 0.852156] IPI shorthand broadcast: enabled1660lb # [ 0.831623] Loading compiled-in X.509 certificates1661worker2 # [ 0.855075] NET: Registered PF_INET6 protocol family1662client # [ 0.828756] IPI shorthand broadcast: enabled1663worker2 # [ 0.859539] Segment Routing with IPv61664worker2 # [ 0.862907] In-situ OAM (IOAM) with IPv61665worker2 # [ 0.864601] IPI shorthand broadcast: enabled1666worker1 # [ 0.864418] sched_clock: Marking stable (702030764, 161441058)->(947428223, -83956401)1667client # [ 0.841083] sched_clock: Marking stable (689035240, 151145799)->(938524057, -98343018)1668worker1 # [ 0.869850] registered taskstats version 11669client # [ 0.844602] registered taskstats version 11670worker1 # [ 0.871287] Loading compiled-in X.509 certificates1671client # [ 0.846010] Loading compiled-in X.509 certificates1672lb # [ 0.853143] Demotion targets for Node 0: null1673lb # [ 0.854184] Key type .fscrypt registered1674lb # [ 0.854918] Key type fscrypt-provisioning registered1675worker2 # [ 0.877011] sched_clock: Marking stable (724023911, 152837431)->(956045949, -79184607)1676lb # [ 0.858245] ima: No TPM chip found, activating TPM-bypass!1677lb # [ 0.859257] ima: Allocated hash algorithm: sha11678worker2 # [ 0.884384] registered taskstats version 11679lb # [ 0.862129] ima: No architecture policies found1680lb # [ 0.863161] PM: Magic number: 10:275:4331681lb # [ 0.864939] RAS: Correctable Errors collector initialized.1682worker2 # [ 0.888067] Loading compiled-in X.509 certificates1683worker1 # [ 0.898043] Demotion targets for Node 0: null1684lb # [ 0.878307] clk: Disabling unused clocks1685lb # [ 0.879156] PM: genpd: Disabling unused power domains1686client # [ 0.874199] Demotion targets for Node 0: null1687worker1 # [ 0.901564] Key type .fscrypt registered1688client # [ 0.875575] Key type .fscrypt registered1689worker1 # [ 0.902583] Key type fscrypt-provisioning registered1690client # [ 0.879206] Key type fscrypt-provisioning registered1691worker1 # [ 0.906660] ima: No TPM chip found, activating TPM-bypass!1692client # [ 0.880606] ima: No TPM chip found, activating TPM-bypass!1693worker1 # [ 0.907759] ima: Allocated hash algorithm: sha11694client # [ 0.883211] ima: Allocated hash algorithm: sha11695client # [ 0.884357] ima: No architecture policies found1696worker1 # [ 0.910517] ima: No architecture policies found1697worker2 # [ 0.913021] Demotion targets for Node 0: null1698worker1 # [ 0.913471] PM: Magic number: 10:275:4331699worker2 # [ 0.917001] Key type .fscrypt registered1700worker1 # [ 0.915303] RAS: Correctable Errors collector initialized.1701client # [ 0.889216] PM: Magic number: 10:275:4331702worker2 # [ 0.917983] Key type fscrypt-provisioning registered1703worker2 # [ 0.921104] ima: No TPM chip found, activating TPM-bypass!1704client # [ 0.893282] RAS: Correctable Errors collector initialized.1705worker2 # [ 0.922369] ima: Allocated hash algorithm: sha11706worker2 # [ 0.924937] ima: No architecture policies found1707worker2 # [ 0.927885] PM: Magic number: 10:275:4331708worker1 # [ 0.926646] clk: Disabling unused clocks1709worker1 # [ 0.929477] PM: genpd: Disabling unused power domains1710worker2 # [ 0.933153] RAS: Correctable Errors collector initialized.1711client # [ 0.909786] clk: Disabling unused clocks1712client # [ 0.910722] PM: genpd: Disabling unused power domains1713worker2 # [ 0.950032] clk: Disabling unused clocks1714worker2 # [ 0.954907] PM: genpd: Disabling unused power domains1715lb # [ 1.093895] Freeing initrd memory: 29092K1716lb # [ 1.097379] Freeing unused decrypted memory: 2028K1717lb # [ 1.100165] Freeing unused kernel image (initmem) memory: 3644K1718lb # [ 1.101330] Write protecting the kernel read-only data: 32768k1719lb # [ 1.103403] Freeing unused kernel image (text/rodata gap) memory: 1208K1720lb # [ 1.104994] Freeing unused kernel image (rodata/data gap) memory: 752K1721worker1 # [ 1.137779] Freeing initrd memory: 29092K1722worker1 # [ 1.141238] Freeing unused decrypted memory: 2028K1723worker1 # [ 1.144039] Freeing unused kernel image (initmem) memory: 3644K1724worker1 # [ 1.145240] Write protecting the kernel read-only data: 32768k1725worker1 # [ 1.147339] Freeing unused kernel image (text/rodata gap) memory: 1208K1726worker1 # [ 1.149039] Freeing unused kernel image (rodata/data gap) memory: 752K1727lb # [ 1.161886] x86/mm: Checked W+X mappings: passed, no W+X pages found.1728lb # [ 1.163141] Run /init as init process1729client # [ 1.169617] Freeing initrd memory: 29096K1730lb # [ 1.176307] systemd[1]: Inserted module 'autofs4'1731client # [ 1.173080] Freeing unused decrypted memory: 2028K1732client # [ 1.175842] Freeing unused kernel image (initmem) memory: 3644K1733client # [ 1.177040] Write protecting the kernel read-only data: 32768k1734client # [ 1.179186] Freeing unused kernel image (text/rodata gap) memory: 1208K1735worker1 # [ 1.205953] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736worker1 # [ 1.207156] Run /init as init process1737client # [ 1.180799] Freeing unused kernel image (rodata/data gap) memory: 752K1738worker2 # [ 1.212741] Freeing initrd memory: 29092K1739worker2 # [ 1.216527] Freeing unused decrypted memory: 2028K1740lb # [ 1.194739] fuse: init (API version 7.45)1741worker2 # [ 1.219671] Freeing unused kernel image (initmem) memory: 3644K1742worker1 # [ 1.218494] systemd[1]: Inserted module 'autofs4'1743worker2 # [ 1.220982] Write protecting the kernel read-only data: 32768k1744worker2 # [ 1.223351] Freeing unused kernel image (text/rodata gap) memory: 1208K1745lb # [ 1.201387] ACPI: \_SB_.GSIG: Enabled at IRQ 221746worker2 # [ 1.225137] Freeing unused kernel image (rodata/data gap) memory: 752K1747lb # [ 1.203988] ACPI: \_SB_.GSIH: Enabled at IRQ 231748lb # [ 1.207348] ACPI: \_SB_.GSIE: Enabled at IRQ 201749lb # [ 1.209634] ACPI: \_SB_.GSIF: Enabled at IRQ 211750worker1 # [ 1.236983] fuse: init (API version 7.45)1751worker1 # [ 1.243749] ACPI: \_SB_.GSIG: Enabled at IRQ 221752worker1 # [ 1.246419] ACPI: \_SB_.GSIH: Enabled at IRQ 231753worker1 # [ 1.250826] ACPI: \_SB_.GSIE: Enabled at IRQ 201754worker1 # [ 1.254557] ACPI: \_SB_.GSIF: Enabled at IRQ 211755lb # [ 1.238349] systemd[1]: Successfully made /usr/ read-only.1756client # [ 1.237542] x86/mm: Checked W+X mappings: passed, no W+X pages found.1757client # [ 1.238770] Run /init as init process1758client # [ 1.250577] systemd[1]: Inserted module 'autofs4'1759worker2 # [ 1.283740] x86/mm: Checked W+X mappings: passed, no W+X pages found.1760worker2 # [ 1.285125] Run /init as init process1761client # [ 1.269434] fuse: init (API version 7.45)1762worker1 # [ 1.298183] systemd[1]: Successfully made /usr/ read-only.1763worker2 # [ 1.301036] systemd[1]: Inserted module 'autofs4'1764client # [ 1.276300] ACPI: \_SB_.GSIG: Enabled at IRQ 221765client # [ 1.278992] ACPI: \_SB_.GSIH: Enabled at IRQ 231766client # [ 1.282418] ACPI: \_SB_.GSIE: Enabled at IRQ 201767client # [ 1.284819] ACPI: \_SB_.GSIF: Enabled at IRQ 211768worker2 # [ 1.324712] fuse: init (API version 7.45)1769worker2 # [ 1.333308] ACPI: \_SB_.GSIG: Enabled at IRQ 221770worker2 # [ 1.336717] ACPI: \_SB_.GSIH: Enabled at IRQ 231771client # [ 1.312376] systemd[1]: Successfully made /usr/ read-only.1772worker2 # [ 1.341080] ACPI: \_SB_.GSIE: Enabled at IRQ 201773worker2 # [ 1.344224] ACPI: \_SB_.GSIF: Enabled at IRQ 211774worker2 # [ 1.380250] systemd[1]: Successfully made /usr/ read-only.1775lb # [ 1.574497] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776lb # [ 1.580040] systemd[1]: Detected virtualization kvm.1777lb # [ 1.581005] systemd[1]: Detected architecture x86-64.1778lb # [ 1.581986] systemd[1]: Running in initrd.1779lb # [ 1.583280] systemd[1]: Initializing machine ID from random generator.1780lb # [ 1.584569] systemd[1]: Hostname set to <lb>.1781worker1 # [ 1.634383] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1782worker1 # [ 1.639774] systemd[1]: Detected virtualization kvm.1783worker1 # [ 1.640770] systemd[1]: Detected architecture x86-64.1784worker1 # [ 1.641711] systemd[1]: Running in initrd.1785worker1 # [ 1.642912] systemd[1]: Initializing machine ID from random generator.1786worker1 # [ 1.644207] systemd[1]: Hostname set to <worker1>.1787client # [ 1.648120] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1788client # [ 1.653407] systemd[1]: Detected virtualization kvm.1789client # [ 1.654333] systemd[1]: Detected architecture x86-64.1790client # [ 1.655254] systemd[1]: Running in initrd.1791client # [ 1.656477] systemd[1]: Initializing machine ID from random generator.1792client # [ 1.657764] systemd[1]: Hostname set to <client>.1793worker2 # [ 1.716232] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1794worker2 # [ 1.723311] systemd[1]: Detected virtualization kvm.1795worker2 # [ 1.725670] systemd[1]: Detected architecture x86-64.1796worker2 # [ 1.727920] systemd[1]: Running in initrd.1797worker2 # [ 1.730627] systemd[1]: Initializing machine ID from random generator.1798worker2 # [ 1.733669] systemd[1]: Hostname set to <worker2>.1799lb # [ 1.784389] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker1 # [ 1.840685] systemd[1]: bpf-restrict-fs: LSM BPF program attached1801worker2 # [ 1.850902] systemd[1]: bpf-restrict-fs: LSM BPF program attached1802lb # [ 1.829756] systemd[1]: Queued start job for default target Initrd Default Target.1803lb # [ 1.834782] systemd[1]: Created slice Slice /system/modprobe.1804lb # [ 1.836159] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1805lb # [ 1.837661] systemd[1]: Expecting device /dev/disk/by-label/nixos...1806lb # [ 1.838865] systemd[1]: Reached target Path Units.1807lb # [ 1.839810] systemd[1]: Reached target Slice Units.1808lb # [ 1.840774] systemd[1]: Reached target Swaps.1809lb # [ 1.841607] systemd[1]: Reached target Timer Units.1810lb # [ 1.842707] systemd[1]: Listening on D-Bus System Message Bus Socket.1811lb # [ 1.844026] systemd[1]: Listening on Journal Socket (/dev/log).1812lb # [ 1.845337] systemd[1]: Listening on Journal Sockets.1813lb # [ 1.846444] systemd[1]: Listening on udev Control Socket.1814lb # [ 1.847533] systemd[1]: Listening on udev Kernel Socket.1815lb # [ 1.848553] systemd[1]: Reached target Socket Units.1816lb # [ 1.851030] systemd[1]: Starting Create List of Static Device Nodes...1817lb # [ 1.857187] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1818worker1 # [ 1.883239] systemd[1]: Queued start job for default target Initrd Default Target.1819lb # [ 1.866446] systemd[1]: Starting Load Kernel Module configfs...1820worker1 # [ 1.888078] systemd[1]: Created slice Slice /system/modprobe.1821client # [ 1.862334] systemd[1]: bpf-restrict-fs: LSM BPF program attached1822worker1 # [ 1.889412] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1823worker1 # [ 1.890978] systemd[1]: Expecting device /dev/disk/by-label/nixos...1824worker1 # [ 1.892179] systemd[1]: Reached target Path Units.1825worker1 # [ 1.893138] systemd[1]: Reached target Slice Units.1826worker2 # [ 1.895075] systemd[1]: Queued start job for default target Initrd Default Target.1827worker1 # [ 1.894080] systemd[1]: Reached target Swaps.1828worker1 # [ 1.894926] systemd[1]: Reached target Timer Units.1829worker1 # [ 1.895992] systemd[1]: Listening on D-Bus System Message Bus Socket.1830worker2 # [ 1.899422] systemd[1]: Created slice Slice /system/modprobe.1831worker1 # [ 1.897432] systemd[1]: Listening on Journal Socket (/dev/log).1832worker1 # [ 1.898763] systemd[1]: Listening on Journal Sockets.1833worker2 # [ 1.900721] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1834worker1 # [ 1.899881] systemd[1]: Listening on udev Control Socket.1835worker2 # [ 1.902170] systemd[1]: Expecting device /dev/disk/by-label/nixos...1836worker1 # [ 1.900971] systemd[1]: Listening on udev Kernel Socket.1837worker2 # [ 1.903319] systemd[1]: Reached target Path Units.1838worker1 # [ 1.901984] systemd[1]: Reached target Socket Units.1839worker2 # [ 1.904333] systemd[1]: Reached target Slice Units.1840worker2 # [ 1.905289] systemd[1]: Reached target Swaps.1841worker2 # [ 1.906144] systemd[1]: Reached target Timer Units.1842worker1 # [ 1.904206] systemd[1]: Starting Create List of Static Device Nodes...1843worker2 # [ 1.907208] systemd[1]: Listening on D-Bus System Message Bus Socket.1844worker2 # [ 1.908468] systemd[1]: Listening on Journal Socket (/dev/log).1845worker2 # [ 1.909702] systemd[1]: Listening on Journal Sockets.1846lb # [ 1.887397] systemd[1]: Starting Journal Service...1847worker1 # [ 1.908550] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1848worker2 # [ 1.910868] systemd[1]: Listening on udev Control Socket.1849worker2 # [ 1.911931] systemd[1]: Listening on udev Kernel Socket.1850worker2 # [ 1.912894] systemd[1]: Reached target Socket Units.1851worker2 # [ 1.915235] systemd[1]: Starting Create List of Static Device Nodes...1852worker1 # [ 1.916800] systemd[1]: Starting Load Kernel Module configfs...1853worker2 # [ 1.920952] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1854worker2 # [ 1.926514] systemd[1]: Starting Load Kernel Module configfs...1855lb # [ 1.905237] systemd[1]: Starting Load Kernel Modules...1856client # [ 1.906424] systemd[1]: Queued start job for default target Initrd Default Target.1857worker1 # [ 1.936565] systemd[1]: Starting Journal Service...1858client # [ 1.910645] systemd[1]: Created slice Slice /system/modprobe.1859client # [ 1.911942] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1860client # [ 1.913476] systemd[1]: Expecting device /dev/disk/by-label/nixos...1861client # [ 1.914650] systemd[1]: Reached target Path Units.1862worker2 # [ 1.943099] systemd[1]: Starting Journal Service...1863client # [ 1.915579] systemd[1]: Reached target Slice Units.1864client # [ 1.916529] systemd[1]: Reached target Swaps.1865client # [ 1.917391] systemd[1]: Reached target Timer Units.1866client # [ 1.918446] systemd[1]: Listening on D-Bus System Message Bus Socket.1867client # [ 1.919729] systemd[1]: Listening on Journal Socket (/dev/log).1868client # [ 1.920941] systemd[1]: Listening on Journal Sockets.1869lb # [ 1.914317] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1870client # [ 1.921987] systemd[1]: Listening on udev Control Socket.1871client # [ 1.923206] systemd[1]: Listening on udev Kernel Socket.1872lb # [ 1.929641] netfs: FS-Cache loaded1873client # [ 1.924234] systemd[1]: Reached target Socket Units.1874client # [ 1.926404] systemd[1]: Starting Create List of Static Device Nodes...1875worker1 # [ 1.953574] systemd[1]: Starting Load Kernel Modules...1876lb # [ 1.935360] 9pnet: Installing 9P2000 support1877client # [ 1.932274] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1878worker2 # [ 1.962963] systemd[1]: Starting Load Kernel Modules...1879client # [ 1.937812] systemd[1]: Starting Load Kernel Module configfs...1880lb # [ 1.948217] systemd[1]: Starting Coldplug All udev Devices...1881worker1 # [ 1.959593] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1882worker2 # [ 1.966969] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1883worker1 # [ 1.978515] netfs: FS-Cache loaded1884worker2 # [ 1.985616] netfs: FS-Cache loaded1885client # [ 1.957286] systemd[1]: Starting Journal Service...1886worker1 # [ 1.984336] 9pnet: Installing 9P2000 support1887lb # [ 1.965215] systemd[1]: Finished Create List of Static Device Nodes.1888worker2 # [ 1.991976] 9pnet: Installing 9P2000 support1889lb # [ 1.972998] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1890worker1 # [ 1.999843] systemd[1]: Starting Coldplug All udev Devices...1891client # [ 1.975309] systemd[1]: Starting Load Kernel Modules...1892lb # [ 1.982676] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1893worker2 # [ 2.005716] systemd[1]: Starting Coldplug All udev Devices...1894lb # [ 1.987678] systemd[1]: modprobe@configfs.service: Deactivated successfully.1895lb # [ 1.993324] systemd-journald[67]: Collecting audit messages is disabled.1896lb # [ 1.996186] systemd[1]: Finished Load Kernel Module configfs.1897worker2 # [ 2.023701] systemd[1]: Finished Create List of Static Device Nodes.1898client # [ 1.982324] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1899worker1 # [ 2.023628] systemd[1]: Finished Create List of Static Device Nodes.1900client # [ 2.000040] netfs: FS-Cache loaded1901lb # [ 2.004795] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1902lb # [ 2.007940] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1903worker2 # [ 2.031502] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1904client # [ 2.005994] 9pnet: Installing 9P2000 support1905worker1 # [ 2.034948] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1906lb # [ 2.017423] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1907worker2 # [ 2.041005] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1908lb # [ 2.019135] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1909worker1 # [ 2.044315] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1910client # [ 2.018297] systemd[1]: Starting Coldplug All udev Devices...1911worker2 # [ 2.047585] systemd[1]: modprobe@configfs.service: Deactivated successfully.1912worker2 # [ 2.049560] systemd-journald[67]: Collecting audit messages is disabled.1913worker1 # [ 2.052254] systemd[1]: modprobe@configfs.service: Deactivated successfully.1914worker1 # [ 2.055478] systemd-journald[67]: Collecting audit messages is disabled.1915worker2 # [ 2.057974] systemd[1]: Finished Load Kernel Module configfs.1916worker1 # [ 2.062178] systemd[1]: Finished Load Kernel Module configfs.1917client # [ 2.035895] systemd[1]: Finished Create List of Static Device Nodes.1918worker2 # [ 2.064597] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1919worker2 # [ 2.067468] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1920client # [ 2.044059] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1921worker1 # [ 2.070313] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1922worker2 # [ 2.077113] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1923lb # [ 2.054811] systemd[1]: Finished Load Kernel Modules.1924worker2 # [ 2.079237] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1925worker1 # [ 2.076519] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1926client # [ 2.054332] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1927worker1 # [ 2.084518] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1928client # [ 2.059737] systemd[1]: modprobe@configfs.service: Deactivated successfully.1929lb # [ 2.066030] systemd[1]: Starting Apply Kernel Variables...1930worker1 # [ 2.087086] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1931client # [ 2.067439] systemd-journald[67]: Collecting audit messages is disabled.1932client # [ 2.069259] systemd[1]: Finished Load Kernel Module configfs.1933client # [ 2.075845] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1934client # [ 2.079011] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1935lb # [ 2.086209] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1936worker2 # [ 2.116959] systemd[1]: Finished Load Kernel Modules.1937client # [ 2.089314] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1938lb # [ 2.098211] systemd[1]: Starting Create Static Device Nodes in /dev...1939client # [ 2.093225] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1940worker1 # [ 2.125297] systemd[1]: Finished Load Kernel Modules.1941worker2 # [ 2.128963] systemd[1]: Starting Apply Kernel Variables...1942worker1 # [ 2.137096] systemd[1]: Starting Apply Kernel Variables...1943worker2 # [ 2.143054] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1944lb # [ 2.125504] systemd[1]: Finished Apply Kernel Variables.1945lb # [ 1.977168] systemd-modules-load[68]: Inserted module 'dm_mod'1946worker1 # [ 2.152785] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1947lb # [ 1.979335] systemd-modules-load[68]: Inserted module 'virtio_balloon'1948lb # [ 2.133894] systemd[1]: Started Journal Service.1949client # [ 2.128881] systemd[1]: Finished Load Kernel Modules.1950lb # [ 1.982421] systemd-modules-load[68]: Inserted module 'virtio_gpu'1951worker2 # [ 2.159992] systemd[1]: Starting Create Static Device Nodes in /dev...1952client # [ 2.139511] systemd[1]: Starting Apply Kernel Variables...1953worker1 # [ 2.169880] systemd[1]: Starting Create Static Device Nodes in /dev...1954lb # [ 2.002187] systemd[1]: Finished Create Static Device Nodes in /dev.1955lb # [ 2.004724] systemd[1]: Reached target Preparation for Local File Systems.1956lb # [ 2.009113] systemd[1]: Reached target Local File Systems.1957lb # [ 2.011125] systemd[1]: Starting Create System Files and Directories...1958worker2 # [ 2.187020] systemd[1]: Finished Apply Kernel Variables.1959client # [ 2.162283] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1960lb # [ 2.016262] systemd[1]: Starting Rule-based Manager for Device Events and Files...1961worker2 # [ 2.192404] systemd[1]: Started Journal Service.1962worker2 # [ 2.042220] systemd-modules-load[68]: Inserted module 'dm_mod'1963worker2 # [ 2.043700] systemd-modules-load[68]: Inserted module 'virtio_balloon'1964worker2 # [ 2.047223] systemd-modules-load[68]: Inserted module 'virtio_gpu'1965worker1 # [ 2.198192] systemd[1]: Finished Apply Kernel Variables.1966client # [ 2.173314] systemd[1]: Starting Create Static Device Nodes in /dev...1967worker1 # [ 2.035910] systemd-modules-load[68]: Inserted module 'dm_mod'1968worker1 # [ 2.041101] systemd-modules-load[68]: Inserted module 'virtio_balloon'1969worker1 # [ 2.203823] systemd[1]: Started Journal Service.1970worker1 # [ 2.043364] systemd-modules-load[68]: Inserted module 'virtio_gpu'1971worker2 # [ 2.067303] systemd[1]: Finished Create Static Device Nodes in /dev.1972worker2 # [ 2.068516] systemd[1]: Reached target Preparation for Local File Systems.1973worker2 # [ 2.069647] systemd[1]: Reached target Local File Systems.1974worker2 # [ 2.074113] systemd[1]: Starting Create System Files and Directories...1975worker1 # [ 2.064166] systemd[1]: Finished Create Static Device Nodes in /dev.1976worker1 # [ 2.065343] systemd[1]: Reached target Preparation for Local File Systems.1977worker1 # [ 2.066367] systemd[1]: Reached target Local File Systems.1978client # [ 2.201370] systemd[1]: Finished Apply Kernel Variables.1979worker1 # [ 2.071164] systemd[1]: Starting Create System Files and Directories...1980worker2 # [ 2.083089] systemd[1]: Starting Rule-based Manager for Device Events and Files...1981lb # [ 2.061905] systemd[1]: Finished Create System Files and Directories.1982client # [ 2.058479] systemd-modules-load[68]: Inserted module 'dm_mod'[ 2.210888] systemd[1]: Started Journal Service.1983client # 1984client # [ 2.062361] systemd-modules-load[68]: Inserted module 'virtio_balloon'1985worker1 # [ 2.079443] systemd[1]: Starting Rule-based Manager for Device Events and Files...1986client # [ 2.063631] systemd-modules-load[68]: Inserted module 'virtio_gpu'1987client # [ 2.081182] systemd[1]: Finished Create Static Device Nodes in /dev.1988lb # [ 2.088520] systemd-udevd[83]: Using default interface naming scheme 'v261'.1989client # [ 2.085104] systemd[1]: Reached target Preparation for Local File Systems.1990client # [ 2.086248] systemd[1]: Reached target Local File Systems.1991client # [ 2.087702] systemd[1]: Starting Create System Files and Directories...1992worker2 # [ 2.117595] systemd[1]: Finished Create System Files and Directories.1993client # [ 2.095105] systemd[1]: Starting Rule-based Manager for Device Events and Files...1994worker1 # [ 2.121110] systemd[1]: Finished Create System Files and Directories.1995lb # [ 2.117267] systemd[1]: Started Rule-based Manager for Device Events and Files.1996worker2 # [ 2.148416] systemd-udevd[82]: Using default interface naming scheme 'v261'.1997worker1 # [ 2.143365] systemd-udevd[83]: Using default interface naming scheme 'v261'.1998client # [ 2.137575] systemd[1]: Finished Create System Files and Directories.1999worker2 # [ 2.175465] systemd[1]: Started Rule-based Manager for Device Events and Files.2000worker1 # [ 2.170764] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 2,018 hidden lines…
4019lb # [ 17.516800] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14020lb # [ 17.518878] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4021lb # [ 17.522166] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4022lb # [ 17.524609] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4023lb # [ 17.526487] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4024lb # [ 17.529429] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4025lb # [ 17.532380] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4026lb # [ 17.534216] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4027lb # [ 17.539514] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4028lb # [ 17.542134] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4029lb # [ 17.544350] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4030lb # [ 17.546723] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4031lb # [ 17.549048] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4032lb # [ 17.551099] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4033lb # [ 17.553369] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4034lb # [ 17.556927] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4035lb # [ 17.558665] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4036lb # [ 17.560254] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4037lb # [ 17.561955] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4038lb # [ 17.565091] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4039lb # [ 17.566959] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4040lb # [ 17.569151] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4041lb # [ 17.573775] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4042lb # [ 17.577221] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4043lb # [ 17.580107] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4044lb # [ 17.584205] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4045lb # [ 17.586086] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4046lb # [ 17.588437] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4047lb # [ 17.591023] envoy[922]: [2026-09-20 13:26:09.502][922][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4048lb # [ 17.593270] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4049lb # [ 17.595820] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4050lb # [ 17.597669] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24051lb # [ 17.600154] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4052lb # [ 17.601821] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4053lb # [ 17.607205] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24054lb # [ 17.609116] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4055lb # [ 17.618785] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4056lb # [ 17.625900] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4057lb # [ 17.629702] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4058lb # [ 17.631918] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4059lb # [ 17.636821] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4060lb # [ 17.639671] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4061lb # [ 17.642313] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4062lb # [ 17.644420] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4063lb # [ 17.646542] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4064lb # [ 17.648903] envoy[922]: [2026-09-20 13:26:09.503][922][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4065lb # [ 17.652153] envoy[922]: [2026-09-20 13:26:09.504][922][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4066lb # [ 17.656731] envoy[922]: [2026-09-20 13:26:09.504][922][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4067lb # [ 17.659420] envoy[922]: [2026-09-20 13:26:09.504][922][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4068lb # [ 17.661319] envoy[922]: [2026-09-20 13:26:09.504][922][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4069lb # [ 17.663381] envoy[922]: [2026-09-20 13:26:09.504][922][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4070lb # [ 17.665072] envoy[922]: [2026-09-20 13:26:09.504][922][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4071lb # [ 17.668270] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4072lb # [ 17.670436] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4073lb # [ 17.672927] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4074lb # [ 17.674489] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4075lb # [ 17.676237] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4076lb # [ 17.678149] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4077lb # [ 17.680422] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4078lb # [ 17.683143] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4079lb # [ 17.685255] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4080lb # [ 17.688506] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4081lb # [ 17.690453] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4082lb # [ 17.695531] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4083lb # [ 17.707358] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4084lb # [ 17.709305] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4085lb # [ 17.711948] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4086lb # [ 17.713890] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4087lb # [ 17.715902] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4088lb # [ 17.717597] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4089lb # [ 17.720105] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4090lb # [ 17.725730] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24091lb # [ 17.728231] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4092lb # [ 17.731421] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4093lb # [ 17.733777] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4094lb # [ 17.739659] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4095lb # [ 17.746924] envoy[922]: [2026-09-20 13:26:09.505][922][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4096lb # [ 17.748899] envoy[922]: [2026-09-20 13:26:09.506][922][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4097lb # [ 17.777492] envoy[922]: [2026-09-20 13:26:09.718][922][info][main] [source/server/server.cc:531] HTTP header map info:4098lb # [ 17.779127] envoy[922]: [2026-09-20 13:26:09.766][922][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4099lb # [ 17.793301] envoy[922]: [2026-09-20 13:26:09.766][922][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4100lb # [ 17.795043] envoy[922]: [2026-09-20 13:26:09.766][922][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4101lb # [ 17.803343] envoy[922]: [2026-09-20 13:26:09.766][922][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4102lb # [ 17.854712] envoy[922]: [2026-09-20 13:26:09.930][922][info][main] [source/server/server.cc:968] runtime: {}4103lb # [ 17.864054] envoy[922]: [2026-09-20 13:26:09.939][922][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4104lb # [ 17.867685] envoy[922]: [2026-09-20 13:26:09.943][922][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014105lb # [ 17.871983] envoy[922]: [2026-09-20 13:26:09.947][922][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014106lb # [ 17.876553] envoy[922]: [2026-09-20 13:26:09.952][922][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4107lb # [ 17.879123] envoy[922]: [2026-09-20 13:26:09.954][922][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4108lb # [ 17.881426] envoy[922]: [2026-09-20 13:26:09.954][922][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4109worker1 # [ 17.979907] systemd[1]: Starting Nix worker-protocol over gRPC...4110lb # [ 17.971611] envoy[922]: [2026-09-20 13:26:10.047][922][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4111worker2 # [ 17.994324] systemd[1]: Reached target Network is Online.4112worker2 # [ 18.000396] systemd[1]: Starting scheduler-token.service...4113lb # [ 18.015723] envoy[922]: [2026-09-20 13:26:10.091][922][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4114lb # [ 18.019699] envoy[922]: [2026-09-20 13:26:10.095][922][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4115lb # [ 18.023971] envoy[922]: [2026-09-20 13:26:10.099][922][info][main] [source/server/server.cc:1077] starting main dispatch loop4116worker2 # [ 18.102116] systemd[1]: scheduler-token.service: Deactivated successfully.4117worker2 # [ 18.103383] systemd[1]: Finished scheduler-token.service.4118worker2 # [ 18.104936] systemd[1]: scheduler-token.service: Consumed 32ms CPU time over 102ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4119worker2 # [ 18.110421] systemd[1]: Starting Nix worker-protocol over gRPC...4120worker1 # [ 18.195106] nix-grpc-daemon[846]: ts=2026-09-20T13:26:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4121worker1 # [ 18.207905] nix-grpc-daemon[846]: ts=2026-09-20T13:26:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4122worker1 # [ 18.214802] nix-grpc-daemon[846]: ts=2026-09-20T13:26:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14123worker1 # [ 18.270097] nix-grpc-daemon[846]: ts=2026-09-20T13:26:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4124worker2 # [ 18.281994] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4125worker1 # [ 18.275378] systemd[1]: Started Nix worker-protocol over gRPC.4126worker2 # [ 18.290225] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4127worker2 # [ 18.296603] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14128worker2 # [ 18.346716] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4129lb # [ 18.325426] envoy[922]: [2026-09-20 13:26:10.401][922][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4130worker1 # [ 18.337644] nix-grpc-daemon[846]: ts=2026-09-20T13:26:09Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4131lb # [ 18.328189] envoy[922]: [2026-09-20 13:26:10.404][922][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4132worker2 # [ 18.353254] systemd[1]: Started Nix worker-protocol over gRPC.4133lb # [ 18.330506] envoy[922]: [2026-09-20 13:26:10.404][922][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4134lb # [ 18.332737] envoy[922]: [2026-09-20 13:26:10.406][922][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4135worker2 # [ 18.413386] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=scheduler_connected addr=lb:500514136worker2 # [ 18.426388] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker1 # [ 18.539084] nix-grpc-daemon[846]: ts=2026-09-20T13:26:10Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4138worker2 # [ 18.629088] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 18.631727] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker2 # [ 18.832795] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 18.835272] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker1 # [ 18.939510] nix-grpc-daemon[846]: ts=2026-09-20T13:26:10Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4143worker2 # [ 19.036515] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=scheduler_connected addr=lb:500514144worker2 # [ 19.039065] nix-grpc-daemon[786]: ts=2026-09-20T13:26:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 19.239897] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 19.242376] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 19.444586] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 19.446649] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 19.645764] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 19.648437] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 19.743189] nix-grpc-daemon[846]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514152worker1 # [ 19.746252] nix-grpc-daemon[846]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker2 # [ 19.849344] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 19.851257] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker1 # [ 19.947126] nix-grpc-daemon[846]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514156worker1 # [ 19.949655] nix-grpc-daemon[846]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 20.052264] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 20.054631] nix-grpc-daemon[786]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 20.150348] nix-grpc-daemon[846]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514160worker1 # [ 20.152964] nix-grpc-daemon[846]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker2 # [ 20.255838] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 20.258265] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker1 # [ 20.353763] nix-grpc-daemon[846]: ts=2026-09-20T13:26:11Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 20.356617] nix-grpc-daemon[846]: ts=2026-09-20T13:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 20.459272] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 20.461106] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker1 # [ 20.557265] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514168worker1 # [ 20.559671] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker2 # [ 20.662130] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 20.664532] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker1 # [ 20.761113] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514172worker1 # [ 20.763853] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker2 # [ 20.865298] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 20.867282] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker1 # [ 20.964827] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514176worker1 # [ 20.967425] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 21.068668] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514178worker2 # [ 21.072636] nix-grpc-daemon[786]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker1 # [ 21.168165] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514180worker1 # [ 21.170477] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker2 # [ 21.273948] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 21.278688] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker1 # [ 21.371437] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 21.373960] nix-grpc-daemon[846]: ts=2026-09-20T13:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker2 # [ 21.479928] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 21.482256] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker1 # [ 21.574966] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 21.577902] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker2 # [ 21.682969] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514190worker2 # [ 21.684942] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker1 # [ 21.779568] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 21.782279] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 21.886404] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 21.890474] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 21.983250] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514196worker1 # [ 21.985838] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker2 # [ 22.091298] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514198worker2 # [ 22.094498] nix-grpc-daemon[786]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker1 # [ 22.186757] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 22.189394] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker2 # [ 22.295440] nix-grpc-daemon[786]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 22.300149] nix-grpc-daemon[786]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker1 # [ 22.390099] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 22.392865] nix-grpc-daemon[846]: ts=2026-09-20T13:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker2 # [ 22.501120] nix-grpc-daemon[786]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 22.503240] nix-grpc-daemon[786]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 22.593537] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514208worker1 # [ 22.595610] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker2 # [ 22.704504] nix-grpc-daemon[786]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 22.706966] nix-grpc-daemon[786]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 22.796530] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 22.798584] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 22.908359] nix-grpc-daemon[786]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 22.911176] nix-grpc-daemon[786]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker1 # [ 22.999347] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 23.001340] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 23.112562] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514218worker2 # [ 23.115364] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker1 # [ 23.202068] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 23.204278] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 23.316397] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 23.318875] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 23.406585] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_connected addr=lb:500514224worker1 # [ 23.409939] nix-grpc-daemon[846]: ts=2026-09-20T13:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 23.520708] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514226worker2 # [ 23.523194] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker1 # [ 23.607422] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 23.609984] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 23.724290] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 23.726522] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker1 # [ 23.812781] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 23.816214] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker2 # [ 23.927561] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 23.929685] nix-grpc-daemon[786]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker1 # [ 24.014064] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 24.018486] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 24.130878] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 24.133722] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239lb # [ 24.114863] rustfs-bucket-start[1003]: mb s3://farm4240lb # [ 24.123603] systemd[1]: Finished rustfs-bucket.service.4241lb # [ 24.133078] systemd[1]: Starting niks3 server...4242worker1 # [ 24.219848] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514243worker1 # [ 24.224337] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker2 # [ 24.334485] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 24.336503] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker1 # [ 24.424820] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_connected addr=lb:500514247worker1 # [ 24.426830] nix-grpc-daemon[846]: ts=2026-09-20T13:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248lb # [ 24.501545] postgres[1022]: [1022] ERROR: relation "goose_db_version" does not exist at character 364249lb # [ 24.503619] postgres[1022]: [1022] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4250worker2 # [ 24.538786] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 24.540760] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252lb # [ 24.569498] niks3-server[1013]: 2026/09/20 13:26:16 OK 20241026095416_initial_model.sql (35.66ms)4253lb # [ 24.577114] niks3-server[1013]: 2026/09/20 13:26:16 OK 20251210153512_drop_unused_gin_index.sql (4.25ms)4254lb # [ 24.585721] niks3-server[1013]: 2026/09/20 13:26:16 OK 20251218171726_add_pins.sql (8.69ms)4255lb # [ 24.597313] niks3-server[1013]: 2026/09/20 13:26:16 OK 20260628120000_add_object_size_and_stats.sql (9.78ms)4256lb # [ 24.605059] niks3-server[1013]: 2026/09/20 13:26:16 OK 20260905000000_add_claims.sql (5.53ms)4257lb # [ 24.613217] niks3-server[1013]: 2026/09/20 13:26:16 OK 20260920000000_drop_claims.sql (7.94ms)4258worker1 # [ 24.627780] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514259lb # [ 24.614938] niks3-server[1013]: 2026/09/20 13:26:16 goose: successfully migrated database to version: 202609200000004260worker1 # [ 24.630620] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261lb # [ 24.624941] niks3-server[1013]: 2026/09/20 13:26:16 OK 1_commit_pending_closure.sql (11.89ms)4262lb # [ 24.628246] niks3-server[1013]: 2026/09/20 13:26:16 OK 2_object_stats_trigger.sql (1.76ms)4263lb # [ 24.629665] niks3-server[1013]: 2026/09/20 13:26:16 goose: up to current file version: 24264lb # [ 24.637134] niks3-server[1013]: 2026/09/20 13:26:16 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4265worker2 # [ 24.739909] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514266worker2 # [ 24.741867] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267lb # [ 24.729776] niks3-server[1013]: 2026/09/20 13:26:16 INFO Read proxy enabled — serving cache objects from S34268lb # [ 24.732108] niks3-server[1013]: 2026/09/20 13:26:16 INFO Using socket-activated listener address=0.0.0.0:57514269lb # [ 24.737988] niks3-server[1013]: 2026/09/20 13:26:16 INFO systemd watchdog enabled interval=15s4270lb # [ 24.740291] systemd[1]: Started niks3 server.4271lb # [ 24.743272] niks3-server[1013]: 2026/09/20 13:26:16 INFO Starting HTTP server address=0.0.0.0:57514272lb # [ 24.745200] systemd[1]: Reached target Multi-User System.4273lb # [ 24.746948] systemd[1]: Startup finished in 1.014s (kernel) + 4.045s (initrd) + 19.687s (userspace) = 24.746s.4274lb # [ 24.755228] niks3-server[1013]: 2026/09/20 13:26:16 INFO lead: acquired remote=192.168.1.4:518924275worker2 # [ 24.784407] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_take_over4276worker1 # [ 24.832367] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514277worker1 # [ 24.834733] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker2 # [ 24.942679] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514279worker2 # [ 24.944732] nix-grpc-daemon[786]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4280worker1 # [ 25.035840] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514281worker1 # [ 25.038419] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker2 # [ 25.145649] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=scheduler_connected addr=lb:500514283worker2 # [ 25.147679] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284worker1 # [ 25.239214] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514285worker1 # [ 25.242095] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4286worker2 # [ 25.349086] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=scheduler_connected addr=lb:500514287worker2 # [ 25.351697] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4288worker1 # [ 25.443057] nix-grpc-daemon[846]: ts=2026-09-20T13:26:16Z level=info event=scheduler_connected addr=lb:500514289worker2 # [ 25.466914] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:432584290worker2 # [ 25.469928] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04291lb: (finished: waiting for unit niks3.service, in 14.13 seconds)4292lb: waiting for TCP port 5751 on localhost4293worker2 # [ 25.553415] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 25.575129] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=oidc_jwks provider=mock keys=14295worker2 # [ 25.578254] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:432584296worker2 # [ 25.580966] nix-grpc-daemon[786]: ts=2026-09-20T13:26:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04297lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4298lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4299lb: waiting for unit envoy.service4300lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4301lb: waiting for TCP port 50051 on localhost4302lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4303lb # [ 25.684706] envoy[922]: conn peer=[::1]:32988 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=44304lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4305worker2 # [ 25.832357] systemd[1]: Starting scheduler-token.service...4306worker2 # [ 25.902385] systemd[1]: scheduler-token.service: Deactivated successfully.4307worker2 # [ 25.904398] systemd[1]: Finished scheduler-token.service.4308worker2 # [ 25.906419] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 72ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4309lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4310lb: must succeed: curl -sf localhost:9901/clusters4311lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4312lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4313subtest: one node schedules and both builders hold a WorkerSession on it4314lb: must succeed: curl -sf localhost:9901/clusters4315lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4316worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4317worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4318lb: must succeed: curl -sf localhost:9901/clusters4319lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4320lb: must succeed: curl -sf localhost:9901/clusters4321lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4322lb: must succeed: curl -sf localhost:9901/clusters4323lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4324lb: must succeed: curl -sf localhost:9901/clusters4325lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4326worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4327worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.03 seconds)4328worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4329worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4330lb: must succeed: curl -sf localhost:9901/clusters4331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4332lb: must succeed: curl -sf localhost:9901/clusters4333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4334worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4335worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4336worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4337worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4338worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4339worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4340worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4341worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4342(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.72 seconds)4343subtest: balancer auth: client cert, bearer token, nothing, unknown CN4344worker1 # [ 27.019221] nix-grpc-daemon[846]: ts=2026-09-20T13:26:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506244345worker1 # [ 27.026218] systemd[1]: Started Nix Daemon.4346worker1 # [ 27.249700] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4347worker1 # [ 27.305253] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)4348worker1 # [ 27.357102] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 2/5)4349worker1 # [ 27.446356] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 391 ms (attempt 3/5)4350worker1 # [ 27.840845] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 4/5)4351worker1 # [ 28.145389] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4352worker2 # [ 28.190413] nix-grpc-daemon[786]: ts=2026-09-20T13:26:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:432704353worker2 # [ 28.199963] systemd[1]: Started Nix Daemon.4354worker2 # [ 28.438598] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4355worker2 # [ 28.476094] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)4356worker2 # [ 28.507460] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)4357worker2 # [ 28.546831] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 300 ms (attempt 3/5)4358worker2 # [ 28.850404] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 380 ms (attempt 4/5)4359worker2 # [ 29.237138] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4360worker2 # [ 29.254956] nix-grpc-daemon[786]: ts=2026-09-20T13:26:21Z level=debug event=path_info_miss path=b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test error="error: path '/nix/store/b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test' is required, but there is no substituter that can build it"4361worker2 # [ 29.263772] nix-grpc-daemon[786]: ts=2026-09-20T13:26:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:432704362worker2 # [ 29.272566] nix-grpc-daemon[786]: ts=2026-09-20T13:26:21Z level=debug event=path_info_miss path=b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test error="error: path '/nix/store/b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test' is required, but there is no substituter that can build it"4363client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4364client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.07 seconds)4365client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4366client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4367worker1 # [ 29.604121] nix-grpc-daemon[846]: ts=2026-09-20T13:26:21Z level=info event=oidc_jwks provider=mock keys=14368worker1 # [ 29.606477] nix-grpc-daemon[846]: ts=2026-09-20T13:26:21Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:506244369worker1 # [ 29.616955] nix-grpc-daemon[846]: ts=2026-09-20T13:26:21Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:506244370worker1 # [ 29.633725] nix-grpc-daemon[846]: ts=2026-09-20T13:26:21Z level=debug event=path_info_miss path=b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test error="error: path '/nix/store/b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test' is required, but there is no substituter that can build it"4371worker2 # [ 29.650281] nix-grpc-daemon[786]: ts=2026-09-20T13:26:21Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:432704372worker2 # [ 29.655907] nix-grpc-daemon[786]: ts=2026-09-20T13:26:21Z level=debug event=path_info_miss path=b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test error="error: path '/nix/store/b3f8wnb0cc9m2wvhfvwcdmiwfsmpklab-nixos-system-client-test' is required, but there is no substituter that can build it"4373client: must succeed: rm -r /var/lib/nix-grpc-store4374client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4375worker1 # [ 29.899660] nix-grpc-daemon[846]: ts=2026-09-20T13:26:21Z level=info event=denied method=QueryMissing cn=- role=none4376worker1 # [ 30.111615] nix-grpc-daemon[846]: ts=2026-09-20T13:26:21Z level=info event=denied method=QueryMissing cn=stranger role=none4377(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.44 seconds)4378subtest: DAG build is scheduled across workers and lands in the cache4379client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24380worker2 # [ 30.360771] nix-grpc-daemon[786]: ts=2026-09-20T13:26:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:432704381client # don't know how to build these paths:4382client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4383worker1 # [ 30.370981] nix-grpc-daemon[846]: ts=2026-09-20T13:26:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244384worker2 # [ 30.427263] nix-grpc-daemon[786]: ts=2026-09-20T13:26:22Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04385worker2 # [ 30.432349] nix-grpc-daemon[786]: ts=2026-09-20T13:26:22Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14386worker1 # [ 30.473478] nix-grpc-daemon[846]: ts=2026-09-20T13:26:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244387worker1 # [ 30.495661] nix-grpc-daemon[846]: ts=2026-09-20T13:26:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244388client # copying 1 paths to 'lb:50051'...4389worker1 # [ 30.509472] nix-grpc-daemon[846]: ts=2026-09-20T13:26:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506244390client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4391worker1 # [ 30.536318] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4392worker1 # [ 30.558064] nix-grpc-daemon[846]: ts=2026-09-20T13:26:22Z level=info event=niks3_push_started pid=9164393worker1 # [ 30.844958] nix-grpc-daemon[916]: time=2026-09-20T13:26:22.381Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4394worker1 # [ 31.060719] nix-daemon[906]: accepted connection from pid 929, user nix-grpc-daemon (trusted)4395worker1 # [ 31.078697] nix-daemon[906]: reaped child process 932, status = succeeded4396worker1 # [ 31.227276] nix-daemon[906]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4397worker1 # [ 31.240695] nix-daemon[906]: reaped child process 937, status = succeeded4398lb # [ 31.239964] niks3-server[1013]: 2026/09/20 13:26:23 INFO Received uploads request method=POST path=/api/pending_closures4399worker1 # [ 31.270123] nix-grpc-daemon[916]: time=2026-09-20T13:26:22.810Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4400worker1 # [ 31.272414] nix-grpc-daemon[916]: time=2026-09-20T13:26:22.812Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4401lb # [ 31.287299] niks3-server[1013]: 2026/09/20 13:26:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4402lb # [ 31.301114] niks3-server[1013]: 2026/09/20 13:26:23 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4403lb # [ 31.314069] niks3-server[1013]: 2026/09/20 13:26:23 INFO Signed narinfos id=1 count=14404worker1 # [ 31.328822] nix-grpc-daemon[916]: time=2026-09-20T13:26:22.868Z level=INFO msg="Uploading 1 narinfos"4405lb # [ 31.325124] niks3-server[1013]: 2026/09/20 13:26:23 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4406lb # [ 31.330870] niks3-server[1013]: 2026/09/20 13:26:23 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4407lb # [ 31.340594] niks3-server[1013]: 2026/09/20 13:26:23 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4408lb # [ 31.352406] niks3-server[1013]: 2026/09/20 13:26:23 INFO Completed upload id=14409worker1 # [ 31.365238] nix-grpc-daemon[916]: time=2026-09-20T13:26:22.905Z level=INFO msg="Upload complete. (492ms)"4410worker1 # [ 31.368363] nix-grpc-daemon[846]: ts=2026-09-20T13:26:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_in=5044411worker1 # [ 31.371722] nix-grpc-daemon[846]: ts=2026-09-20T13:26:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244412worker1 # [ 31.380201] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4413worker1 # [ 31.381752] nix-daemon[906]: reaped child process 914, status = succeeded4414client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4415worker1 # [ 31.387934] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4416client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4417client # farm-a> LOG-farm-a-t14418worker1 # [ 32.603543] nix-daemon[906]: reaped child process 941, status = succeeded4419worker1 # [ 32.750168] nix-daemon[906]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4420worker1 # [ 32.763698] nix-daemon[906]: reaped child process 960, status = succeeded4421lb # [ 32.767588] niks3-server[1013]: 2026/09/20 13:26:24 INFO Received uploads request method=POST path=/api/pending_closures4422worker1 # [ 32.790925] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.330Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4423worker1 # [ 32.792766] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.332Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4424lb # [ 32.804530] niks3-server[1013]: 2026/09/20 13:26:24 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4425lb # [ 32.809730] niks3-server[1013]: 2026/09/20 13:26:24 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4426lb # [ 32.814714] niks3-server[1013]: 2026/09/20 13:26:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4427lb # [ 32.817937] niks3-server[1013]: 2026/09/20 13:26:24 INFO Signed narinfos id=2 count=14428worker1 # [ 32.831274] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.370Z level=INFO msg="Uploading 1 narinfos"4429lb # [ 32.820935] niks3-server[1013]: 2026/09/20 13:26:24 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4430lb # [ 32.830300] niks3-server[1013]: 2026/09/20 13:26:24 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4431lb # [ 32.833738] niks3-server[1013]: 2026/09/20 13:26:24 INFO Completed upload id=24432worker1 # [ 32.846577] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.386Z level=INFO msg="Upload complete. (249ms)"4433lb # [ 32.835556] niks3-server[1013]: 2026/09/20 13:26:24 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4434worker1 # [ 32.850379] nix-grpc-daemon[846]: ts=2026-09-20T13:26:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14435worker1 # [ 32.857091] nix-daemon[906]: reaped child process 939, status = succeeded4436worker2 # [ 32.873344] nix-grpc-daemon[786]: ts=2026-09-20T13:26:24Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04437worker2 # [ 32.875096] nix-grpc-daemon[786]: ts=2026-09-20T13:26:24Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14438worker1 # [ 32.906862] nix-grpc-daemon[846]: ts=2026-09-20T13:26:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244439worker1 # [ 32.916085] nix-grpc-daemon[846]: ts=2026-09-20T13:26:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244440worker1 # [ 32.941980] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.481Z level=INFO msg="All 1 paths already cached"4441client # copying 1 paths to 'lb:50051'...4442client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4443worker1 # [ 32.947324] nix-grpc-daemon[846]: ts=2026-09-20T13:26:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506244444worker1 # [ 32.953074] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4445worker1 # [ 33.119334] nix-daemon[906]: accepted connection from pid 964, user nix-grpc-daemon (trusted)4446worker1 # [ 33.139734] nix-daemon[906]: reaped child process 967, status = succeeded4447worker1 # [ 33.294344] nix-daemon[906]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4448worker1 # [ 33.307177] nix-daemon[906]: reaped child process 973, status = succeeded4449lb # [ 33.308776] niks3-server[1013]: 2026/09/20 13:26:25 INFO Received uploads request method=POST path=/api/pending_closures4450worker1 # [ 33.336359] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.876Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4451worker1 # [ 33.337884] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.877Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4452lb # [ 33.353452] niks3-server[1013]: 2026/09/20 13:26:25 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4453lb # [ 33.366454] niks3-server[1013]: 2026/09/20 13:26:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4454lb # [ 33.372361] niks3-server[1013]: 2026/09/20 13:26:25 INFO Signed narinfos id=3 count=14455worker1 # [ 33.387903] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.927Z level=INFO msg="Uploading 1 narinfos"4456lb # [ 33.378354] niks3-server[1013]: 2026/09/20 13:26:25 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4457lb # [ 33.390287] niks3-server[1013]: 2026/09/20 13:26:25 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4458lb # [ 33.397760] niks3-server[1013]: 2026/09/20 13:26:25 INFO Completed upload id=34459worker1 # [ 33.410829] nix-grpc-daemon[916]: time=2026-09-20T13:26:24.950Z level=INFO msg="Upload complete. (450ms)"4460lb # [ 33.399884] niks3-server[1013]: 2026/09/20 13:26:25 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4461worker1 # [ 33.415696] nix-grpc-daemon[846]: ts=2026-09-20T13:26:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_in=6244462worker1 # [ 33.422537] nix-grpc-daemon[846]: ts=2026-09-20T13:26:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244463worker1 # [ 33.432153] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4464worker1 # [ 33.441800] nix-daemon[906]: reaped child process 962, status = succeeded4465client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4466client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4467worker1 # [ 33.452589] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4468client # farm-b> LOG-farm-b-t14469worker1 # [ 34.669214] nix-daemon[906]: reaped child process 977, status = succeeded4470worker1 # [ 34.817087] nix-daemon[906]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4471worker1 # [ 34.829641] nix-daemon[906]: reaped child process 995, status = succeeded4472lb # [ 34.832941] niks3-server[1013]: 2026/09/20 13:26:26 INFO Received uploads request method=POST path=/api/pending_closures4473worker1 # [ 34.862256] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.401Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4474worker1 # [ 34.864858] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.403Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4475lb # [ 34.878965] niks3-server[1013]: 2026/09/20 13:26:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4476lb # [ 34.883075] niks3-server[1013]: 2026/09/20 13:26:26 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4477lb # [ 34.886915] niks3-server[1013]: 2026/09/20 13:26:26 INFO Signed narinfos id=4 count=14478worker1 # [ 34.899643] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.439Z level=INFO msg="Uploading 1 narinfos"4479lb # [ 34.891374] niks3-server[1013]: 2026/09/20 13:26:26 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4480lb # [ 34.895072] niks3-server[1013]: 2026/09/20 13:26:26 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4481lb # [ 34.897263] niks3-server[1013]: 2026/09/20 13:26:26 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4482lb # [ 34.905556] niks3-server[1013]: 2026/09/20 13:26:26 INFO Completed upload id=44483worker1 # [ 34.918342] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.458Z level=INFO msg="Upload complete. (259ms)"4484lb # [ 34.906847] niks3-server[1013]: 2026/09/20 13:26:26 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4485worker1 # [ 34.922175] nix-grpc-daemon[846]: ts=2026-09-20T13:26:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14486worker1 # [ 34.928416] nix-daemon[906]: reaped child process 975, status = succeeded4487worker2 # [ 34.940356] nix-grpc-daemon[786]: ts=2026-09-20T13:26:26Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04488worker2 # [ 34.942303] nix-grpc-daemon[786]: ts=2026-09-20T13:26:26Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14489worker1 # [ 34.975595] nix-grpc-daemon[846]: ts=2026-09-20T13:26:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244490worker1 # [ 34.984689] nix-grpc-daemon[846]: ts=2026-09-20T13:26:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244491worker1 # [ 34.994961] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.534Z level=INFO msg="All 2 paths already cached"4492client # copying 1 paths to 'lb:50051'...4493worker1 # [ 35.002865] nix-grpc-daemon[846]: ts=2026-09-20T13:26:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506244494client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4495worker1 # [ 35.013185] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4496worker1 # [ 35.193280] nix-daemon[906]: accepted connection from pid 999, user nix-grpc-daemon (trusted)4497worker1 # [ 35.208893] nix-daemon[906]: reaped child process 1002, status = succeeded4498worker1 # [ 35.357146] nix-daemon[906]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)4499worker1 # [ 35.371668] nix-daemon[906]: reaped child process 1007, status = succeeded4500lb # [ 35.375574] niks3-server[1013]: 2026/09/20 13:26:27 INFO Received uploads request method=POST path=/api/pending_closures4501worker1 # [ 35.393765] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.933Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4502worker1 # [ 35.395431] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.935Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4503lb # [ 35.394693] niks3-server[1013]: 2026/09/20 13:26:27 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4504lb # [ 35.399791] niks3-server[1013]: 2026/09/20 13:26:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4505lb # [ 35.402170] niks3-server[1013]: 2026/09/20 13:26:27 INFO Signed narinfos id=5 count=14506worker1 # [ 35.415117] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.955Z level=INFO msg="Uploading 1 narinfos"4507lb # [ 35.407929] niks3-server[1013]: 2026/09/20 13:26:27 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4508lb # [ 35.412947] niks3-server[1013]: 2026/09/20 13:26:27 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4509lb # [ 35.417537] niks3-server[1013]: 2026/09/20 13:26:27 INFO Completed upload id=54510worker1 # [ 35.431743] nix-grpc-daemon[916]: time=2026-09-20T13:26:26.971Z level=INFO msg="Upload complete. (404ms)"4511lb # [ 35.420265] niks3-server[1013]: 2026/09/20 13:26:27 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4512worker1 # [ 35.435091] nix-grpc-daemon[846]: ts=2026-09-20T13:26:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_in=7524513worker1 # [ 35.438532] nix-grpc-daemon[846]: ts=2026-09-20T13:26:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244514worker1 # [ 35.449145] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4515worker1 # [ 35.452446] nix-daemon[906]: reaped child process 997, status = succeeded4516client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4517client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4518worker1 # [ 35.459408] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4519client # farm-top> LOG-farm-top-t14520worker1 # [ 36.082595] nix-daemon[906]: reaped child process 1011, status = succeeded4521worker1 # [ 36.226587] nix-daemon[906]: accepted connection from pid 1026, user nix-grpc-daemon (trusted)4522worker1 # [ 36.246345] nix-daemon[906]: reaped child process 1029, status = succeeded4523lb # [ 36.242802] niks3-server[1013]: 2026/09/20 13:26:28 INFO Received uploads request method=POST path=/api/pending_closures4524worker1 # [ 36.275114] nix-grpc-daemon[916]: time=2026-09-20T13:26:27.809Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4525worker1 # [ 36.276932] nix-grpc-daemon[916]: time=2026-09-20T13:26:27.810Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4526lb # [ 36.280154] niks3-server[1013]: 2026/09/20 13:26:28 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4527lb # [ 36.286861] niks3-server[1013]: 2026/09/20 13:26:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4528lb # [ 36.288819] niks3-server[1013]: 2026/09/20 13:26:28 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4529lb # [ 36.293021] niks3-server[1013]: 2026/09/20 13:26:28 INFO Signed narinfos id=6 count=14530worker1 # [ 36.305813] nix-grpc-daemon[916]: time=2026-09-20T13:26:27.845Z level=INFO msg="Uploading 1 narinfos"4531lb # [ 36.298634] niks3-server[1013]: 2026/09/20 13:26:28 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4532lb # [ 36.302888] niks3-server[1013]: 2026/09/20 13:26:28 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4533lb # [ 36.307634] niks3-server[1013]: 2026/09/20 13:26:28 INFO Completed upload id=64534lb # [ 36.309739] niks3-server[1013]: 2026/09/20 13:26:28 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4535worker1 # [ 36.322616] nix-grpc-daemon[916]: time=2026-09-20T13:26:27.861Z level=INFO msg="Upload complete. (252ms)"4536worker1 # [ 36.326434] nix-grpc-daemon[846]: ts=2026-09-20T13:26:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14537worker1 # [ 36.339224] nix-daemon[906]: reaped child process 1009, status = succeeded4538client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.22 seconds)4539client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4540client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4541client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4542client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4543client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4544client # copying 3 paths...4545client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4546client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4547client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4548client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4549(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.78 seconds)4550subtest: repeat is answered Cached by the scheduler without building4551worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4552worker1 # finding garbage collector roots...4553worker1 # deleting specified paths...4554worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4555worker1 # deleting unused links...4556worker1 # note: hard linking is currently saving 0.0 KiB4557worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4558worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4559worker2 # finding garbage collector roots...4560worker2 # deleting specified paths...4561worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4562worker2 # deleting unused links...4563worker2 # note: hard linking is currently saving 0.0 KiB4564worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.31 seconds)4565client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24566worker1 # [ 37.697327] nix-grpc-daemon[846]: ts=2026-09-20T13:26:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506244567client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4568client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4569worker1 # [ 37.719013] nix-grpc-daemon[846]: ts=2026-09-20T13:26:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244570worker1 # [ 37.731688] nix-grpc-daemon[916]: time=2026-09-20T13:26:29.270Z level=INFO msg="All 2 paths already cached"4571worker2 # [ 37.757481] nix-grpc-daemon[786]: ts=2026-09-20T13:26:29Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14572client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.29 seconds)4573client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4574client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4575lb: must succeed: curl -sf localhost:9901/clusters4576lb # [ 37.986547] envoy[922]: conn peer=[2001:db8:1::1]:42502 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=22964577lb # [ 37.989747] envoy[922]: conn peer=[2001:db8:1::1]:42514 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=363 ms=664578lb # [ 37.992414] envoy[922]: conn peer=[2001:db8:1::1]:42520 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=124579lb # [ 37.996789] envoy[922]: conn peer=[2001:db8:1::1]:42530 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134580lb # [ 38.000432] envoy[922]: conn peer=[2001:db8:1::1]:42532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9494 tx=3831 ms=60224581lb # [ 38.004348] envoy[922]: conn peer=[2001:db8:1::1]:58966 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=634582lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4583lb: must succeed: curl -sf localhost:9901/clusters4584lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4585worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4586worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4587(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.26 seconds)4588subtest: inputs already in the cache are not sent to the scheduler4589lb: must succeed: curl -sf localhost:9901/clusters4590lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4591lb: must succeed: curl -sf localhost:9901/clusters4592lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4593worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4594worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4595client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24596worker1 # [ 38.487713] nix-grpc-daemon[846]: ts=2026-09-20T13:26:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506244597client # don't know how to build these paths:4598client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4599worker1 # [ 38.493532] nix-grpc-daemon[846]: ts=2026-09-20T13:26:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244600worker1 # [ 38.501666] nix-grpc-daemon[916]: time=2026-09-20T13:26:30.041Z level=INFO msg="All 2 paths already cached"4601worker2 # [ 38.522541] nix-grpc-daemon[786]: ts=2026-09-20T13:26:30Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04602worker2 # [ 38.524421] nix-grpc-daemon[786]: ts=2026-09-20T13:26:30Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14603worker1 # [ 38.556379] nix-grpc-daemon[846]: ts=2026-09-20T13:26:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244604worker1 # [ 38.567123] nix-grpc-daemon[846]: ts=2026-09-20T13:26:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244605worker1 # [ 38.577046] nix-grpc-daemon[916]: time=2026-09-20T13:26:30.116Z level=INFO msg="All 2 paths already cached"4606client # copying 1 paths to 'lb:50051'...4607worker1 # [ 38.583471] nix-grpc-daemon[846]: ts=2026-09-20T13:26:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506244608client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4609worker1 # [ 38.590346] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4610worker1 # [ 38.765646] nix-daemon[906]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4611worker1 # [ 38.780107] nix-daemon[906]: reaped child process 1050, status = succeeded4612worker1 # [ 38.948085] nix-daemon[906]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4613worker1 # [ 38.962202] nix-daemon[906]: reaped child process 1055, status = succeeded4614lb # [ 38.962792] niks3-server[1013]: 2026/09/20 13:26:31 INFO Received uploads request method=POST path=/api/pending_closures4615worker1 # [ 38.988958] nix-grpc-daemon[916]: time=2026-09-20T13:26:30.528Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4616worker1 # [ 38.990394] nix-grpc-daemon[916]: time=2026-09-20T13:26:30.528Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4617lb # [ 38.996319] niks3-server[1013]: 2026/09/20 13:26:31 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4618lb # [ 39.010294] niks3-server[1013]: 2026/09/20 13:26:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4619worker1 # [ 39.027723] nix-grpc-daemon[916]: time=2026-09-20T13:26:30.567Z level=INFO msg="Uploading 1 narinfos"4620lb # [ 39.016123] niks3-server[1013]: 2026/09/20 13:26:31 INFO Signed narinfos id=7 count=14621lb # [ 39.021168] niks3-server[1013]: 2026/09/20 13:26:31 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4622lb # [ 39.031274] niks3-server[1013]: 2026/09/20 13:26:31 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4623lb # [ 39.035871] niks3-server[1013]: 2026/09/20 13:26:31 INFO Completed upload id=74624worker1 # [ 39.048673] nix-grpc-daemon[916]: time=2026-09-20T13:26:30.588Z level=INFO msg="Upload complete. (451ms)"4625lb # [ 39.037662] niks3-server[1013]: 2026/09/20 13:26:31 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4626worker1 # [ 39.054337] nix-grpc-daemon[846]: ts=2026-09-20T13:26:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_in=7524627worker1 # [ 39.059616] nix-grpc-daemon[846]: ts=2026-09-20T13:26:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244628worker1 # [ 39.070057] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4629worker1 # [ 39.074357] nix-daemon[906]: reaped child process 1045, status = succeeded4630client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4631worker1 # [ 39.079423] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4632client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4633client # farm-top> LOG-farm-top-t1b4634worker1 # [ 40.292791] nix-daemon[906]: reaped child process 1059, status = succeeded4635worker1 # [ 40.449629] nix-daemon[906]: accepted connection from pid 1075, user nix-grpc-daemon (trusted)4636worker1 # [ 40.464584] nix-daemon[906]: reaped child process 1078, status = succeeded4637lb # [ 40.464406] niks3-server[1013]: 2026/09/20 13:26:32 INFO Received uploads request method=POST path=/api/pending_closures4638worker1 # [ 40.488893] nix-grpc-daemon[916]: time=2026-09-20T13:26:32.028Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4639worker1 # [ 40.491761] nix-grpc-daemon[916]: time=2026-09-20T13:26:32.030Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4640lb # [ 40.509870] niks3-server[1013]: 2026/09/20 13:26:32 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4641lb # [ 40.516543] niks3-server[1013]: 2026/09/20 13:26:32 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4642lb # [ 40.520058] niks3-server[1013]: 2026/09/20 13:26:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4643lb # [ 40.522887] niks3-server[1013]: 2026/09/20 13:26:32 INFO Signed narinfos id=8 count=14644worker1 # [ 40.537321] nix-grpc-daemon[916]: time=2026-09-20T13:26:32.076Z level=INFO msg="Uploading 1 narinfos"4645lb # [ 40.528082] niks3-server[1013]: 2026/09/20 13:26:32 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4646lb # [ 40.533263] niks3-server[1013]: 2026/09/20 13:26:32 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4647lb # [ 40.536814] niks3-server[1013]: 2026/09/20 13:26:32 INFO Completed upload id=84648lb # [ 40.538551] niks3-server[1013]: 2026/09/20 13:26:32 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4649worker1 # [ 40.551057] nix-grpc-daemon[916]: time=2026-09-20T13:26:32.090Z level=INFO msg="Upload complete. (263ms)"4650worker1 # [ 40.556432] nix-grpc-daemon[846]: ts=2026-09-20T13:26:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14651worker1 # [ 40.568307] nix-daemon[906]: reaped child process 1057, status = succeeded4652client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.27 seconds)4653client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4654client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds)4655lb: must succeed: curl -sf localhost:9901/clusters4656lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4657lb: must succeed: curl -sf localhost:9901/clusters4658lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4659worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4660worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4661(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.77 seconds)4662subtest: two clients wanting the same drv build it once4663client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4664client # [ 40.978316] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4665client # Running as unit: dedup1.service; invocation ID: 8efd75b801134a11babc4ed92b452ea34666client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.06 seconds)4667client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4668worker2 # [ 41.184290] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:432704669client # [ 41.163093] nix[888]: don't know how to build these paths:4670client # [ 41.163969] nix[888]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4671worker2 # [ 41.196313] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04672worker2 # [ 41.203732] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14673worker2 # [ 41.208987] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432704674worker2 # [ 41.226210] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432704675client # [ 41.210133] nix[888]: copying 1 paths to 'lb:50051'...4676client # [ 41.211149] nix[888]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4677worker2 # [ 41.238227] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432704678worker2 # [ 41.248876] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4679worker2 # [ 41.272947] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=info event=niks3_push_started pid=9174680worker2 # [ 41.527321] nix-grpc-daemon[917]: time=2026-09-20T13:26:33.420Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4681worker2 # [ 41.700199] nix-daemon[866]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4682worker2 # [ 41.718878] nix-daemon[866]: reaped child process 933, status = succeeded4683worker2 # [ 41.883734] nix-daemon[866]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4684worker2 # [ 41.899481] nix-daemon[866]: reaped child process 938, status = succeeded4685lb # [ 41.891695] niks3-server[1013]: 2026/09/20 13:26:33 INFO Received uploads request method=POST path=/api/pending_closures4686worker2 # [ 41.921936] nix-grpc-daemon[917]: time=2026-09-20T13:26:33.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4687worker2 # [ 41.924787] nix-grpc-daemon[917]: time=2026-09-20T13:26:33.821Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4688lb # [ 41.923168] niks3-server[1013]: 2026/09/20 13:26:33 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4689lb # [ 41.928840] niks3-server[1013]: 2026/09/20 13:26:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4690worker2 # [ 41.954911] nix-grpc-daemon[917]: time=2026-09-20T13:26:33.851Z level=INFO msg="Uploading 1 narinfos"4691lb # [ 41.931900] niks3-server[1013]: 2026/09/20 13:26:34 INFO Signed narinfos id=9 count=14692lb # [ 41.940203] niks3-server[1013]: 2026/09/20 13:26:34 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4693lb # [ 41.944135] niks3-server[1013]: 2026/09/20 13:26:34 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4694lb # [ 41.950517] niks3-server[1013]: 2026/09/20 13:26:34 INFO Completed upload id=94695worker2 # [ 41.973971] nix-grpc-daemon[917]: time=2026-09-20T13:26:33.871Z level=INFO msg="Upload complete. (434ms)"4696lb # [ 41.952294] niks3-server[1013]: 2026/09/20 13:26:34 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4697worker2 # [ 41.976665] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=0 paths=1 nar_bytes_in=4324698worker2 # [ 41.980863] nix-grpc-daemon[786]: ts=2026-09-20T13:26:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432704699worker2 # [ 41.988136] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4700worker2 # [ 41.990913] nix-daemon[866]: reaped child process 915, status = succeeded4701client # [ 41.969482] nix[888]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4702worker2 # [ 41.997340] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4703client # [ 42.035663] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4704client # Running as unit: dedup2.service; invocation ID: b4adec24bdc94ae687fb6507881e95954705client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.05 seconds)4706worker1 # [ 42.275339] nix-grpc-daemon[846]: ts=2026-09-20T13:26:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506244707client # [ 42.265532] nix[904]: don't know how to build these paths:4708client # [ 42.266915] nix[904]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4709worker2 # [ 42.300030] nix-grpc-daemon[786]: ts=2026-09-20T13:26:34Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04710worker2 # [ 42.305322] nix-grpc-daemon[786]: ts=2026-09-20T13:26:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432704711client # [ 42.281396] nix[904]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4712client: must succeed: sleep 24713client: (finished: must succeed: sleep 2, in 2.04 seconds)4714client: waiting for success: ! systemctl is-active dedup1 dedup24715worker2 # [ 45.619675] nix-daemon[866]: reaped child process 942, status = succeeded4716worker2 # [ 45.772205] nix-daemon[866]: accepted connection from pid 974, user nix-grpc-daemon (trusted)4717worker2 # [ 45.789993] nix-daemon[866]: reaped child process 977, status = succeeded4718lb # [ 45.776737] niks3-server[1013]: 2026/09/20 13:26:37 INFO Received uploads request method=POST path=/api/pending_closures4719worker2 # [ 45.808118] nix-grpc-daemon[917]: time=2026-09-20T13:26:37.704Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4720worker2 # [ 45.811674] nix-grpc-daemon[917]: time=2026-09-20T13:26:37.704Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4721lb # [ 45.809066] niks3-server[1013]: 2026/09/20 13:26:37 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4722lb # [ 45.816100] niks3-server[1013]: 2026/09/20 13:26:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4723lb # [ 45.819772] niks3-server[1013]: 2026/09/20 13:26:37 INFO Signed narinfos id=10 count=14724worker2 # [ 45.843116] nix-grpc-daemon[917]: time=2026-09-20T13:26:37.740Z level=INFO msg="Uploading 1 narinfos"4725lb # [ 45.820898] niks3-server[1013]: 2026/09/20 13:26:37 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4726lb # [ 45.826505] niks3-server[1013]: 2026/09/20 13:26:37 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4727lb # [ 45.832072] niks3-server[1013]: 2026/09/20 13:26:37 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4728lb # [ 45.837639] niks3-server[1013]: 2026/09/20 13:26:37 INFO Completed upload id=104729worker2 # [ 45.861114] nix-grpc-daemon[917]: time=2026-09-20T13:26:37.758Z level=INFO msg="Upload complete. (249ms)"4730lb # [ 45.839151] niks3-server[1013]: 2026/09/20 13:26:37 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4731worker2 # [ 45.865843] nix-grpc-daemon[786]: ts=2026-09-20T13:26:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14732worker2 # [ 45.870268] nix-grpc-daemon[786]: ts=2026-09-20T13:26:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14733worker2 # [ 45.877500] nix-daemon[866]: reaped child process 940, status = succeeded4734client # [ 45.867299] systemd[1]: dedup2.service: Deactivated successfully.4735client # [ 45.869324] systemd[1]: dedup2.service: Consumed 125ms CPU time over 3.831s wall clock time, 33.2M memory peak, 7.4K incoming IP traffic, 6.4K outgoing IP traffic.4736client # [ 45.911568] systemd[1]: dedup1.service: Deactivated successfully.4737client # [ 45.913476] systemd[1]: dedup1.service: Consumed 115ms CPU time over 4.935s wall clock time, 39.3M memory peak, 8.5K incoming IP traffic, 9.1K outgoing IP traffic.4738client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4739client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4740client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4741worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4742worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4743worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4744worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4745worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4746worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4747worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4748worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4749(finished: subtest: two clients wanting the same drv build it once, in 5.77 seconds)4750subtest: build hook: nix-daemon with builders = grpc://lb4751client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4752client # [ 46.904802] systemd[1]: Started Nix Daemon.4753client # [ 47.103739] nix-daemon[947]: accepted connection from pid 944, user root (trusted)4754worker1 # [ 47.301829] nix-grpc-daemon[846]: ts=2026-09-20T13:26:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:506244755worker2 # [ 47.319577] nix-grpc-daemon[786]: ts=2026-09-20T13:26:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432704756worker2 # [ 47.331567] nix-grpc-daemon[786]: ts=2026-09-20T13:26:39Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04757worker2 # [ 47.334135] nix-grpc-daemon[786]: ts=2026-09-20T13:26:39Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14758worker1 # [ 47.365713] nix-grpc-daemon[846]: ts=2026-09-20T13:26:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244759worker1 # [ 47.378675] nix-grpc-daemon[846]: ts=2026-09-20T13:26:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244760worker1 # [ 47.393430] nix-grpc-daemon[846]: ts=2026-09-20T13:26:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506244761worker1 # [ 47.405628] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4762worker1 # [ 47.582363] nix-daemon[906]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4763worker1 # [ 47.598358] nix-daemon[906]: reaped child process 1107, status = succeeded4764worker1 # [ 47.750578] nix-daemon[906]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4765worker1 # [ 47.761548] nix-daemon[906]: reaped child process 1112, status = succeeded4766lb # [ 47.765714] niks3-server[1013]: 2026/09/20 13:26:39 INFO Received uploads request method=POST path=/api/pending_closures4767worker1 # [ 47.786193] nix-grpc-daemon[916]: time=2026-09-20T13:26:39.325Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4768worker1 # [ 47.788236] nix-grpc-daemon[916]: time=2026-09-20T13:26:39.327Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4769lb # [ 47.797974] niks3-server[1013]: 2026/09/20 13:26:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4770worker1 # [ 47.818427] nix-grpc-daemon[916]: time=2026-09-20T13:26:39.358Z level=INFO msg="Uploading 1 narinfos"4771lb # [ 47.806203] niks3-server[1013]: 2026/09/20 13:26:39 INFO Signed narinfos id=11 count=14772lb # [ 47.810278] niks3-server[1013]: 2026/09/20 13:26:39 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4773lb # [ 47.818425] niks3-server[1013]: 2026/09/20 13:26:39 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4774lb # [ 47.829778] niks3-server[1013]: 2026/09/20 13:26:39 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4775worker1 # [ 47.849454] nix-grpc-daemon[916]: time=2026-09-20T13:26:39.389Z level=INFO msg="Upload complete. (427ms)"4776lb # [ 47.836883] niks3-server[1013]: 2026/09/20 13:26:39 INFO Completed upload id=114777worker1 # [ 47.852086] nix-grpc-daemon[846]: ts=2026-09-20T13:26:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_in=5124778lb # [ 47.840345] niks3-server[1013]: 2026/09/20 13:26:39 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4779worker1 # [ 47.858326] nix-grpc-daemon[846]: ts=2026-09-20T13:26:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244780worker1 # [ 47.860372] nix-daemon[906]: reaped child process 1102, status = succeeded4781worker1 # [ 47.863329] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4782worker1 # [ 47.869938] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4783lb # [ 47.986877] envoy[922]: conn peer=[2001:db8:1::1]:58970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1710 ms=20764784lb # [ 47.991278] envoy[922]: conn peer=[2001:db8:1::1]:58992 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=36054785lb # [ 47.995209] envoy[922]: conn peer=[2001:db8:1::1]:58980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=1349 ms=47384786worker1 # [ 48.684296] nix-daemon[906]: reaped child process 1116, status = succeeded4787worker1 # [ 48.822343] nix-daemon[906]: accepted connection from pid 1131, user nix-grpc-daemon (trusted)4788worker1 # [ 48.835159] nix-daemon[906]: reaped child process 1134, status = succeeded4789lb # [ 48.833879] niks3-server[1013]: 2026/09/20 13:26:40 INFO Received uploads request method=POST path=/api/pending_closures4790worker1 # [ 48.852904] nix-grpc-daemon[916]: time=2026-09-20T13:26:40.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4791worker1 # [ 48.856029] nix-grpc-daemon[916]: time=2026-09-20T13:26:40.394Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4792lb # [ 48.860883] niks3-server[1013]: 2026/09/20 13:26:40 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4793lb # [ 48.863867] niks3-server[1013]: 2026/09/20 13:26:40 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4794lb # [ 48.870088] niks3-server[1013]: 2026/09/20 13:26:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4795lb # [ 48.873378] niks3-server[1013]: 2026/09/20 13:26:40 INFO Signed narinfos id=12 count=14796worker1 # [ 48.886355] nix-grpc-daemon[916]: time=2026-09-20T13:26:40.426Z level=INFO msg="Uploading 1 narinfos"4797lb # [ 48.876119] niks3-server[1013]: 2026/09/20 13:26:40 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4798lb # [ 48.881703] niks3-server[1013]: 2026/09/20 13:26:40 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4799lb # [ 48.886282] niks3-server[1013]: 2026/09/20 13:26:40 INFO Completed upload id=124800worker1 # [ 48.898868] nix-grpc-daemon[916]: time=2026-09-20T13:26:40.438Z level=INFO msg="Upload complete. (222ms)"4801lb # [ 48.888061] niks3-server[1013]: 2026/09/20 13:26:40 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4802worker1 # [ 48.901450] nix-grpc-daemon[846]: ts=2026-09-20T13:26:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14803worker1 # [ 48.908330] nix-daemon[906]: reaped child process 1114, status = succeeded4804worker2 # [ 48.959413] nix-grpc-daemon[786]: ts=2026-09-20T13:26:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:432704805worker2 # [ 48.972982] nix-grpc-daemon[786]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4806worker1 # [ 48.993079] nix-grpc-daemon[846]: ts=2026-09-20T13:26:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:506244807worker1 # [ 48.999219] nix-grpc-daemon[846]: ts=2026-09-20T13:26:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_out=1284808worker2 # [ 49.200165] nix-grpc-daemon[786]: ts=2026-09-20T13:26:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:432704809worker1 # [ 49.194607] nix-grpc-daemon[846]: ts=2026-09-20T13:26:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244810worker1 # [ 49.199735] nix-grpc-daemon[916]: time=2026-09-20T13:26:40.739Z level=INFO msg="All 1 paths already cached"4811worker2 # [ 49.222435] nix-grpc-daemon[786]: ts=2026-09-20T13:26:41Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04812worker2 # [ 49.224939] nix-grpc-daemon[786]: ts=2026-09-20T13:26:41Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14813worker1 # [ 49.216096] nix-grpc-daemon[846]: ts=2026-09-20T13:26:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244814worker1 # [ 49.222494] nix-grpc-daemon[846]: ts=2026-09-20T13:26:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244815worker1 # [ 49.229476] nix-grpc-daemon[916]: time=2026-09-20T13:26:40.769Z level=INFO msg="All 1 paths already cached"4816worker1 # [ 49.232966] nix-grpc-daemon[846]: ts=2026-09-20T13:26:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506244817worker1 # [ 49.238091] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4818worker1 # [ 49.376261] nix-daemon[906]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)4819worker1 # [ 49.389528] nix-daemon[906]: reaped child process 1141, status = succeeded4820worker1 # [ 49.525643] nix-daemon[906]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4821worker1 # [ 49.538340] nix-daemon[906]: reaped child process 1146, status = succeeded4822lb # [ 49.537200] niks3-server[1013]: 2026/09/20 13:26:41 INFO Received uploads request method=POST path=/api/pending_closures4823worker1 # [ 49.555310] nix-grpc-daemon[916]: time=2026-09-20T13:26:41.094Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4824worker1 # [ 49.556754] nix-grpc-daemon[916]: time=2026-09-20T13:26:41.094Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4825lb # [ 49.553392] niks3-server[1013]: 2026/09/20 13:26:41 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4826lb # [ 49.559752] niks3-server[1013]: 2026/09/20 13:26:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4827lb # [ 49.563337] niks3-server[1013]: 2026/09/20 13:26:41 INFO Signed narinfos id=13 count=14828worker1 # [ 49.577960] nix-grpc-daemon[916]: time=2026-09-20T13:26:41.117Z level=INFO msg="Uploading 1 narinfos"4829lb # [ 49.566330] niks3-server[1013]: 2026/09/20 13:26:41 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4830lb # [ 49.575932] niks3-server[1013]: 2026/09/20 13:26:41 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4831lb # [ 49.579694] niks3-server[1013]: 2026/09/20 13:26:41 INFO Completed upload id=134832worker1 # [ 49.592445] nix-grpc-daemon[916]: time=2026-09-20T13:26:41.132Z level=INFO msg="Upload complete. (349ms)"4833lb # [ 49.582322] niks3-server[1013]: 2026/09/20 13:26:41 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4834worker1 # [ 49.594518] nix-grpc-daemon[846]: ts=2026-09-20T13:26:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_in=6244835worker1 # [ 49.598049] nix-grpc-daemon[846]: ts=2026-09-20T13:26:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244836worker1 # [ 49.605093] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4837worker1 # [ 49.607369] nix-daemon[906]: reaped child process 1136, status = succeeded4838worker1 # [ 49.613568] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4839worker1 # [ 50.827470] nix-daemon[906]: reaped child process 1150, status = succeeded4840worker1 # [ 50.946542] nix-daemon[906]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)4841worker1 # [ 50.957585] nix-daemon[906]: reaped child process 1168, status = succeeded4842lb # [ 50.956370] niks3-server[1013]: 2026/09/20 13:26:43 INFO Received uploads request method=POST path=/api/pending_closures4843worker1 # [ 50.974764] nix-grpc-daemon[916]: time=2026-09-20T13:26:42.514Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4844worker1 # [ 50.976202] nix-grpc-daemon[916]: time=2026-09-20T13:26:42.516Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4845lb # [ 50.978765] niks3-server[1013]: 2026/09/20 13:26:43 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4846lb # [ 50.987708] niks3-server[1013]: 2026/09/20 13:26:43 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4847lb # [ 50.991580] niks3-server[1013]: 2026/09/20 13:26:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4848lb # [ 50.993915] niks3-server[1013]: 2026/09/20 13:26:43 INFO Signed narinfos id=14 count=14849worker1 # [ 51.008436] nix-grpc-daemon[916]: time=2026-09-20T13:26:42.548Z level=INFO msg="Uploading 1 narinfos"4850lb # [ 51.000179] niks3-server[1013]: 2026/09/20 13:26:43 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4851lb # [ 51.005078] niks3-server[1013]: 2026/09/20 13:26:43 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4852lb # [ 51.010143] niks3-server[1013]: 2026/09/20 13:26:43 INFO Completed upload id=144853worker1 # [ 51.022872] nix-grpc-daemon[916]: time=2026-09-20T13:26:42.562Z level=INFO msg="Upload complete. (203ms)"4854worker1 # [ 51.026241] nix-grpc-daemon[846]: ts=2026-09-20T13:26:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14855lb # [ 51.014027] niks3-server[1013]: 2026/09/20 13:26:43 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4856worker1 # [ 51.032110] nix-daemon[906]: reaped child process 1148, status = succeeded4857worker2 # [ 51.042787] nix-grpc-daemon[786]: ts=2026-09-20T13:26:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:432704858worker2 # [ 51.053154] nix-grpc-daemon[786]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4859worker1 # [ 51.065792] nix-grpc-daemon[846]: ts=2026-09-20T13:26:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:506244860worker1 # [ 51.067513] nix-grpc-daemon[846]: ts=2026-09-20T13:26:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_out=1764861worker2 # [ 51.284502] nix-grpc-daemon[786]: ts=2026-09-20T13:26:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:432704862worker2 # [ 51.292628] nix-grpc-daemon[786]: ts=2026-09-20T13:26:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432704863worker2 # [ 51.301220] nix-grpc-daemon[917]: time=2026-09-20T13:26:43.197Z level=INFO msg="All 2 paths already cached"4864worker1 # [ 51.307432] nix-grpc-daemon[846]: ts=2026-09-20T13:26:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244865worker2 # [ 51.315993] nix-grpc-daemon[786]: ts=2026-09-20T13:26:43Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04866worker2 # [ 51.320262] nix-grpc-daemon[786]: ts=2026-09-20T13:26:43Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14867worker1 # [ 51.314468] nix-grpc-daemon[846]: ts=2026-09-20T13:26:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506244868worker1 # [ 51.320718] nix-grpc-daemon[916]: time=2026-09-20T13:26:42.860Z level=INFO msg="All 2 paths already cached"4869worker1 # [ 51.324575] nix-grpc-daemon[846]: ts=2026-09-20T13:26:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506244870worker1 # [ 51.329434] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4871worker1 # [ 51.468672] nix-daemon[906]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)4872worker1 # [ 51.480912] nix-daemon[906]: reaped child process 1175, status = succeeded4873worker1 # [ 51.629878] nix-daemon[906]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)4874worker1 # [ 51.642134] nix-daemon[906]: reaped child process 1180, status = succeeded4875lb # [ 51.645457] niks3-server[1013]: 2026/09/20 13:26:43 INFO Received uploads request method=POST path=/api/pending_closures4876worker1 # [ 51.663903] nix-grpc-daemon[916]: time=2026-09-20T13:26:43.203Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4877worker1 # [ 51.665292] nix-grpc-daemon[916]: time=2026-09-20T13:26:43.203Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4878lb # [ 51.662040] niks3-server[1013]: 2026/09/20 13:26:43 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4879lb # [ 51.667906] niks3-server[1013]: 2026/09/20 13:26:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4880lb # [ 51.671611] niks3-server[1013]: 2026/09/20 13:26:43 INFO Signed narinfos id=15 count=14881worker1 # [ 51.686216] nix-grpc-daemon[916]: time=2026-09-20T13:26:43.225Z level=INFO msg="Uploading 1 narinfos"4882lb # [ 51.673644] niks3-server[1013]: 2026/09/20 13:26:43 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4883lb # [ 51.681550] niks3-server[1013]: 2026/09/20 13:26:43 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4884worker1 # [ 51.698630] nix-grpc-daemon[916]: time=2026-09-20T13:26:43.238Z level=INFO msg="Upload complete. (364ms)"4885lb # [ 51.686070] niks3-server[1013]: 2026/09/20 13:26:43 INFO Completed upload id=154886worker1 # [ 51.700715] nix-grpc-daemon[846]: ts=2026-09-20T13:26:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_in=7524887lb # [ 51.688753] niks3-server[1013]: 2026/09/20 13:26:43 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4888worker1 # [ 51.704533] nix-grpc-daemon[846]: ts=2026-09-20T13:26:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506244889worker1 # [ 51.710870] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4890worker1 # [ 51.713129] nix-daemon[906]: reaped child process 1170, status = succeeded4891worker1 # [ 51.718286] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)4892worker1 # [ 53.133700] nix-daemon[906]: reaped child process 1184, status = succeeded4893worker1 # [ 53.261304] nix-daemon[906]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)4894worker1 # [ 53.273956] nix-daemon[906]: reaped child process 1202, status = succeeded4895lb # [ 53.272089] niks3-server[1013]: 2026/09/20 13:26:45 INFO Received uploads request method=POST path=/api/pending_closures4896worker1 # [ 53.338187] nix-grpc-daemon[916]: time=2026-09-20T13:26:44.877Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4897worker1 # [ 53.341065] nix-grpc-daemon[916]: time=2026-09-20T13:26:44.879Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4898lb # [ 53.346839] niks3-server[1013]: 2026/09/20 13:26:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4899lb # [ 53.348844] niks3-server[1013]: 2026/09/20 13:26:45 INFO Signed narinfos id=16 count=14900worker1 # [ 53.363920] nix-grpc-daemon[916]: time=2026-09-20T13:26:44.903Z level=INFO msg="Uploading 1 narinfos"4901lb # [ 53.358456] niks3-server[1013]: 2026/09/20 13:26:45 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4902lb # [ 53.406313] niks3-server[1013]: 2026/09/20 13:26:45 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4903lb # [ 53.443695] niks3-server[1013]: 2026/09/20 13:26:45 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4904lb # [ 53.464756] niks3-server[1013]: 2026/09/20 13:26:45 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4905lb # [ 53.473461] niks3-server[1013]: 2026/09/20 13:26:45 INFO Completed upload id=164906worker1 # [ 53.486615] nix-grpc-daemon[916]: time=2026-09-20T13:26:45.026Z level=INFO msg="Upload complete. (360ms)"4907lb # [ 53.481778] niks3-server[1013]: 2026/09/20 13:26:45 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4908worker1 # [ 53.492836] nix-grpc-daemon[846]: ts=2026-09-20T13:26:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14909worker1 # [ 53.502314] nix-grpc-daemon[846]: ts=2026-09-20T13:26:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:506244910worker1 # [ 53.512696] nix-daemon[906]: reaped child process 1182, status = succeeded4911worker2 # [ 53.527969] nix-grpc-daemon[786]: ts=2026-09-20T13:26:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:432704912worker2 # [ 53.535494] nix-grpc-daemon[786]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4913worker2 # [ 53.544774] nix-grpc-daemon[786]: ts=2026-09-20T13:26:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=0 paths=1 nar_bytes_out=2324914client # [ 53.610954] nix-daemon[947]: reaped child process 949, status = succeeded4915client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.93 seconds)4916client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4917client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4918client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4919client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)4920client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4921client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4922(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.01 seconds)4923subtest: an input only one worker has still reaches the builder4924client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4925client # this derivation will be built:4926client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4927client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4928client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.40 seconds)4929client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4930client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.16 seconds)4931worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4932worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.22 seconds)4933worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4934worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4935client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24936client # [ 54.660864] nix-daemon[947]: accepted connection from pid 1030, user root (trusted)4937client # this derivation will be built:4938client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4939worker1 # [ 54.838364] nix-grpc-daemon[846]: ts=2026-09-20T13:26:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:506244940client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...4941worker2 # [ 54.852823] nix-grpc-daemon[786]: ts=2026-09-20T13:26:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432704942client # copying 1 paths...4943client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4944worker2 # [ 54.865408] nix-grpc-daemon[786]: ts=2026-09-20T13:26:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432704945worker2 # [ 54.876330] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4946worker2 # [ 55.056685] nix-daemon[866]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4947worker2 # [ 55.069698] nix-daemon[866]: reaped child process 1005, status = succeeded4948lb # [ 55.058201] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received uploads request method=POST path=/api/pending_closures4949worker2 # [ 55.114479] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.011Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4950worker2 # [ 55.116387] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.013Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4951lb # [ 55.103225] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4952lb # [ 55.106320] niks3-server[1013]: 2026/09/20 13:26:47 INFO Signed narinfos id=17 count=14953worker2 # [ 55.129651] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.026Z level=INFO msg="Uploading 1 narinfos"4954lb # [ 55.108664] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4955lb # [ 55.111811] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4956lb # [ 55.115746] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4957lb # [ 55.119641] niks3-server[1013]: 2026/09/20 13:26:47 INFO Completed upload id=174958worker2 # [ 55.144252] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.041Z level=INFO msg="Upload complete. (257ms)"4959lb # [ 55.122485] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4960worker2 # [ 55.147855] nix-grpc-daemon[786]: ts=2026-09-20T13:26:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=0 paths=1 nar_bytes_in=1204961worker2 # [ 55.158191] nix-daemon[866]: reaped child process 1000, status = succeeded4962worker2 # [ 55.161425] nix-grpc-daemon[786]: ts=2026-09-20T13:26:47Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04963worker2 # [ 55.163486] nix-grpc-daemon[786]: ts=2026-09-20T13:26:47Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14964worker2 # [ 55.207358] nix-grpc-daemon[786]: ts=2026-09-20T13:26:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432704965worker2 # [ 55.215696] nix-grpc-daemon[786]: ts=2026-09-20T13:26:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432704966client # copying 2 paths to 'lb:50051'...4967client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4968worker2 # [ 55.226788] nix-grpc-daemon[786]: ts=2026-09-20T13:26:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432704969client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4970worker2 # [ 55.230929] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4971worker2 # [ 55.373061] nix-daemon[866]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4972worker2 # [ 55.386348] nix-daemon[866]: reaped child process 1014, status = succeeded4973worker2 # [ 55.509445] nix-daemon[866]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)4974worker2 # [ 55.521344] nix-daemon[866]: reaped child process 1019, status = succeeded4975lb # [ 55.508854] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received uploads request method=POST path=/api/pending_closures4976lb # [ 55.517362] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received uploads request method=POST path=/api/pending_closures4977worker2 # [ 55.547729] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.444Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4978worker2 # [ 55.549944] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.446Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4979worker2 # [ 55.551515] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.446Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4980lb # [ 55.539706] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4981lb # [ 55.542357] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst4982lb # [ 55.547839] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4983lb # [ 55.550783] niks3-server[1013]: 2026/09/20 13:26:47 INFO Signed narinfos id=18 count=14984lb # [ 55.552093] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls4985lb # [ 55.554350] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4986lb # [ 55.556489] niks3-server[1013]: 2026/09/20 13:26:47 INFO Signed narinfos id=19 count=24987worker2 # [ 55.580363] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.477Z level=INFO msg="Uploading 3 narinfos"4988lb # [ 55.558314] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls4989lb # [ 55.570716] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4990lb # [ 55.573072] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4991lb # [ 55.576823] niks3-server[1013]: 2026/09/20 13:26:47 INFO Completed upload id=184992lb # [ 55.578873] niks3-server[1013]: 2026/09/20 13:26:47 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4993lb # [ 55.580776] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4994lb # [ 55.582875] niks3-server[1013]: 2026/09/20 13:26:47 INFO Completed upload id=194995worker2 # [ 55.606211] nix-grpc-daemon[917]: time=2026-09-20T13:26:47.503Z level=INFO msg="Upload complete. (366ms)"4996lb # [ 55.584571] niks3-server[1013]: 2026/09/20 13:26:47 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo4997worker2 # [ 55.608339] nix-grpc-daemon[786]: ts=2026-09-20T13:26:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=0 paths=2 nar_bytes_in=9764998worker2 # [ 55.611589] nix-grpc-daemon[786]: ts=2026-09-20T13:26:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432704999worker2 # [ 55.617139] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5000worker2 # [ 55.620293] nix-daemon[866]: reaped child process 1009, status = succeeded5001client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5002worker2 # [ 55.625068] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5003client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5004worker2 # [ 56.839524] nix-daemon[866]: reaped child process 1023, status = succeeded5005worker2 # [ 56.962936] nix-daemon[866]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5006worker2 # [ 56.974958] nix-daemon[866]: reaped child process 1041, status = succeeded5007lb # [ 56.965187] niks3-server[1013]: 2026/09/20 13:26:49 INFO Received uploads request method=POST path=/api/pending_closures5008worker2 # [ 56.997255] nix-grpc-daemon[917]: time=2026-09-20T13:26:48.894Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5009lb # [ 56.997091] niks3-server[1013]: 2026/09/20 13:26:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5010lb # [ 56.999612] niks3-server[1013]: 2026/09/20 13:26:49 INFO Signed narinfos id=20 count=15011worker2 # [ 57.024236] nix-grpc-daemon[917]: time=2026-09-20T13:26:48.921Z level=INFO msg="Uploading 1 narinfos"5012lb # [ 57.006384] niks3-server[1013]: 2026/09/20 13:26:49 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5013lb # [ 57.017655] niks3-server[1013]: 2026/09/20 13:26:49 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5014lb # [ 57.031941] niks3-server[1013]: 2026/09/20 13:26:49 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5015lb # [ 57.041854] niks3-server[1013]: 2026/09/20 13:26:49 INFO Completed upload id=205016worker2 # [ 57.065255] nix-grpc-daemon[917]: time=2026-09-20T13:26:48.962Z level=INFO msg="Upload complete. (233ms)"5017worker2 # [ 57.068761] nix-grpc-daemon[786]: ts=2026-09-20T13:26:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15018lb # [ 57.051266] niks3-server[1013]: 2026/09/20 13:26:49 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5019worker2 # [ 57.076262] nix-daemon[866]: reaped child process 1021, status = succeeded5020client # copying 1 paths...5021worker1 # [ 57.102802] nix-grpc-daemon[846]: ts=2026-09-20T13:26:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:506245022worker1 # [ 57.113089] nix-grpc-daemon[846]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5023client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5024worker2 # [ 57.148422] nix-grpc-daemon[786]: ts=2026-09-20T13:26:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:432705025worker2 # [ 57.150772] nix-grpc-daemon[786]: ts=2026-09-20T13:26:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=0 paths=1 nar_bytes_out=1205026client # [ 57.213719] nix-daemon[947]: reaped child process 1033, status = succeeded5027client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.71 seconds)5028client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25029client # [ 57.352451] nix-daemon[947]: accepted connection from pid 1052, user root (trusted)5030client # this derivation will be built:5031client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5032worker2 # [ 57.526881] nix-grpc-daemon[786]: ts=2026-09-20T13:26:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:432705033client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5034worker2 # [ 57.531725] nix-grpc-daemon[786]: ts=2026-09-20T13:26:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432705035worker2 # [ 57.599954] nix-grpc-daemon[917]: time=2026-09-20T13:26:49.497Z level=INFO msg="All 1 paths already cached"5036worker2 # [ 57.614164] nix-grpc-daemon[786]: ts=2026-09-20T13:26:49Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05037worker2 # [ 57.618760] nix-grpc-daemon[786]: ts=2026-09-20T13:26:49Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15038worker1 # [ 57.615461] nix-grpc-daemon[846]: ts=2026-09-20T13:26:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506245039worker1 # [ 57.625471] nix-grpc-daemon[846]: ts=2026-09-20T13:26:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506245040client # copying 1 paths to 'lb:50051'...5041client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5042worker1 # [ 57.636929] nix-grpc-daemon[846]: ts=2026-09-20T13:26:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506245043worker1 # [ 57.648337] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)5044worker1 # [ 57.657146] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)5045worker1 # [ 57.667519] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)5046worker1 # [ 57.674970] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 3/5)5047worker1 # [ 57.690565] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 771 ms (attempt 4/5)5048lb # [ 57.986578] envoy[922]: conn peer=[2001:db8:1::1]:48852 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3384 tx=1727 ms=17065049lb # [ 57.988255] envoy[922]: conn peer=[2001:db8:1::1]:48862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=333 ms=155050lb # [ 57.989852] envoy[922]: conn peer=[2001:db8:1::1]:48866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3837 tx=1945 ms=18835051lb # [ 57.991286] envoy[922]: conn peer=[2001:db8:1::1]:48876 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=85052lb # [ 57.992758] envoy[922]: conn peer=[2001:db8:1::1]:48878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4328 tx=2080 ms=23095053lb # [ 57.994185] envoy[922]: conn peer=[2001:db8:1::1]:38652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=695054lb # [ 57.995553] envoy[922]: conn peer=[2001:db8:1::1]:38674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=505055lb # [ 57.996959] envoy[922]: conn peer=[2001:db8:1::1]:38668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4221 tx=1861 ms=23545056worker1 # [ 58.465773] nix-grpc-daemon[846]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5057worker1 # [ 58.471296] nix-grpc-daemon[846]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5058worker1 # [ 58.675370] nix-daemon[906]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)5059worker1 # [ 58.688782] nix-daemon[906]: reaped child process 1219, status = succeeded5060worker1 # [ 58.850660] nix-daemon[906]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5061worker1 # [ 58.863366] nix-daemon[906]: reaped child process 1224, status = succeeded5062lb # [ 58.863437] niks3-server[1013]: 2026/09/20 13:26:50 INFO Received uploads request method=POST path=/api/pending_closures5063worker1 # [ 59.161331] nix-grpc-daemon[916]: time=2026-09-20T13:26:50.699Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5064worker1 # [ 59.164795] nix-grpc-daemon[916]: time=2026-09-20T13:26:50.700Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5065lb # [ 59.168968] niks3-server[1013]: 2026/09/20 13:26:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5066worker1 # [ 59.188734] nix-grpc-daemon[916]: time=2026-09-20T13:26:50.728Z level=INFO msg="Uploading 1 narinfos"5067lb # [ 59.176156] niks3-server[1013]: 2026/09/20 13:26:51 INFO Signed narinfos id=21 count=15068lb # [ 59.182313] niks3-server[1013]: 2026/09/20 13:26:51 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5069lb # [ 59.193919] niks3-server[1013]: 2026/09/20 13:26:51 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5070lb # [ 59.201263] niks3-server[1013]: 2026/09/20 13:26:51 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5071lb # [ 59.207249] niks3-server[1013]: 2026/09/20 13:26:51 INFO Completed upload id=215072worker1 # [ 59.220284] nix-grpc-daemon[916]: time=2026-09-20T13:26:50.760Z level=INFO msg="Upload complete. (729ms)"5073worker1 # [ 59.225086] nix-grpc-daemon[846]: ts=2026-09-20T13:26:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=1 paths=1 nar_bytes_in=5605074lb # [ 59.215846] niks3-server[1013]: 2026/09/20 13:26:51 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5075worker1 # [ 59.231673] nix-grpc-daemon[846]: ts=2026-09-20T13:26:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506245076worker1 # [ 59.242156] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)5077worker1 # [ 59.258854] nix-daemon[906]: reaped child process 1210, status = succeeded5078client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5079worker1 # [ 59.265363] nix-daemon[906]: accepted connection from pid 846, user nix-grpc-daemon (trusted)5080client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5081worker1 # [ 60.493536] nix-daemon[906]: reaped child process 1228, status = succeeded5082worker1 # [ 60.635421] nix-daemon[906]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5083worker1 # [ 60.648151] nix-daemon[906]: reaped child process 1246, status = succeeded5084lb # [ 60.649369] niks3-server[1013]: 2026/09/20 13:26:52 INFO Received uploads request method=POST path=/api/pending_closures5085worker1 # [ 60.670440] nix-grpc-daemon[916]: time=2026-09-20T13:26:52.210Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5086lb # [ 60.680331] niks3-server[1013]: 2026/09/20 13:26:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5087lb # [ 60.685137] niks3-server[1013]: 2026/09/20 13:26:52 INFO Signed narinfos id=22 count=15088worker1 # [ 60.700963] nix-grpc-daemon[916]: time=2026-09-20T13:26:52.240Z level=INFO msg="Uploading 1 narinfos"5089lb # [ 60.694715] niks3-server[1013]: 2026/09/20 13:26:52 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5090lb # [ 60.700380] niks3-server[1013]: 2026/09/20 13:26:52 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5091lb # [ 60.712293] niks3-server[1013]: 2026/09/20 13:26:52 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5092lb # [ 60.719254] niks3-server[1013]: 2026/09/20 13:26:52 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5093worker1 # [ 60.735845] nix-grpc-daemon[916]: time=2026-09-20T13:26:52.275Z level=INFO msg="Upload complete. (256ms)"5094lb # [ 60.723227] niks3-server[1013]: 2026/09/20 13:26:52 INFO Completed upload id=225095worker1 # [ 60.738561] nix-grpc-daemon[846]: ts=2026-09-20T13:26:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15096client # copying 1 paths...5097worker1 # [ 60.744408] nix-grpc-daemon[846]: ts=2026-09-20T13:26:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:506245098worker1 # [ 60.747684] nix-daemon[906]: reaped child process 1226, status = succeeded5099client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5100worker1 # [ 60.763594] nix-grpc-daemon[846]: ts=2026-09-20T13:26:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:506245101worker1 # [ 60.765789] nix-grpc-daemon[846]: ts=2026-09-20T13:26:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50624 duration_s=0 paths=1 nar_bytes_out=1205102client # [ 60.837503] nix-daemon[947]: reaped child process 1055, status = succeeded5103client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.63 seconds)5104client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25105client # [ 61.006104] nix-daemon[947]: accepted connection from pid 1073, user root (trusted)5106client # this derivation will be built:5107client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5108worker2 # [ 61.197044] nix-grpc-daemon[786]: ts=2026-09-20T13:26:53Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:432705109worker1 # [ 61.189344] nix-grpc-daemon[846]: ts=2026-09-20T13:26:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506245110client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5111worker1 # [ 61.508083] nix-grpc-daemon[916]: time=2026-09-20T13:26:53.047Z level=INFO msg="All 1 paths already cached"5112worker2 # [ 61.537588] nix-grpc-daemon[786]: ts=2026-09-20T13:26:53Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05113worker2 # [ 61.544510] nix-grpc-daemon[786]: ts=2026-09-20T13:26:53Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15114worker2 # [ 61.549625] nix-grpc-daemon[786]: ts=2026-09-20T13:26:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432705115worker2 # [ 61.562174] nix-grpc-daemon[786]: ts=2026-09-20T13:26:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432705116worker2 # [ 61.578145] nix-grpc-daemon[917]: time=2026-09-20T13:26:53.474Z level=INFO msg="All 1 paths already cached"5117client # copying 1 paths to 'lb:50051'...5118client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5119worker2 # [ 61.587792] nix-grpc-daemon[786]: ts=2026-09-20T13:26:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432705120worker2 # [ 61.596894] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5121worker2 # [ 61.769744] nix-daemon[866]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5122worker2 # [ 61.785174] nix-daemon[866]: reaped child process 1052, status = succeeded5123worker2 # [ 61.915567] nix-daemon[866]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)5124worker2 # [ 61.929033] nix-daemon[866]: reaped child process 1057, status = succeeded5125lb # [ 61.919517] niks3-server[1013]: 2026/09/20 13:26:53 INFO Received uploads request method=POST path=/api/pending_closures5126worker2 # [ 61.947402] nix-grpc-daemon[917]: time=2026-09-20T13:26:53.844Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5127worker2 # [ 61.948971] nix-grpc-daemon[917]: time=2026-09-20T13:26:53.846Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5128lb # [ 61.937076] niks3-server[1013]: 2026/09/20 13:26:54 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5129lb # [ 61.945530] niks3-server[1013]: 2026/09/20 13:26:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5130lb # [ 61.951311] niks3-server[1013]: 2026/09/20 13:26:54 INFO Signed narinfos id=23 count=15131worker2 # [ 61.978454] nix-grpc-daemon[917]: time=2026-09-20T13:26:53.875Z level=INFO msg="Uploading 1 narinfos"5132lb # [ 61.956090] niks3-server[1013]: 2026/09/20 13:26:54 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5133lb # [ 61.966850] niks3-server[1013]: 2026/09/20 13:26:54 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5134lb # [ 61.974900] niks3-server[1013]: 2026/09/20 13:26:54 INFO Completed upload id=235135worker2 # [ 61.998274] nix-grpc-daemon[917]: time=2026-09-20T13:26:53.895Z level=INFO msg="Upload complete. (386ms)"5136lb # [ 61.976835] niks3-server[1013]: 2026/09/20 13:26:54 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5137worker2 # [ 62.000867] nix-grpc-daemon[786]: ts=2026-09-20T13:26:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=0 paths=1 nar_bytes_in=5605138worker2 # [ 62.004297] nix-grpc-daemon[786]: ts=2026-09-20T13:26:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432705139worker2 # [ 62.011158] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5140worker2 # [ 62.014695] nix-daemon[866]: reaped child process 1047, status = succeeded5141worker2 # [ 62.020273] nix-daemon[866]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5142client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5143client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5144worker2 # [ 62.636952] nix-daemon[866]: reaped child process 1061, status = succeeded5145worker2 # [ 62.751234] nix-daemon[866]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)5146worker2 # [ 62.764501] nix-daemon[866]: reaped child process 1079, status = succeeded5147lb # [ 62.753913] niks3-server[1013]: 2026/09/20 13:26:54 INFO Received uploads request method=POST path=/api/pending_closures5148worker2 # [ 62.786292] nix-grpc-daemon[917]: time=2026-09-20T13:26:54.682Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5149lb # [ 62.778057] niks3-server[1013]: 2026/09/20 13:26:54 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5150lb # [ 62.786305] niks3-server[1013]: 2026/09/20 13:26:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5151lb # [ 62.790353] niks3-server[1013]: 2026/09/20 13:26:54 INFO Signed narinfos id=24 count=15152worker2 # [ 62.816491] nix-grpc-daemon[917]: time=2026-09-20T13:26:54.713Z level=INFO msg="Uploading 1 narinfos"5153lb # [ 62.800872] niks3-server[1013]: 2026/09/20 13:26:54 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5154lb # [ 62.809714] niks3-server[1013]: 2026/09/20 13:26:54 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5155worker2 # [ 62.839494] nix-grpc-daemon[917]: time=2026-09-20T13:26:54.736Z level=INFO msg="Upload complete. (213ms)"5156lb # [ 62.816179] niks3-server[1013]: 2026/09/20 13:26:54 INFO Completed upload id=245157lb # [ 62.819220] niks3-server[1013]: 2026/09/20 13:26:54 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5158worker2 # [ 62.842917] nix-grpc-daemon[786]: ts=2026-09-20T13:26:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=0 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15159worker2 # [ 62.850150] nix-daemon[866]: reaped child process 1059, status = succeeded5160client # copying 1 paths...5161worker2 # [ 62.887767] nix-grpc-daemon[786]: ts=2026-09-20T13:26:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:432705162client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5163worker2 # [ 62.905648] nix-grpc-daemon[786]: ts=2026-09-20T13:26:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:432705164worker2 # [ 62.907249] nix-grpc-daemon[786]: ts=2026-09-20T13:26:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43270 duration_s=0 paths=1 nar_bytes_out=1205165client # [ 62.968113] nix-daemon[947]: reaped child process 1076, status = succeeded5166client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.13 seconds)5167(finished: subtest: an input only one worker has still reaches the builder, in 9.27 seconds)5168subtest: upload whose reference the worker lacks is completed from the cache5169client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5170client # this derivation will be built:5171client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5172client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5173client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.34 seconds)5174client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5175client # this derivation will be built:5176client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5177client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5178client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.34 seconds)5179client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5180client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.15 seconds)5181worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5182worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5183worker2 # [ 64.078779] systemd[1]: Stopping Nix worker-protocol over gRPC...5184worker2 # [ 64.368905] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=signal_exit5185worker2 # [ 64.623857] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=scheduler_yield5186worker2 # [ 64.628612] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=scheduler_restarting addr=lb:500515187lb # [ 64.603463] niks3-server[1013]: 2026/09/20 13:26:56 INFO lead: released remote=192.168.1.4:518925188worker1 # [ 64.625578] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_restarting addr=lb:500515189worker2 # [ 64.635730] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:432585190worker2 # [ 64.640753] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=worker_gone addr=192.168.1.4:500515191worker2 # [ 64.642282] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:432585192worker2 # [ 64.644042] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=worker_gone addr=192.168.1.3:500515193lb # [ 64.752029] niks3-server[1013]: 2026/09/20 13:26:56 INFO lead: acquired remote=192.168.1.3:597185194worker1 # [ 64.765128] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_take_over5195worker2 # [ 64.829214] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=scheduler_connected addr=lb:500515196worker2 # [ 64.830938] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5197worker1 # [ 64.825538] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_connected addr=lb:500515198worker1 # [ 64.827151] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5199worker2 # [ 65.031564] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=scheduler_connected addr=lb:500515200worker2 # [ 65.033373] nix-grpc-daemon[786]: ts=2026-09-20T13:26:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5201worker1 # [ 65.027607] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_connected addr=lb:500515202worker1 # [ 65.029450] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5203worker2 # [ 65.234188] nix-grpc-daemon[786]: ts=2026-09-20T13:26:57Z level=info event=scheduler_connected addr=lb:500515204worker2 # [ 65.236268] nix-grpc-daemon[786]: ts=2026-09-20T13:26:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5205worker1 # [ 65.229910] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_connected addr=lb:500515206worker1 # [ 65.231731] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5207worker2 # [ 65.437384] nix-grpc-daemon[786]: ts=2026-09-20T13:26:57Z level=info event=scheduler_connected addr=lb:500515208worker2 # [ 65.439797] nix-grpc-daemon[786]: ts=2026-09-20T13:26:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5209worker1 # [ 65.432146] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_connected addr=lb:500515210worker1 # [ 65.433818] nix-grpc-daemon[846]: ts=2026-09-20T13:26:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5211worker2 # [ 65.628703] nix-grpc-daemon[786]: ts=2026-09-20T13:26:57Z level=info event=scheduler_restarting5212worker1 # [ 65.636352] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_connected addr=lb:500515213worker1 # [ 65.639516] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5214worker2 # [ 65.653681] nix-grpc-daemon[786]: ts=2026-09-20T13:26:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5215worker2 # [ 65.666787] nix-daemon[866]: reaped child process 868, status = succeeded5216worker1 # [ 65.836510] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_connected addr=lb:500515217worker1 # [ 65.838217] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5218worker1 # [ 66.039121] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_connected addr=lb:500515219worker1 # [ 66.042962] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5220worker1 # [ 66.243167] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_connected addr=lb:500515221worker1 # [ 66.244861] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5222worker1 # [ 66.445291] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=scheduler_connected addr=lb:500515223worker1 # [ 66.455846] nix-grpc-daemon[846]: ts=2026-09-20T13:26:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:549725224worker1 # [ 66.460745] nix-grpc-daemon[846]: ts=2026-09-20T13:26:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05225worker2 # [ 66.479252] nix-grpc-daemon[786]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5226worker2 # [ 66.481311] nix-grpc-daemon[786]: E0000 00:00:1789910818.375197 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5227worker2 # [ 66.483603] nix-grpc-daemon[786]: E0000 00:00:1789910818.378391 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5228worker2 # [ 66.758489] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5229worker2 # [ 66.761431] systemd[1]: Stopped Nix worker-protocol over gRPC.5230worker2 # [ 66.762744] systemd[1]: nix-grpc-daemon.service: Consumed 1.462s CPU time over 48.647s wall clock time, 50.1M memory peak, 56.9K incoming IP traffic, 155.8K outgoing IP traffic.5231worker2 # [ 66.768461] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5232worker2 # [ 66.770652] systemd[1]: Closed Nix worker-protocol over gRPC.5233worker2 # [ 66.772536] systemd[1]: nix-grpc-daemon.socket: Consumed 310us CPU time over 56.227s wall clock time, 256K memory peak, 70.9K incoming IP traffic, 59.8K outgoing IP traffic.5234lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5235lb: must succeed: curl -sf localhost:9901/clusters5236lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5237lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5238client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25239worker1 # [ 66.978937] nix-grpc-daemon[846]: ts=2026-09-20T13:26:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506245240client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5241worker1 # [ 66.981341] nix-grpc-daemon[846]: ts=2026-09-20T13:26:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:506245242client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5243worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5244worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.01 seconds)5245worker2 # [ 67.057793] systemd[1]: Listening on Nix worker-protocol over gRPC.5246worker1 # [ 67.082410] systemd[1]: Stopping Nix worker-protocol over gRPC...5247worker1 # [ 67.290346] nix-grpc-daemon[846]: ts=2026-09-20T13:26:58Z level=info event=signal_exit5248worker1 # [ 67.561634] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_yield5249lb # [ 67.551497] niks3-server[1013]: 2026/09/20 13:26:59 INFO lead: released remote=192.168.1.3:597185250worker1 # [ 67.566977] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_restarting addr=lb:500515251worker1 # [ 67.571621] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:549725252worker1 # [ 67.575188] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=worker_gone addr=192.168.1.3:500515253worker2 # [ 67.756682] systemd[1]: Starting scheduler-token.service...5254worker1 # [ 67.768168] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_connected addr=lb:500515255worker1 # [ 67.771859] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5256worker2 # [ 67.831864] systemd[1]: scheduler-token.service: Deactivated successfully.5257worker2 # [ 67.833808] systemd[1]: Finished scheduler-token.service.5258worker2 # [ 67.835299] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 77ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5259worker2 # [ 67.839997] systemd[1]: Starting Nix worker-protocol over gRPC...5260worker1 # [ 67.970252] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_connected addr=lb:500515261worker2 # [ 67.985445] nix-grpc-daemon[1103]: ts=2026-09-20T13:26:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5262worker1 # [ 67.974352] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5263worker2 # [ 67.987502] nix-grpc-daemon[1103]: ts=2026-09-20T13:26:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5264worker2 # [ 67.991562] nix-grpc-daemon[1103]: ts=2026-09-20T13:26:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15265worker2 # [ 67.998444] nix-grpc-daemon[1103]: ts=2026-09-20T13:26:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5266worker2 # [ 68.003148] systemd[1]: Started Nix worker-protocol over gRPC.5267lb # [ 67.984547] niks3-server[1013]: 2026/09/20 13:27:00 INFO lead: acquired remote=192.168.1.4:363005268worker2 # [ 68.008216] nix-grpc-daemon[1103]: ts=2026-09-20T13:26:59Z level=info event=scheduler_take_over5269lb # [ 67.987652] envoy[922]: conn peer=[2001:db8:1::1]:38688 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3731 tx=1817 ms=32985270lb # [ 67.989226] envoy[922]: conn peer=[2001:db8:1::1]:38704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=535271lb # [ 67.992021] envoy[922]: conn peer=[2001:db8:1::1]:38710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3719 tx=1799 ms=17595272lb # [ 67.993541] envoy[922]: conn peer=[2001:db8:1::1]:36388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=495273lb # [ 67.994942] envoy[922]: conn peer=[2001:db8:1::4]:42892 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=33981 tx=1810 ms=472545274lb # [ 67.996470] envoy[922]: conn peer=[2001:db8:1::1]:36400 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=105275worker2 # [ 68.022642] nix-grpc-daemon[1103]: ts=2026-09-20T13:26:59Z level=info event=scheduler_connected addr=lb:500515276worker2 # [ 68.024308] nix-grpc-daemon[1103]: ts=2026-09-20T13:26:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5277worker1 # [ 68.174962] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_connected addr=lb:500515278worker1 # [ 68.176823] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5279worker2 # [ 68.225191] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_connected addr=lb:500515280worker2 # [ 68.226905] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5281worker1 # [ 68.377720] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_connected addr=lb:500515282worker1 # [ 68.381506] nix-grpc-daemon[846]: ts=2026-09-20T13:26:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker2 # [ 68.428380] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_connected addr=lb:500515284worker2 # [ 68.432249] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5285worker1 # [ 68.564588] nix-grpc-daemon[846]: ts=2026-09-20T13:27:00Z level=info event=scheduler_restarting5286worker1 # [ 68.595923] nix-grpc-daemon[846]: ts=2026-09-20T13:27:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5287worker1 # [ 68.606796] nix-daemon[906]: reaped child process 908, status = succeeded5288worker2 # [ 68.633233] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_connected addr=lb:500515289worker2 # [ 68.637207] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5290worker1 # [ 68.715738] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5291worker1 # [ 68.719641] systemd[1]: Stopped Nix worker-protocol over gRPC.5292worker1 # [ 68.722966] systemd[1]: nix-grpc-daemon.service: Consumed 2.506s CPU time over 50.735s wall clock time, 55.3M memory peak, 84.3K incoming IP traffic, 154.4K outgoing IP traffic.5293worker1 # [ 68.734578] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5294worker1 # [ 68.737865] systemd[1]: Closed Nix worker-protocol over gRPC.5295worker1 # [ 68.740502] systemd[1]: nix-grpc-daemon.socket: Consumed 352us CPU time over 58.843s wall clock time, 256K memory peak, 75.6K incoming IP traffic, 64.2K outgoing IP traffic.5296lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5297lb: must succeed: curl -sf localhost:9901/clusters5298lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5299lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5300client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25301worker2 # [ 68.838181] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_connected addr=lb:500515302worker2 # [ 68.842424] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5303client # lb:50051: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, retrying5304worker2 # [ 69.043554] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=scheduler_connected addr=lb:500515305worker2 # [ 69.056139] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=oidc_jwks provider=mock keys=15306worker2 # [ 69.059097] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:590945307worker2 # [ 69.063450] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05308worker2 # [ 69.448898] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591105309worker2 # [ 69.454646] nix-daemon[866]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)5310worker2 # [ 69.462852] nix-grpc-daemon[1103]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)5311worker2 # [ 69.514275] nix-grpc-daemon[1103]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 2/5)5312worker2 # [ 69.657860] nix-grpc-daemon[1103]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 3/5)5313worker2 # [ 69.825425] nix-grpc-daemon[1103]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 715 ms (attempt 4/5)5314worker2 # [ 70.541527] nix-grpc-daemon[1103]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5315client # copying 2 paths...5316worker2 # [ 70.556741] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591105317client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5318client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5319worker2 # [ 70.566678] nix-daemon[866]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)5320worker2 # [ 70.577975] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:02Z level=info event=niks3_push_started pid=11595321worker2 # [ 70.733804] nix-grpc-daemon[1159]: time=2026-09-20T13:27:02.630Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5322worker2 # [ 70.888153] nix-daemon[866]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)5323worker2 # [ 70.899919] nix-daemon[866]: reaped child process 1175, status = succeeded5324lb # [ 70.890597] niks3-server[1013]: 2026/09/20 13:27:02 INFO Received uploads request method=POST path=/api/pending_closures5325lb # [ 70.895758] niks3-server[1013]: 2026/09/20 13:27:02 INFO Received uploads request method=POST path=/api/pending_closures5326worker2 # [ 70.922669] nix-grpc-daemon[1159]: time=2026-09-20T13:27:02.819Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5327worker2 # [ 70.924290] nix-grpc-daemon[1159]: time=2026-09-20T13:27:02.821Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5328worker2 # [ 70.926201] nix-grpc-daemon[1159]: time=2026-09-20T13:27:02.823Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5329lb # [ 70.934523] niks3-server[1013]: 2026/09/20 13:27:03 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5330lb # [ 70.949873] niks3-server[1013]: 2026/09/20 13:27:03 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5331lb # [ 70.953779] niks3-server[1013]: 2026/09/20 13:27:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5332lb # [ 70.956699] niks3-server[1013]: 2026/09/20 13:27:03 INFO Signed narinfos id=26 count=25333lb # [ 70.959445] niks3-server[1013]: 2026/09/20 13:27:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5334lb # [ 70.966390] niks3-server[1013]: 2026/09/20 13:27:03 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5335lb # [ 70.974110] niks3-server[1013]: 2026/09/20 13:27:03 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5336lb # [ 70.978071] niks3-server[1013]: 2026/09/20 13:27:03 INFO Signed narinfos id=25 count=15337worker2 # [ 71.003281] nix-grpc-daemon[1159]: time=2026-09-20T13:27:02.900Z level=INFO msg="Uploading 3 narinfos"5338lb # [ 71.005109] niks3-server[1013]: 2026/09/20 13:27:03 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5339lb # [ 71.007153] niks3-server[1013]: 2026/09/20 13:27:03 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5340lb # [ 71.013119] niks3-server[1013]: 2026/09/20 13:27:03 INFO Completed upload id=255341lb # [ 71.015251] niks3-server[1013]: 2026/09/20 13:27:03 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5342lb # [ 71.017693] niks3-server[1013]: 2026/09/20 13:27:03 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5343lb # [ 71.020696] niks3-server[1013]: 2026/09/20 13:27:03 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5344lb # [ 71.023185] niks3-server[1013]: 2026/09/20 13:27:03 INFO Completed upload id=265345worker2 # [ 71.048500] nix-grpc-daemon[1159]: time=2026-09-20T13:27:02.945Z level=INFO msg="Upload complete. (310ms)"5346worker2 # [ 71.053842] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59110 duration_s=0 paths=2 nar_bytes_in=3125347worker2 # [ 71.066700] nix-daemon[866]: reaped child process 1157, status = succeeded5348client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 2.28 seconds)5349worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5350worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5351worker1 # [ 71.160804] systemd[1]: Listening on Nix worker-protocol over gRPC.5352worker1 # [ 71.168905] systemd[1]: Starting Nix worker-protocol over gRPC...5353worker1 # [ 71.337870] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:02Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5354worker1 # [ 71.339696] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5355worker1 # [ 71.343397] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15356worker1 # [ 71.350795] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:02Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5357worker1 # [ 71.352406] systemd[1]: Started Nix worker-protocol over gRPC.5358lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5359lb: must succeed: curl -sf localhost:9901/clusters5360worker1 # [ 71.371120] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:02Z level=info event=scheduler_connected addr=lb:500515361worker2 # [ 71.384819] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:590945362worker2 # [ 71.386562] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05363lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5364lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5365(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 8.45 seconds)5366subtest: scheduler down: the other node takes the lock and keeps it5367lb: must succeed: curl -sf localhost:9901/clusters5368lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5369lb: must succeed: curl -sf localhost:9901/clusters5370lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5371lb: must succeed: curl -sf localhost:9901/clusters5372lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5373lb: must succeed: curl -sf localhost:9901/clusters5374lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5375worker2 # [ 71.668947] systemd[1]: Stopping Nix worker-protocol over gRPC...5376worker2 # [ 72.003440] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=signal_exit5377worker2 # [ 72.013287] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=scheduler_yield5378lb # [ 71.992321] niks3-server[1013]: 2026/09/20 13:27:04 INFO lead: released remote=192.168.1.4:363005379worker2 # [ 72.017360] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590945380worker1 # [ 72.005065] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:03Z level=info event=scheduler_restarting addr=lb:500515381worker2 # [ 72.019358] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=worker_gone addr=192.168.1.3:500515382worker2 # [ 72.020992] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=scheduler_restarting addr=lb:500515383worker2 # [ 72.022295] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590945384worker2 # [ 72.023636] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:03Z level=info event=worker_gone addr=192.168.1.4:500515385worker1 # [ 72.205599] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:03Z level=info event=scheduler_connected addr=lb:500515386worker1 # [ 72.207492] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker2 # [ 72.220803] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515388worker2 # [ 72.224605] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5389worker1 # [ 72.408279] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:03Z level=info event=scheduler_connected addr=lb:500515390worker1 # [ 72.410046] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5391worker2 # [ 72.422411] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515392worker2 # [ 72.424052] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5393worker1 # [ 72.611092] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515394worker2 # [ 72.624726] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515395worker2 # [ 72.626406] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5396worker1 # [ 72.614950] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5397worker1 # [ 72.815586] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515398worker2 # [ 72.827712] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515399worker1 # [ 72.819636] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5400worker2 # [ 72.831767] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5401worker2 # [ 73.019690] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=scheduler_restarting5402worker1 # [ 73.020424] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515403worker1 # [ 73.024769] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5404worker2 # [ 73.042885] nix-grpc-daemon[1103]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5405worker2 # [ 73.046724] nix-grpc-daemon[1103]: E0000 00:00:1789910824.934519 1146 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5406worker2 # [ 73.051804] nix-grpc-daemon[1103]: E0000 00:00:1789910824.934644 1146 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5407worker2 # [ 73.056724] nix-grpc-daemon[1103]: ts=2026-09-20T13:27:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5408worker2 # [ 73.064605] nix-daemon[866]: reaped child process 1152, status = succeeded5409worker1 # [ 73.225709] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515410worker1 # [ 73.229622] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker1 # [ 73.430299] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_connected addr=lb:500515412worker1 # [ 73.432033] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5413worker1 # [ 73.632516] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_connected addr=lb:500515414worker1 # [ 73.634252] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5415worker1 # [ 73.835158] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_connected addr=lb:500515416worker1 # [ 73.838860] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5417worker2 # [ 74.024721] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5418worker2 # [ 74.030495] systemd[1]: Stopped Nix worker-protocol over gRPC.5419worker2 # [ 74.032556] systemd[1]: nix-grpc-daemon.service: Consumed 307ms CPU time over 6.183s wall clock time, 30.6M memory peak, 20K incoming IP traffic, 31.6K outgoing IP traffic.5420worker2 # [ 74.041392] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5421worker2 # [ 74.045374] systemd[1]: Closed Nix worker-protocol over gRPC.5422worker2 # [ 74.048383] systemd[1]: nix-grpc-daemon.socket: Consumed 320us CPU time over 6.990s wall clock time, 256K memory peak, 18.1K incoming IP traffic, 12.9K outgoing IP traffic.5423worker1 # [ 74.039718] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_connected addr=lb:500515424worker1 # [ 74.043961] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5426worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5427worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5428worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5429worker1 # [ 74.244603] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_connected addr=lb:500515430worker1 # [ 74.248375] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker1 # [ 74.450681] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_connected addr=lb:500515432worker1 # [ 74.454162] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker1 # [ 74.650972] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:06Z level=info event=scheduler_connected addr=lb:500515434worker1 # [ 74.652752] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker1 # [ 74.853710] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:06Z level=info event=scheduler_connected addr=lb:500515436worker1 # [ 74.857537] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker1 # [ 75.058439] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:06Z level=info event=scheduler_connected addr=lb:500515438worker1 # [ 75.062660] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5439worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5440worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5441worker1 # [ 75.263553] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:06Z level=info event=scheduler_connected addr=lb:500515442worker1 # [ 75.268127] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5443worker1 # [ 75.470444] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_connected addr=lb:500515444worker1 # [ 75.474372] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5445worker1 # [ 75.671520] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_connected addr=lb:500515446worker1 # [ 75.676185] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5447worker1 # [ 75.876963] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_connected addr=lb:500515448worker1 # [ 75.881357] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5449worker1 # [ 76.081689] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_connected addr=lb:500515450worker1 # [ 76.083596] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5451worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5452worker1 # [ 76.286078] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_connected addr=lb:500515453worker1 # [ 76.288604] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5454worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5455worker1 # [ 76.362715] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:07Z level=info event=scheduler_take_over5456lb # [ 76.349979] niks3-server[1013]: 2026/09/20 13:27:08 INFO lead: acquired remote=192.168.1.3:405125457worker1 # [ 76.487414] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:08Z level=info event=scheduler_connected addr=lb:500515458worker1 # [ 76.492114] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:375285459worker1 # [ 76.494692] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05460worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5461worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5462client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25463worker1 # [ 77.570405] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375445464worker1 # [ 77.583926] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5465client # don't know how to build these paths:5466client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5467worker1 # [ 77.600796] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375445468worker1 # [ 77.615122] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5)5469worker1 # [ 77.643994] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 148 ms (attempt 2/5)5470worker1 # [ 77.792973] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 246 ms (attempt 3/5)5471lb # [ 77.989076] envoy[922]: conn peer=[2001:db8:1::3]:37542 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10331 tx=2333 ms=492265472lb # [ 77.993602] envoy[922]: conn peer=[2001:db8:1::1]:36416 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1165 tx=403 ms=21205473lb # [ 77.999307] envoy[922]: conn peer=[2001:db8:1::4]:56216 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8603 tx=535 ms=50195474worker1 # [ 78.044839] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 666 ms (attempt 4/5)5475worker1 # [ 78.711957] nix-grpc-daemon[1270]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5476worker1 # [ 78.738919] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:10Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05477worker1 # [ 78.744555] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:10Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15478worker1 # [ 78.784188] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375445479worker1 # [ 78.795257] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375445480client # copying 1 paths to 'lb:50051'...5481worker1 # [ 78.805643] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375445482client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5483worker1 # [ 78.815902] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5484worker1 # [ 79.065155] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:10Z level=info event=niks3_push_started pid=13465485worker1 # [ 79.295510] nix-grpc-daemon[1346]: time=2026-09-20T13:27:10.834Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5486worker1 # [ 79.460207] nix-daemon[906]: accepted connection from pid 1359, user nix-grpc-daemon (trusted)5487worker1 # [ 79.473774] nix-daemon[906]: reaped child process 1362, status = succeeded5488worker1 # [ 79.647295] nix-daemon[906]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5489worker1 # [ 79.659853] nix-daemon[906]: reaped child process 1367, status = succeeded5490lb # [ 79.666576] niks3-server[1013]: 2026/09/20 13:27:11 INFO Received uploads request method=POST path=/api/pending_closures5491worker1 # [ 79.695076] nix-grpc-daemon[1346]: time=2026-09-20T13:27:11.234Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5492worker1 # [ 79.696567] nix-grpc-daemon[1346]: time=2026-09-20T13:27:11.234Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5493lb # [ 79.707549] niks3-server[1013]: 2026/09/20 13:27:11 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5494lb # [ 79.716398] niks3-server[1013]: 2026/09/20 13:27:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5495lb # [ 79.721504] niks3-server[1013]: 2026/09/20 13:27:11 INFO Signed narinfos id=27 count=15496worker1 # [ 79.740107] nix-grpc-daemon[1346]: time=2026-09-20T13:27:11.279Z level=INFO msg="Uploading 1 narinfos"5497lb # [ 79.731820] niks3-server[1013]: 2026/09/20 13:27:11 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5498lb # [ 79.737155] niks3-server[1013]: 2026/09/20 13:27:11 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5499lb # [ 79.743223] niks3-server[1013]: 2026/09/20 13:27:11 INFO Completed upload id=275500worker1 # [ 79.756102] nix-grpc-daemon[1346]: time=2026-09-20T13:27:11.296Z level=INFO msg="Upload complete. (456ms)"5501worker1 # [ 79.758649] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37544 duration_s=0 paths=1 nar_bytes_in=5205502lb # [ 79.746964] niks3-server[1013]: 2026/09/20 13:27:11 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5503worker1 # [ 79.762597] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375445504worker1 # [ 79.772053] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5505worker1 # [ 79.774336] nix-daemon[906]: reaped child process 1344, status = succeeded5506client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5507worker1 # [ 79.782585] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5508client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5509client # farm-a> LOG-farm-a-on-standby5510worker1 # [ 81.212905] nix-daemon[906]: reaped child process 1371, status = succeeded5511worker1 # [ 81.358235] nix-daemon[906]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)5512worker1 # [ 81.369743] nix-daemon[906]: reaped child process 1389, status = succeeded5513lb # [ 81.370145] niks3-server[1013]: 2026/09/20 13:27:13 INFO Received uploads request method=POST path=/api/pending_closures5514worker1 # [ 81.537661] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.075Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5515worker1 # [ 81.550243] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.076Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5516lb # [ 81.561542] niks3-server[1013]: 2026/09/20 13:27:13 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5517lb # [ 81.572431] niks3-server[1013]: 2026/09/20 13:27:13 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5518lb # [ 81.582853] niks3-server[1013]: 2026/09/20 13:27:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5519lb # [ 81.589827] niks3-server[1013]: 2026/09/20 13:27:13 INFO Signed narinfos id=28 count=15520worker1 # [ 81.603980] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.142Z level=INFO msg="Uploading 1 narinfos"5521lb # [ 81.593234] niks3-server[1013]: 2026/09/20 13:27:13 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5522lb # [ 81.602879] niks3-server[1013]: 2026/09/20 13:27:13 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5523lb # [ 81.611113] niks3-server[1013]: 2026/09/20 13:27:13 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5524lb # [ 81.614144] niks3-server[1013]: 2026/09/20 13:27:13 INFO Completed upload id=285525worker1 # [ 81.627133] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.166Z level=INFO msg="Upload complete. (435ms)"5526worker1 # [ 81.638508] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37544 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15527worker1 # [ 81.655872] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:13Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05528worker1 # [ 81.660561] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:13Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15529worker1 # [ 81.670363] nix-daemon[906]: reaped child process 1369, status = succeeded5530worker1 # [ 81.699702] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375445531worker1 # [ 81.706600] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375445532worker1 # [ 81.712992] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.252Z level=INFO msg="All 1 paths already cached"5533client # copying 1 paths to 'lb:50051'...5534worker1 # [ 81.718366] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375445535client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5536worker1 # [ 81.728210] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5537worker1 # [ 81.900366] nix-daemon[906]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5538worker1 # [ 81.915844] nix-daemon[906]: reaped child process 1398, status = succeeded5539worker1 # [ 82.066904] nix-daemon[906]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5540worker1 # [ 82.079292] nix-daemon[906]: reaped child process 1403, status = succeeded5541lb # [ 82.079305] niks3-server[1013]: 2026/09/20 13:27:14 INFO Received uploads request method=POST path=/api/pending_closures5542worker1 # [ 82.097739] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.637Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5543worker1 # [ 82.099500] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.637Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5544lb # [ 82.096102] niks3-server[1013]: 2026/09/20 13:27:14 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5545lb # [ 82.099691] niks3-server[1013]: 2026/09/20 13:27:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5546lb # [ 82.102666] niks3-server[1013]: 2026/09/20 13:27:14 INFO Signed narinfos id=29 count=15547worker1 # [ 82.115417] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.655Z level=INFO msg="Uploading 1 narinfos"5548lb # [ 82.106202] niks3-server[1013]: 2026/09/20 13:27:14 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5549lb # [ 82.112601] niks3-server[1013]: 2026/09/20 13:27:14 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5550lb # [ 82.116276] niks3-server[1013]: 2026/09/20 13:27:14 INFO Completed upload id=295551worker1 # [ 82.130594] nix-grpc-daemon[1346]: time=2026-09-20T13:27:13.670Z level=INFO msg="Upload complete. (396ms)"5552lb # [ 82.118409] niks3-server[1013]: 2026/09/20 13:27:14 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5553worker1 # [ 82.132960] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37544 duration_s=0 paths=1 nar_bytes_in=6405554worker1 # [ 82.136261] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375445555worker1 # [ 82.141293] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5556worker1 # [ 82.149144] nix-daemon[906]: reaped child process 1393, status = succeeded5557client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5558client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5559worker1 # [ 82.153599] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5560worker1 # [ 82.965479] nix-daemon[906]: reaped child process 1407, status = succeeded5561client # farm-b> LOG-farm-b-on-standby5562worker1 # [ 83.104684] nix-daemon[906]: accepted connection from pid 1422, user nix-grpc-daemon (trusted)5563worker1 # [ 83.116904] nix-daemon[906]: reaped child process 1425, status = succeeded5564lb # [ 83.118498] niks3-server[1013]: 2026/09/20 13:27:15 INFO Received uploads request method=POST path=/api/pending_closures5565worker1 # [ 83.140768] nix-grpc-daemon[1346]: time=2026-09-20T13:27:14.679Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5566worker1 # [ 83.142205] nix-grpc-daemon[1346]: time=2026-09-20T13:27:14.679Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5567lb # [ 83.153660] niks3-server[1013]: 2026/09/20 13:27:15 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5568lb # [ 83.162347] niks3-server[1013]: 2026/09/20 13:27:15 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5569lb # [ 83.173074] niks3-server[1013]: 2026/09/20 13:27:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5570lb # [ 83.178646] niks3-server[1013]: 2026/09/20 13:27:15 INFO Signed narinfos id=30 count=15571worker1 # [ 83.194526] nix-grpc-daemon[1346]: time=2026-09-20T13:27:14.734Z level=INFO msg="Uploading 1 narinfos"5572lb # [ 83.183740] niks3-server[1013]: 2026/09/20 13:27:15 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5573lb # [ 83.194766] niks3-server[1013]: 2026/09/20 13:27:15 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5574lb # [ 83.198342] niks3-server[1013]: 2026/09/20 13:27:15 INFO Completed upload id=305575worker1 # [ 83.212381] nix-grpc-daemon[1346]: time=2026-09-20T13:27:14.752Z level=INFO msg="Upload complete. (253ms)"5576lb # [ 83.199817] niks3-server[1013]: 2026/09/20 13:27:15 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5577worker1 # [ 83.215621] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37544 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15578worker1 # [ 83.221593] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:14Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05579worker1 # [ 83.223527] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:14Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15580worker1 # [ 83.228399] nix-daemon[906]: reaped child process 1405, status = succeeded5581worker1 # [ 83.264726] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375445582worker1 # [ 83.273443] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375445583worker1 # [ 83.288627] nix-grpc-daemon[1346]: time=2026-09-20T13:27:14.828Z level=INFO msg="All 2 paths already cached"5584client # copying 1 paths to 'lb:50051'...5585client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5586worker1 # [ 83.293481] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375445587worker1 # [ 83.298288] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5588worker1 # [ 83.423473] nix-daemon[906]: accepted connection from pid 1431, user nix-grpc-daemon (trusted)5589worker1 # [ 83.435928] nix-daemon[906]: reaped child process 1434, status = succeeded5590worker1 # [ 83.568527] nix-daemon[906]: accepted connection from pid 1436, user nix-grpc-daemon (trusted)5591worker1 # [ 83.580681] nix-daemon[906]: reaped child process 1439, status = succeeded5592lb # [ 83.576317] niks3-server[1013]: 2026/09/20 13:27:15 INFO Received uploads request method=POST path=/api/pending_closures5593worker1 # [ 83.807551] nix-grpc-daemon[1346]: time=2026-09-20T13:27:15.346Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5594worker1 # [ 83.809300] nix-grpc-daemon[1346]: time=2026-09-20T13:27:15.347Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5595lb # [ 83.815300] niks3-server[1013]: 2026/09/20 13:27:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5596lb # [ 83.820324] niks3-server[1013]: 2026/09/20 13:27:15 INFO Signed narinfos id=31 count=15597worker1 # [ 83.836592] nix-grpc-daemon[1346]: time=2026-09-20T13:27:15.375Z level=INFO msg="Uploading 1 narinfos"5598lb # [ 83.824533] niks3-server[1013]: 2026/09/20 13:27:15 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5599lb # [ 83.837331] niks3-server[1013]: 2026/09/20 13:27:15 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5600lb # [ 83.845362] niks3-server[1013]: 2026/09/20 13:27:15 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5601lb # [ 83.850785] niks3-server[1013]: 2026/09/20 13:27:15 INFO Completed upload id=315602worker1 # [ 83.866770] nix-grpc-daemon[1346]: time=2026-09-20T13:27:15.406Z level=INFO msg="Upload complete. (562ms)"5603lb # [ 83.856094] niks3-server[1013]: 2026/09/20 13:27:15 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5604worker1 # [ 83.872156] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37544 duration_s=0 paths=1 nar_bytes_in=7685605worker1 # [ 83.877888] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375445606worker1 # [ 83.891133] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5607worker1 # [ 83.900772] nix-daemon[906]: reaped child process 1429, status = succeeded5608client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5609client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5610worker1 # [ 83.911941] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5611client # farm-top> LOG-farm-top-on-standby5612worker1 # [ 84.930203] nix-daemon[906]: reaped child process 1443, status = succeeded5613worker1 # [ 85.065832] nix-daemon[906]: accepted connection from pid 1458, user nix-grpc-daemon (trusted)5614worker1 # [ 85.077696] nix-daemon[906]: reaped child process 1461, status = succeeded5615lb # [ 85.075222] niks3-server[1013]: 2026/09/20 13:27:17 INFO Received uploads request method=POST path=/api/pending_closures5616worker1 # [ 85.093353] nix-grpc-daemon[1346]: time=2026-09-20T13:27:16.633Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5617worker1 # [ 85.094844] nix-grpc-daemon[1346]: time=2026-09-20T13:27:16.634Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5618lb # [ 85.095903] niks3-server[1013]: 2026/09/20 13:27:17 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5619lb # [ 85.098524] niks3-server[1013]: 2026/09/20 13:27:17 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5620lb # [ 85.102642] niks3-server[1013]: 2026/09/20 13:27:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5621lb # [ 85.104506] niks3-server[1013]: 2026/09/20 13:27:17 INFO Signed narinfos id=32 count=15622worker1 # [ 85.117209] nix-grpc-daemon[1346]: time=2026-09-20T13:27:16.656Z level=INFO msg="Uploading 1 narinfos"5623lb # [ 85.108233] niks3-server[1013]: 2026/09/20 13:27:17 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5624lb # [ 85.112061] niks3-server[1013]: 2026/09/20 13:27:17 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5625lb # [ 85.115561] niks3-server[1013]: 2026/09/20 13:27:17 INFO Completed upload id=325626worker1 # [ 85.128188] nix-grpc-daemon[1346]: time=2026-09-20T13:27:16.668Z level=INFO msg="Upload complete. (203ms)"5627lb # [ 85.117054] niks3-server[1013]: 2026/09/20 13:27:17 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5628worker1 # [ 85.131329] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37544 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15629worker1 # [ 85.137861] nix-daemon[906]: reaped child process 1441, status = succeeded5630client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.80 seconds)5631client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5632client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5633worker2 # [ 85.445226] systemd[1]: Listening on Nix worker-protocol over gRPC.5634worker2 # [ 85.449875] systemd[1]: Starting scheduler-token.service...5635worker2 # [ 85.515537] systemd[1]: scheduler-token.service: Deactivated successfully.5636worker2 # [ 85.517405] systemd[1]: Finished scheduler-token.service.5637worker2 # [ 85.520878] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 67ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5638worker2 # [ 85.524130] systemd[1]: Starting Nix worker-protocol over gRPC...5639worker2 # [ 85.689482] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5640worker2 # [ 85.692270] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5641worker2 # [ 85.696133] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15642worker2 # [ 85.703685] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5643worker2 # [ 85.705298] systemd[1]: Started Nix worker-protocol over gRPC.5644lb: must succeed: curl -sf localhost:9901/clusters5645worker2 # [ 85.724225] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:17Z level=info event=scheduler_connected addr=lb:500515646worker1 # [ 85.721841] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:17Z level=info event=oidc_jwks provider=mock keys=15647worker1 # [ 85.723252] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:375285648worker1 # [ 85.724918] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05649lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5650worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5651worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5652lb: must succeed: curl -sf localhost:9901/clusters5653lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5654lb: must succeed: curl -sf localhost:9901/clusters5655lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5656worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5657worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5658(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.52 seconds)5659subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5660client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5661client # [ 86.003685] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5662client # Running as unit: rs.service; invocation ID: 8f929c86a0b2495bb87665505e6652e85663client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5664worker2 # [ 86.237170] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:547625665worker2 # [ 86.250270] nix-daemon[866]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)5666client # [ 86.238890] nix[1176]: don't know how to build these paths:5667client # [ 86.239966] nix[1176]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5668worker1 # [ 86.263731] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:17Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05669worker1 # [ 86.265554] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:17Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15670worker1 # [ 86.307133] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375445671worker1 # [ 86.318506] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375445672client # [ 86.311440] nix[1176]: copying 1 paths to 'lb:50051'...5673worker1 # [ 86.327648] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375445674client # [ 86.312454] nix[1176]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5675worker1 # [ 86.334135] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5676worker1 # [ 86.464790] nix-daemon[906]: accepted connection from pid 1474, user nix-grpc-daemon (trusted)5677worker1 # [ 86.476820] nix-daemon[906]: reaped child process 1477, status = succeeded5678worker1 # [ 86.642795] nix-daemon[906]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)5679worker1 # [ 86.654553] nix-daemon[906]: reaped child process 1482, status = succeeded5680lb # [ 86.659679] niks3-server[1013]: 2026/09/20 13:27:18 INFO Received uploads request method=POST path=/api/pending_closures5681worker1 # [ 86.678846] nix-grpc-daemon[1346]: time=2026-09-20T13:27:18.218Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5682worker1 # [ 86.680687] nix-grpc-daemon[1346]: time=2026-09-20T13:27:18.220Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5683lb # [ 86.676787] niks3-server[1013]: 2026/09/20 13:27:18 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5684lb # [ 86.679890] niks3-server[1013]: 2026/09/20 13:27:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5685lb # [ 86.681570] niks3-server[1013]: 2026/09/20 13:27:18 INFO Signed narinfos id=33 count=15686worker1 # [ 86.694642] nix-grpc-daemon[1346]: time=2026-09-20T13:27:18.234Z level=INFO msg="Uploading 1 narinfos"5687lb # [ 86.685937] niks3-server[1013]: 2026/09/20 13:27:18 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5688lb # [ 86.690307] niks3-server[1013]: 2026/09/20 13:27:18 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5689lb # [ 86.693541] niks3-server[1013]: 2026/09/20 13:27:18 INFO Completed upload id=335690worker1 # [ 86.707502] nix-grpc-daemon[1346]: time=2026-09-20T13:27:18.247Z level=INFO msg="Upload complete. (368ms)"5691lb # [ 86.695491] niks3-server[1013]: 2026/09/20 13:27:18 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5692worker1 # [ 86.709716] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37544 duration_s=0 paths=1 nar_bytes_in=4165693worker1 # [ 86.712675] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375445694worker1 # [ 86.719413] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5695worker1 # [ 86.721432] nix-daemon[906]: reaped child process 1472, status = succeeded5696worker1 # [ 86.726106] nix-daemon[906]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5697client # [ 86.709306] nix[1176]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5698lb # [ 87.988217] envoy[922]: conn peer=[2001:db8:1::1]:34958 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9564 tx=3867 ms=75865699lb: must succeed: curl -sf localhost:9901/clusters5700lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5701lb: must succeed: curl -sf localhost:9901/clusters5702lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5703lb: must succeed: curl -sf localhost:9901/clusters5704lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5705lb: must succeed: curl -sf localhost:9901/clusters5706lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5707worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5708worker2 # [ 88.814031] systemd-journald[285]: Received client request to rotate journal, rotating.5709worker2 # [ 88.832563] systemd-journald[285]: Vacuuming done, freed 0B of archived journals from /var/log/journal/6bccd763220b4c6980ad21df1ed0c48f.5710worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.05 seconds)5711worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5712worker1 # [ 88.725623] systemd[1]: Stopping Nix worker-protocol over gRPC...5713worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5714client: must succeed: sleep 5.05715worker1 # [ 89.360740] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:20Z level=info event=draining5716client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5717worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5718worker1 # [ 94.164788] nix-daemon[906]: reaped child process 1486, status = succeeded5719worker1 # [ 94.343171] nix-daemon[906]: accepted connection from pid 1526, user nix-grpc-daemon (trusted)5720worker1 # [ 94.356978] nix-daemon[906]: reaped child process 1529, status = succeeded5721lb # [ 94.363515] niks3-server[1013]: 2026/09/20 13:27:26 INFO Received uploads request method=POST path=/api/pending_closures5722worker1 # [ 94.385744] nix-grpc-daemon[1346]: time=2026-09-20T13:27:25.925Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5723worker1 # [ 94.387474] nix-grpc-daemon[1346]: time=2026-09-20T13:27:25.927Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5724lb # [ 94.405212] niks3-server[1013]: 2026/09/20 13:27:26 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5725lb # [ 94.416440] niks3-server[1013]: 2026/09/20 13:27:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5726lb # [ 94.425876] niks3-server[1013]: 2026/09/20 13:27:26 INFO Signed narinfos id=34 count=15727worker1 # [ 94.441663] nix-grpc-daemon[1346]: time=2026-09-20T13:27:25.981Z level=INFO msg="Uploading 1 narinfos"5728lb # [ 94.438974] niks3-server[1013]: 2026/09/20 13:27:26 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5729lb # [ 94.448972] niks3-server[1013]: 2026/09/20 13:27:26 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5730lb # [ 94.456781] niks3-server[1013]: 2026/09/20 13:27:26 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5731lb # [ 94.459453] niks3-server[1013]: 2026/09/20 13:27:26 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5732lb # [ 94.462037] niks3-server[1013]: 2026/09/20 13:27:26 INFO Completed upload id=345733worker1 # [ 94.474841] nix-grpc-daemon[1346]: time=2026-09-20T13:27:26.014Z level=INFO msg="Upload complete. (334ms)"5734worker1 # [ 94.480055] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37544 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15735worker1 # [ 94.485960] nix-daemon[906]: reaped child process 1484, status = succeeded5736client # [ 94.486340] systemd[1]: rs.service: Deactivated successfully.5737client # [ 94.488084] systemd[1]: rs.service: Consumed 110ms CPU time over 8.484s wall clock time, 39.1M memory peak, 8.5K incoming IP traffic, 9.1K outgoing IP traffic.5738worker1 # [ 95.364291] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:26Z level=info event=signal_exit5739lb # [ 95.572477] niks3-server[1013]: 2026/09/20 13:27:27 INFO lead: released remote=192.168.1.3:405125740worker1 # [ 95.585289] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_yield5741worker2 # [ 95.600138] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:27Z level=info event=scheduler_restarting addr=lb:500515742worker1 # [ 95.592183] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_restarting addr=lb:500515743worker1 # [ 95.595344] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:375285744worker1 # [ 95.598626] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=worker_gone addr=192.168.1.4:500515745worker1 # [ 95.601555] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:375285746worker1 # [ 95.604835] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=worker_gone addr=192.168.1.3:500515747lb # [ 95.690649] niks3-server[1013]: 2026/09/20 13:27:27 INFO lead: acquired remote=192.168.1.4:540585748worker2 # [ 95.714268] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:27Z level=info event=scheduler_take_over5749worker2 # [ 95.800223] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:27Z level=info event=scheduler_connected addr=lb:500515750worker1 # [ 95.791763] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_connected addr=lb:500515751worker2 # [ 95.804117] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5752worker1 # [ 95.795756] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5753worker2 # [ 96.005319] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:27Z level=info event=scheduler_connected addr=lb:500515754worker1 # [ 95.996665] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_connected addr=lb:500515755worker2 # [ 96.009295] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5756worker1 # [ 96.000513] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5757worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.12 seconds)5758client: waiting for success: ! systemctl is-active rs5759client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)5760client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5761worker2 # [ 96.209947] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_connected addr=lb:500515762worker2 # [ 96.212261] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5763client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5764client: must fail: journalctl -u rs | grep -q 'reconnecting'5765worker1 # [ 96.202322] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_connected addr=lb:500515766worker1 # [ 96.205688] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5767client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5768lb: must succeed: curl -sf localhost:9901/clusters5769lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5770worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5771worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5772worker2 # [ 96.413600] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_connected addr=lb:500515773worker1 # [ 96.402959] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_connected addr=lb:500515774worker2 # [ 96.417265] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5775worker1 # [ 96.407094] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5776worker1 # [ 96.590114] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:28Z level=info event=scheduler_restarting5777worker2 # [ 96.619562] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_connected addr=lb:500515778worker2 # [ 96.622840] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5779worker1 # [ 96.620798] nix-daemon[906]: reaped child process 1337, status = succeeded5780worker1 # [ 96.623290] nix-grpc-daemon[1270]: ts=2026-09-20T13:27:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5781worker2 # [ 96.819744] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_connected addr=lb:500515782worker2 # [ 96.821555] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5783worker2 # [ 97.022611] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_connected addr=lb:500515784worker2 # [ 97.026589] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5785worker2 # [ 97.227651] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:29Z level=info event=scheduler_connected addr=lb:500515786worker2 # [ 97.234919] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:29Z level=info event=oidc_jwks provider=mock keys=15787worker2 # [ 97.236475] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:422625788worker2 # [ 97.238105] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05789lb: must succeed: curl -sf localhost:9901/clusters5790lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5791worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5792worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5793worker1 # [ 97.471160] nix-grpc-daemon[1270]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5794worker1 # [ 97.473130] nix-grpc-daemon[1270]: E0000 00:00:1789910849.010635 1313 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5795worker1 # [ 97.476055] nix-grpc-daemon[1270]: E0000 00:00:1789910849.015966 1313 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5796worker1 # [ 97.862571] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5797worker1 # [ 97.866154] systemd[1]: Stopped Nix worker-protocol over gRPC.5798worker1 # [ 97.868314] systemd[1]: nix-grpc-daemon.service: Consumed 1.338s CPU time over 26.691s wall clock time, 33.5M memory peak, 51.4K incoming IP traffic, 101.9K outgoing IP traffic.5799worker1 # [ 97.884626] systemd[1]: Starting Nix worker-protocol over gRPC...5800lb # [ 97.987970] envoy[922]: conn peer=[2001:db8:1::1]:52996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=1325 ms=82685801lb # [ 97.989715] envoy[922]: conn peer=[2001:db8:1::3]:38986 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=6606 tx=1560 ms=252455802worker1 # [ 98.038499] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5803worker1 # [ 98.040310] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5804worker1 # [ 98.044058] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15805worker1 # [ 98.051639] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5806worker1 # [ 98.054888] systemd[1]: Started Nix worker-protocol over gRPC.5807worker1 # [ 98.067231] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:29Z level=info event=scheduler_connected addr=lb:500515808worker2 # [ 98.079975] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:422625809worker2 # [ 98.084888] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05810lb: must succeed: curl -sf localhost:9901/clusters5811lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5812worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5813worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5814(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.59 seconds)5815subtest: scheduler restart mid-build: build finishes, no double build5816client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5817client # [ 98.572283] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5818client # Running as unit: mid.service; invocation ID: 11430a5400824fe2a0f6a8df5e2cb2be5819client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.04 seconds)5820worker2 # [ 98.784258] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:547625821client # [ 98.763258] nix[1212]: don't know how to build these paths:5822client # [ 98.764409] nix[1212]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5823worker2 # [ 98.798661] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:30Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05824worker2 # [ 98.804000] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:30Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15825worker2 # [ 98.842833] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547625826worker2 # [ 98.858303] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)5827worker2 # [ 98.885723] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 2/5)5828worker2 # [ 98.963518] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 3/5)5829worker2 # [ 99.108361] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 426 ms (attempt 4/5)5830worker2 # [ 99.535128] nix-grpc-daemon[1199]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5831worker2 # [ 99.550465] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:547625832client # [ 99.533987] nix[1212]: copying 1 paths to 'lb:50051'...5833client # [ 99.538231] nix[1212]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5834worker2 # [ 99.564616] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:547625835worker2 # [ 99.575821] nix-daemon[866]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)5836worker2 # [ 99.594272] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:31Z level=info event=niks3_push_started pid=13205837worker2 # [ 99.776252] nix-grpc-daemon[1320]: time=2026-09-20T13:27:31.671Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5838worker2 # [ 99.929104] nix-daemon[866]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5839worker2 # [ 99.941839] nix-daemon[866]: reaped child process 1340, status = succeeded5840worker2 # [ 100.091468] nix-daemon[866]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5841worker2 # [ 100.103759] nix-daemon[866]: reaped child process 1345, status = succeeded5842lb # [ 100.093275] niks3-server[1013]: 2026/09/20 13:27:32 INFO Received uploads request method=POST path=/api/pending_closures5843worker2 # [ 100.122162] nix-grpc-daemon[1320]: time=2026-09-20T13:27:32.019Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5844worker2 # [ 100.123701] nix-grpc-daemon[1320]: time=2026-09-20T13:27:32.020Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5845lb # [ 100.113859] niks3-server[1013]: 2026/09/20 13:27:32 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5846lb # [ 100.118819] niks3-server[1013]: 2026/09/20 13:27:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5847lb # [ 100.122113] niks3-server[1013]: 2026/09/20 13:27:32 INFO Signed narinfos id=35 count=15848worker2 # [ 100.145527] nix-grpc-daemon[1320]: time=2026-09-20T13:27:32.042Z level=INFO msg="Uploading 1 narinfos"5849lb # [ 100.125614] niks3-server[1013]: 2026/09/20 13:27:32 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5850lb # [ 100.130521] niks3-server[1013]: 2026/09/20 13:27:32 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5851lb # [ 100.134209] niks3-server[1013]: 2026/09/20 13:27:32 INFO Completed upload id=355852worker2 # [ 100.157674] nix-grpc-daemon[1320]: time=2026-09-20T13:27:32.054Z level=INFO msg="Upload complete. (379ms)"5853lb # [ 100.135974] niks3-server[1013]: 2026/09/20 13:27:32 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5854worker2 # [ 100.159888] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54762 duration_s=0 paths=1 nar_bytes_in=4245855worker2 # [ 100.164103] nix-grpc-daemon[1199]: ts=2026-09-20T13:27:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547625856worker2 # [ 100.169548] nix-daemon[866]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)5857worker2 # [ 100.171901] nix-daemon[866]: reaped child process 1318, status = succeeded5858worker2 # [ 100.176780] nix-daemon[866]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)5859client # [ 100.150328] nix[1212]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5860lb: must succeed: curl -sf localhost:9901/clusters5861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5862lb: must succeed: curl -sf localhost:9901/clusters5863lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5864worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5865worker2 # [ 102.237259] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1199 (nix-grpc-daemon) on client request.5866worker2 # [ 102.240399] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5867lb # [ 102.223380] niks3-server[1013]: 2026/09/20 13:27:34 INFO lead: released remote=192.168.1.4:540585868worker1 # [ 102.236151] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:33Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5869client # [ 102.221771] nix[1212]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5870client # [ 102.226678] nix[1212]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5871worker2 # [ 102.260325] systemd[1]: Starting scheduler-token.service...5872worker2 # [ 102.270688] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5873worker2 # [ 102.272191] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5874worker2 # [ 102.274186] systemd[1]: nix-grpc-daemon.service: Consumed 436ms CPU time over 16.742s wall clock time, 31.3M memory peak, 16.9K incoming IP traffic, 37K outgoing IP traffic.5875worker2 # [ 102.285067] nix-daemon[866]: reaped child process 1347, status = succeeded5876worker2 # [ 102.286960] nix-daemon[866]: reaped child process 1255, status = succeeded5877worker2 # [ 102.294685] nix-daemon[1349]: unexpected Nix daemon error: error: interrupted by the user5878worker2 # [ 102.301950] nix-daemon[866]: reaped child process 1349, status = failed with exit code 15879worker2 # [ 102.338217] systemd[1]: scheduler-token.service: Deactivated successfully.5880worker2 # [ 102.340679] systemd[1]: Finished scheduler-token.service.5881worker2 # [ 102.341833] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5882worker2 # [ 102.343630] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5883worker2 # [ 102.349858] systemd[1]: Starting scheduler-token.service...5884worker2 # [ 102.395747] systemd[1]: scheduler-token.service: Deactivated successfully.5885worker2 # [ 102.397382] systemd[1]: Finished scheduler-token.service.5886worker2 # [ 102.398511] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 47ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5887worker2 # [ 102.403102] systemd[1]: Starting Nix worker-protocol over gRPC...5888worker1 # [ 102.436774] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:33Z level=info event=scheduler_connected addr=lb:500515889worker2 # [ 102.531337] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5890worker2 # [ 102.533886] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5891worker2 # [ 102.537668] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15892worker2 # [ 102.544846] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5893worker2 # [ 102.550146] systemd[1]: Started Nix worker-protocol over gRPC.5894lb # [ 102.533038] niks3-server[1013]: 2026/09/20 13:27:34 INFO lead: acquired remote=192.168.1.4:410365895worker2 # [ 102.556433] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=scheduler_take_over5896worker1 # [ 102.545625] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5897worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.36 seconds)5898client: must succeed: sleep 5.05899worker2 # [ 102.563398] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=scheduler_connected addr=lb:500515900worker2 # [ 102.567531] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=oidc_jwks provider=mock keys=15901worker2 # [ 102.569044] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:422685902worker2 # [ 102.570725] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05903worker1 # [ 102.746626] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:34Z level=info event=scheduler_connected addr=lb:500515904worker2 # [ 102.758655] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:422685905worker2 # [ 102.763780] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05906worker2 # [ 102.773469] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05907worker2 # [ 102.778166] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15908worker2 # [ 102.824953] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:464725909worker2 # [ 102.833433] nix-daemon[866]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5910worker2 # [ 102.849286] nix-daemon[866]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5911client # [ 102.835601] nix[1212]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5912worker2 # [ 102.863546] nix-daemon[866]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5913client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5914client: waiting for success: ! systemctl is-active mid5915worker2 # [ 107.882800] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:39Z level=info event=niks3_push_started pid=14705916worker2 # [ 107.904276] nix-daemon[866]: reaped child process 1446, status = succeeded5917lb # [ 107.987939] envoy[922]: conn peer=[2001:db8:1::4]:44526 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7821 tx=567 ms=165285918worker2 # [ 108.050505] nix-grpc-daemon[1470]: time=2026-09-20T13:27:39.946Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5919worker2 # [ 108.189218] nix-daemon[866]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)5920worker2 # [ 108.201938] nix-daemon[866]: reaped child process 1486, status = succeeded5921lb # [ 108.190248] niks3-server[1013]: 2026/09/20 13:27:40 INFO Received uploads request method=POST path=/api/pending_closures5922worker2 # [ 108.219434] nix-grpc-daemon[1470]: time=2026-09-20T13:27:40.116Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5923worker2 # [ 108.220953] nix-grpc-daemon[1470]: time=2026-09-20T13:27:40.118Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5924lb # [ 108.216285] niks3-server[1013]: 2026/09/20 13:27:40 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5925lb # [ 108.220395] niks3-server[1013]: 2026/09/20 13:27:40 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5926lb # [ 108.223382] niks3-server[1013]: 2026/09/20 13:27:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5927lb # [ 108.225099] niks3-server[1013]: 2026/09/20 13:27:40 INFO Signed narinfos id=36 count=15928worker2 # [ 108.249620] nix-grpc-daemon[1470]: time=2026-09-20T13:27:40.146Z level=INFO msg="Uploading 1 narinfos"5929lb # [ 108.228434] niks3-server[1013]: 2026/09/20 13:27:40 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5930lb # [ 108.233554] niks3-server[1013]: 2026/09/20 13:27:40 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5931lb # [ 108.237071] niks3-server[1013]: 2026/09/20 13:27:40 INFO Completed upload id=365932worker2 # [ 108.260375] nix-grpc-daemon[1470]: time=2026-09-20T13:27:40.157Z level=INFO msg="Upload complete. (206ms)"5933worker2 # [ 108.264067] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46472 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15934worker2 # [ 108.267562] nix-grpc-daemon[1470]: time=2026-09-20T13:27:40.163Z level=WARN msg="Failed to register uploaded object" key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo error="server returned 404: object is not pending upload\n"5935worker2 # [ 108.273725] nix-daemon[866]: reaped child process 1444, status = succeeded5936client # [ 108.278402] systemd[1]: mid.service: Deactivated successfully.5937client # [ 108.281979] systemd[1]: mid.service: Consumed 122ms CPU time over 9.708s wall clock time, 37.3M memory peak, 9.6K incoming IP traffic, 10.6K outgoing IP traffic.5938client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)5939client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5940client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)5941(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.35 seconds)5942subtest: interrupting the client stops the build on the worker5943client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5944client # [ 108.918261] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5945client # Running as unit: intr.service; invocation ID: 72937129443043a287c97724cef34fef5946client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5947worker1 # [ 109.123435] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566405948worker1 # [ 109.134836] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5949client # [ 109.130605] nix[1248]: don't know how to build these paths:5950client # [ 109.131480] nix[1248]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5951worker2 # [ 109.170410] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:41Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05952worker2 # [ 109.174646] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:41Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15953worker1 # [ 109.202745] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566405954worker1 # [ 109.216462] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)5955worker1 # [ 109.274329] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 2/5)5956worker1 # [ 109.339942] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 3/5)5957worker1 # [ 109.372566] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 382 ms (attempt 4/5)5958worker1 # [ 109.761780] nix-grpc-daemon[1536]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5959worker1 # [ 109.780700] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566405960client # [ 109.775315] nix[1248]: copying 1 paths to 'lb:50051'...5961client # [ 109.778991] nix[1248]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5962worker1 # [ 109.796297] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566405963worker1 # [ 109.807161] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5964worker1 # [ 109.827566] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:41Z level=info event=niks3_push_started pid=16245965worker1 # [ 110.024591] nix-grpc-daemon[1624]: time=2026-09-20T13:27:41.564Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5966worker1 # [ 110.187617] nix-daemon[906]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)5967worker1 # [ 110.199501] nix-daemon[906]: reaped child process 1644, status = succeeded5968worker1 # [ 110.352793] nix-daemon[906]: accepted connection from pid 1646, user nix-grpc-daemon (trusted)5969worker1 # [ 110.364645] nix-daemon[906]: reaped child process 1649, status = succeeded5970lb # [ 110.365260] niks3-server[1013]: 2026/09/20 13:27:42 INFO Received uploads request method=POST path=/api/pending_closures5971worker1 # [ 110.412940] nix-grpc-daemon[1624]: time=2026-09-20T13:27:41.951Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5972worker1 # [ 110.414383] nix-grpc-daemon[1624]: time=2026-09-20T13:27:41.951Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5973lb # [ 110.435683] niks3-server[1013]: 2026/09/20 13:27:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5974worker1 # [ 110.454507] nix-grpc-daemon[1624]: time=2026-09-20T13:27:41.994Z level=INFO msg="Uploading 1 narinfos"5975lb # [ 110.440987] niks3-server[1013]: 2026/09/20 13:27:42 INFO Signed narinfos id=37 count=15976lb # [ 110.446680] niks3-server[1013]: 2026/09/20 13:27:42 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5977lb # [ 110.452714] niks3-server[1013]: 2026/09/20 13:27:42 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5978lb # [ 110.460854] niks3-server[1013]: 2026/09/20 13:27:42 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5979lb # [ 110.466714] niks3-server[1013]: 2026/09/20 13:27:42 INFO Completed upload id=375980worker1 # [ 110.479651] nix-grpc-daemon[1624]: time=2026-09-20T13:27:42.019Z level=INFO msg="Upload complete. (451ms)"5981lb # [ 110.469720] niks3-server[1013]: 2026/09/20 13:27:42 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5982worker1 # [ 110.484942] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=4245983worker1 # [ 110.491861] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566405984worker1 # [ 110.501474] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5985worker1 # [ 110.513310] nix-daemon[906]: reaped child process 1622, status = succeeded5986worker1 # [ 110.517779] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5987client # [ 110.501781] nix[1248]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5988client: must succeed: systemctl kill -s INT intr5989client # [ 112.439883] systemd[1]: intr.service: Sent signal SIGINT to main process 1248 (nix) on client request.5990worker1 # [ 112.463435] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:44Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5991client # [ 112.448223] nix[1248]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5992client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)5993client # [ 112.454779] nix[1248]: error: interrupted by the user5994client # [ 112.485940] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5995client # [ 112.487307] systemd[1]: intr.service: Failed with result 'exit-code'.5996client # [ 112.489071] systemd[1]: intr.service: Consumed 108ms CPU time over 3.567s wall clock time, 39.1M memory peak, 7.5K incoming IP traffic, 8.6K outgoing IP traffic.5997worker1 # [ 112.538240] nix-daemon[906]: reaped child process 1651, status = succeeded5998worker1 # [ 112.544279] nix-daemon[1653]: unexpected Nix daemon error: error: interrupted by the user5999worker1 # [ 112.547848] nix-daemon[906]: reaped child process 1653, status = failed with exit code 16000(finished: subtest: interrupting the client stops the build on the worker, in 4.79 seconds)6001subtest: a deploy mid-build drains: the build finishes, the new generation takes over6002client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6003client # [ 113.729145] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6004client # Running as unit: sw.service; invocation ID: 108052f74e3046baaa7d7f0972756ad86005client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6006client # [ 113.924618] nix[1269]: don't know how to build these paths:6007client # [ 113.925527] nix[1269]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6008worker2 # [ 113.947997] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:464726009worker2 # [ 113.961216] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:45Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06010worker2 # [ 113.965723] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:45Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16011worker2 # [ 114.009525] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:464726012worker2 # [ 114.016338] nix-grpc-daemon[1391]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)6013worker2 # [ 114.054076] nix-grpc-daemon[1391]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 2/5)6014worker2 # [ 114.180381] nix-grpc-daemon[1391]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 3/5)6015worker2 # [ 114.364860] nix-grpc-daemon[1391]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 628 ms (attempt 4/5)6016worker2 # [ 114.996336] nix-grpc-daemon[1391]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6017worker2 # [ 115.010821] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:464726018client # [ 114.993118] nix[1269]: copying 1 paths to 'lb:50051'...6019client # [ 114.993865] nix[1269]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6020worker2 # [ 115.020540] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:464726021worker2 # [ 115.024899] nix-daemon[866]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)6022worker2 # [ 115.196554] nix-daemon[866]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)6023worker2 # [ 115.211771] nix-daemon[866]: reaped child process 1532, status = succeeded6024worker2 # [ 115.385584] nix-daemon[866]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6025worker2 # [ 115.397934] nix-daemon[866]: reaped child process 1537, status = succeeded6026lb # [ 115.387579] niks3-server[1013]: 2026/09/20 13:27:47 INFO Received uploads request method=POST path=/api/pending_closures6027worker2 # [ 115.416219] nix-grpc-daemon[1470]: time=2026-09-20T13:27:47.312Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6028worker2 # [ 115.417607] nix-grpc-daemon[1470]: time=2026-09-20T13:27:47.312Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6029lb # [ 115.403416] niks3-server[1013]: 2026/09/20 13:27:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6030lb # [ 115.406176] niks3-server[1013]: 2026/09/20 13:27:47 INFO Signed narinfos id=38 count=16031worker2 # [ 115.429728] nix-grpc-daemon[1470]: time=2026-09-20T13:27:47.326Z level=INFO msg="Uploading 1 narinfos"6032lb # [ 115.411205] niks3-server[1013]: 2026/09/20 13:27:47 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6033lb # [ 115.415065] niks3-server[1013]: 2026/09/20 13:27:47 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6034lb # [ 115.420107] niks3-server[1013]: 2026/09/20 13:27:47 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6035lb # [ 115.423897] niks3-server[1013]: 2026/09/20 13:27:47 INFO Completed upload id=386036worker2 # [ 115.448433] nix-grpc-daemon[1470]: time=2026-09-20T13:27:47.345Z level=INFO msg="Upload complete. (416ms)"6037lb # [ 115.426292] niks3-server[1013]: 2026/09/20 13:27:47 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6038worker2 # [ 115.450456] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46472 duration_s=0 paths=1 nar_bytes_in=4166039worker2 # [ 115.453963] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:464726040worker2 # [ 115.459946] nix-daemon[866]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)6041worker2 # [ 115.462168] nix-daemon[866]: reaped child process 1527, status = succeeded6042worker2 # [ 115.467697] nix-daemon[866]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)6043client # [ 115.440592] nix[1269]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6044worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26045worker2 # Checking switch inhibitors... done6046lb # [ 117.988615] envoy[922]: conn peer=[2001:db8:1::1]:60526 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3848 tx=1662 ms=94966047lb # [ 117.992549] envoy[922]: conn peer=[2001:db8:1::1]:56732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=728 ms=33656048worker2 # [ 118.092996] nixos[1572]: switching to system configuration /nix/store/1s7hk7v6xhwd69k2r25jlpix4jrc15kf-nixos-system-worker2-test6049worker2 # [ 118.096397] systemd[1]: Stopped target Remote File Systems.6050worker2 # activating the configuration...6051worker2 # [ 118.098986] systemd[1]: Stopped target Local File Systems.6052worker2 # setting up /etc...6053worker2 # [ 118.746159] systemd[1]: Reload requested from client PID 1572 ('.switch-to-conf') (unit backdoor.service)...6054worker2 # [ 118.748096] systemd[1]: Reloading...6055worker2 # [ 119.419370] systemd[1]: Reloading finished in 670 ms.6056worker2 # restarting sysinit-reactivation.target6057worker2 # [ 119.474212] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6058worker2 # [ 119.591351] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6059worker2 # [ 119.593462] systemd[1]: Reached target Reactivate sysinit units.6060worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6061worker2 # [ 119.599296] systemd[1]: Reloading D-Bus System Message Bus...6062worker2 # [ 119.601626] dbus-broker-launch[532]: Caught SIGHUP, trigger reload.6063worker2 # [ 119.605142] systemd[1]: Reloading Nix worker-protocol over gRPC...6064worker2 # [ 119.636570] dbus-broker-launch[532]: Looking up NSS user entry for 'systemd-timesync'...6065worker2 # [ 119.640200] dbus-broker-launch[532]: NSS returned no entry for 'systemd-timesync'6066worker2 # [ 119.641324] dbus-broker-launch[532]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6067worker2 # [ 119.644667] systemd[1]: Reloaded Nix worker-protocol over gRPC.6068worker2 # [ 119.662330] systemd[1]: Reloaded D-Bus System Message Bus.6069worker2 # [ 119.665472] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6070worker2 # [ 119.667314] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6071worker2 # [ 119.675704] systemd[1]: Reached target Remote File Systems.6072worker2 # [ 119.679673] systemd[1]: Reached target Local File Systems.6073worker2 # [ 119.682109] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6074worker2 # [ 119.687650] systemd[1]: Starting Create SUID/SGID Wrappers...6075worker2 # [ 119.688457] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6076worker2 # [ 119.689558] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676077worker2 # [ 119.691461] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6078worker2 # [ 119.692862] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6079worker2 # [ 119.695562] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6080worker2 # [ 120.091771] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6081worker2 # [ 120.094404] systemd[1]: Finished Create SUID/SGID Wrappers.6082worker2 # [ 120.097455] systemd[1]: Starting linger-users.service...6083worker2 # [ 120.138352] systemd[1]: linger-users.service: Deactivated successfully.6084worker2 # [ 120.140796] systemd[1]: Finished linger-users.service.6085worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6086worker2 # [ 120.396765] nixos[1572]: finished switching to system configuration /nix/store/1s7hk7v6xhwd69k2r25jlpix4jrc15kf-nixos-system-worker2-test6087worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.05 seconds)6088worker2: must succeed: pgrep -f 'read -t [9]0 x'6089worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6090client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26091worker2 # [ 120.557667] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:52Z level=info event=draining6092worker1 # [ 120.655632] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566406093client # don't know how to build these paths:6094client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6095worker1 # [ 120.661492] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406096worker2 # [ 120.688802] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:52Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06097worker2 # [ 120.691140] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:52Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16098worker1 # [ 120.726575] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406099worker1 # [ 120.738112] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406100client # copying 1 paths to 'lb:50051'...6101worker1 # [ 120.748187] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406102client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6103worker1 # [ 120.765916] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6104worker1 # [ 120.934629] nix-daemon[906]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6105worker1 # [ 120.946396] nix-daemon[906]: reaped child process 1710, status = succeeded6106worker1 # [ 121.116074] nix-daemon[906]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6107worker1 # [ 121.127588] nix-daemon[906]: reaped child process 1715, status = succeeded6108lb # [ 121.130443] niks3-server[1013]: 2026/09/20 13:27:53 INFO Received uploads request method=POST path=/api/pending_closures6109worker1 # [ 121.147601] nix-grpc-daemon[1624]: time=2026-09-20T13:27:52.687Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6110worker1 # [ 121.149126] nix-grpc-daemon[1624]: time=2026-09-20T13:27:52.689Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6111lb # [ 121.147814] niks3-server[1013]: 2026/09/20 13:27:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6112lb # [ 121.149260] niks3-server[1013]: 2026/09/20 13:27:53 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6113lb # [ 121.151596] niks3-server[1013]: 2026/09/20 13:27:53 INFO Signed narinfos id=39 count=16114worker1 # [ 121.164369] nix-grpc-daemon[1624]: time=2026-09-20T13:27:52.704Z level=INFO msg="Uploading 1 narinfos"6115lb # [ 121.154575] niks3-server[1013]: 2026/09/20 13:27:53 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6116lb # [ 121.159105] niks3-server[1013]: 2026/09/20 13:27:53 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6117lb # [ 121.164192] niks3-server[1013]: 2026/09/20 13:27:53 INFO Completed upload id=396118worker1 # [ 121.176949] nix-grpc-daemon[1624]: time=2026-09-20T13:27:52.716Z level=INFO msg="Upload complete. (397ms)"6119lb # [ 121.165883] niks3-server[1013]: 2026/09/20 13:27:53 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6120worker1 # [ 121.178892] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=5286121worker1 # [ 121.184407] nix-daemon[906]: reaped child process 1705, status = succeeded6122worker1 # [ 121.188885] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406123worker1 # [ 121.193168] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6124worker1 # [ 121.198327] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6125client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6126client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6127worker1 # [ 122.210545] nix-daemon[906]: reaped child process 1719, status = succeeded6128client # farm-a> LOG-farm-a-during-drain6129worker1 # [ 122.360560] nix-daemon[906]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6130worker1 # [ 122.372129] nix-daemon[906]: reaped child process 1737, status = succeeded6131lb # [ 122.372445] niks3-server[1013]: 2026/09/20 13:27:54 INFO Received uploads request method=POST path=/api/pending_closures6132worker1 # [ 122.393811] nix-grpc-daemon[1624]: time=2026-09-20T13:27:53.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6133worker1 # [ 122.395294] nix-grpc-daemon[1624]: time=2026-09-20T13:27:53.935Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6134lb # [ 122.407304] niks3-server[1013]: 2026/09/20 13:27:54 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6135lb # [ 122.425862] niks3-server[1013]: 2026/09/20 13:27:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6136worker1 # [ 122.443872] nix-grpc-daemon[1624]: time=2026-09-20T13:27:53.983Z level=INFO msg="Uploading 1 narinfos"6137lb # [ 122.430739] niks3-server[1013]: 2026/09/20 13:27:54 INFO Signed narinfos id=40 count=16138lb # [ 122.436157] niks3-server[1013]: 2026/09/20 13:27:54 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6139lb # [ 122.442831] niks3-server[1013]: 2026/09/20 13:27:54 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6140lb # [ 122.450695] niks3-server[1013]: 2026/09/20 13:27:54 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6141lb # [ 122.454083] niks3-server[1013]: 2026/09/20 13:27:54 INFO Completed upload id=406142worker1 # [ 122.466902] nix-grpc-daemon[1624]: time=2026-09-20T13:27:54.006Z level=INFO msg="Upload complete. (262ms)"6143lb # [ 122.456204] niks3-server[1013]: 2026/09/20 13:27:54 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6144worker1 # [ 122.470334] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16145worker1 # [ 122.476319] nix-daemon[906]: reaped child process 1717, status = succeeded6146worker2 # [ 122.487043] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:54Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06147worker2 # [ 122.488795] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:54Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16148worker1 # [ 122.518097] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406149worker1 # [ 122.527778] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406150worker1 # [ 122.536335] nix-grpc-daemon[1624]: time=2026-09-20T13:27:54.076Z level=INFO msg="All 1 paths already cached"6151client # copying 1 paths to 'lb:50051'...6152worker1 # [ 122.540508] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406153client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6154worker1 # [ 122.549071] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6155worker1 # [ 122.703426] nix-daemon[906]: accepted connection from pid 1741, user nix-grpc-daemon (trusted)6156worker1 # [ 122.715488] nix-daemon[906]: reaped child process 1744, status = succeeded6157worker1 # [ 122.878554] nix-daemon[906]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6158worker1 # [ 122.890237] nix-daemon[906]: reaped child process 1749, status = succeeded6159lb # [ 122.892652] niks3-server[1013]: 2026/09/20 13:27:54 INFO Received uploads request method=POST path=/api/pending_closures6160worker1 # [ 122.910488] nix-grpc-daemon[1624]: time=2026-09-20T13:27:54.450Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6161worker1 # [ 122.911921] nix-grpc-daemon[1624]: time=2026-09-20T13:27:54.450Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6162lb # [ 122.905701] niks3-server[1013]: 2026/09/20 13:27:54 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6163lb # [ 122.910877] niks3-server[1013]: 2026/09/20 13:27:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6164worker1 # [ 122.927055] nix-grpc-daemon[1624]: time=2026-09-20T13:27:54.466Z level=INFO msg="Uploading 1 narinfos"6165lb # [ 122.914365] niks3-server[1013]: 2026/09/20 13:27:54 INFO Signed narinfos id=41 count=16166lb # [ 122.917400] niks3-server[1013]: 2026/09/20 13:27:54 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6167lb # [ 122.921878] niks3-server[1013]: 2026/09/20 13:27:54 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6168lb # [ 122.926142] niks3-server[1013]: 2026/09/20 13:27:55 INFO Completed upload id=416169worker1 # [ 122.941022] nix-grpc-daemon[1624]: time=2026-09-20T13:27:54.480Z level=INFO msg="Upload complete. (386ms)"6170lb # [ 122.928859] niks3-server[1013]: 2026/09/20 13:27:55 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6171worker1 # [ 122.942814] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=6406172worker1 # [ 122.946149] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406173worker1 # [ 122.950140] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6174worker1 # [ 122.954745] nix-daemon[906]: reaped child process 1739, status = succeeded6175client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6176client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6177worker1 # [ 122.959551] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6178worker1 # [ 123.771877] nix-daemon[906]: reaped child process 1753, status = succeeded6179client # farm-b> LOG-farm-b-during-drain6180worker1 # [ 123.921726] nix-daemon[906]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)6181worker1 # [ 123.933376] nix-daemon[906]: reaped child process 1771, status = succeeded6182lb # [ 123.937495] niks3-server[1013]: 2026/09/20 13:27:56 INFO Received uploads request method=POST path=/api/pending_closures6183worker1 # [ 123.958527] nix-grpc-daemon[1624]: time=2026-09-20T13:27:55.498Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6184worker1 # [ 123.960092] nix-grpc-daemon[1624]: time=2026-09-20T13:27:55.500Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6185lb # [ 123.974850] niks3-server[1013]: 2026/09/20 13:27:56 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6186lb # [ 123.982645] niks3-server[1013]: 2026/09/20 13:27:56 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6187lb # [ 123.990171] niks3-server[1013]: 2026/09/20 13:27:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6188lb # [ 123.994979] niks3-server[1013]: 2026/09/20 13:27:56 INFO Signed narinfos id=42 count=16189worker1 # [ 124.010764] nix-grpc-daemon[1624]: time=2026-09-20T13:27:55.550Z level=INFO msg="Uploading 1 narinfos"6190lb # [ 124.005882] niks3-server[1013]: 2026/09/20 13:27:56 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6191lb # [ 124.011455] niks3-server[1013]: 2026/09/20 13:27:56 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6192lb # [ 124.017341] niks3-server[1013]: 2026/09/20 13:27:56 INFO Completed upload id=426193worker1 # [ 124.030456] nix-grpc-daemon[1624]: time=2026-09-20T13:27:55.570Z level=INFO msg="Upload complete. (264ms)"6194lb # [ 124.019685] niks3-server[1013]: 2026/09/20 13:27:56 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6195worker2 # [ 124.050922] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:55Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06196worker1 # [ 124.036613] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16197worker2 # [ 124.052690] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:55Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16198worker1 # [ 124.049635] nix-daemon[906]: reaped child process 1751, status = succeeded6199worker1 # [ 124.084314] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406200worker1 # [ 124.092266] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406201worker1 # [ 124.100204] nix-grpc-daemon[1624]: time=2026-09-20T13:27:55.640Z level=INFO msg="All 2 paths already cached"6202client # copying 1 paths to 'lb:50051'...6203worker1 # [ 124.105780] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406204client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6205worker1 # [ 124.121484] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6206worker1 # [ 124.256539] nix-daemon[906]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6207worker1 # [ 124.268368] nix-daemon[906]: reaped child process 1778, status = succeeded6208worker1 # [ 124.424680] nix-daemon[906]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6209worker1 # [ 124.436528] nix-daemon[906]: reaped child process 1783, status = succeeded6210lb # [ 124.436849] niks3-server[1013]: 2026/09/20 13:27:56 INFO Received uploads request method=POST path=/api/pending_closures6211worker1 # [ 124.458828] nix-grpc-daemon[1624]: time=2026-09-20T13:27:55.998Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6212worker1 # [ 124.460177] nix-grpc-daemon[1624]: time=2026-09-20T13:27:55.998Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6213lb # [ 124.464701] niks3-server[1013]: 2026/09/20 13:27:56 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6214lb # [ 124.474412] niks3-server[1013]: 2026/09/20 13:27:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6215lb # [ 124.479513] niks3-server[1013]: 2026/09/20 13:27:56 INFO Signed narinfos id=43 count=16216worker1 # [ 124.495244] nix-grpc-daemon[1624]: time=2026-09-20T13:27:56.035Z level=INFO msg="Uploading 1 narinfos"6217lb # [ 124.486613] niks3-server[1013]: 2026/09/20 13:27:56 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6218lb # [ 124.496348] niks3-server[1013]: 2026/09/20 13:27:56 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6219lb # [ 124.502243] niks3-server[1013]: 2026/09/20 13:27:56 INFO Completed upload id=436220worker1 # [ 124.517163] nix-grpc-daemon[1624]: time=2026-09-20T13:27:56.057Z level=INFO msg="Upload complete. (390ms)"6221lb # [ 124.506224] niks3-server[1013]: 2026/09/20 13:27:56 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6222worker1 # [ 124.520154] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=7686223worker1 # [ 124.525389] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406224worker1 # [ 124.534964] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6225worker1 # [ 124.540658] nix-daemon[906]: reaped child process 1773, status = succeeded6226client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6227worker1 # [ 124.543592] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6228client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6229client # farm-top> LOG-farm-top-during-drain6230worker1 # [ 125.755884] nix-daemon[906]: reaped child process 1787, status = succeeded6231worker1 # [ 125.889092] nix-daemon[906]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6232worker1 # [ 125.901197] nix-daemon[906]: reaped child process 1805, status = succeeded6233lb # [ 125.902760] niks3-server[1013]: 2026/09/20 13:27:57 INFO Received uploads request method=POST path=/api/pending_closures6234worker1 # [ 125.923387] nix-grpc-daemon[1624]: time=2026-09-20T13:27:57.463Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6235worker1 # [ 125.924984] nix-grpc-daemon[1624]: time=2026-09-20T13:27:57.464Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6236lb # [ 125.933288] niks3-server[1013]: 2026/09/20 13:27:58 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6237lb # [ 125.947642] niks3-server[1013]: 2026/09/20 13:27:58 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6238lb # [ 125.958790] niks3-server[1013]: 2026/09/20 13:27:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6239worker1 # [ 125.976320] nix-grpc-daemon[1624]: time=2026-09-20T13:27:57.516Z level=INFO msg="Uploading 1 narinfos"6240lb # [ 125.963766] niks3-server[1013]: 2026/09/20 13:27:58 INFO Signed narinfos id=44 count=16241lb # [ 125.968189] niks3-server[1013]: 2026/09/20 13:27:58 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6242lb # [ 125.977922] niks3-server[1013]: 2026/09/20 13:27:58 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6243lb # [ 125.983576] niks3-server[1013]: 2026/09/20 13:27:58 INFO Completed upload id=446244lb # [ 125.985033] niks3-server[1013]: 2026/09/20 13:27:58 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6245worker1 # [ 125.996534] nix-grpc-daemon[1624]: time=2026-09-20T13:27:57.536Z level=INFO msg="Upload complete. (245ms)"6246worker1 # [ 126.001769] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16247worker1 # [ 126.014821] nix-daemon[906]: reaped child process 1785, status = succeeded6248client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.61 seconds)6249client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6250client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6251worker2: must succeed: pkill -f 'read -t [9]0 x'6252worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6253client: waiting for success: ! systemctl is-active sw6254worker2 # [ 126.505229] nix-daemon[866]: reaped child process 1541, status = succeeded6255worker2 # [ 126.653042] nix-daemon[866]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6256worker2 # [ 126.665528] nix-daemon[866]: reaped child process 1721, status = succeeded6257lb # [ 126.653121] niks3-server[1013]: 2026/09/20 13:27:58 INFO Received uploads request method=POST path=/api/pending_closures6258worker2 # [ 126.684553] nix-grpc-daemon[1470]: time=2026-09-20T13:27:58.581Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6259worker2 # [ 126.686096] nix-grpc-daemon[1470]: time=2026-09-20T13:27:58.583Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6260lb # [ 126.685619] niks3-server[1013]: 2026/09/20 13:27:58 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6261lb # [ 126.702069] niks3-server[1013]: 2026/09/20 13:27:58 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6262lb # [ 126.708777] niks3-server[1013]: 2026/09/20 13:27:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6263worker2 # [ 126.737599] nix-grpc-daemon[1470]: time=2026-09-20T13:27:58.634Z level=INFO msg="Uploading 1 narinfos"6264lb # [ 126.714110] niks3-server[1013]: 2026/09/20 13:27:58 INFO Signed narinfos id=45 count=16265lb # [ 126.718823] niks3-server[1013]: 2026/09/20 13:27:58 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6266lb # [ 126.729099] niks3-server[1013]: 2026/09/20 13:27:58 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6267lb # [ 126.734184] niks3-server[1013]: 2026/09/20 13:27:58 INFO Completed upload id=456268worker2 # [ 126.759366] nix-grpc-daemon[1470]: time=2026-09-20T13:27:58.656Z level=INFO msg="Upload complete. (273ms)"6269lb # [ 126.737125] niks3-server[1013]: 2026/09/20 13:27:58 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6270worker2 # [ 126.764112] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46472 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16271worker2 # [ 126.771360] nix-daemon[866]: reaped child process 1539, status = succeeded6272client # [ 126.777188] systemd[1]: sw.service: Deactivated successfully.6273client # [ 126.780198] systemd[1]: sw.service: Consumed 113ms CPU time over 13.047s wall clock time, 37.2M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.6274client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6275client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6276client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6277worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6278worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.05 seconds)6279lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6280lb: must succeed: curl -sf localhost:9901/clusters6281lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6282worker2 # [ 127.559163] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=signal_exit6283worker2 # [ 127.778506] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=scheduler_yield6284lb # [ 127.756804] niks3-server[1013]: 2026/09/20 13:27:59 INFO lead: released remote=192.168.1.4:410366285worker1 # [ 127.774659] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:59Z level=info event=scheduler_restarting addr=lb:500516286worker2 # [ 127.785378] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:422686287worker2 # [ 127.790756] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=worker_gone addr=192.168.1.3:500516288worker2 # [ 127.793835] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=scheduler_restarting addr=lb:500516289worker2 # [ 127.796755] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:422686290worker2 # [ 127.800102] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=worker_gone addr=192.168.1.4:500516291worker1 # [ 127.974476] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:59Z level=info event=scheduler_connected addr=lb:500516292worker1 # [ 127.978452] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6293worker2 # [ 127.991238] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=scheduler_connected addr=lb:500516294worker2 # [ 127.995941] nix-grpc-daemon[1391]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6295lb # [ 127.989229] envoy[922]: conn peer=[2001:db8:1::1]:33176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9581 tx=3822 ms=53596296lb # [ 127.992963] envoy[922]: conn peer=[2001:db8:1::1]:33166 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1376 ms=128336297lb # [ 128.048366] niks3-server[1013]: 2026/09/20 13:28:00 INFO lead: acquired remote=192.168.1.3:590066298worker1 # [ 128.064718] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:59Z level=info event=scheduler_take_over6299worker1 # [ 128.179101] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:59Z level=info event=scheduler_connected addr=lb:500516300worker1 # [ 128.183273] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6301worker2 # [ 128.197066] nix-grpc-daemon[1391]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500516302worker2 # [ 128.200704] nix-grpc-daemon[1391]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6303worker1 # [ 128.383992] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:59Z level=info event=scheduler_connected addr=lb:500516304worker2 # [ 128.398602] nix-grpc-daemon[1391]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500516305worker1 # [ 128.387972] nix-grpc-daemon[1536]: ts=2026-09-20T13:27:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6306worker2 # [ 128.400581] nix-grpc-daemon[1391]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6307lb: must succeed: curl -sf localhost:9901/clusters6308worker1 # [ 128.588320] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500516309worker1 # [ 128.590274] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6310worker2 # [ 128.601180] nix-grpc-daemon[1391]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500516311lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6312worker2 # [ 128.603138] nix-grpc-daemon[1391]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6313worker2 # [ 128.778413] nix-grpc-daemon[1391]: ts=2026-09-20T13:28:00Z level=info event=scheduler_restarting6314worker2 # [ 128.791605] nix-daemon[866]: reaped child process 1442, status = succeeded6315worker1 # [ 128.791952] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500516316worker2 # [ 128.807220] nix-grpc-daemon[1391]: ts=2026-09-20T13:28:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6317worker1 # [ 128.795928] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6318worker1 # [ 128.996570] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500516319worker1 # [ 129.000422] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6320worker2 # [ 129.104912] nix-grpc-daemon[1391]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6321worker2 # [ 129.108456] nix-grpc-daemon[1391]: E0000 00:00:1789910881.001623 1434 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6322worker2 # [ 129.113152] nix-grpc-daemon[1391]: E0000 00:00:1789910881.001894 1434 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6323worker1 # [ 129.201346] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500516324worker1 # [ 129.205107] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6325worker1 # [ 129.405944] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_connected addr=lb:500516326worker1 # [ 129.409728] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6327lb: must succeed: curl -sf localhost:9901/clusters6328worker1 # [ 129.610472] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500516329worker1 # [ 129.614252] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6331worker1 # [ 129.816241] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500516332worker1 # [ 129.819423] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6333worker1 # [ 130.017122] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:01Z level=info event=scheduler_connected addr=lb:500516334worker1 # [ 130.028135] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:400706335worker1 # [ 130.031699] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06336worker2 # [ 130.592959] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6337worker2 # [ 130.596597] systemd[1]: nix-grpc-daemon.service: Consumed 672ms CPU time over 28.324s wall clock time, 31M memory peak, 26K incoming IP traffic, 36.5K outgoing IP traffic.6338lb: must succeed: curl -sf localhost:9901/clusters6339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6340worker2 # [ 131.582484] systemd[1]: Starting scheduler-token.service...6341worker2 # [ 131.672970] systemd[1]: scheduler-token.service: Deactivated successfully.6342worker2 # [ 131.674951] systemd[1]: Finished scheduler-token.service.6343worker2 # [ 131.676124] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 92ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6344worker2 # [ 131.681461] systemd[1]: Starting Nix worker-protocol over gRPC...6345lb: must succeed: curl -sf localhost:9901/clusters6346lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6347worker2 # [ 131.839515] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6348worker2 # [ 131.841297] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6349worker2 # [ 131.844657] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16350worker2 # [ 131.851931] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6351worker2 # [ 131.853566] systemd[1]: Started Nix worker-protocol over gRPC.6352worker2 # [ 131.868448] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:03Z level=info event=scheduler_connected addr=lb:500516353worker1 # [ 131.864804] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:03Z level=info event=oidc_jwks provider=mock keys=16354worker1 # [ 131.868805] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:400706355worker1 # [ 131.872584] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06356lb: must succeed: curl -sf localhost:9901/clusters6357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6358lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 5.38 seconds)6359lb: must succeed: curl -sf localhost:9901/clusters6360lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6361worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6362worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6363(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 19.24 seconds)6364subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6365client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16366worker2 # [ 133.156474] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578926367worker2 # [ 133.168240] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6368worker1 # [ 133.173204] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406369worker1 # [ 133.212118] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:04Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06370worker1 # [ 133.214205] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:04Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16371worker1 # [ 133.216081] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:04Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06372worker1 # [ 133.217595] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:04Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06373worker1 # [ 133.255948] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406374worker1 # [ 133.264064] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406375worker1 # [ 133.271317] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406376worker1 # [ 133.276738] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6377worker1 # [ 133.433921] nix-daemon[906]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)6378worker1 # [ 133.445955] nix-daemon[906]: reaped child process 1817, status = succeeded6379worker1 # [ 133.604178] nix-daemon[906]: accepted connection from pid 1819, user nix-grpc-daemon (trusted)6380worker1 # [ 133.615797] nix-daemon[906]: reaped child process 1822, status = succeeded6381lb # [ 133.618489] niks3-server[1013]: 2026/09/20 13:28:05 INFO Received uploads request method=POST path=/api/pending_closures6382worker1 # [ 133.641110] nix-grpc-daemon[1624]: time=2026-09-20T13:28:05.180Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6383worker1 # [ 133.642429] nix-grpc-daemon[1624]: time=2026-09-20T13:28:05.180Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6384lb # [ 133.647819] niks3-server[1013]: 2026/09/20 13:28:05 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6385lb # [ 133.660733] niks3-server[1013]: 2026/09/20 13:28:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6386worker1 # [ 133.681298] nix-grpc-daemon[1624]: time=2026-09-20T13:28:05.221Z level=INFO msg="Uploading 1 narinfos"6387lb # [ 133.668291] niks3-server[1013]: 2026/09/20 13:28:05 INFO Signed narinfos id=46 count=16388lb # [ 133.686914] niks3-server[1013]: 2026/09/20 13:28:05 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6389lb # [ 133.695217] niks3-server[1013]: 2026/09/20 13:28:05 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6390lb # [ 133.698045] niks3-server[1013]: 2026/09/20 13:28:05 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6391lb # [ 133.704284] niks3-server[1013]: 2026/09/20 13:28:05 INFO Completed upload id=466392worker1 # [ 133.718558] nix-grpc-daemon[1624]: time=2026-09-20T13:28:05.258Z level=INFO msg="Upload complete. (436ms)"6393worker1 # [ 133.723186] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=5046394worker1 # [ 133.729314] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406395worker1 # [ 133.741186] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6396worker1 # [ 133.750284] nix-daemon[906]: reaped child process 1812, status = succeeded6397worker1 # [ 133.757152] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6398worker1 # [ 134.383843] nix-daemon[906]: reaped child process 1826, status = succeeded6399worker1 # [ 134.521360] nix-daemon[906]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)6400worker1 # [ 134.532334] nix-daemon[906]: reaped child process 1844, status = succeeded6401lb # [ 134.531617] niks3-server[1013]: 2026/09/20 13:28:06 INFO Received uploads request method=POST path=/api/pending_closures6402worker1 # [ 134.552069] nix-grpc-daemon[1624]: time=2026-09-20T13:28:06.090Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6403worker1 # [ 134.553436] nix-grpc-daemon[1624]: time=2026-09-20T13:28:06.091Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6404lb # [ 134.560376] niks3-server[1013]: 2026/09/20 13:28:06 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6405lb # [ 134.566775] niks3-server[1013]: 2026/09/20 13:28:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6406lb # [ 134.570571] niks3-server[1013]: 2026/09/20 13:28:06 INFO Signed narinfos id=47 count=16407worker1 # [ 134.583239] nix-grpc-daemon[1624]: time=2026-09-20T13:28:06.123Z level=INFO msg="Uploading 1 narinfos"6408lb # [ 134.572825] niks3-server[1013]: 2026/09/20 13:28:06 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6409lb # [ 134.575727] niks3-server[1013]: 2026/09/20 13:28:06 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6410lb # [ 134.582160] niks3-server[1013]: 2026/09/20 13:28:06 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6411worker1 # [ 134.599034] nix-grpc-daemon[1624]: time=2026-09-20T13:28:06.138Z level=INFO msg="Upload complete. (233ms)"6412lb # [ 134.586565] niks3-server[1013]: 2026/09/20 13:28:06 INFO Completed upload id=476413lb # [ 134.589240] niks3-server[1013]: 2026/09/20 13:28:06 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6414worker1 # [ 134.603345] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16415worker1 # [ 134.606714] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:06Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16416worker1 # [ 134.609522] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406417worker1 # [ 134.613223] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:06Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06418worker1 # [ 134.618345] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406419worker1 # [ 134.620835] nix-daemon[906]: reaped child process 1824, status = succeeded6420worker1 # [ 134.624292] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406421worker1 # [ 134.628850] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6422worker1 # [ 134.789040] nix-daemon[906]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6423worker1 # [ 134.800960] nix-daemon[906]: reaped child process 1853, status = succeeded6424worker1 # [ 134.978117] nix-daemon[906]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6425worker1 # [ 134.992218] nix-daemon[906]: reaped child process 1858, status = succeeded6426lb # [ 134.997958] niks3-server[1013]: 2026/09/20 13:28:07 INFO Received uploads request method=POST path=/api/pending_closures6427worker1 # [ 135.021982] nix-grpc-daemon[1624]: time=2026-09-20T13:28:06.561Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6428worker1 # [ 135.023486] nix-grpc-daemon[1624]: time=2026-09-20T13:28:06.563Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6429lb # [ 135.031291] niks3-server[1013]: 2026/09/20 13:28:07 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6430lb # [ 135.040972] niks3-server[1013]: 2026/09/20 13:28:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6431lb # [ 135.047063] niks3-server[1013]: 2026/09/20 13:28:07 INFO Signed narinfos id=48 count=16432worker1 # [ 135.063026] nix-grpc-daemon[1624]: time=2026-09-20T13:28:06.602Z level=INFO msg="Uploading 1 narinfos"6433lb # [ 135.050791] niks3-server[1013]: 2026/09/20 13:28:07 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6434lb # [ 135.063852] niks3-server[1013]: 2026/09/20 13:28:07 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6435worker1 # [ 135.082974] nix-grpc-daemon[1624]: time=2026-09-20T13:28:06.622Z level=INFO msg="Upload complete. (449ms)"6436lb # [ 135.070511] niks3-server[1013]: 2026/09/20 13:28:07 INFO Completed upload id=486437lb # [ 135.073163] niks3-server[1013]: 2026/09/20 13:28:07 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6438worker1 # [ 135.085327] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=5046439worker1 # [ 135.093343] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406440worker1 # [ 135.102053] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6441worker1 # [ 135.105399] nix-daemon[906]: reaped child process 1848, status = succeeded6442worker1 # [ 135.110361] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6443worker1 # [ 136.325789] nix-daemon[906]: reaped child process 1862, status = succeeded6444worker1 # [ 136.486811] nix-daemon[906]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)6445worker1 # [ 136.497241] nix-daemon[906]: reaped child process 1880, status = succeeded6446lb # [ 136.503722] niks3-server[1013]: 2026/09/20 13:28:08 INFO Received uploads request method=POST path=/api/pending_closures6447worker1 # [ 136.524755] nix-grpc-daemon[1624]: time=2026-09-20T13:28:08.064Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6448worker1 # [ 136.527056] nix-grpc-daemon[1624]: time=2026-09-20T13:28:08.066Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6449lb # [ 136.538148] niks3-server[1013]: 2026/09/20 13:28:08 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6450lb # [ 136.553151] niks3-server[1013]: 2026/09/20 13:28:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6451worker1 # [ 136.570572] nix-grpc-daemon[1624]: time=2026-09-20T13:28:08.110Z level=INFO msg="Uploading 1 narinfos"6452lb # [ 136.558481] niks3-server[1013]: 2026/09/20 13:28:08 INFO Signed narinfos id=49 count=16453lb # [ 136.562448] niks3-server[1013]: 2026/09/20 13:28:08 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6454lb # [ 136.573781] niks3-server[1013]: 2026/09/20 13:28:08 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6455lb # [ 136.579452] niks3-server[1013]: 2026/09/20 13:28:08 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6456lb # [ 136.588198] niks3-server[1013]: 2026/09/20 13:28:08 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6457lb # [ 136.591227] niks3-server[1013]: 2026/09/20 13:28:08 INFO Completed upload id=496458worker1 # [ 136.604356] nix-grpc-daemon[1624]: time=2026-09-20T13:28:08.144Z level=INFO msg="Upload complete. (286ms)"6459worker1 # [ 136.614562] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16460worker1 # [ 136.621988] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:08Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16461worker1 # [ 136.627473] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:08Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06462worker1 # [ 136.635783] nix-daemon[906]: reaped child process 1860, status = succeeded6463worker1 # [ 136.654933] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406464worker1 # [ 136.665786] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406465worker1 # [ 136.676331] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406466worker1 # [ 136.686132] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6467worker1 # [ 136.836335] nix-daemon[906]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6468worker1 # [ 136.848440] nix-daemon[906]: reaped child process 1889, status = succeeded6469worker1 # [ 137.005334] nix-daemon[906]: accepted connection from pid 1891, user nix-grpc-daemon (trusted)6470worker1 # [ 137.016788] nix-daemon[906]: reaped child process 1894, status = succeeded6471lb # [ 137.020348] niks3-server[1013]: 2026/09/20 13:28:09 INFO Received uploads request method=POST path=/api/pending_closures6472worker1 # [ 137.040797] nix-grpc-daemon[1624]: time=2026-09-20T13:28:08.580Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6473worker1 # [ 137.042651] nix-grpc-daemon[1624]: time=2026-09-20T13:28:08.582Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6474lb # [ 137.047737] niks3-server[1013]: 2026/09/20 13:28:09 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6475lb # [ 137.058748] niks3-server[1013]: 2026/09/20 13:28:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6476worker1 # [ 137.076086] nix-grpc-daemon[1624]: time=2026-09-20T13:28:08.615Z level=INFO msg="Uploading 1 narinfos"6477lb # [ 137.063138] niks3-server[1013]: 2026/09/20 13:28:09 INFO Signed narinfos id=50 count=16478lb # [ 137.074808] niks3-server[1013]: 2026/09/20 13:28:09 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6479lb # [ 137.080489] niks3-server[1013]: 2026/09/20 13:28:09 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6480lb # [ 137.086764] niks3-server[1013]: 2026/09/20 13:28:09 INFO Completed upload id=506481worker1 # [ 137.101692] nix-grpc-daemon[1624]: time=2026-09-20T13:28:08.641Z level=INFO msg="Upload complete. (406ms)"6482lb # [ 137.089972] niks3-server[1013]: 2026/09/20 13:28:09 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6483worker1 # [ 137.103927] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=5046484worker1 # [ 137.108837] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406485worker1 # [ 137.119070] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6486worker1 # [ 137.122838] nix-daemon[906]: reaped child process 1884, status = succeeded6487worker1 # [ 137.127764] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6488lb # [ 137.988279] envoy[922]: conn peer=[2001:db8:1::4]:44358 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4876 tx=568 ms=262436489worker1 # [ 138.141646] nix-daemon[906]: reaped child process 1898, status = succeeded6490worker1 # [ 138.306283] nix-daemon[906]: accepted connection from pid 1913, user nix-grpc-daemon (trusted)6491worker1 # [ 138.318147] nix-daemon[906]: reaped child process 1916, status = succeeded6492lb # [ 138.322469] niks3-server[1013]: 2026/09/20 13:28:10 INFO Received uploads request method=POST path=/api/pending_closures6493worker1 # [ 138.342959] nix-grpc-daemon[1624]: time=2026-09-20T13:28:09.882Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6494worker1 # [ 138.345417] nix-grpc-daemon[1624]: time=2026-09-20T13:28:09.884Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6495lb # [ 138.356789] niks3-server[1013]: 2026/09/20 13:28:10 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6496lb # [ 138.363390] niks3-server[1013]: 2026/09/20 13:28:10 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6497lb # [ 138.376559] niks3-server[1013]: 2026/09/20 13:28:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6498worker1 # [ 138.394705] nix-grpc-daemon[1624]: time=2026-09-20T13:28:09.934Z level=INFO msg="Uploading 1 narinfos"6499lb # [ 138.381434] niks3-server[1013]: 2026/09/20 13:28:10 INFO Signed narinfos id=51 count=16500lb # [ 138.386964] niks3-server[1013]: 2026/09/20 13:28:10 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6501lb # [ 138.395685] niks3-server[1013]: 2026/09/20 13:28:10 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6502lb # [ 138.400194] niks3-server[1013]: 2026/09/20 13:28:10 INFO Completed upload id=516503worker1 # [ 138.412795] nix-grpc-daemon[1624]: time=2026-09-20T13:28:09.952Z level=WARN msg="Failed to register uploaded object" key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo error="server returned 404: object is not pending upload\n"6504worker1 # [ 138.422622] nix-grpc-daemon[1624]: time=2026-09-20T13:28:09.958Z level=INFO msg="Upload complete. (283ms)"6505worker1 # [ 138.432265] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16506worker1 # [ 138.436627] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:09Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16507worker1 # [ 138.440578] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406508worker1 # [ 138.444366] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:09Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06509worker1 # [ 138.452429] nix-daemon[906]: reaped child process 1896, status = succeeded6510worker1 # [ 138.455199] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406511worker1 # [ 138.463328] nix-grpc-daemon[1624]: time=2026-09-20T13:28:10.003Z level=INFO msg="All 1 paths already cached"6512worker1 # [ 138.467682] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406513worker1 # [ 138.471441] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6514worker1 # [ 138.630420] nix-daemon[906]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6515worker1 # [ 138.642199] nix-daemon[906]: reaped child process 1925, status = succeeded6516worker1 # [ 138.810503] nix-daemon[906]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6517worker1 # [ 138.822109] nix-daemon[906]: reaped child process 1930, status = succeeded6518lb # [ 138.827132] niks3-server[1013]: 2026/09/20 13:28:10 INFO Received uploads request method=POST path=/api/pending_closures6519worker1 # [ 138.847281] nix-grpc-daemon[1624]: time=2026-09-20T13:28:10.387Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6520worker1 # [ 138.848765] nix-grpc-daemon[1624]: time=2026-09-20T13:28:10.388Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6521lb # [ 138.855453] niks3-server[1013]: 2026/09/20 13:28:10 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6522lb # [ 138.867850] niks3-server[1013]: 2026/09/20 13:28:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6523worker1 # [ 138.884786] nix-grpc-daemon[1624]: time=2026-09-20T13:28:10.424Z level=INFO msg="Uploading 1 narinfos"6524lb # [ 138.872113] niks3-server[1013]: 2026/09/20 13:28:10 INFO Signed narinfos id=52 count=16525lb # [ 138.879980] niks3-server[1013]: 2026/09/20 13:28:10 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6526lb # [ 138.888745] niks3-server[1013]: 2026/09/20 13:28:10 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6527worker1 # [ 138.907208] nix-grpc-daemon[1624]: time=2026-09-20T13:28:10.447Z level=INFO msg="Upload complete. (429ms)"6528lb # [ 138.894454] niks3-server[1013]: 2026/09/20 13:28:10 INFO Completed upload id=526529worker1 # [ 138.909253] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=6246530worker1 # [ 138.911933] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406531lb # [ 138.899482] niks3-server[1013]: 2026/09/20 13:28:10 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6532worker1 # [ 138.920366] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6533worker1 # [ 138.924343] nix-daemon[906]: reaped child process 1920, status = succeeded6534worker1 # [ 138.929328] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6535worker1 # [ 139.941421] nix-daemon[906]: reaped child process 1934, status = succeeded6536worker1 # [ 140.086591] nix-daemon[906]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6537worker1 # [ 140.097290] nix-daemon[906]: reaped child process 1952, status = succeeded6538lb # [ 140.100919] niks3-server[1013]: 2026/09/20 13:28:12 INFO Received uploads request method=POST path=/api/pending_closures6539worker1 # [ 140.124098] nix-grpc-daemon[1624]: time=2026-09-20T13:28:11.663Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6540worker1 # [ 140.125669] nix-grpc-daemon[1624]: time=2026-09-20T13:28:11.665Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6541lb # [ 140.141693] niks3-server[1013]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6542lb # [ 140.150155] niks3-server[1013]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6543lb # [ 140.157282] niks3-server[1013]: 2026/09/20 13:28:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6544worker1 # [ 140.173564] nix-grpc-daemon[1624]: time=2026-09-20T13:28:11.713Z level=INFO msg="Uploading 1 narinfos"6545lb # [ 140.161094] niks3-server[1013]: 2026/09/20 13:28:12 INFO Signed narinfos id=53 count=16546lb # [ 140.165319] niks3-server[1013]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6547lb # [ 140.171323] niks3-server[1013]: 2026/09/20 13:28:12 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6548lb # [ 140.174059] niks3-server[1013]: 2026/09/20 13:28:12 INFO Completed upload id=536549worker1 # [ 140.186813] nix-grpc-daemon[1624]: time=2026-09-20T13:28:11.726Z level=WARN msg="Failed to register uploaded object" key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo error="server returned 404: object is not pending upload\n"6550worker1 # [ 140.192539] nix-grpc-daemon[1624]: time=2026-09-20T13:28:11.731Z level=INFO msg="Upload complete. (254ms)"6551worker1 # [ 140.202456] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16552worker1 # [ 140.206084] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:11Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16553worker1 # [ 140.208101] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406554worker1 # [ 140.209769] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:11Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06555worker1 # [ 140.215382] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406556worker1 # [ 140.217700] nix-daemon[906]: reaped child process 1932, status = succeeded6557worker1 # [ 140.221820] nix-grpc-daemon[1624]: time=2026-09-20T13:28:11.761Z level=INFO msg="All 1 paths already cached"6558worker1 # [ 140.225053] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406559worker1 # [ 140.229724] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6560worker1 # [ 140.382551] nix-daemon[906]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6561worker1 # [ 140.395438] nix-daemon[906]: reaped child process 1961, status = succeeded6562worker1 # [ 140.562540] nix-daemon[906]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6563worker1 # [ 140.574277] nix-daemon[906]: reaped child process 1966, status = succeeded6564lb # [ 140.578932] niks3-server[1013]: 2026/09/20 13:28:12 INFO Received uploads request method=POST path=/api/pending_closures6565worker1 # [ 140.600993] nix-grpc-daemon[1624]: time=2026-09-20T13:28:12.140Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6566worker1 # [ 140.602563] nix-grpc-daemon[1624]: time=2026-09-20T13:28:12.142Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6567lb # [ 140.607949] niks3-server[1013]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6568lb # [ 140.620806] niks3-server[1013]: 2026/09/20 13:28:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6569worker1 # [ 140.637340] nix-grpc-daemon[1624]: time=2026-09-20T13:28:12.177Z level=INFO msg="Uploading 1 narinfos"6570lb # [ 140.624910] niks3-server[1013]: 2026/09/20 13:28:12 INFO Signed narinfos id=54 count=16571lb # [ 140.629104] niks3-server[1013]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6572lb # [ 140.639304] niks3-server[1013]: 2026/09/20 13:28:12 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6573lb # [ 140.642893] niks3-server[1013]: 2026/09/20 13:28:12 INFO Completed upload id=546574worker1 # [ 140.655748] nix-grpc-daemon[1624]: time=2026-09-20T13:28:12.195Z level=INFO msg="Upload complete. (421ms)"6575lb # [ 140.645769] niks3-server[1013]: 2026/09/20 13:28:12 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6576worker1 # [ 140.657947] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=6246577worker1 # [ 140.662964] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406578worker1 # [ 140.671294] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6579worker1 # [ 140.675630] nix-daemon[906]: reaped child process 1956, status = succeeded6580worker1 # [ 140.680447] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6581worker1 # [ 141.894261] nix-daemon[906]: reaped child process 1970, status = succeeded6582worker1 # [ 142.042536] nix-daemon[906]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6583worker1 # [ 142.055329] nix-daemon[906]: reaped child process 1988, status = succeeded6584lb # [ 142.059421] niks3-server[1013]: 2026/09/20 13:28:14 INFO Received uploads request method=POST path=/api/pending_closures6585worker1 # [ 142.079926] nix-grpc-daemon[1624]: time=2026-09-20T13:28:13.619Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6586worker1 # [ 142.081420] nix-grpc-daemon[1624]: time=2026-09-20T13:28:13.621Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6587lb # [ 142.094791] niks3-server[1013]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6588lb # [ 142.104729] niks3-server[1013]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6589lb # [ 142.112846] niks3-server[1013]: 2026/09/20 13:28:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6590lb # [ 142.118360] niks3-server[1013]: 2026/09/20 13:28:14 INFO Signed narinfos id=55 count=16591worker1 # [ 142.138499] nix-grpc-daemon[1624]: time=2026-09-20T13:28:13.678Z level=INFO msg="Uploading 1 narinfos"6592lb # [ 142.137566] niks3-server[1013]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6593lb # [ 142.139160] niks3-server[1013]: 2026/09/20 13:28:14 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6594lb # [ 142.141842] niks3-server[1013]: 2026/09/20 13:28:14 INFO Completed upload id=556595worker1 # [ 142.154938] nix-grpc-daemon[1624]: time=2026-09-20T13:28:13.694Z level=INFO msg="Upload complete. (267ms)"6596lb # [ 142.144070] niks3-server[1013]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6597worker1 # [ 142.165835] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16598worker1 # [ 142.173678] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:13Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16599worker1 # [ 142.178459] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406600worker1 # [ 142.182210] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:13Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06601worker1 # [ 142.192328] nix-daemon[906]: reaped child process 1968, status = succeeded6602worker1 # [ 142.206655] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406603worker1 # [ 142.217574] nix-grpc-daemon[1624]: time=2026-09-20T13:28:13.757Z level=INFO msg="All 1 paths already cached"6604worker1 # [ 142.223713] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406605worker1 # [ 142.230547] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6606worker1 # [ 142.382340] nix-daemon[906]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6607worker1 # [ 142.395250] nix-daemon[906]: reaped child process 1997, status = succeeded6608worker1 # [ 142.560556] nix-daemon[906]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)6609worker1 # [ 142.572710] nix-daemon[906]: reaped child process 2002, status = succeeded6610lb # [ 142.576350] niks3-server[1013]: 2026/09/20 13:28:14 INFO Received uploads request method=POST path=/api/pending_closures6611worker1 # [ 142.597059] nix-grpc-daemon[1624]: time=2026-09-20T13:28:14.136Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6612worker1 # [ 142.598510] nix-grpc-daemon[1624]: time=2026-09-20T13:28:14.136Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6613lb # [ 142.601136] niks3-server[1013]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6614lb # [ 142.613218] niks3-server[1013]: 2026/09/20 13:28:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6615lb # [ 142.619749] niks3-server[1013]: 2026/09/20 13:28:14 INFO Signed narinfos id=56 count=16616worker1 # [ 142.635831] nix-grpc-daemon[1624]: time=2026-09-20T13:28:14.175Z level=INFO msg="Uploading 1 narinfos"6617lb # [ 142.624561] niks3-server[1013]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6618lb # [ 142.635065] niks3-server[1013]: 2026/09/20 13:28:14 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6619worker1 # [ 142.654854] nix-grpc-daemon[1624]: time=2026-09-20T13:28:14.194Z level=INFO msg="Upload complete. (415ms)"6620lb # [ 142.642307] niks3-server[1013]: 2026/09/20 13:28:14 INFO Completed upload id=566621worker1 # [ 142.657340] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=6246622lb # [ 142.647549] niks3-server[1013]: 2026/09/20 13:28:14 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6623worker1 # [ 142.664762] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406624worker1 # [ 142.673601] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6625worker1 # [ 142.676389] nix-daemon[906]: reaped child process 1992, status = succeeded6626worker1 # [ 142.681663] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6627worker1 # [ 144.306909] nix-daemon[906]: reaped child process 2006, status = succeeded6628worker1 # [ 144.470916] nix-daemon[906]: accepted connection from pid 2021, user nix-grpc-daemon (trusted)6629worker1 # [ 144.482653] nix-daemon[906]: reaped child process 2024, status = succeeded6630lb # [ 144.485805] niks3-server[1013]: 2026/09/20 13:28:16 INFO Received uploads request method=POST path=/api/pending_closures6631worker1 # [ 144.504408] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.044Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6632worker1 # [ 144.507159] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.045Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6633lb # [ 144.505890] niks3-server[1013]: 2026/09/20 13:28:16 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6634lb # [ 144.512880] niks3-server[1013]: 2026/09/20 13:28:16 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6635lb # [ 144.519915] niks3-server[1013]: 2026/09/20 13:28:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6636lb # [ 144.524616] niks3-server[1013]: 2026/09/20 13:28:16 INFO Signed narinfos id=57 count=16637worker1 # [ 144.538835] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.078Z level=INFO msg="Uploading 1 narinfos"6638lb # [ 144.527810] niks3-server[1013]: 2026/09/20 13:28:16 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6639lb # [ 144.532675] niks3-server[1013]: 2026/09/20 13:28:16 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6640lb # [ 144.536086] niks3-server[1013]: 2026/09/20 13:28:16 INFO Completed upload id=576641worker1 # [ 144.548720] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.088Z level=INFO msg="Upload complete. (259ms)"6642lb # [ 144.537539] niks3-server[1013]: 2026/09/20 13:28:16 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6643worker1 # [ 144.551144] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16644worker1 # [ 144.555938] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:16Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16645worker1 # [ 144.558784] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406646worker1 # [ 144.561933] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:16Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06647worker1 # [ 144.567690] nix-daemon[906]: reaped child process 2004, status = succeeded6648worker1 # [ 144.569445] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406649worker1 # [ 144.575761] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.115Z level=INFO msg="All 2 paths already cached"6650worker1 # [ 144.579407] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406651worker1 # [ 144.584529] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6652worker1 # [ 144.740546] nix-daemon[906]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)6653worker1 # [ 144.753698] nix-daemon[906]: reaped child process 2033, status = succeeded6654worker1 # [ 144.932692] nix-daemon[906]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)6655worker1 # [ 144.946406] nix-daemon[906]: reaped child process 2038, status = succeeded6656lb # [ 144.948720] niks3-server[1013]: 2026/09/20 13:28:17 INFO Received uploads request method=POST path=/api/pending_closures6657worker1 # [ 144.969383] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.509Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6658worker1 # [ 144.970866] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.510Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6659lb # [ 144.976828] niks3-server[1013]: 2026/09/20 13:28:17 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6660lb # [ 144.987865] niks3-server[1013]: 2026/09/20 13:28:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6661lb # [ 144.992930] niks3-server[1013]: 2026/09/20 13:28:17 INFO Signed narinfos id=58 count=16662worker1 # [ 145.008639] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.548Z level=INFO msg="Uploading 1 narinfos"6663lb # [ 144.999405] niks3-server[1013]: 2026/09/20 13:28:17 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6664lb # [ 145.011237] niks3-server[1013]: 2026/09/20 13:28:17 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6665worker1 # [ 145.030720] nix-grpc-daemon[1624]: time=2026-09-20T13:28:16.570Z level=INFO msg="Upload complete. (441ms)"6666lb # [ 145.017971] niks3-server[1013]: 2026/09/20 13:28:17 INFO Completed upload id=586667worker1 # [ 145.032776] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=7526668lb # [ 145.022069] niks3-server[1013]: 2026/09/20 13:28:17 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6669worker1 # [ 145.038866] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406670worker1 # [ 145.047733] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6671worker1 # [ 145.050041] nix-daemon[906]: reaped child process 2028, status = succeeded6672worker1 # [ 145.055435] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6673worker1 # [ 146.281805] nix-daemon[906]: reaped child process 2042, status = succeeded6674worker1 # [ 146.414246] nix-daemon[906]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)6675worker1 # [ 146.426274] nix-daemon[906]: reaped child process 2060, status = succeeded6676lb # [ 146.429652] niks3-server[1013]: 2026/09/20 13:28:18 INFO Received uploads request method=POST path=/api/pending_closures6677worker1 # [ 146.451203] nix-grpc-daemon[1624]: time=2026-09-20T13:28:17.990Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6678worker1 # [ 146.452912] nix-grpc-daemon[1624]: time=2026-09-20T13:28:17.992Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6679lb # [ 146.469284] niks3-server[1013]: 2026/09/20 13:28:18 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6680lb # [ 146.476733] niks3-server[1013]: 2026/09/20 13:28:18 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6681lb # [ 146.484066] niks3-server[1013]: 2026/09/20 13:28:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6682lb # [ 146.487069] niks3-server[1013]: 2026/09/20 13:28:18 INFO Signed narinfos id=59 count=16683worker1 # [ 146.500026] nix-grpc-daemon[1624]: time=2026-09-20T13:28:18.039Z level=INFO msg="Uploading 1 narinfos"6684lb # [ 146.490502] niks3-server[1013]: 2026/09/20 13:28:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6685lb # [ 146.495247] niks3-server[1013]: 2026/09/20 13:28:18 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6686lb # [ 146.498481] niks3-server[1013]: 2026/09/20 13:28:18 INFO Completed upload id=596687lb # [ 146.500179] niks3-server[1013]: 2026/09/20 13:28:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6688worker1 # [ 146.512649] nix-grpc-daemon[1624]: time=2026-09-20T13:28:18.052Z level=INFO msg="Upload complete. (249ms)"6689worker1 # [ 146.518861] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16690worker1 # [ 146.524215] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:18Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16691worker1 # [ 146.532848] nix-daemon[906]: reaped child process 2040, status = succeeded6692worker1 # [ 146.561325] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406693worker1 # [ 146.572931] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406694worker1 # [ 146.585703] nix-grpc-daemon[1624]: time=2026-09-20T13:28:18.125Z level=INFO msg="All 2 paths already cached"6695worker1 # [ 146.593496] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406696worker1 # [ 146.602366] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6697worker1 # [ 146.759734] nix-daemon[906]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)6698worker1 # [ 146.774984] nix-daemon[906]: reaped child process 2067, status = succeeded6699worker1 # [ 146.937433] nix-daemon[906]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6700worker1 # [ 146.952349] nix-daemon[906]: reaped child process 2072, status = succeeded6701lb # [ 146.954143] niks3-server[1013]: 2026/09/20 13:28:19 INFO Received uploads request method=POST path=/api/pending_closures6702worker1 # [ 146.974382] nix-grpc-daemon[1624]: time=2026-09-20T13:28:18.514Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6703worker1 # [ 146.975969] nix-grpc-daemon[1624]: time=2026-09-20T13:28:18.515Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6704lb # [ 146.981590] niks3-server[1013]: 2026/09/20 13:28:19 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6705lb # [ 146.993815] niks3-server[1013]: 2026/09/20 13:28:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6706worker1 # [ 147.011512] nix-grpc-daemon[1624]: time=2026-09-20T13:28:18.551Z level=INFO msg="Uploading 1 narinfos"6707lb # [ 146.999067] niks3-server[1013]: 2026/09/20 13:28:19 INFO Signed narinfos id=60 count=16708lb # [ 147.005521] niks3-server[1013]: 2026/09/20 13:28:19 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6709lb # [ 147.014858] niks3-server[1013]: 2026/09/20 13:28:19 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6710worker1 # [ 147.034538] nix-grpc-daemon[1624]: time=2026-09-20T13:28:18.574Z level=INFO msg="Upload complete. (420ms)"6711lb # [ 147.021968] niks3-server[1013]: 2026/09/20 13:28:19 INFO Completed upload id=606712worker1 # [ 147.036780] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=7526713lb # [ 147.025785] niks3-server[1013]: 2026/09/20 13:28:19 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6714worker1 # [ 147.042586] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406715worker1 # [ 147.048655] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6716worker1 # [ 147.051134] nix-daemon[906]: reaped child process 2062, status = succeeded6717worker1 # [ 147.056715] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6718worker1 # [ 148.481508] nix-daemon[906]: reaped child process 2076, status = succeeded6719worker1 # [ 148.652831] nix-daemon[906]: accepted connection from pid 2091, user nix-grpc-daemon (trusted)6720worker1 # [ 148.665880] nix-daemon[906]: reaped child process 2094, status = succeeded6721lb # [ 148.668745] niks3-server[1013]: 2026/09/20 13:28:20 INFO Received uploads request method=POST path=/api/pending_closures6722worker1 # [ 148.691029] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.230Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6723worker1 # [ 148.692690] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.232Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6724lb # [ 148.704137] niks3-server[1013]: 2026/09/20 13:28:20 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6725lb # [ 148.710821] niks3-server[1013]: 2026/09/20 13:28:20 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6726lb # [ 148.725127] niks3-server[1013]: 2026/09/20 13:28:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6727worker1 # [ 148.742497] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.282Z level=INFO msg="Uploading 1 narinfos"6728lb # [ 148.729448] niks3-server[1013]: 2026/09/20 13:28:20 INFO Signed narinfos id=61 count=16729lb # [ 148.738174] niks3-server[1013]: 2026/09/20 13:28:20 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6730lb # [ 148.744927] niks3-server[1013]: 2026/09/20 13:28:20 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6731lb # [ 148.750320] niks3-server[1013]: 2026/09/20 13:28:20 INFO Completed upload id=616732worker1 # [ 148.763048] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.302Z level=INFO msg="Upload complete. (297ms)"6733lb # [ 148.752960] niks3-server[1013]: 2026/09/20 13:28:20 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6734worker1 # [ 148.772210] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16735worker1 # [ 148.777634] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:20Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16736worker1 # [ 148.788221] nix-daemon[906]: reaped child process 2074, status = succeeded6737worker1 # [ 148.813593] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406738worker1 # [ 148.825389] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406739worker1 # [ 148.837092] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.376Z level=INFO msg="All 2 paths already cached"6740worker1 # [ 148.843736] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566406741worker1 # [ 148.852456] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6742worker1 # [ 149.032328] nix-daemon[906]: accepted connection from pid 2098, user nix-grpc-daemon (trusted)6743worker1 # [ 149.048086] nix-daemon[906]: reaped child process 2101, status = succeeded6744worker1 # [ 149.225256] nix-daemon[906]: accepted connection from pid 2103, user nix-grpc-daemon (trusted)6745worker1 # [ 149.240061] nix-daemon[906]: reaped child process 2106, status = succeeded6746lb # [ 149.242808] niks3-server[1013]: 2026/09/20 13:28:21 INFO Received uploads request method=POST path=/api/pending_closures6747worker1 # [ 149.266304] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.806Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6748worker1 # [ 149.268349] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.807Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6749lb # [ 149.266132] niks3-server[1013]: 2026/09/20 13:28:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6750lb # [ 149.267660] niks3-server[1013]: 2026/09/20 13:28:21 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6751lb # [ 149.270196] niks3-server[1013]: 2026/09/20 13:28:21 INFO Signed narinfos id=62 count=16752worker1 # [ 149.283307] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.823Z level=INFO msg="Uploading 1 narinfos"6753lb # [ 149.272380] niks3-server[1013]: 2026/09/20 13:28:21 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6754lb # [ 149.277336] niks3-server[1013]: 2026/09/20 13:28:21 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6755lb # [ 149.281221] niks3-server[1013]: 2026/09/20 13:28:21 INFO Completed upload id=626756worker1 # [ 149.293977] nix-grpc-daemon[1624]: time=2026-09-20T13:28:20.833Z level=INFO msg="Upload complete. (429ms)"6757lb # [ 149.282467] niks3-server[1013]: 2026/09/20 13:28:21 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6758worker1 # [ 149.296239] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=0 paths=1 nar_bytes_in=7526759worker1 # [ 149.299615] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566406760worker1 # [ 149.306592] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6761worker1 # [ 149.309848] nix-daemon[906]: reaped child process 2096, status = succeeded6762worker1 # [ 149.315444] nix-daemon[906]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)6763worker1 # [ 150.733333] nix-daemon[906]: reaped child process 2110, status = succeeded6764worker1 # [ 150.879480] nix-daemon[906]: accepted connection from pid 2125, user nix-grpc-daemon (trusted)6765worker1 # [ 150.894670] nix-daemon[906]: reaped child process 2128, status = succeeded6766lb # [ 150.899584] niks3-server[1013]: 2026/09/20 13:28:22 INFO Received uploads request method=POST path=/api/pending_closures6767worker1 # [ 150.922187] nix-grpc-daemon[1624]: time=2026-09-20T13:28:22.461Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6768worker1 # [ 150.923789] nix-grpc-daemon[1624]: time=2026-09-20T13:28:22.463Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6769lb # [ 150.939454] niks3-server[1013]: 2026/09/20 13:28:23 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6770lb # [ 150.947850] niks3-server[1013]: 2026/09/20 13:28:23 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6771lb # [ 150.951859] niks3-server[1013]: 2026/09/20 13:28:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6772lb # [ 150.955833] niks3-server[1013]: 2026/09/20 13:28:23 INFO Signed narinfos id=63 count=16773worker1 # [ 150.971186] nix-grpc-daemon[1624]: time=2026-09-20T13:28:22.510Z level=INFO msg="Uploading 1 narinfos"6774lb # [ 150.963246] niks3-server[1013]: 2026/09/20 13:28:23 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6775lb # [ 150.967788] niks3-server[1013]: 2026/09/20 13:28:23 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6776lb # [ 150.970219] niks3-server[1013]: 2026/09/20 13:28:23 INFO Completed upload id=636777worker1 # [ 150.983108] nix-grpc-daemon[1624]: time=2026-09-20T13:28:22.522Z level=INFO msg="Upload complete. (258ms)"6778lb # [ 150.972101] niks3-server[1013]: 2026/09/20 13:28:23 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6779worker1 # [ 150.987077] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56640 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16780worker1 # [ 150.995435] nix-daemon[906]: reaped child process 2108, status = succeeded6781client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.12 seconds)6782worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6783worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6784client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'6785client # [ 151.153934] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6786client # Running as unit: gpu.service; invocation ID: 8aabb5b6b9eb46298d9dacc9ea6b943b6787client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.07 seconds)6788lb: must succeed: curl -sf localhost:9901/clusters6789lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6790lb: must succeed: curl -sf localhost:9901/clusters6791lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6792worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6793worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6794worker2 # [ 151.388789] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578926795client # [ 151.368039] nix[1349]: don't know how to build these paths:6796client # [ 151.369552] nix[1349]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6797worker1 # [ 151.387349] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406798worker1 # [ 151.399336] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:22Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06799client # [ 151.383539] nix[1349]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6800lb: must succeed: curl -sf localhost:9901/clusters6801lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6802lb: must succeed: curl -sf localhost:9901/clusters6803lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6804worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6805worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6806client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6807lb # [ 157.989444] envoy[922]: conn peer=[2001:db8:1::1]:33912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27167 tx=10635 ms=178646808client # [ 181.384669] nix[1349]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6809client # [ 181.393513] nix[1349]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6810client # [ 181.426330] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6811client # [ 181.429752] systemd[1]: gpu.service: Failed with result 'exit-code'.6812client # [ 181.434315] systemd[1]: gpu.service: Consumed 129ms CPU time over 30.279s wall clock time, 33.4M memory peak, 6.6K incoming IP traffic, 5.8K outgoing IP traffic.6813client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.57 seconds)6814client: must succeed: journalctl -u gpu -o cat6815client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6816lb: must succeed: curl -sf localhost:9901/clusters6817lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6818lb: must succeed: curl -sf localhost:9901/clusters6819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6820worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6821worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6822(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.38 seconds)6823subtest: low disk drains a worker and builds go to the other6824worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6825worker1 # [ 183.084512] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:54Z level=info event=unhealthy reason=min_free6826worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.92 seconds)6827worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6828worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds)6829client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26830client # don't know how to build these paths:6831client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6832worker2 # [ 183.457482] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:578926833worker1 # [ 183.450784] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566406834worker1 # [ 183.469358] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:55Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06835worker1 # [ 183.473514] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:55Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16836worker2 # [ 183.523745] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578926837worker2 # [ 183.535379] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)6838worker2 # [ 183.593262] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 160 ms (attempt 2/5)6839worker2 # [ 183.759137] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 3/5)6840worker2 # [ 183.785556] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 596 ms (attempt 4/5)6841worker2 # [ 184.389244] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6842worker2 # [ 184.404680] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578926843client # copying 1 paths to 'lb:50051'...6844client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6845worker2 # [ 184.416656] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578926846worker2 # [ 184.428306] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6847worker2 # [ 184.447125] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:56Z level=info event=niks3_push_started pid=17926848worker2 # [ 184.605552] nix-grpc-daemon[1792]: time=2026-09-20T13:28:56.501Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6849worker2 # [ 184.753851] nix-daemon[866]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)6850worker2 # [ 184.767780] nix-daemon[866]: reaped child process 1808, status = succeeded6851worker2 # [ 184.928278] nix-daemon[866]: accepted connection from pid 1810, user nix-grpc-daemon (trusted)6852worker2 # [ 184.942271] nix-daemon[866]: reaped child process 1813, status = succeeded6853lb # [ 184.932070] niks3-server[1013]: 2026/09/20 13:28:57 INFO Received uploads request method=POST path=/api/pending_closures6854worker2 # [ 184.963074] nix-grpc-daemon[1792]: time=2026-09-20T13:28:56.859Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6855worker2 # [ 184.964454] nix-grpc-daemon[1792]: time=2026-09-20T13:28:56.859Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6856lb # [ 184.955655] niks3-server[1013]: 2026/09/20 13:28:57 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6857lb # [ 184.962447] niks3-server[1013]: 2026/09/20 13:28:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6858lb # [ 184.965737] niks3-server[1013]: 2026/09/20 13:28:57 INFO Signed narinfos id=64 count=16859worker2 # [ 184.988944] nix-grpc-daemon[1792]: time=2026-09-20T13:28:56.885Z level=INFO msg="Uploading 1 narinfos"6860lb # [ 184.973071] niks3-server[1013]: 2026/09/20 13:28:57 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6861lb # [ 184.976806] niks3-server[1013]: 2026/09/20 13:28:57 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6862lb # [ 184.982228] niks3-server[1013]: 2026/09/20 13:28:57 INFO Completed upload id=646863worker2 # [ 185.005669] nix-grpc-daemon[1792]: time=2026-09-20T13:28:56.902Z level=INFO msg="Upload complete. (400ms)"6864worker2 # [ 185.009098] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57892 duration_s=0 paths=1 nar_bytes_in=5126865lb # [ 184.985923] niks3-server[1013]: 2026/09/20 13:28:57 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6866worker2 # [ 185.012518] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578926867worker2 # [ 185.019210] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6868worker2 # [ 185.021604] nix-daemon[866]: reaped child process 1790, status = succeeded6869worker2 # [ 185.026872] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6870client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6871client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6872client # farm-a> LOG-farm-a-drain6873worker2 # [ 186.055515] nix-daemon[866]: reaped child process 1817, status = succeeded6874worker2 # [ 186.208199] nix-daemon[866]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)6875worker2 # [ 186.220529] nix-daemon[866]: reaped child process 1835, status = succeeded6876lb # [ 186.212603] niks3-server[1013]: 2026/09/20 13:28:58 INFO Received uploads request method=POST path=/api/pending_closures6877worker2 # [ 186.246233] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.143Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6878worker2 # [ 186.250632] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.144Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6879lb # [ 186.250144] niks3-server[1013]: 2026/09/20 13:28:58 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6880lb # [ 186.261265] niks3-server[1013]: 2026/09/20 13:28:58 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6881lb # [ 186.272549] niks3-server[1013]: 2026/09/20 13:28:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6882lb # [ 186.276440] niks3-server[1013]: 2026/09/20 13:28:58 INFO Signed narinfos id=65 count=16883worker2 # [ 186.302612] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.199Z level=INFO msg="Uploading 1 narinfos"6884lb # [ 186.281076] niks3-server[1013]: 2026/09/20 13:28:58 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6885lb # [ 186.286491] niks3-server[1013]: 2026/09/20 13:28:58 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6886lb # [ 186.290090] niks3-server[1013]: 2026/09/20 13:28:58 INFO Completed upload id=656887worker2 # [ 186.313786] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.210Z level=INFO msg="Upload complete. (279ms)"6888lb # [ 186.292821] niks3-server[1013]: 2026/09/20 13:28:58 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6889worker2 # [ 186.318955] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57892 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16890worker1 # [ 186.315890] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:57Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06891worker2 # [ 186.331540] nix-daemon[866]: reaped child process 1815, status = succeeded6892worker1 # [ 186.319977] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:57Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16893worker2 # [ 186.369744] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578926894worker2 # [ 186.380285] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578926895worker2 # [ 186.391643] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.288Z level=INFO msg="All 1 paths already cached"6896client # copying 1 paths to 'lb:50051'...6897client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6898worker2 # [ 186.399610] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578926899worker2 # [ 186.408320] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6900worker2 # [ 186.560551] nix-daemon[866]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6901worker2 # [ 186.572995] nix-daemon[866]: reaped child process 1842, status = succeeded6902worker2 # [ 186.712151] nix-daemon[866]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6903worker2 # [ 186.724438] nix-daemon[866]: reaped child process 1847, status = succeeded6904lb # [ 186.712938] niks3-server[1013]: 2026/09/20 13:28:58 INFO Received uploads request method=POST path=/api/pending_closures6905worker2 # [ 186.744221] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.641Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6906worker2 # [ 186.745700] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.642Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6907lb # [ 186.743747] niks3-server[1013]: 2026/09/20 13:28:58 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6908lb # [ 186.752903] niks3-server[1013]: 2026/09/20 13:28:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6909lb # [ 186.757558] niks3-server[1013]: 2026/09/20 13:28:58 INFO Signed narinfos id=66 count=16910worker2 # [ 186.783661] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.680Z level=INFO msg="Uploading 1 narinfos"6911lb # [ 186.763741] niks3-server[1013]: 2026/09/20 13:28:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6912lb # [ 186.775386] niks3-server[1013]: 2026/09/20 13:28:58 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6913lb # [ 186.781984] niks3-server[1013]: 2026/09/20 13:28:58 INFO Completed upload id=666914worker2 # [ 186.805312] nix-grpc-daemon[1792]: time=2026-09-20T13:28:58.702Z level=INFO msg="Upload complete. (384ms)"6915lb # [ 186.784368] niks3-server[1013]: 2026/09/20 13:28:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6916worker2 # [ 186.808992] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57892 duration_s=0 paths=1 nar_bytes_in=6246917worker2 # [ 186.813837] nix-grpc-daemon[1736]: ts=2026-09-20T13:28:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578926918worker2 # [ 186.820842] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6919worker2 # [ 186.826804] nix-daemon[866]: reaped child process 1837, status = succeeded6920client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6921client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6922worker2 # [ 186.831085] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6923client # farm-b> LOG-farm-b-drain6924lb # [ 187.991321] envoy[922]: conn peer=[2001:db8:1::1]:44782 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=494 ms=300446925worker2 # [ 188.059800] nix-daemon[866]: reaped child process 1851, status = succeeded6926worker2 # [ 188.214478] nix-daemon[866]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)6927worker2 # [ 188.225875] nix-daemon[866]: reaped child process 1869, status = succeeded6928lb # [ 188.217395] niks3-server[1013]: 2026/09/20 13:29:00 INFO Received uploads request method=POST path=/api/pending_closures6929worker2 # [ 188.249090] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.146Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6930worker2 # [ 188.250566] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.147Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6931lb # [ 188.252974] niks3-server[1013]: 2026/09/20 13:29:00 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6932lb # [ 188.259212] niks3-server[1013]: 2026/09/20 13:29:00 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6933lb # [ 188.271535] niks3-server[1013]: 2026/09/20 13:29:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6934lb # [ 188.277886] niks3-server[1013]: 2026/09/20 13:29:00 INFO Signed narinfos id=67 count=16935worker2 # [ 188.303941] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.200Z level=INFO msg="Uploading 1 narinfos"6936lb # [ 188.280722] niks3-server[1013]: 2026/09/20 13:29:00 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6937lb # [ 188.292252] niks3-server[1013]: 2026/09/20 13:29:00 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6938lb # [ 188.299041] niks3-server[1013]: 2026/09/20 13:29:00 INFO Completed upload id=676939lb # [ 188.300780] niks3-server[1013]: 2026/09/20 13:29:00 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6940worker2 # [ 188.322622] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.219Z level=INFO msg="Upload complete. (283ms)"6941worker2 # [ 188.329339] nix-grpc-daemon[1736]: ts=2026-09-20T13:29:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57892 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16942worker1 # [ 188.324955] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:59Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06943worker1 # [ 188.329192] nix-grpc-daemon[1536]: ts=2026-09-20T13:28:59Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16944worker2 # [ 188.343198] nix-daemon[866]: reaped child process 1849, status = succeeded6945worker2 # [ 188.379166] nix-grpc-daemon[1736]: ts=2026-09-20T13:29:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578926946worker2 # [ 188.391423] nix-grpc-daemon[1736]: ts=2026-09-20T13:29:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:578926947worker2 # [ 188.403605] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.300Z level=INFO msg="All 2 paths already cached"6948client # copying 1 paths to 'lb:50051'...6949client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6950worker2 # [ 188.411577] nix-grpc-daemon[1736]: ts=2026-09-20T13:29:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:578926951worker2 # [ 188.421166] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6952worker2 # [ 188.576358] nix-daemon[866]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6953worker2 # [ 188.589242] nix-daemon[866]: reaped child process 1876, status = succeeded6954worker2 # [ 188.738291] nix-daemon[866]: accepted connection from pid 1878, user nix-grpc-daemon (trusted)6955worker2 # [ 188.751167] nix-daemon[866]: reaped child process 1881, status = succeeded6956lb # [ 188.740990] niks3-server[1013]: 2026/09/20 13:29:00 INFO Received uploads request method=POST path=/api/pending_closures6957worker2 # [ 188.771909] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.668Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6958worker2 # [ 188.773290] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.668Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6959lb # [ 188.767602] niks3-server[1013]: 2026/09/20 13:29:00 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6960lb # [ 188.776397] niks3-server[1013]: 2026/09/20 13:29:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6961worker2 # [ 188.804714] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.701Z level=INFO msg="Uploading 1 narinfos"6962lb # [ 188.781503] niks3-server[1013]: 2026/09/20 13:29:00 INFO Signed narinfos id=68 count=16963lb # [ 188.786648] niks3-server[1013]: 2026/09/20 13:29:00 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6964lb # [ 188.798297] niks3-server[1013]: 2026/09/20 13:29:00 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6965worker2 # [ 188.827737] nix-grpc-daemon[1792]: time=2026-09-20T13:29:00.724Z level=INFO msg="Upload complete. (394ms)"6966lb # [ 188.804344] niks3-server[1013]: 2026/09/20 13:29:00 INFO Completed upload id=686967worker2 # [ 188.829933] nix-grpc-daemon[1736]: ts=2026-09-20T13:29:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57892 duration_s=0 paths=1 nar_bytes_in=7526968lb # [ 188.807504] niks3-server[1013]: 2026/09/20 13:29:00 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6969worker2 # [ 188.834801] nix-grpc-daemon[1736]: ts=2026-09-20T13:29:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:578926970worker2 # [ 188.841279] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6971worker2 # [ 188.846428] nix-daemon[866]: reaped child process 1871, status = succeeded6972client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6973worker2 # [ 188.852162] nix-daemon[866]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6974client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6975client # farm-top> LOG-farm-top-drain6976worker2 # [ 189.479592] nix-daemon[866]: reaped child process 1885, status = succeeded6977worker2 # [ 189.629135] nix-daemon[866]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)6978worker2 # [ 189.641254] nix-daemon[866]: reaped child process 1903, status = succeeded6979lb # [ 189.632924] niks3-server[1013]: 2026/09/20 13:29:01 INFO Received uploads request method=POST path=/api/pending_closures6980worker2 # [ 189.664697] nix-grpc-daemon[1792]: time=2026-09-20T13:29:01.561Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6981worker2 # [ 189.666206] nix-grpc-daemon[1792]: time=2026-09-20T13:29:01.563Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6982lb # [ 189.668145] niks3-server[1013]: 2026/09/20 13:29:01 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6983lb # [ 189.678943] niks3-server[1013]: 2026/09/20 13:29:01 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6984lb # [ 189.687801] niks3-server[1013]: 2026/09/20 13:29:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6985lb # [ 189.691773] niks3-server[1013]: 2026/09/20 13:29:01 INFO Signed narinfos id=69 count=16986worker2 # [ 189.718303] nix-grpc-daemon[1792]: time=2026-09-20T13:29:01.615Z level=INFO msg="Uploading 1 narinfos"6987lb # [ 189.696483] niks3-server[1013]: 2026/09/20 13:29:01 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6988lb # [ 189.705555] niks3-server[1013]: 2026/09/20 13:29:01 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6989lb # [ 189.709046] niks3-server[1013]: 2026/09/20 13:29:01 INFO Completed upload id=696990lb # [ 189.710778] niks3-server[1013]: 2026/09/20 13:29:01 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6991worker2 # [ 189.733865] nix-grpc-daemon[1792]: time=2026-09-20T13:29:01.630Z level=INFO msg="Upload complete. (274ms)"6992worker2 # [ 189.739645] nix-grpc-daemon[1736]: ts=2026-09-20T13:29:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57892 duration_s=0 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16993worker2 # [ 189.752395] nix-daemon[866]: reaped child process 1883, status = succeeded6994client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.50 seconds)6995client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6996client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.18 seconds)6997worker1: must succeed: rm /nix/.rw-store/fill6998worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)6999worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7000worker1 # [ 191.088393] nix-grpc-daemon[1536]: ts=2026-09-20T13:29:02Z level=info event=healthy reason=min_free7001worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)7002(finished: subtest: low disk drains a worker and builds go to the other, in 8.94 seconds)7003(finished: run the VM test script, in 192.42 seconds)7004test script finished in 192.66s7005cleanup7006kill QemuMachine (pid 52)7007client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7008kill QemuMachine (pid 50)7009lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7010kill QemuMachine (pid 49)7011worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7012kill QemuMachine (pid 51)7013worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7014(finished: cleanup, in 0.94 seconds)7015additionally exposed symbols:7016 client, lb, worker1, worker2,7017 vlan1,7018 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh