vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #308
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1: QEMU running (pid 51)13client # Disk image does not exist, creating the virtualisation disk image...14worker1 # Disk image does not exist, creating the virtualisation disk image...15client # Formatting '/build/vm-state-client/tmp.wbZtuS4V3g', fmt=raw size=107374182416worker1 # Formatting '/build/vm-state-worker1/tmp.KqMSh0VmPl', fmt=raw size=107374182417client # mke2fs 1.47.4 (6-Mar-2025)18worker1 # mke2fs 1.47.4 (6-Mar-2025)19client # Discarding device blocks: 0/262144 done20worker1 # Discarding device blocks: 0/262144 done21client # Creating filesystem with 262144 4k blocks and 65536 inodes22worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes23client # Filesystem UUID: 97b49305-9ae6-4b20-8fc3-9079b82544bc24worker1 # Filesystem UUID: 13c772ce-064d-40c1-b2ef-2e710803a95325client # Superblock backups stored on blocks:26worker1 # Superblock backups stored on blocks:27client # 32768, 98304, 163840, 22937628worker1 # 32768, 98304, 163840, 22937629client # 30worker1 # 31client # Allocating group tables: 0/8 done32worker1 # Allocating group tables: 0/8 done33client # Writing inode tables: 0/8 done34worker1 # Writing inode tables: 0/8 done35client # Creating journal (8192 blocks): done36worker1 # Creating journal (8192 blocks): done37client # Writing superblocks and filesystem accounting information: 0/8 done38worker1 # Writing superblocks and filesystem accounting information: 0/8 done39client # 40worker1 # 41client # Virtualisation disk image created.42worker1 # Virtualisation disk image created.43lb # Disk image does not exist, creating the virtualisation disk image...44lb: QEMU running (pid 49)45lb # Formatting '/build/vm-state-lb/tmp.S2O7YZbYcA', fmt=raw size=107374182446client: QEMU running (pid 48)47lb # mke2fs 1.47.4 (6-Mar-2025)48lb # Discarding device blocks: 0/262144 done49lb # Creating filesystem with 262144 4k blocks and 65536 inodes50lb # Filesystem UUID: c51adeee-92d5-463f-b33b-0567d77adb6151lb # Superblock backups stored on blocks:52lb # 32768, 98304, 163840, 22937653lb # 54lb # Allocating group tables: 0/8 done55lb # Writing inode tables: 0/8 done56lb # Creating journal (8192 blocks): done57lb # Writing superblocks and filesystem accounting information: 0/8 done58lb # 59lb # Virtualisation disk image created.60worker2: QEMU running (pid 52)61worker2 # Disk image does not exist, creating the virtualisation disk image...62worker2 # Formatting '/build/vm-state-worker2/tmp.VWVo7SKU6w', fmt=raw size=107374182463(finished: start all VMs, in 0.32 seconds)64worker2 # mke2fs 1.47.4 (6-Mar-2025)65client: waiting for unit network-addresses-eth1.service66worker2 # Discarding device blocks: 0/262144 done67client: waiting for the VM to finish booting68worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes69worker2 # Filesystem UUID: e62c9af9-6a1f-4d3c-9a06-1d057a6c8f2270worker2 # Superblock backups stored on blocks:71worker2 # 32768, 98304, 163840, 22937672worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker1 # 84worker1 # 85worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086lb # 87lb # 88worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...89worker1 # 90lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0091worker1 # 92worker1 # 93worker1 # 94worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0095lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...96lb # 97lb # 98lb # 99lb # 100worker2 # 101worker2 # 102lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00103worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00104worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105worker1 # 106worker1 # 107worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108worker2 # 109worker2 # 110worker2 # 111worker2 # 112lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...113lb # 114lb # 115worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117worker2 # 118worker2 # 119client # 120client # 121client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123client # 124client # 125client # 126client # 127client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129client # 130client # 131worker1 # Booting from ROM...132lb # Booting from ROM...133worker2 # Booting from ROM...134client # Booting from ROM...135lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026137lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0138lb # [ 0.000000] BIOS-provided physical RAM map:139lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable140worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/22mkqaxiafgbafaik2ihxp37g0lk30n2-nixos-system-worker1-test/init regInfo=/nix/store/3kzmr37iw9w01p23x75f20i38c2j1zw7-closure-info/registration console=ttyS0,115200n8 console=tty0141worker1 # [ 0.000000] BIOS-provided physical RAM map:142lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved143worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable144lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved145worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved146lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable147lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved148worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved149worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026150lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved151worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable152lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved153worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved154lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved155worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved156lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved157worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved158lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved159worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jir2qfj8vkn5wjzhcf27xbm6m2miphhh-nixos-system-worker2-test/init regInfo=/nix/store/h5kl2w2kcawlhgm49r82hqfc61bxy0v0-closure-info/registration console=ttyS0,115200n8 console=tty0160worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved161lb # [ 0.000000] NX (Execute Disable) protection: active162worker2 # [ 0.000000] BIOS-provided physical RAM map:163lb # [ 0.000000] APIC: Static calls initialized164worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved165worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable166lb # [ 0.000000] SMBIOS 2.8 present.167worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved168worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved169worker1 # [ 0.000000] NX (Execute Disable) protection: active170lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014171worker1 # [ 0.000000] APIC: Static calls initialized172worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved173lb # [ 0.000000] DMI: Memory slots populated: 1/1174worker1 # [ 0.000000] SMBIOS 2.8 present.175lb # [ 0.000000] Hypervisor detected: KVM176worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable177lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000178worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014179worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved180lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00181worker1 # [ 0.000000] DMI: Memory slots populated: 1/1182worker1 # [ 0.000000] Hypervisor detected: KVM183lb # [ 0.000000] kvm-clock: using sched offset of 483480103 cycles184worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved185worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000186worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved187worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00188lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns189worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved190lb # [ 0.000006] tsc: Detected 2400.012 MHz processor191worker1 # [ 0.000001] kvm-clock: using sched offset of 468896075 cycles192lb # [ 0.000847] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000193worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved194client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026195worker1 # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns196lb # [ 0.000877] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs197worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved198worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor199worker2 # [ 0.000000] NX (Execute Disable) protection: active200lb # [ 0.000880] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT201worker2 # [ 0.000000] APIC: Static calls initialized202worker1 # [ 0.000830] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000203lb # [ 0.002864] found SMP MP-table at [mem 0x000f5450-0x000f545f]204worker2 # [ 0.000000] SMBIOS 2.8 present.205lb # [ 0.002879] Using GB pages for direct mapping206worker1 # [ 0.000857] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs207lb # [ 0.003014] RAMDISK: [mem 0x5e367000-0x5ffcffff]208worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014209worker1 # [ 0.000860] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT210worker2 # [ 0.000000] DMI: Memory slots populated: 1/1211lb # [ 0.003023] ACPI: Early table checksum verification disabled212client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test/init regInfo=/nix/store/m6q6h80vyi40rq32z35r3qi0wpyx4p2s-closure-info/registration console=ttyS0,115200n8 console=tty0213worker2 # [ 0.000000] Hypervisor detected: KVM214client # [ 0.000000] BIOS-provided physical RAM map:215worker1 # [ 0.002812] found SMP MP-table at [mem 0x000f5450-0x000f545f]216lb # [ 0.003027] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )217worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000218worker1 # [ 0.002824] Using GB pages for direct mapping219client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable220worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00221worker1 # [ 0.002897] RAMDISK: [mem 0x5e367000-0x5ffcffff]222lb # [ 0.003031] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)223client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved224worker2 # [ 0.000000] kvm-clock: using sched offset of 490921706 cycles225worker1 # [ 0.002905] ACPI: Early table checksum verification disabled226lb # [ 0.003036] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)227client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved228worker1 # [ 0.002908] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )229lb # [ 0.003044] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)230worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns231client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable232lb # [ 0.003046] ACPI: FACS 0x000000005FFE0000 000040233worker1 # [ 0.002912] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)234worker2 # [ 0.000006] tsc: Detected 2400.012 MHz processor235client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved236worker2 # [ 0.000852] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000237lb # [ 0.003048] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)238worker1 # [ 0.002917] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)239client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved240worker2 # [ 0.000890] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs241lb # [ 0.003049] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)242client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved243worker1 # [ 0.002925] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)244worker2 # [ 0.000892] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT245worker1 # [ 0.002927] ACPI: FACS 0x000000005FFE0000 000040246client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved247lb # [ 0.003051] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)248worker2 # [ 0.002907] found SMP MP-table at [mem 0x000f5450-0x000f545f]249client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved250worker2 # [ 0.002921] Using GB pages for direct mapping251worker1 # [ 0.002929] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)252lb # [ 0.003052] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)253worker2 # [ 0.003003] RAMDISK: [mem 0x5e367000-0x5ffcffff]254client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved255lb # [ 0.003054] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]256worker2 # [ 0.003011] ACPI: Early table checksum verification disabled257worker1 # [ 0.002930] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)258client # [ 0.000000] NX (Execute Disable) protection: active259client # [ 0.000000] APIC: Static calls initialized260worker2 # [ 0.003016] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )261lb # [ 0.003055] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]262client # [ 0.000000] SMBIOS 2.8 present.263worker1 # [ 0.002932] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)264lb # [ 0.003055] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]265worker2 # [ 0.003022] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)266worker1 # [ 0.002933] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)267client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014268lb # [ 0.003056] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]269worker2 # [ 0.003027] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)270client # [ 0.000000] DMI: Memory slots populated: 1/1271worker1 # [ 0.002935] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]272lb # [ 0.003056] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]273client # [ 0.000000] Hypervisor detected: KVM274worker2 # [ 0.003035] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)275client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000276worker1 # [ 0.002936] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]277lb # [ 0.003057] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]278worker2 # [ 0.003038] ACPI: FACS 0x000000005FFE0000 000040279client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00280lb # [ 0.003057] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]281worker1 # [ 0.002936] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]282client # [ 0.000000] kvm-clock: using sched offset of 510281610 cycles283lb # [ 0.003296] No NUMA configuration found284worker2 # [ 0.003039] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)285worker1 # [ 0.002937] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]286lb # [ 0.003297] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]287worker2 # [ 0.003041] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)288client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns289worker1 # [ 0.002937] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]290lb # [ 0.003300] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]291client # [ 0.000006] tsc: Detected 2400.012 MHz processor292lb # [ 0.005527] Zone ranges:293worker2 # [ 0.003043] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)294worker1 # [ 0.002938] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]295client # [ 0.000854] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000296lb # [ 0.005529] DMA [mem 0x0000000000001000-0x0000000000ffffff]297worker1 # [ 0.002938] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]298worker2 # [ 0.003044] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)299lb # [ 0.005530] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]300client # [ 0.000883] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs301lb # [ 0.005532] Normal empty302worker1 # [ 0.003176] No NUMA configuration found303lb # [ 0.005532] Device empty304worker2 # [ 0.003046] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]305client # [ 0.000886] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT306lb # [ 0.005533] Movable zone start for each node307worker1 # [ 0.003177] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]308lb # [ 0.005534] Early memory node ranges309worker2 # [ 0.003047] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]310client # [ 0.002938] found SMP MP-table at [mem 0x000f5450-0x000f545f]311worker1 # [ 0.003179] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]312client # [ 0.002954] Using GB pages for direct mapping313worker1 # [ 0.005461] Zone ranges:314lb # [ 0.005534] node 0: [mem 0x0000000000001000-0x000000000009efff]315worker2 # [ 0.003047] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]316client # [ 0.003061] RAMDISK: [mem 0x5e366000-0x5ffcffff]317worker1 # [ 0.005462] DMA [mem 0x0000000000001000-0x0000000000ffffff]318lb # [ 0.005535] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]319worker2 # [ 0.003048] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]320client # [ 0.003070] ACPI: Early table checksum verification disabled321worker1 # [ 0.005465] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]322lb # [ 0.005537] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]323client # [ 0.003074] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )324worker1 # [ 0.005466] Normal empty325worker2 # [ 0.003048] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]326worker1 # [ 0.005466] Device empty327lb # [ 0.005556] On node 0, zone DMA: 1 pages in unavailable ranges328client # [ 0.003079] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)329worker2 # [ 0.003049] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]330worker1 # [ 0.005467] Movable zone start for each node331lb # [ 0.005821] On node 0, zone DMA: 97 pages in unavailable ranges332worker1 # [ 0.005468] Early memory node ranges333worker2 # [ 0.003049] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]334lb # [ 0.033055] On node 0, zone DMA32: 40 pages in unavailable ranges335client # [ 0.003085] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)336worker1 # [ 0.005468] node 0: [mem 0x0000000000001000-0x000000000009efff]337worker2 # [ 0.003288] No NUMA configuration found338lb # [ 0.033554] ACPI: PM-Timer IO Port: 0x608339worker1 # [ 0.005469] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]340client # [ 0.003092] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)341worker2 # [ 0.003289] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]342lb # [ 0.033570] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])343client # [ 0.003094] ACPI: FACS 0x000000005FFE0000 000040344worker1 # [ 0.005470] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]345worker2 # [ 0.003291] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]346lb # [ 0.033603] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23347worker2 # [ 0.005532] Zone ranges:348worker1 # [ 0.005490] On node 0, zone DMA: 1 pages in unavailable ranges349client # [ 0.003096] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)350lb # [ 0.033606] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)351worker2 # [ 0.005533] DMA [mem 0x0000000000001000-0x0000000000ffffff]352worker1 # [ 0.005755] On node 0, zone DMA: 97 pages in unavailable ranges353lb # [ 0.033608] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)354client # [ 0.003098] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)355worker2 # [ 0.005535] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]356worker1 # [ 0.032761] On node 0, zone DMA32: 40 pages in unavailable ranges357worker2 # [ 0.005536] Normal empty358worker2 # [ 0.005537] Device empty359lb # [ 0.033609] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)360worker1 # [ 0.033264] ACPI: PM-Timer IO Port: 0x608361client # [ 0.003099] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)362worker2 # [ 0.005538] Movable zone start for each node363worker1 # [ 0.033280] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])364lb # [ 0.033611] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)365worker2 # [ 0.005539] Early memory node ranges366client # [ 0.003101] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)367lb # [ 0.033611] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)368worker1 # [ 0.033318] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23369worker2 # [ 0.005539] node 0: [mem 0x0000000000001000-0x000000000009efff]370client # [ 0.003102] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]371worker2 # [ 0.005540] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]372worker1 # [ 0.033321] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)373lb # [ 0.033614] ACPI: Using ACPI (MADT) for SMP configuration information374client # [ 0.003103] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]375lb # [ 0.033615] ACPI: HPET id: 0x8086a201 base: 0xfed00000376worker2 # [ 0.005541] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]377worker1 # [ 0.033323] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)378lb # [ 0.033621] TSC deadline timer available379client # [ 0.003104] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]380worker2 # [ 0.005563] On node 0, zone DMA: 1 pages in unavailable ranges381worker1 # [ 0.033325] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)382lb # [ 0.033625] CPU topo: Max. logical packages: 1383lb # [ 0.033626] CPU topo: Max. logical dies: 1384worker2 # [ 0.005828] On node 0, zone DMA: 97 pages in unavailable ranges385client # [ 0.003104] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]386worker1 # [ 0.033326] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)387lb # [ 0.033626] CPU topo: Max. dies per package: 1388worker2 # [ 0.033886] On node 0, zone DMA32: 40 pages in unavailable ranges389client # [ 0.003105] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]390worker1 # [ 0.033327] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)391lb # [ 0.033632] CPU topo: Max. threads per core: 1392worker2 # [ 0.034471] ACPI: PM-Timer IO Port: 0x608393lb # [ 0.033633] CPU topo: Num. cores per package: 1394client # [ 0.003105] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]395worker1 # [ 0.033330] ACPI: Using ACPI (MADT) for SMP configuration information396worker2 # [ 0.034487] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])397lb # [ 0.033633] CPU topo: Num. threads per package: 1398worker1 # [ 0.033331] ACPI: HPET id: 0x8086a201 base: 0xfed00000399client # [ 0.003106] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]400worker2 # [ 0.034518] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23401worker1 # [ 0.033336] TSC deadline timer available402client # [ 0.003353] No NUMA configuration found403lb # [ 0.033634] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs404worker2 # [ 0.034521] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)405worker1 # [ 0.033341] CPU topo: Max. logical packages: 1406client # [ 0.003354] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]407worker1 # [ 0.033342] CPU topo: Max. logical dies: 1408lb # [ 0.033657] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()409client # [ 0.003356] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]410worker1 # [ 0.033342] CPU topo: Max. dies per package: 1411worker2 # [ 0.034523] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)412client # [ 0.005628] Zone ranges:413lb # [ 0.033693] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]414worker1 # [ 0.033348] CPU topo: Max. threads per core: 1415worker2 # [ 0.034524] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)416client # [ 0.005630] DMA [mem 0x0000000000001000-0x0000000000ffffff]417lb # [ 0.033695] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]418worker1 # [ 0.033349] CPU topo: Num. cores per package: 1419worker2 # [ 0.034526] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)420client # [ 0.005632] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]421worker1 # [ 0.033349] CPU topo: Num. threads per package: 1422lb # [ 0.033697] [mem 0x60000000-0xafffffff] available for PCI devices423client # [ 0.005633] Normal empty424worker2 # [ 0.034526] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)425lb # [ 0.033698] Booting paravirtualized kernel on KVM426client # [ 0.005634] Device empty427worker1 # [ 0.033349] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs428client # [ 0.005635] Movable zone start for each node429worker2 # [ 0.034529] ACPI: Using ACPI (MADT) for SMP configuration information430worker1 # [ 0.033374] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()431client # [ 0.005636] Early memory node ranges432lb # [ 0.033703] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns433worker2 # [ 0.034530] ACPI: HPET id: 0x8086a201 base: 0xfed00000434worker2 # [ 0.034537] TSC deadline timer available435client # [ 0.005636] node 0: [mem 0x0000000000001000-0x000000000009efff]436worker1 # [ 0.033413] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]437lb # [ 0.038662] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1438worker2 # [ 0.034541] CPU topo: Max. logical packages: 1439client # [ 0.005637] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]440worker1 # [ 0.033415] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]441lb # [ 0.041068] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152442worker2 # [ 0.034542] CPU topo: Max. logical dies: 1443worker1 # [ 0.033416] [mem 0x60000000-0xafffffff] available for PCI devices444client # [ 0.005639] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]445worker2 # [ 0.034543] CPU topo: Max. dies per package: 1446lb # [ 0.041130] kvm-guest: PV spinlocks disabled, single CPU447worker1 # [ 0.033418] Booting paravirtualized kernel on KVM448worker2 # [ 0.034547] CPU topo: Max. threads per core: 1449client # [ 0.005659] On node 0, zone DMA: 1 pages in unavailable ranges450worker2 # [ 0.034548] CPU topo: Num. cores per package: 1451client # [ 0.005935] On node 0, zone DMA: 97 pages in unavailable ranges452worker1 # [ 0.033422] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns453worker2 # [ 0.034548] CPU topo: Num. threads per package: 1454client # [ 0.034877] On node 0, zone DMA32: 40 pages in unavailable ranges455worker2 # [ 0.034548] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs456client # [ 0.035411] ACPI: PM-Timer IO Port: 0x608457worker1 # [ 0.038039] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1458client # [ 0.035426] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])459worker2 # [ 0.034573] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()460worker1 # [ 0.040355] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152461lb # [ 0.041132] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0462client # [ 0.035461] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23463worker2 # [ 0.034614] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]464worker1 # [ 0.040404] kvm-guest: PV spinlocks disabled, single CPU465client # [ 0.035465] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)466worker2 # [ 0.034616] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]467lb # [ 0.041249] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.468client # [ 0.035467] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)469worker2 # [ 0.034617] [mem 0x60000000-0xafffffff] available for PCI devices470lb # [ 0.041263] random: crng init done471worker2 # [ 0.034619] Booting paravirtualized kernel on KVM472client # [ 0.035468] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)473lb # [ 0.041264] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes474client # [ 0.035469] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)475worker2 # [ 0.034623] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns476lb # [ 0.043504] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)477worker1 # [ 0.040406] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/22mkqaxiafgbafaik2ihxp37g0lk30n2-nixos-system-worker1-test/init regInfo=/nix/store/3kzmr37iw9w01p23x75f20i38c2j1zw7-closure-info/registration console=ttyS0,115200n8 console=tty0478client # [ 0.035470] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)479worker2 # [ 0.039480] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1480lb # [ 0.045025] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)481client # [ 0.035473] ACPI: Using ACPI (MADT) for SMP configuration information482lb # [ 0.045074] Fallback order for Node 0: 0483worker2 # [ 0.041909] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152484client # [ 0.035474] ACPI: HPET id: 0x8086a201 base: 0xfed00000485worker1 # [ 0.040510] Unknown kernel command line parameters "regInfo=/nix/store/3kzmr37iw9w01p23x75f20i38c2j1zw7-closure-info/registration", will be passed to user space.486lb # [ 0.045078] Built 1 zonelists, mobility grouping on. Total pages: 393078487client # [ 0.035480] TSC deadline timer available488worker2 # [ 0.041963] kvm-guest: PV spinlocks disabled, single CPU489worker1 # [ 0.040526] random: crng init done490lb # [ 0.045080] Policy zone: DMA32491client # [ 0.035485] CPU topo: Max. logical packages: 1492client # [ 0.035486] CPU topo: Max. logical dies: 1493worker1 # [ 0.040526] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes494lb # [ 0.048303] mem auto-init: stack:all(zero), heap alloc:on, heap free:off495client # [ 0.035486] CPU topo: Max. dies per package: 1496lb # [ 0.051716] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1497worker1 # [ 0.042793] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)498client # [ 0.035490] CPU topo: Max. threads per core: 1499lb # [ 0.055187] allocated 3145728 bytes of page_ext500client # [ 0.035490] CPU topo: Num. cores per package: 1501worker1 # [ 0.044205] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)502lb # [ 0.066233] ftrace: allocating 48731 entries in 192 pages503client # [ 0.035491] CPU topo: Num. threads per package: 1504worker1 # [ 0.044245] Fallback order for Node 0: 0505lb # [ 0.066238] ftrace: allocated 192 pages with 2 groups506worker2 # [ 0.041965] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/jir2qfj8vkn5wjzhcf27xbm6m2miphhh-nixos-system-worker2-test/init regInfo=/nix/store/h5kl2w2kcawlhgm49r82hqfc61bxy0v0-closure-info/registration console=ttyS0,115200n8 console=tty0507lb # [ 0.067203] Dynamic Preempt: lazy508worker1 # [ 0.044249] Built 1 zonelists, mobility grouping on. Total pages: 393078509client # [ 0.035491] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs510worker1 # [ 0.044250] Policy zone: DMA32511lb # [ 0.067350] rcu: Preemptible hierarchical RCU implementation.512client # [ 0.035520] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()513lb # [ 0.067350] rcu: RCU event tracing is enabled.514worker1 # [ 0.046835] mem auto-init: stack:all(zero), heap alloc:on, heap free:off515worker2 # [ 0.042070] Unknown kernel command line parameters "regInfo=/nix/store/h5kl2w2kcawlhgm49r82hqfc61bxy0v0-closure-info/registration", will be passed to user space.516client # [ 0.035567] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]517worker2 # [ 0.042084] random: crng init done518worker1 # [ 0.050499] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1519lb # [ 0.067351] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.520worker1 # [ 0.054094] allocated 3145728 bytes of page_ext521client # [ 0.035569] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]522lb # [ 0.067353] Trampoline variant of Tasks RCU enabled.523worker2 # [ 0.042085] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes524lb # [ 0.067353] Rude variant of Tasks RCU enabled.525worker1 # [ 0.065172] ftrace: allocating 48731 entries in 192 pages526client # [ 0.035572] [mem 0x60000000-0xafffffff] available for PCI devices527worker1 # [ 0.065174] ftrace: allocated 192 pages with 2 groups528lb # [ 0.067354] Tracing variant of Tasks RCU enabled.529worker2 # [ 0.044533] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)530client # [ 0.035573] Booting paravirtualized kernel on KVM531worker1 # [ 0.066200] Dynamic Preempt: lazy532lb # [ 0.067354] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.533worker2 # [ 0.045630] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)534worker1 # [ 0.066389] rcu: Preemptible hierarchical RCU implementation.535client # [ 0.035578] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns536worker2 # [ 0.045670] Fallback order for Node 0: 0537worker1 # [ 0.066390] rcu: RCU event tracing is enabled.538lb # [ 0.067355] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1539worker2 # [ 0.045674] Built 1 zonelists, mobility grouping on. Total pages: 393078540client # [ 0.040584] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1541worker1 # [ 0.066391] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.542worker2 # [ 0.045675] Policy zone: DMA32543lb # [ 0.067372] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.544worker1 # [ 0.066393] Trampoline variant of Tasks RCU enabled.545client # [ 0.042987] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152546worker2 # [ 0.048658] mem auto-init: stack:all(zero), heap alloc:on, heap free:off547worker1 # [ 0.066394] Rude variant of Tasks RCU enabled.548client # [ 0.043043] kvm-guest: PV spinlocks disabled, single CPU549lb # [ 0.067374] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.550worker1 # [ 0.066394] Tracing variant of Tasks RCU enabled.551worker2 # [ 0.052211] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1552worker2 # [ 0.056025] allocated 3145728 bytes of page_ext553worker1 # [ 0.066395] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.554lb # [ 0.067375] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.555worker2 # [ 0.066778] ftrace: allocating 48731 entries in 192 pages556lb # [ 0.072780] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16557worker1 # [ 0.066396] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1558worker2 # [ 0.066781] ftrace: allocated 192 pages with 2 groups559lb # [ 0.073081] rcu: srcu_init: Setting srcu_struct sizes based on contention.560worker2 # [ 0.067729] Dynamic Preempt: lazy561worker1 # [ 0.066416] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.562worker2 # [ 0.067945] rcu: Preemptible hierarchical RCU implementation.563client # [ 0.043045] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test/init regInfo=/nix/store/m6q6h80vyi40rq32z35r3qi0wpyx4p2s-closure-info/registration console=ttyS0,115200n8 console=tty0564lb # [ 0.073089] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns565worker2 # [ 0.067946] rcu: RCU event tracing is enabled.566worker1 # [ 0.066418] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.567worker2 # [ 0.067946] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.568lb # [ 0.073206] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)569worker2 # [ 0.067948] Trampoline variant of Tasks RCU enabled.570worker1 # [ 0.066419] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.571client # [ 0.043150] Unknown kernel command line parameters "regInfo=/nix/store/m6q6h80vyi40rq32z35r3qi0wpyx4p2s-closure-info/registration", will be passed to user space.572lb # [ 0.077175] Console: colour VGA+ 80x25573client # [ 0.043165] random: crng init done574worker2 # [ 0.067948] Rude variant of Tasks RCU enabled.575lb # [ 0.077179] printk: legacy console [tty0] enabled576worker1 # [ 0.072036] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16577worker2 # [ 0.067949] Tracing variant of Tasks RCU enabled.578lb # [ 0.108650] printk: legacy console [ttyS0] enabled579client # [ 0.043166] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes580worker1 # [ 0.072327] rcu: srcu_init: Setting srcu_struct sizes based on contention.581worker2 # [ 0.067950] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.582lb # [ 0.208613] ACPI: Core revision 20250807583client # [ 0.045442] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)584worker1 # [ 0.072333] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns585worker2 # [ 0.067950] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1586client # [ 0.046704] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)587client # [ 0.046749] Fallback order for Node 0: 0588lb # [ 0.209630] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns589worker2 # [ 0.067977] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.590worker1 # [ 0.072452] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)591worker1 # [ 0.076481] Console: colour VGA+ 80x25592client # [ 0.046753] Built 1 zonelists, mobility grouping on. Total pages: 393078593lb # [ 0.211276] APIC: Switch to symmetric I/O mode setup594client # [ 0.046755] Policy zone: DMA32595worker2 # [ 0.067979] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.596worker1 # [ 0.076486] printk: legacy console [tty0] enabled597lb # [ 0.212321] x2apic enabled598worker1 # [ 0.112260] printk: legacy console [ttyS0] enabled599client # [ 0.049364] mem auto-init: stack:all(zero), heap alloc:on, heap free:off600worker2 # [ 0.067980] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.601worker1 # [ 0.215921] ACPI: Core revision 20250807602client # [ 0.052788] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1603lb # [ 0.213104] APIC: Switched APIC routing to: physical x2apic604worker2 # [ 0.072885] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16605client # [ 0.056669] allocated 3145728 bytes of page_ext606worker2 # [ 0.073203] rcu: srcu_init: Setting srcu_struct sizes based on contention.607client # [ 0.067574] ftrace: allocating 48731 entries in 192 pages608worker1 # [ 0.216869] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns609client # [ 0.067577] ftrace: allocated 192 pages with 2 groups610lb # [ 0.215037] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1611worker2 # [ 0.073211] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns612client # [ 0.068508] Dynamic Preempt: lazy613worker1 # [ 0.218530] APIC: Switch to symmetric I/O mode setup614client # [ 0.068656] rcu: Preemptible hierarchical RCU implementation.615worker1 # [ 0.219541] x2apic enabled616worker2 # [ 0.073332] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)617client # [ 0.068657] rcu: RCU event tracing is enabled.618lb # [ 0.216080] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns619worker2 # [ 0.077227] Console: colour VGA+ 80x25620worker1 # [ 0.220308] APIC: Switched APIC routing to: physical x2apic621client # [ 0.068658] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.622worker2 # [ 0.077231] printk: legacy console [tty0] enabled623client # [ 0.068659] Trampoline variant of Tasks RCU enabled.624worker2 # [ 0.108827] printk: legacy console [ttyS0] enabled625lb # [ 0.217826] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)626client # [ 0.068660] Rude variant of Tasks RCU enabled.627worker2 # [ 0.210241] ACPI: Core revision 20250807628client # [ 0.068660] Tracing variant of Tasks RCU enabled.629worker1 # [ 0.222235] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1630lb # [ 0.218915] x86/cpu: User Mode Instruction Prevention (UMIP) activated631client # [ 0.068661] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.632lb # [ 0.219955] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127633worker2 # [ 0.211139] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns634worker1 # [ 0.223243] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns635client # [ 0.068662] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1636worker2 # [ 0.212723] APIC: Switch to symmetric I/O mode setup637lb # [ 0.220823] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0638worker2 # [ 0.213681] x2apic enabled639client # [ 0.068712] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.640worker1 # [ 0.224944] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)641client # [ 0.068713] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.642lb # [ 0.221828] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto643worker2 # [ 0.214454] APIC: Switched APIC routing to: physical x2apic644worker1 # [ 0.226030] x86/cpu: User Mode Instruction Prevention (UMIP) activated645client # [ 0.068715] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.646worker1 # [ 0.228008] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127647lb # [ 0.222823] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl648worker2 # [ 0.216277] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1649client # [ 0.073466] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16650worker1 # [ 0.228940] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0651lb # [ 0.223823] Transient Scheduler Attacks: Vulnerable: No microcode652client # [ 0.073756] rcu: srcu_init: Setting srcu_struct sizes based on contention.653lb # [ 0.224823] Spectre V2 : Mitigation: Enhanced / Automatic IBRS654worker2 # [ 0.217246] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns655client # [ 0.073764] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns656worker1 # [ 0.229944] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto657lb # [ 0.225823] Speculative Return Stack Overflow: Mitigation: Safe RET658worker2 # [ 0.218895] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)659client # [ 0.073884] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)660worker1 # [ 0.230941] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl661client # [ 0.077561] Console: colour VGA+ 80x25662worker2 # [ 0.219980] x86/cpu: User Mode Instruction Prevention (UMIP) activated663client # [ 0.077564] printk: legacy console [tty0] enabled664worker1 # [ 0.232940] Transient Scheduler Attacks: Vulnerable: No microcode665client # [ 0.108643] printk: legacy console [ttyS0] enabled666worker2 # [ 0.221063] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127667worker1 # [ 0.233940] Spectre V2 : Mitigation: Enhanced / Automatic IBRS668client # [ 0.209179] ACPI: Core revision 20250807669worker2 # [ 0.221891] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0670worker1 # [ 0.234940] Speculative Return Stack Overflow: Mitigation: Safe RET671lb # [ 0.226823] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization672lb # [ 0.228831] Spectre V2 : Enabling IBPB for BPF673client # [ 0.210084] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns674worker2 # [ 0.222895] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto675client # [ 0.211714] APIC: Switch to symmetric I/O mode setup676lb # [ 0.229823] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier677client # [ 0.212704] x2apic enabled678worker2 # [ 0.224891] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl679lb # [ 0.230824] active return thunk: srso_alias_return_thunk680worker2 # [ 0.225891] Transient Scheduler Attacks: Vulnerable: No microcode681client # [ 0.213501] APIC: Switched APIC routing to: physical x2apic682worker1 # [ 0.235940] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization683lb # [ 0.232506] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'684worker2 # [ 0.226891] Spectre V2 : Mitigation: Enhanced / Automatic IBRS685worker1 # [ 0.237949] Spectre V2 : Enabling IBPB for BPF686lb # [ 0.233757] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'687worker2 # [ 0.227891] Speculative Return Stack Overflow: Mitigation: Safe RET688client # [ 0.215455] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1689worker1 # [ 0.238721] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier690lb # [ 0.234607] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'691worker1 # [ 0.239941] active return thunk: srso_alias_return_thunk692client # [ 0.216453] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns693lb # [ 0.235823] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'694worker1 # [ 0.240962] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'695lb # [ 0.236822] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'696client # [ 0.218193] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)697worker1 # [ 0.241940] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'698worker2 # [ 0.228891] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization699lb # [ 0.237822] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'700client # [ 0.219281] x86/cpu: User Mode Instruction Prevention (UMIP) activated701worker2 # [ 0.230900] Spectre V2 : Enabling IBPB for BPF702worker1 # [ 0.242940] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'703lb # [ 0.238823] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'704client # [ 0.221317] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127705worker1 # [ 0.243940] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'706worker2 # [ 0.231892] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier707lb # [ 0.239823] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256708client # [ 0.222191] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0709worker1 # [ 0.244940] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'710worker2 # [ 0.232892] active return thunk: srso_alias_return_thunk711lb # [ 0.240823] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64712worker1 # [ 0.245940] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'713worker2 # [ 0.234572] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'714client # [ 0.223195] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto715lb # [ 0.241823] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512716worker1 # [ 0.246940] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'717worker2 # [ 0.235891] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'718lb # [ 0.242822] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024719client # [ 0.224190] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl720worker1 # [ 0.248940] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256721worker2 # [ 0.236891] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'722lb # [ 0.243823] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8723client # [ 0.226190] Transient Scheduler Attacks: Vulnerable: No microcode724worker1 # [ 0.249940] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64725worker2 # [ 0.237891] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'726client # [ 0.227190] Spectre V2 : Mitigation: Enhanced / Automatic IBRS727lb # [ 0.244823] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.728worker1 # [ 0.250940] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512729worker2 # [ 0.238891] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'730client # [ 0.228190] Speculative Return Stack Overflow: Mitigation: Safe RET731worker1 # [ 0.251940] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024732worker2 # [ 0.239891] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'733worker1 # [ 0.252940] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8734worker2 # [ 0.240891] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'735worker2 # [ 0.241893] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256736worker1 # [ 0.253940] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.737worker2 # [ 0.242891] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64738client # [ 0.229190] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization739worker2 # [ 0.243891] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512740client # [ 0.231199] Spectre V2 : Enabling IBPB for BPF741worker2 # [ 0.244891] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024742client # [ 0.231993] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier743worker2 # [ 0.246655] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8744client # [ 0.233191] active return thunk: srso_alias_return_thunk745worker2 # [ 0.247644] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.746client # [ 0.234213] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'747client # [ 0.235190] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748client # [ 0.236190] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749client # [ 0.237190] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750client # [ 0.239022] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751client # [ 0.240016] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752client # [ 0.241190] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753client # [ 0.242190] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754client # [ 0.243190] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755client # [ 0.244190] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756client # [ 0.245190] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757client # [ 0.246190] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758client # [ 0.247190] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759lb # [ 0.283368] Freeing SMP alternatives memory: 44K760lb # [ 0.283828] pid_max: default: 32768 minimum: 301761lb # [ 0.284917] LSM: initializing lsm=capability,landlock,yama,bpf,ima762lb # [ 0.285937] landlock: Up and running.763worker1 # [ 0.291166] Freeing SMP alternatives memory: 44K764lb # [ 0.286823] Yama: becoming mindful.765worker1 # [ 0.291944] pid_max: default: 32768 minimum: 301766lb # [ 0.288014] LSM support for eBPF active767worker1 # [ 0.292960] LSM: initializing lsm=capability,landlock,yama,bpf,ima768lb # [ 0.288814] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)769worker1 # [ 0.294064] landlock: Up and running.770worker1 # [ 0.294734] Yama: becoming mindful.771lb # [ 0.289865] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)772worker1 # [ 0.295638] LSM support for eBPF active773worker1 # [ 0.296073] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)774lb # [ 0.292625] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)775worker2 # [ 0.286149] Freeing SMP alternatives memory: 44K776worker1 # [ 0.297941] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)777worker2 # [ 0.286896] pid_max: default: 32768 minimum: 301778lb # [ 0.293434] Performance Events: Fam17h+ core perfctr, AMD PMU driver.779lb # [ 0.293829] ... version: 2780worker2 # [ 0.288013] LSM: initializing lsm=capability,landlock,yama,bpf,ima781worker1 # [ 0.300354] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)782lb # [ 0.294581] ... bit width: 48783worker2 # [ 0.289049] landlock: Up and running.784lb # [ 0.294826] ... generic counters: 6785worker2 # [ 0.289893] Yama: becoming mindful.786worker1 # [ 0.301569] Performance Events: Fam17h+ core perfctr, AMD PMU driver.787lb # [ 0.295642] ... generic bitmap: 000000000000003f788worker2 # [ 0.291150] LSM support for eBPF active789worker1 # [ 0.301951] ... version: 2790lb # [ 0.296626] ... fixed-purpose counters: 0791worker1 # [ 0.302750] ... bit width: 48792worker2 # [ 0.292036] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)793lb # [ 0.297403] ... fixed-purpose bitmap: 0000000000000000794worker1 # [ 0.303552] ... generic counters: 6795lb # [ 0.297825] ... value mask: 0000ffffffffffff796worker2 # [ 0.292928] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)797worker1 # [ 0.303943] ... generic bitmap: 000000000000003f798lb # [ 0.298785] ... max period: 00007fffffffffff799worker1 # [ 0.304906] ... fixed-purpose counters: 0800lb # [ 0.299569] ... global_ctrl mask: 000000000000003f801worker1 # [ 0.305522] ... fixed-purpose bitmap: 0000000000000000802worker2 # [ 0.295778] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)803lb # [ 0.299965] signal: max sigframe size: 3376804worker1 # [ 0.305943] ... value mask: 0000ffffffffffff805lb # [ 0.300837] rcu: Hierarchical SRCU implementation.806worker1 # [ 0.306926] ... max period: 00007fffffffffff807worker2 # [ 0.296429] Performance Events: Fam17h+ core perfctr, AMD PMU driver.808lb # [ 0.301694] rcu: Max phase no-delay instances is 400.809worker1 # [ 0.307686] ... global_ctrl mask: 000000000000003f810worker2 # [ 0.296896] ... version: 2811worker2 # [ 0.297630] ... bit width: 48812worker1 # [ 0.308067] signal: max sigframe size: 3376813worker2 # [ 0.297894] ... generic counters: 6814client # [ 0.286628] Freeing SMP alternatives memory: 44K815worker1 # [ 0.308962] rcu: Hierarchical SRCU implementation.816client # [ 0.287194] pid_max: default: 32768 minimum: 301817worker2 # [ 0.298643] ... generic bitmap: 000000000000003f818worker1 # [ 0.309794] rcu: Max phase no-delay instances is 400.819worker2 # [ 0.298941] ... fixed-purpose counters: 0820client # [ 0.288287] LSM: initializing lsm=capability,landlock,yama,bpf,ima821worker2 # [ 0.299682] ... fixed-purpose bitmap: 0000000000000000822client # [ 0.289305] landlock: Up and running.823client # [ 0.290191] Yama: becoming mindful.824worker2 # [ 0.300582] ... value mask: 0000ffffffffffff825lb # [ 0.307428] smp: Bringing up secondary CPUs ...826client # [ 0.291250] LSM support for eBPF active827worker2 # [ 0.300894] ... max period: 00007fffffffffff828lb # [ 0.307841] smp: Brought up 1 node, 1 CPU829worker2 # [ 0.301803] ... global_ctrl mask: 000000000000003f830lb # [ 0.308533] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)831client # [ 0.292045] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)832worker2 # [ 0.302719] signal: max sigframe size: 3376833worker1 # [ 0.315582] smp: Bringing up secondary CPUs ...834worker2 # [ 0.303523] rcu: Hierarchical SRCU implementation.835client # [ 0.293231] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)836worker1 # [ 0.315960] smp: Brought up 1 node, 1 CPU837worker2 # [ 0.303897] rcu: Max phase no-delay instances is 400.838lb # [ 0.309000] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)839worker1 # [ 0.316675] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)840lb # [ 0.310103] devtmpfs: initialized841client # [ 0.295752] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)842lb # [ 0.311022] x86/mm: Memory block size: 128MB843worker1 # [ 0.317111] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)844client # [ 0.296732] Performance Events: Fam17h+ core perfctr, AMD PMU driver.845worker1 # [ 0.318223] devtmpfs: initialized846client # [ 0.297196] ... version: 2847lb # [ 0.312927] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)848client # [ 0.297928] ... bit width: 48849worker1 # [ 0.319091] x86/mm: Memory block size: 128MB850worker2 # [ 0.309675] smp: Bringing up secondary CPUs ...851client # [ 0.298193] ... generic counters: 6852lb # [ 0.313857] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).853worker2 # [ 0.310499] smp: Brought up 1 node, 1 CPU854client # [ 0.298968] ... generic bitmap: 000000000000003f855lb # [ 0.314921] pinctrl core: initialized pinctrl subsystem856worker1 # [ 0.321022] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)857client # [ 0.299205] ... fixed-purpose counters: 0858worker2 # [ 0.310896] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)859lb # [ 0.316126] PM: RTC time: 12:19:55, date: 2026-09-20860client # [ 0.299932] ... fixed-purpose bitmap: 0000000000000000861worker1 # [ 0.321972] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).862client # [ 0.300193] ... value mask: 0000ffffffffffff863worker1 # [ 0.323054] pinctrl core: initialized pinctrl subsystem864worker2 # [ 0.312174] Memory: 1454548K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)865client # [ 0.301139] ... max period: 00007fffffffffff866worker2 # [ 0.313172] devtmpfs: initialized867worker1 # [ 0.324212] PM: RTC time: 12:19:55, date: 2026-09-20868client # [ 0.301935] ... global_ctrl mask: 000000000000003f869worker2 # [ 0.314016] x86/mm: Memory block size: 128MB870client # [ 0.302319] signal: max sigframe size: 3376871lb # [ 0.319677] NET: Registered PF_NETLINK/PF_ROUTE protocol family872client # [ 0.303237] rcu: Hierarchical SRCU implementation.873lb # [ 0.321167] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations874client # [ 0.304109] rcu: Max phase no-delay instances is 400.875worker2 # [ 0.315925] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)876worker1 # [ 0.327923] NET: Registered PF_NETLINK/PF_ROUTE protocol family877lb # [ 0.321846] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations878worker2 # [ 0.316925] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).879worker2 # [ 0.318000] pinctrl core: initialized pinctrl subsystem880worker1 # [ 0.329275] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations881lb # [ 0.323118] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations882worker2 # [ 0.319177] PM: RTC time: 12:19:55, date: 2026-09-20883lb # [ 0.323840] audit: initializing netlink subsys (disabled)884worker1 # [ 0.329968] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations885lb # [ 0.325114] thermal_sys: Registered thermal governor 'fair_share'886worker1 # [ 0.331236] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations887client # [ 0.309828] smp: Bringing up secondary CPUs ...888lb # [ 0.325116] thermal_sys: Registered thermal governor 'bang_bang'889worker1 # [ 0.331958] audit: initializing netlink subsys (disabled)890client # [ 0.310212] smp: Brought up 1 node, 1 CPU891worker2 # [ 0.322665] NET: Registered PF_NETLINK/PF_ROUTE protocol family892client # [ 0.310944] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)893worker1 # [ 0.333165] thermal_sys: Registered thermal governor 'fair_share'894lb # [ 0.325830] audit: type=2000 audit(1789906796.251:1): state=initialized audit_enabled=0 res=1895worker1 # [ 0.333167] thermal_sys: Registered thermal governor 'bang_bang'896lb # [ 0.327829] thermal_sys: Registered thermal governor 'step_wise'897worker2 # [ 0.324233] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations898worker1 # [ 0.333945] thermal_sys: Registered thermal governor 'step_wise'899lb # [ 0.327831] thermal_sys: Registered thermal governor 'user_space'900client # [ 0.312269] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)901worker2 # [ 0.324913] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations902worker1 # [ 0.334922] thermal_sys: Registered thermal governor 'user_space'903client # [ 0.313450] devtmpfs: initialized904lb # [ 0.328826] thermal_sys: Registered thermal governor 'power_allocator'905lb # [ 0.329849] cpuidle: using governor menu906client # [ 0.314297] x86/mm: Memory block size: 128MB907worker2 # [ 0.326176] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations908worker1 # [ 0.335713] audit: type=2000 audit(1789906796.249:1): state=initialized audit_enabled=0 res=1909worker2 # [ 0.326907] audit: initializing netlink subsys (disabled)910worker1 # [ 0.337951] thermal_sys: Registered thermal governor 'power_allocator'911lb # [ 0.332156] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5912worker1 # [ 0.337975] cpuidle: using governor menu913client # [ 0.316252] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)914worker2 # [ 0.327896] audit: type=2000 audit(1789906796.260:1): state=initialized audit_enabled=0 res=1915lb # [ 0.333104] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]916worker1 # [ 0.340296] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5917worker2 # [ 0.329106] thermal_sys: Registered thermal governor 'fair_share'918client # [ 0.317233] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).919lb # [ 0.333830] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry920client # [ 0.318279] pinctrl core: initialized pinctrl subsystem921worker2 # [ 0.329108] thermal_sys: Registered thermal governor 'bang_bang'922lb # [ 0.334841] PCI: Using configuration type 1 for base access923worker1 # [ 0.341233] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]924worker2 # [ 0.329895] thermal_sys: Registered thermal governor 'step_wise'925client # [ 0.319490] PM: RTC time: 12:19:55, date: 2026-09-20926worker1 # [ 0.341948] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry927worker2 # [ 0.330885] thermal_sys: Registered thermal governor 'user_space'928lb # [ 0.336038] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.929worker1 # [ 0.342962] PCI: Using configuration type 1 for base access930worker2 # [ 0.331644] thermal_sys: Registered thermal governor 'power_allocator'931worker2 # [ 0.331912] cpuidle: using governor menu932worker1 # [ 0.344144] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.933client # [ 0.322987] NET: Registered PF_NETLINK/PF_ROUTE protocol family934worker2 # [ 0.334175] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5935client # [ 0.324492] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations936worker2 # [ 0.335161] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]937worker2 # [ 0.335897] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry938client # [ 0.325211] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations939worker2 # [ 0.336908] PCI: Using configuration type 1 for base access940lb # [ 0.342930] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages941worker1 # [ 0.349082] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages942client # [ 0.326460] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations943lb # [ 0.343828] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page944client # [ 0.327205] audit: initializing netlink subsys (disabled)945worker2 # [ 0.338033] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.946worker1 # [ 0.349944] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page947client # [ 0.328195] audit: type=2000 audit(1789906796.276:1): state=initialized audit_enabled=0 res=1948client # [ 0.329363] thermal_sys: Registered thermal governor 'fair_share'949client # [ 0.329365] thermal_sys: Registered thermal governor 'bang_bang'950client # [ 0.330194] thermal_sys: Registered thermal governor 'step_wise'951lb # [ 0.348827] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages952worker1 # [ 0.354945] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages953worker2 # [ 0.343175] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages954client # [ 0.331183] thermal_sys: Registered thermal governor 'user_space'955lb # [ 0.349826] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page956worker1 # [ 0.355943] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page957worker2 # [ 0.343896] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page958client # [ 0.331919] thermal_sys: Registered thermal governor 'power_allocator'959client # [ 0.332218] cpuidle: using governor menu960client # [ 0.334474] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5961client # [ 0.335464] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]962worker2 # [ 0.348896] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages963client # [ 0.336197] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry964worker2 # [ 0.349894] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page965client # [ 0.337207] PCI: Using configuration type 1 for base access966client # [ 0.338364] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.967lb # [ 0.360234] ACPI: Added _OSI(Module Device)968worker1 # [ 0.366361] ACPI: Added _OSI(Module Device)969lb # [ 0.360827] ACPI: Added _OSI(Processor Device)970worker1 # [ 0.366944] ACPI: Added _OSI(Processor Device)971lb # [ 0.361560] ACPI: Added _OSI(Processor Aggregator Device)972worker1 # [ 0.367673] ACPI: Added _OSI(Processor Aggregator Device)973client # [ 0.345473] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages974worker2 # [ 0.360612] ACPI: Added _OSI(Module Device)975client # [ 0.346194] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page976worker2 # [ 0.360895] ACPI: Added _OSI(Processor Device)977worker2 # [ 0.361616] ACPI: Added _OSI(Processor Aggregator Device)978lb # [ 0.370368] ACPI: 1 ACPI AML tables successfully acquired and loaded979worker1 # [ 0.376734] ACPI: 1 ACPI AML tables successfully acquired and loaded980client # [ 0.351194] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages981lb # [ 0.371812] ACPI: Interpreter enabled982client # [ 0.352194] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page983lb # [ 0.372270] ACPI: PM: (supports S0 S3 S4 S5)984worker1 # [ 0.380720] ACPI: Interpreter enabled985worker1 # [ 0.380977] ACPI: PM: (supports S0 S3 S4 S5)986lb # [ 0.374826] ACPI: Using IOAPIC for interrupt routing987worker1 # [ 0.381696] ACPI: Using IOAPIC for interrupt routing988lb # [ 0.375698] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug989worker2 # [ 0.370534] ACPI: 1 ACPI AML tables successfully acquired and loaded990worker1 # [ 0.382032] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug991lb # [ 0.378826] PCI: Using E820 reservations for host bridge windows992client # [ 0.360854] ACPI: Added _OSI(Module Device)993lb # [ 0.379932] ACPI: Enabled 2 GPEs in block 00 to 3F994worker2 # [ 0.373932] ACPI: Interpreter enabled995client # [ 0.361195] ACPI: Added _OSI(Processor Device)996worker2 # [ 0.374556] ACPI: PM: (supports S0 S3 S4 S5)997client # [ 0.361948] ACPI: Added _OSI(Processor Aggregator Device)998worker2 # [ 0.374895] ACPI: Using IOAPIC for interrupt routing999worker1 # [ 0.386944] PCI: Using E820 reservations for host bridge windows1000worker1 # [ 0.388051] ACPI: Enabled 2 GPEs in block 00 to 3F1001worker2 # [ 0.375747] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1002worker2 # [ 0.380895] PCI: Using E820 reservations for host bridge windows1003lb # [ 0.388551] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1004client # [ 0.368759] ACPI: 1 ACPI AML tables successfully acquired and loaded1005worker2 # [ 0.381988] ACPI: Enabled 2 GPEs in block 00 to 3F1006lb # [ 0.388896] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1007worker1 # [ 0.396706] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1008lb # [ 0.389904] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1009client # [ 0.372126] ACPI: Interpreter enabled1010lb # [ 0.390948] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1011worker1 # [ 0.396952] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1012client # [ 0.372670] ACPI: PM: (supports S0 S3 S4 S5)1013lb # [ 0.392184] PCI host bridge to bus 0000:001014client # [ 0.373194] ACPI: Using IOAPIC for interrupt routing1015worker1 # [ 0.398025] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1016lb # [ 0.392830] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1017worker1 # [ 0.399086] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1018client # [ 0.374000] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1019lb # [ 0.393826] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1020worker1 # [ 0.400333] PCI host bridge to bus 0000:001021lb # [ 0.394825] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1022worker1 # [ 0.400949] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1023worker2 # [ 0.390509] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1024lb # [ 0.395825] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1025worker1 # [ 0.401944] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1026worker2 # [ 0.390904] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1027lb # [ 0.396833] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1028worker1 # [ 0.402944] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1029worker2 # [ 0.391985] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1030client # [ 0.379193] PCI: Using E820 reservations for host bridge windows1031lb # [ 0.397856] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1032worker1 # [ 0.403951] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1033client # [ 0.380288] ACPI: Enabled 2 GPEs in block 00 to 3F1034worker2 # [ 0.393020] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1035lb # [ 0.398826] pci_bus 0000:00: root bus resource [bus 00-ff]1036worker1 # [ 0.404944] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1037worker2 # [ 0.394248] PCI host bridge to bus 0000:001038lb # [ 0.399787] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1039worker1 # [ 0.405976] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1040worker2 # [ 0.394899] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1041worker1 # [ 0.406945] pci_bus 0000:00: root bus resource [bus 00-ff]1042worker2 # [ 0.395895] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1043lb # [ 0.401236] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1044worker1 # [ 0.407966] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1045worker2 # [ 0.396894] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1046worker2 # [ 0.397894] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1047worker1 # [ 0.409437] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1048worker2 # [ 0.398895] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1049worker2 # [ 0.399894] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1050worker2 # [ 0.400896] pci_bus 0000:00: root bus resource [bus 00-ff]1051client # [ 0.389027] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1052worker2 # [ 0.401949] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1053client # [ 0.390653] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1054lb # [ 0.403916] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1055client # [ 0.391273] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1056lb # [ 0.404838] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1057worker2 # [ 0.403354] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1058client # [ 0.392317] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1059lb # [ 0.405793] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1060worker1 # [ 0.412021] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1061client # [ 0.393546] PCI host bridge to bus 0000:001062worker1 # [ 0.412956] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1063lb # [ 0.406907] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1064client # [ 0.394197] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1065worker1 # [ 0.413970] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1066client # [ 0.395193] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1067lb # [ 0.408641] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1068client # [ 0.396193] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1069worker1 # [ 0.415254] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1070client # [ 0.397193] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1071client # [ 0.398193] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1072worker1 # [ 0.416932] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1073lb # [ 0.411865] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1074worker2 # [ 0.405974] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1075lb # [ 0.412720] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1076client # [ 0.399193] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1077worker2 # [ 0.407054] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1078client # [ 0.400193] pci_bus 0000:00: root bus resource [bus 00-ff]1079lb # [ 0.413572] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1080worker2 # [ 0.407917] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1081lb # [ 0.413832] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1082client # [ 0.401165] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1083worker1 # [ 0.419979] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1084worker2 # [ 0.409394] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1085worker1 # [ 0.420927] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1086client # [ 0.402651] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1087lb # [ 0.415730] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1088worker1 # [ 0.421689] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1089worker2 # [ 0.410680] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1090worker1 # [ 0.422932] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1091lb # [ 0.417862] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1092lb # [ 0.418694] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1093worker2 # [ 0.413895] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1094worker2 # [ 0.414777] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1095worker1 # [ 0.424721] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1096lb # [ 0.419616] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1097worker2 # [ 0.415645] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1098lb # [ 0.421225] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1099worker2 # [ 0.415901] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1100client # [ 0.405388] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1101worker1 # [ 0.427957] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1102worker2 # [ 0.417696] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1103client # [ 0.406254] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1104worker1 # [ 0.428957] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1105client # [ 0.407216] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1106lb # [ 0.424002] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1107worker1 # [ 0.429977] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1108worker2 # [ 0.419903] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1109lb # [ 0.424833] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1110client # [ 0.408619] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1111worker2 # [ 0.420761] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1112lb # [ 0.425781] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1113worker2 # [ 0.421685] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1114client # [ 0.409969] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1115worker1 # [ 0.432071] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1116lb # [ 0.427218] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1117worker2 # [ 0.423385] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118client # [ 0.412228] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1119lb # [ 0.428861] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1120client # [ 0.413099] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1121lb # [ 0.429765] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1122worker1 # [ 0.434960] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1123client # [ 0.413994] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1124lb # [ 0.430582] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1125worker1 # [ 0.435954] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1126worker2 # [ 0.425931] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1127client # [ 0.415079] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1128worker2 # [ 0.426902] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1129worker1 # [ 0.436976] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1130lb # [ 0.431400] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1131worker2 # [ 0.427899] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1132client # [ 0.416847] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1133worker2 # [ 0.429523] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1134lb # [ 0.433851] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1135worker1 # [ 0.438778] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1136client # [ 0.418229] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1137lb # [ 0.434724] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1138client # [ 0.419108] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1139lb # [ 0.435578] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1140worker2 # [ 0.430926] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1141client # [ 0.419974] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1142worker2 # [ 0.431813] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1143worker1 # [ 0.441991] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1144lb # [ 0.436467] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1145worker2 # [ 0.432662] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1146client # [ 0.421736] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1147worker1 # [ 0.442958] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1148worker1 # [ 0.443977] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1149worker2 # [ 0.434785] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150lb # [ 0.438862] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1151lb # [ 0.439833] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1152client # [ 0.424406] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1153worker1 # [ 0.445749] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1154worker2 # [ 0.436920] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1155lb # [ 0.440805] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1156client # [ 0.425199] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1157worker2 # [ 0.437806] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1158client # [ 0.426178] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1159lb # [ 0.442281] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1160worker2 # [ 0.438689] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1161client # [ 0.427697] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1162worker2 # [ 0.440461] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1163lb # [ 0.443861] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1164worker1 # [ 0.449958] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1165lb # [ 0.444737] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1166client # [ 0.429225] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1167worker1 # [ 0.450967] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1168lb # [ 0.445577] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1169client # [ 0.430090] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1170worker1 # [ 0.451975] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1171lb # [ 0.445832] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1172client # [ 0.430949] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1173worker2 # [ 0.442904] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1174worker2 # [ 0.443902] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1175worker1 # [ 0.453773] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1176lb # [ 0.447640] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1177worker2 # [ 0.444917] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1178client # [ 0.432178] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1179lb # [ 0.449838] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1180worker2 # [ 0.446490] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1181client # [ 0.434202] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1182lb # [ 0.450802] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1183client # [ 0.435067] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1184client # [ 0.435930] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1185worker2 # [ 0.447922] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1186worker1 # [ 0.457387] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1187lb # [ 0.452250] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1188worker2 # [ 0.448775] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1189worker1 # [ 0.458569] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1190client # [ 0.436803] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1191worker2 # [ 0.449640] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1192worker1 # [ 0.458975] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1193lb # [ 0.453867] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1194worker2 # [ 0.449901] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1195lb # [ 0.454728] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1196worker1 # [ 0.460803] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1197client # [ 0.439228] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1198worker2 # [ 0.451534] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1199lb # [ 0.455848] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1200client # [ 0.440199] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1201client # [ 0.441209] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1202lb # [ 0.457408] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1203worker2 # [ 0.452908] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1204worker1 # [ 0.463968] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1205worker2 # [ 0.453917] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1206client # [ 0.442809] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1207lb # [ 0.458862] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1208lb # [ 0.459741] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1209worker1 # [ 0.464957] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1210worker2 # [ 0.455522] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1211client # [ 0.444201] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1212lb # [ 0.460565] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1213worker1 # [ 0.466009] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1214client # [ 0.445079] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1215worker1 # [ 0.466956] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1216client # [ 0.445952] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1217lb # [ 0.461460] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1218worker2 # [ 0.456952] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1219client # [ 0.446199] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1220worker2 # [ 0.457901] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1221worker1 # [ 0.468748] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1222lb # [ 0.462862] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1223worker2 # [ 0.458917] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1224client # [ 0.448292] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1225lb # [ 0.463947] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1226worker2 # [ 0.460485] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1227worker1 # [ 0.471635] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1228client # [ 0.450067] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1229lb # [ 0.465488] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1230worker1 # [ 0.471977] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1231client # [ 0.450938] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1232worker2 # [ 0.461902] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1233worker2 # [ 0.462791] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1234client # [ 0.451755] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1235worker2 # [ 0.463714] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1236worker1 # [ 0.474360] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1237client # [ 0.453238] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1238worker2 # [ 0.465447] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1239client # [ 0.454114] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1240worker1 # [ 0.476972] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1241client # [ 0.455215] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1242worker2 # [ 0.467586] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1243worker1 # [ 0.477955] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1244worker2 # [ 0.468138] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1245client # [ 0.456806] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1246worker1 # [ 0.478976] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1247worker2 # [ 0.469548] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1248client # [ 0.458223] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1249client # [ 0.459080] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1250worker1 # [ 0.480690] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1251lb # [ 0.466031] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1252client # [ 0.459912] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1253lb # [ 0.468495] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1254client # [ 0.460779] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1255lb # [ 0.469107] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1256worker1 # [ 0.483977] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1257worker1 # [ 0.484956] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1258client # [ 0.461859] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1259lb # [ 0.470423] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1260worker1 # [ 0.485975] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1261client # [ 0.462453] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1262lb # [ 0.471116] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1263client # [ 0.463777] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1264lb # [ 0.472122] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1265worker1 # [ 0.487757] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1266lb # [ 0.473108] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1267worker2 # [ 0.470099] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1268worker1 # [ 0.489928] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1269lb # [ 0.474785] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1270lb # [ 0.475461] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1271worker2 # [ 0.472583] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1272worker1 # [ 0.491083] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1273lb # [ 0.476146] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1274worker2 # [ 0.473213] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1275worker1 # [ 0.492980] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1276lb # [ 0.478511] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1277worker2 # [ 0.474546] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1278worker2 # [ 0.475168] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1279worker2 # [ 0.476209] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1280lb # [ 0.482083] ACPI: PCI: Interrupt link LNKA configured for IRQ 101281client # [ 0.464402] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1282lb # [ 0.482949] ACPI: PCI: Interrupt link LNKB configured for IRQ 101283worker2 # [ 0.477205] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1284client # [ 0.466873] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1285lb # [ 0.483952] ACPI: PCI: Interrupt link LNKC configured for IRQ 111286lb # [ 0.484958] ACPI: PCI: Interrupt link LNKD configured for IRQ 111287client # [ 0.467498] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1288worker2 # [ 0.479659] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1289lb # [ 0.485961] ACPI: PCI: Interrupt link LNKE configured for IRQ 101290worker2 # [ 0.480766] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1291client # [ 0.468823] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1292lb # [ 0.486943] ACPI: PCI: Interrupt link LNKF configured for IRQ 101293lb # [ 0.488062] ACPI: PCI: Interrupt link LNKG configured for IRQ 111294client # [ 0.469517] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1295worker2 # [ 0.482372] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1296lb # [ 0.488954] ACPI: PCI: Interrupt link LNKH configured for IRQ 111297client # [ 0.470535] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1298worker2 # [ 0.483689] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1299lb # [ 0.489895] ACPI: PCI: Interrupt link GSIA configured for IRQ 161300lb # [ 0.490849] ACPI: PCI: Interrupt link GSIB configured for IRQ 171301client # [ 0.471472] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1302lb # [ 0.491839] ACPI: PCI: Interrupt link GSIC configured for IRQ 181303worker1 # [ 0.494283] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1304client # [ 0.473211] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1305lb # [ 0.492842] ACPI: PCI: Interrupt link GSID configured for IRQ 191306lb # [ 0.493843] ACPI: PCI: Interrupt link GSIE configured for IRQ 201307client # [ 0.474115] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1308worker1 # [ 0.496982] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1309worker2 # [ 0.488805] ACPI: PCI: Interrupt link LNKA configured for IRQ 101310lb # [ 0.494838] ACPI: PCI: Interrupt link GSIF configured for IRQ 211311worker2 # [ 0.489924] ACPI: PCI: Interrupt link LNKB configured for IRQ 101312lb # [ 0.495833] ACPI: PCI: Interrupt link GSIG configured for IRQ 221313client # [ 0.475573] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1314worker1 # [ 0.498311] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1315lb # [ 0.496813] ACPI: PCI: Interrupt link GSIH configured for IRQ 231316worker2 # [ 0.491026] ACPI: PCI: Interrupt link LNKC configured for IRQ 111317client # [ 0.476830] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1318worker2 # [ 0.492009] ACPI: PCI: Interrupt link LNKD configured for IRQ 111319worker1 # [ 0.499904] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1320lb # [ 0.498836] iommu: Default domain type: Translated1321worker2 # [ 0.493013] ACPI: PCI: Interrupt link LNKE configured for IRQ 101322lb # [ 0.499700] iommu: DMA domain TLB invalidation policy: lazy mode1323worker2 # [ 0.494037] ACPI: PCI: Interrupt link LNKF configured for IRQ 101324lb # [ 0.500878] ACPI: bus type USB registered1325worker1 # [ 0.501386] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1326client # [ 0.480366] ACPI: PCI: Interrupt link LNKA configured for IRQ 101327lb # [ 0.501652] usbcore: registered new interface driver usbfs1328worker2 # [ 0.495032] ACPI: PCI: Interrupt link LNKG configured for IRQ 111329lb # [ 0.502657] usbcore: registered new interface driver hub1330client # [ 0.481310] ACPI: PCI: Interrupt link LNKB configured for IRQ 101331lb # [ 0.503591] usbcore: registered new device driver usb1332worker2 # [ 0.496023] ACPI: PCI: Interrupt link LNKH configured for IRQ 111333client # [ 0.482322] ACPI: PCI: Interrupt link LNKC configured for IRQ 111334worker1 # [ 0.503175] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1335worker2 # [ 0.496995] ACPI: PCI: Interrupt link GSIA configured for IRQ 161336client # [ 0.483298] ACPI: PCI: Interrupt link LNKD configured for IRQ 111337lb # [ 0.504805] NetLabel: Initializing1338lb # [ 0.505249] NetLabel: domain hash size = 1281339worker2 # [ 0.497918] ACPI: PCI: Interrupt link GSIB configured for IRQ 171340client # [ 0.484292] ACPI: PCI: Interrupt link LNKE configured for IRQ 101341lb # [ 0.505825] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1342worker1 # [ 0.504370] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1343worker2 # [ 0.498916] ACPI: PCI: Interrupt link GSIC configured for IRQ 181344client # [ 0.485298] ACPI: PCI: Interrupt link LNKF configured for IRQ 101345lb # [ 0.506761] NetLabel: unlabeled traffic allowed by default1346lb # [ 0.507507] PCI: Using ACPI for IRQ routing1347client # [ 0.486296] ACPI: PCI: Interrupt link LNKG configured for IRQ 111348worker2 # [ 0.499923] ACPI: PCI: Interrupt link GSID configured for IRQ 191349worker2 # [ 0.500918] ACPI: PCI: Interrupt link GSIE configured for IRQ 201350client # [ 0.487293] ACPI: PCI: Interrupt link LNKH configured for IRQ 111351worker2 # [ 0.501909] ACPI: PCI: Interrupt link GSIF configured for IRQ 211352client # [ 0.488232] ACPI: PCI: Interrupt link GSIA configured for IRQ 161353worker2 # [ 0.502907] ACPI: PCI: Interrupt link GSIG configured for IRQ 221354client # [ 0.489206] ACPI: PCI: Interrupt link GSIB configured for IRQ 171355worker1 # [ 0.506997] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1356worker2 # [ 0.503908] ACPI: PCI: Interrupt link GSIH configured for IRQ 231357client # [ 0.490209] ACPI: PCI: Interrupt link GSIC configured for IRQ 181358worker1 # [ 0.508057] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1359client # [ 0.491193] ACPI: PCI: Interrupt link GSID configured for IRQ 191360worker2 # [ 0.506115] iommu: Default domain type: Translated1361client # [ 0.492208] ACPI: PCI: Interrupt link GSIE configured for IRQ 201362worker2 # [ 0.506909] iommu: DMA domain TLB invalidation policy: lazy mode1363client # [ 0.493206] ACPI: PCI: Interrupt link GSIF configured for IRQ 211364worker1 # [ 0.509767] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1365worker2 # [ 0.508141] ACPI: bus type USB registered1366client # [ 0.494195] ACPI: PCI: Interrupt link GSIG configured for IRQ 221367worker2 # [ 0.508939] usbcore: registered new interface driver usbfs1368client # [ 0.495205] ACPI: PCI: Interrupt link GSIH configured for IRQ 231369worker1 # [ 0.511981] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1370worker2 # [ 0.509888] usbcore: registered new interface driver hub1371worker2 # [ 0.510779] usbcore: registered new device driver usb1372client # [ 0.497312] iommu: Default domain type: Translated1373client # [ 0.498165] iommu: DMA domain TLB invalidation policy: lazy mode1374worker2 # [ 0.512543] NetLabel: Initializing1375worker2 # [ 0.512895] NetLabel: domain hash size = 1281376client # [ 0.499289] ACPI: bus type USB registered1377worker2 # [ 0.513680] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1378client # [ 0.500086] usbcore: registered new interface driver usbfs1379worker2 # [ 0.514675] NetLabel: unlabeled traffic allowed by default1380client # [ 0.500945] usbcore: registered new interface driver hub1381worker2 # [ 0.515634] PCI: Using ACPI for IRQ routing1382worker1 # [ 0.517912] ACPI: PCI: Interrupt link LNKA configured for IRQ 101383client # [ 0.501911] usbcore: registered new device driver usb1384worker1 # [ 0.518900] ACPI: PCI: Interrupt link LNKB configured for IRQ 101385client # [ 0.503112] NetLabel: Initializing1386client # [ 0.503648] NetLabel: domain hash size = 1281387worker1 # [ 0.519981] ACPI: PCI: Interrupt link LNKC configured for IRQ 111388client # [ 0.504192] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1389worker1 # [ 0.521089] ACPI: PCI: Interrupt link LNKD configured for IRQ 111390client # [ 0.505237] NetLabel: unlabeled traffic allowed by default1391worker1 # [ 0.522096] ACPI: PCI: Interrupt link LNKE configured for IRQ 101392client # [ 0.506206] PCI: Using ACPI for IRQ routing1393worker1 # [ 0.523083] ACPI: PCI: Interrupt link LNKF configured for IRQ 101394worker1 # [ 0.524119] ACPI: PCI: Interrupt link LNKG configured for IRQ 111395worker1 # [ 0.525092] ACPI: PCI: Interrupt link LNKH configured for IRQ 111396worker1 # [ 0.526014] ACPI: PCI: Interrupt link GSIA configured for IRQ 161397worker1 # [ 0.526977] ACPI: PCI: Interrupt link GSIB configured for IRQ 171398worker1 # [ 0.527974] ACPI: PCI: Interrupt link GSIC configured for IRQ 181399worker1 # [ 0.528975] ACPI: PCI: Interrupt link GSID configured for IRQ 191400worker1 # [ 0.529968] ACPI: PCI: Interrupt link GSIE configured for IRQ 201401worker1 # [ 0.530969] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402worker1 # [ 0.531969] ACPI: PCI: Interrupt link GSIG configured for IRQ 221403worker1 # [ 0.532966] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404worker1 # [ 0.535410] iommu: Default domain type: Translated1405worker1 # [ 0.535959] iommu: DMA domain TLB invalidation policy: lazy mode1406worker1 # [ 0.537334] ACPI: bus type USB registered1407worker1 # [ 0.538067] usbcore: registered new interface driver usbfs1408worker1 # [ 0.538972] usbcore: registered new interface driver hub1409worker1 # [ 0.539967] usbcore: registered new device driver usb1410lb # [ 0.552087] pci 0000:00:01.0: vgaarb: setting as boot VGA device1411lb # [ 0.552821] pci 0000:00:01.0: vgaarb: bridge control possible1412worker1 # [ 0.542097] NetLabel: Initializing1413worker1 # [ 0.542901] NetLabel: domain hash size = 1281414lb # [ 0.552821] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1415lb # [ 0.552830] vgaarb: loaded1416worker1 # [ 0.543619] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1417lb # [ 0.553564] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01418worker1 # [ 0.544030] NetLabel: unlabeled traffic allowed by default1419lb # [ 0.553833] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1420worker1 # [ 0.544970] PCI: Using ACPI for IRQ routing1421lb # [ 0.558924] clocksource: Switched to clocksource kvm-clock1422lb # [ 0.560585] VFS: Disk quotas dquot_6.6.01423lb # [ 0.563286] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1424lb # [ 0.564659] pnp: PnP ACPI init1425worker2 # [ 0.560890] pci 0000:00:01.0: vgaarb: setting as boot VGA device1426lb # [ 0.565507] ACPI: IRQ 4 override to edge(!), high(!)1427worker2 # [ 0.560890] pci 0000:00:01.0: vgaarb: bridge control possible1428lb # [ 0.566541] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1429worker2 # [ 0.560890] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1430lb # [ 0.568165] pnp: PnP ACPI: found 6 devices1431worker2 # [ 0.560908] vgaarb: loaded1432worker2 # [ 0.561719] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01433worker2 # [ 0.562569] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1434worker2 # [ 0.567003] clocksource: Switched to clocksource kvm-clock1435lb # [ 0.575950] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1436lb # [ 0.577398] clocksource: Switched to clocksource acpi_pm1437client # [ 0.558494] pci 0000:00:01.0: vgaarb: setting as boot VGA device1438lb # [ 0.578392] NET: Registered PF_INET protocol family1439client # [ 0.559189] pci 0000:00:01.0: vgaarb: bridge control possible1440lb # [ 0.579637] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1441worker2 # [ 0.571368] VFS: Disk quotas dquot_6.6.01442client # [ 0.559189] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1443client # [ 0.559215] vgaarb: loaded1444worker2 # [ 0.572323] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1445worker2 # [ 0.573824] pnp: PnP ACPI init1446client # [ 0.560316] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01447worker2 # [ 0.574632] ACPI: IRQ 4 override to edge(!), high(!)1448client # [ 0.561196] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1449worker2 # [ 0.575646] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1450worker2 # [ 0.577331] pnp: PnP ACPI: found 6 devices1451client # [ 0.566418] clocksource: Switched to clocksource kvm-clock1452worker2 # [ 0.585143] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1453client # [ 0.571903] VFS: Disk quotas dquot_6.6.01454worker2 # [ 0.586712] clocksource: Switched to clocksource acpi_pm1455worker2 # [ 0.587726] NET: Registered PF_INET protocol family1456lb # [ 0.597136] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1457client # [ 0.573262] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1458worker2 # [ 0.589052] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1459client # [ 0.576104] pnp: PnP ACPI init1460lb # [ 0.598820] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1461client # [ 0.577034] ACPI: IRQ 4 override to edge(!), high(!)1462lb # [ 0.600137] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1463client # [ 0.578168] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1464lb # [ 0.601496] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1465client # [ 0.579793] pnp: PnP ACPI: found 6 devices1466lb # [ 0.602798] TCP: Hash tables configured (established 16384 bind 16384)1467lb # [ 0.604045] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1468lb # [ 0.605390] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1469lb # [ 0.606580] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1470lb # [ 0.607917] NET: Registered PF_UNIX/PF_LOCAL protocol family1471lb # [ 0.608888] NET: Registered PF_XDP protocol family1472worker1 # [ 0.599673] pci 0000:00:01.0: vgaarb: setting as boot VGA device1473lb # [ 0.609752] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1474worker1 # [ 0.599939] pci 0000:00:01.0: vgaarb: bridge control possible1475lb # [ 0.610759] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1476client # [ 0.587741] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1477lb # [ 0.611776] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1478worker1 # [ 0.599939] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1479client # [ 0.589210] clocksource: Switched to clocksource acpi_pm1480worker1 # [ 0.599955] vgaarb: loaded1481lb # [ 0.612861] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1482client # [ 0.590252] NET: Registered PF_INET protocol family1483worker1 # [ 0.600882] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01484lb # [ 0.613938] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1485client # [ 0.591552] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1486worker1 # [ 0.601687] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1487lb # [ 0.615017] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1488lb # [ 0.616799] ACPI: \_SB_.GSIA: Enabled at IRQ 161489lb # [ 0.619141] ACPI: \_SB_.GSIB: Enabled at IRQ 171490worker2 # [ 0.609639] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1491worker2 # [ 0.611362] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1492lb # [ 0.621161] ACPI: \_SB_.GSIC: Enabled at IRQ 181493worker1 # [ 0.607101] clocksource: Switched to clocksource kvm-clock1494worker2 # [ 0.612768] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1495lb # [ 0.623173] ACPI: \_SB_.GSID: Enabled at IRQ 191496worker2 # [ 0.614194] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1497lb # [ 0.624908] PCI: CLS 0 bytes, default 641498worker2 # [ 0.615551] TCP: Hash tables configured (established 16384 bind 16384)1499lb # [ 0.625882] Trying to unpack rootfs image as initramfs...1500worker1 # [ 0.611929] VFS: Disk quotas dquot_6.6.01501worker2 # [ 0.616904] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1502worker2 # [ 0.618325] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1503worker1 # [ 0.612859] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1504worker2 # [ 0.619406] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1505worker1 # [ 0.614744] pnp: PnP ACPI init1506worker2 # [ 0.620610] NET: Registered PF_UNIX/PF_LOCAL protocol family1507worker1 # [ 0.615893] ACPI: IRQ 4 override to edge(!), high(!)1508worker2 # [ 0.621594] NET: Registered PF_XDP protocol family1509lb # [ 0.630221] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1510worker2 # [ 0.622591] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1511worker1 # [ 0.617130] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1512worker2 # [ 0.623720] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1513worker1 # [ 0.619058] pnp: PnP ACPI: found 6 devices1514worker2 # [ 0.624720] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1515client # [ 0.610202] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1516worker2 # [ 0.625809] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1517client # [ 0.611853] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1518worker2 # [ 0.626896] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1519client # [ 0.613185] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1520worker2 # [ 0.627978] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1521client # [ 0.614691] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1522worker2 # [ 0.629718] ACPI: \_SB_.GSIA: Enabled at IRQ 161523client # [ 0.615989] TCP: Hash tables configured (established 16384 bind 16384)1524client # [ 0.617210] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1525worker2 # [ 0.632003] ACPI: \_SB_.GSIB: Enabled at IRQ 171526client # [ 0.618589] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1527worker1 # [ 0.627504] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1528worker2 # [ 0.634192] ACPI: \_SB_.GSIC: Enabled at IRQ 181529client # [ 0.619735] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1530worker1 # [ 0.629300] clocksource: Switched to clocksource acpi_pm1531client # [ 0.621020] NET: Registered PF_UNIX/PF_LOCAL protocol family1532worker1 # [ 0.630497] NET: Registered PF_INET protocol family1533worker2 # [ 0.636272] ACPI: \_SB_.GSID: Enabled at IRQ 191534client # [ 0.622061] NET: Registered PF_XDP protocol family1535client # [ 0.622912] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1536worker2 # [ 0.637974] PCI: CLS 0 bytes, default 641537worker1 # [ 0.631988] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1538client # [ 0.623975] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1539client # [ 0.625010] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1540worker2 # [ 0.638945] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1541client # [ 0.626133] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1542worker2 # [ 0.640725] Trying to unpack rootfs image as initramfs...1543client # [ 0.627265] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1544client # [ 0.628381] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1545client # [ 0.630247] ACPI: \_SB_.GSIA: Enabled at IRQ 161546client # [ 0.632817] ACPI: \_SB_.GSIB: Enabled at IRQ 171547client # [ 0.634842] ACPI: \_SB_.GSIC: Enabled at IRQ 181548client # [ 0.636860] ACPI: \_SB_.GSID: Enabled at IRQ 191549client # [ 0.638599] PCI: CLS 0 bytes, default 641550client # [ 0.639620] Trying to unpack rootfs image as initramfs...1551client # [ 0.643987] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1552worker1 # [ 0.652919] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1553worker1 # [ 0.654928] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1554worker1 # [ 0.656600] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1555worker1 # [ 0.658378] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1556lb # [ 0.674201] Initialise system trusted keyrings1557worker1 # [ 0.660012] TCP: Hash tables configured (established 16384 bind 16384)1558lb # [ 0.675162] workingset: timestamp_bits=40 max_order=19 bucket_order=01559worker1 # [ 0.661438] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1560worker1 # [ 0.663021] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1561worker1 # [ 0.664360] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1562worker1 # [ 0.665902] NET: Registered PF_UNIX/PF_LOCAL protocol family1563worker1 # [ 0.667109] NET: Registered PF_XDP protocol family1564worker1 # [ 0.668268] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1565worker1 # [ 0.669567] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1566worker1 # [ 0.670786] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1567worker1 # [ 0.672227] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1568worker1 # [ 0.673667] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1569worker1 # [ 0.675094] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1570worker1 # [ 0.677481] ACPI: \_SB_.GSIA: Enabled at IRQ 161571worker1 # [ 0.680392] ACPI: \_SB_.GSIB: Enabled at IRQ 171572worker2 # [ 0.686549] Initialise system trusted keyrings1573worker2 # [ 0.687528] workingset: timestamp_bits=40 max_order=19 bucket_order=01574worker1 # [ 0.683238] ACPI: \_SB_.GSIC: Enabled at IRQ 181575lb # [ 0.699246] Key type asymmetric registered1576lb # [ 0.699988] Asymmetric key parser 'x509' registered1577worker1 # [ 0.686022] ACPI: \_SB_.GSID: Enabled at IRQ 191578worker1 # [ 0.688354] PCI: CLS 0 bytes, default 641579lb # [ 0.702826] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1580worker1 # [ 0.689592] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1581lb # [ 0.705818] io scheduler mq-deadline registered1582worker1 # [ 0.691849] Trying to unpack rootfs image as initramfs...1583lb # [ 0.706606] io scheduler kyber registered1584lb # [ 0.709398] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1585lb # [ 0.710681] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1586client # [ 0.689996] Initialise system trusted keyrings1587client # [ 0.691062] workingset: timestamp_bits=40 max_order=19 bucket_order=01588lb # [ 0.715620] Linux agpgart interface v0.1031589lb # [ 0.716453] ACPI: bus type drm_connector registered1590lb # [ 0.719241] usbcore: registered new interface driver usbserial_generic1591lb # [ 0.720303] usbserial: USB Serial support registered for generic1592worker2 # [ 0.714100] Key type asymmetric registered1593worker2 # [ 0.714874] Asymmetric key parser 'x509' registered1594lb # [ 0.723758] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1595lb # [ 0.723758] Please enable it if your BIOS has the CPPC option.1596lb # [ 0.725927] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1597worker2 # [ 0.717806] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1598worker2 # [ 0.719331] io scheduler mq-deadline registered1599lb # [ 0.729944] drop_monitor: Initializing network drop monitor service1600worker2 # [ 0.721769] io scheduler kyber registered1601lb # [ 0.731129] NET: Registered PF_INET6 protocol family1602worker2 # [ 0.724740] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1603lb # [ 0.734293] Segment Routing with IPv61604worker2 # [ 0.726170] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1605lb # [ 0.737788] In-situ OAM (IOAM) with IPv61606client # [ 0.714702] Key type asymmetric registered1607client # [ 0.715501] Asymmetric key parser 'x509' registered1608lb # [ 0.738945] IPI shorthand broadcast: enabled1609client # [ 0.716385] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1610worker2 # [ 0.732174] Linux agpgart interface v0.1031611worker2 # [ 0.733006] ACPI: bus type drm_connector registered1612client # [ 0.719644] io scheduler mq-deadline registered1613client # [ 0.720528] io scheduler kyber registered1614worker2 # [ 0.736185] usbcore: registered new interface driver usbserial_generic1615worker2 # [ 0.737403] usbserial: USB Serial support registered for generic1616lb # [ 0.747179] sched_clock: Marking stable (607019480, 139732967)->(822964875, -76212428)1617client # [ 0.724278] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1618worker2 # [ 0.740724] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1619lb # [ 0.750863] registered taskstats version 11620worker2 # [ 0.740724] Please enable it if your BIOS has the CPPC option.1621client # [ 0.727729] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1622lb # [ 0.751864] Loading compiled-in X.509 certificates1623worker2 # [ 0.743116] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1624client # [ 0.732405] Linux agpgart interface v0.1031625worker2 # [ 0.746912] drop_monitor: Initializing network drop monitor service1626client # [ 0.733239] ACPI: bus type drm_connector registered1627worker2 # [ 0.748152] NET: Registered PF_INET6 protocol family1628client # [ 0.736042] usbcore: registered new interface driver usbserial_generic1629client # [ 0.737215] usbserial: USB Serial support registered for generic1630worker2 # [ 0.752283] Segment Routing with IPv61631client # [ 0.740460] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1632worker2 # [ 0.755717] In-situ OAM (IOAM) with IPv61633worker1 # [ 0.750160] Initialise system trusted keyrings1634client # [ 0.740460] Please enable it if your BIOS has the CPPC option.1635worker2 # [ 0.756897] IPI shorthand broadcast: enabled1636client # [ 0.742749] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1637worker1 # [ 0.753715] workingset: timestamp_bits=40 max_order=19 bucket_order=01638client # [ 0.746717] drop_monitor: Initializing network drop monitor service1639lb # [ 0.770778] Demotion targets for Node 0: null1640client # [ 0.748022] NET: Registered PF_INET6 protocol family1641lb # [ 0.772814] Key type .fscrypt registered1642lb # [ 0.773494] Key type fscrypt-provisioning registered1643lb # [ 0.774463] ima: No TPM chip found, activating TPM-bypass!1644client # [ 0.752037] Segment Routing with IPv61645worker2 # [ 0.765549] sched_clock: Marking stable (623019822, 141680481)->(840818956, -76118653)1646client # [ 0.755466] In-situ OAM (IOAM) with IPv61647lb # [ 0.778769] ima: Allocated hash algorithm: sha11648lb # [ 0.779579] ima: No architecture policies found1649client # [ 0.756616] IPI shorthand broadcast: enabled1650worker2 # [ 0.771165] registered taskstats version 11651worker2 # [ 0.772277] Loading compiled-in X.509 certificates1652lb # [ 0.781770] PM: Magic number: 10:66:3301653lb # [ 0.782477] tty tty51: hash matches1654lb # [ 0.786107] RAS: Correctable Errors collector initialized.1655client # [ 0.764983] sched_clock: Marking stable (625017672, 139426038)->(841721011, -77277301)1656client # [ 0.768567] registered taskstats version 11657client # [ 0.769590] Loading compiled-in X.509 certificates1658lb # [ 0.795023] clk: Disabling unused clocks1659lb # [ 0.797767] PM: genpd: Disabling unused power domains1660worker1 # [ 0.783247] Key type asymmetric registered1661worker2 # [ 0.792138] Demotion targets for Node 0: null1662worker2 # [ 0.793113] Key type .fscrypt registered1663worker1 # [ 0.787650] Asymmetric key parser 'x509' registered1664worker1 # [ 0.788912] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1665worker2 # [ 0.795702] Key type fscrypt-provisioning registered1666worker2 # [ 0.796653] ima: No TPM chip found, activating TPM-bypass!1667worker2 # [ 0.797627] ima: Allocated hash algorithm: sha11668worker1 # [ 0.793814] io scheduler mq-deadline registered1669worker2 # [ 0.799740] ima: No architecture policies found1670worker1 # [ 0.796650] io scheduler kyber registered1671worker2 # [ 0.802716] PM: Magic number: 10:66:3301672worker2 # [ 0.803474] tty tty51: hash matches1673client # [ 0.790455] Demotion targets for Node 0: null1674worker1 # [ 0.799437] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1675client # [ 0.791520] Key type .fscrypt registered1676client # [ 0.792195] Key type fscrypt-provisioning registered1677worker2 # [ 0.807115] RAS: Correctable Errors collector initialized.1678client # [ 0.793259] ima: No TPM chip found, activating TPM-bypass!1679worker1 # [ 0.803003] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1680client # [ 0.797469] ima: Allocated hash algorithm: sha11681client # [ 0.798323] ima: No architecture policies found1682worker1 # [ 0.808870] Linux agpgart interface v0.1031683client # [ 0.800454] PM: Magic number: 10:66:3301684client # [ 0.801212] tty tty51: hash matches1685worker1 # [ 0.809884] ACPI: bus type drm_connector registered1686worker2 # [ 0.816005] clk: Disabling unused clocks1687worker2 # [ 0.818712] PM: genpd: Disabling unused power domains1688client # [ 0.805132] RAS: Correctable Errors collector initialized.1689worker1 # [ 0.815273] usbcore: registered new interface driver usbserial_generic1690worker1 # [ 0.819659] usbserial: USB Serial support registered for generic1691worker1 # [ 0.820964] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1692worker1 # [ 0.820964] Please enable it if your BIOS has the CPPC option.1693client # [ 0.814254] clk: Disabling unused clocks1694client # [ 0.816458] PM: genpd: Disabling unused power domains1695worker1 # [ 0.829655] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1696worker1 # [ 0.831518] drop_monitor: Initializing network drop monitor service1697worker1 # [ 0.834826] NET: Registered PF_INET6 protocol family1698worker1 # [ 0.839188] Segment Routing with IPv61699worker1 # [ 0.842691] In-situ OAM (IOAM) with IPv61700worker1 # [ 0.844187] IPI shorthand broadcast: enabled1701worker1 # [ 0.855981] sched_clock: Marking stable (708020237, 147606057)->(948998885, -93372591)1702worker1 # [ 0.861803] registered taskstats version 11703worker1 # [ 0.863089] Loading compiled-in X.509 certificates1704worker1 # [ 0.887226] Demotion targets for Node 0: null1705worker1 # [ 0.888488] Key type .fscrypt registered1706worker1 # [ 0.891635] Key type fscrypt-provisioning registered1707worker1 # [ 0.892865] ima: No TPM chip found, activating TPM-bypass!1708worker1 # [ 0.895642] ima: Allocated hash algorithm: sha11709worker1 # [ 0.896691] ima: No architecture policies found1710worker1 # [ 0.899873] PM: Magic number: 10:66:3301711worker1 # [ 0.900768] tty tty51: hash matches1712worker1 # [ 0.904966] RAS: Correctable Errors collector initialized.1713worker1 # [ 0.918575] clk: Disabling unused clocks1714worker1 # [ 0.921649] PM: genpd: Disabling unused power domains1715lb # [ 0.987324] Freeing initrd memory: 29092K1716lb # [ 0.990574] Freeing unused decrypted memory: 2028K1717lb # [ 0.993220] Freeing unused kernel image (initmem) memory: 3644K1718lb # [ 0.994293] Write protecting the kernel read-only data: 32768k1719lb # [ 0.996190] Freeing unused kernel image (text/rodata gap) memory: 1208K1720lb # [ 0.997681] Freeing unused kernel image (rodata/data gap) memory: 752K1721worker2 # [ 1.009006] Freeing initrd memory: 29092K1722worker2 # [ 1.012241] Freeing unused decrypted memory: 2028K1723worker2 # [ 1.014928] Freeing unused kernel image (initmem) memory: 3644K1724worker2 # [ 1.016019] Write protecting the kernel read-only data: 32768k1725worker2 # [ 1.017894] Freeing unused kernel image (text/rodata gap) memory: 1208K1726worker2 # [ 1.019375] Freeing unused kernel image (rodata/data gap) memory: 752K1727client # [ 1.005729] Freeing initrd memory: 29096K1728client # [ 1.010292] Freeing unused decrypted memory: 2028K1729client # [ 1.014168] Freeing unused kernel image (initmem) memory: 3644K1730client # [ 1.015349] Write protecting the kernel read-only data: 32768k1731client # [ 1.017455] Freeing unused kernel image (text/rodata gap) memory: 1208K1732client # [ 1.019146] Freeing unused kernel image (rodata/data gap) memory: 752K1733lb # [ 1.049565] x86/mm: Checked W+X mappings: passed, no W+X pages found.1734lb # [ 1.050670] Run /init as init process1735lb # [ 1.060890] systemd[1]: Inserted module 'autofs4'1736lb # [ 1.077320] fuse: init (API version 7.45)1737worker2 # [ 1.071788] x86/mm: Checked W+X mappings: passed, no W+X pages found.1738worker2 # [ 1.072921] Run /init as init process1739lb # [ 1.083266] ACPI: \_SB_.GSIG: Enabled at IRQ 221740lb # [ 1.085629] ACPI: \_SB_.GSIH: Enabled at IRQ 231741lb # [ 1.088568] ACPI: \_SB_.GSIE: Enabled at IRQ 201742lb # [ 1.090851] ACPI: \_SB_.GSIF: Enabled at IRQ 211743worker2 # [ 1.083855] systemd[1]: Inserted module 'autofs4'1744client # [ 1.070863] x86/mm: Checked W+X mappings: passed, no W+X pages found.1745client # [ 1.072103] Run /init as init process1746client # [ 1.083816] systemd[1]: Inserted module 'autofs4'1747worker2 # [ 1.100978] fuse: init (API version 7.45)1748worker2 # [ 1.107036] ACPI: \_SB_.GSIG: Enabled at IRQ 221749worker2 # [ 1.109662] ACPI: \_SB_.GSIH: Enabled at IRQ 231750worker2 # [ 1.112634] ACPI: \_SB_.GSIE: Enabled at IRQ 201751worker2 # [ 1.115050] ACPI: \_SB_.GSIF: Enabled at IRQ 211752client # [ 1.101613] fuse: init (API version 7.45)1753lb # [ 1.125725] systemd[1]: Successfully made /usr/ read-only.1754client # [ 1.108094] ACPI: \_SB_.GSIG: Enabled at IRQ 221755worker1 # [ 1.119652] Freeing initrd memory: 29092K1756client # [ 1.111090] ACPI: \_SB_.GSIH: Enabled at IRQ 231757worker1 # [ 1.122916] Freeing unused decrypted memory: 2028K1758client # [ 1.115043] ACPI: \_SB_.GSIE: Enabled at IRQ 201759worker1 # [ 1.125682] Freeing unused kernel image (initmem) memory: 3644K1760client # [ 1.117730] ACPI: \_SB_.GSIF: Enabled at IRQ 211761worker1 # [ 1.126922] Write protecting the kernel read-only data: 32768k1762worker1 # [ 1.128927] Freeing unused kernel image (text/rodata gap) memory: 1208K1763worker1 # [ 1.130518] Freeing unused kernel image (rodata/data gap) memory: 752K1764worker2 # [ 1.144585] systemd[1]: Successfully made /usr/ read-only.1765client # [ 1.150528] systemd[1]: Successfully made /usr/ read-only.1766worker1 # [ 1.182835] x86/mm: Checked W+X mappings: passed, no W+X pages found.1767worker1 # [ 1.183989] Run /init as init process1768worker1 # [ 1.195325] systemd[1]: Inserted module 'autofs4'1769worker1 # [ 1.212853] fuse: init (API version 7.45)1770worker1 # [ 1.219309] ACPI: \_SB_.GSIG: Enabled at IRQ 221771worker1 # [ 1.222070] ACPI: \_SB_.GSIH: Enabled at IRQ 231772worker1 # [ 1.226436] ACPI: \_SB_.GSIE: Enabled at IRQ 201773worker1 # [ 1.229095] ACPI: \_SB_.GSIF: Enabled at IRQ 211774worker1 # [ 1.260482] systemd[1]: Successfully made /usr/ read-only.1775lb # [ 1.462351] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776lb # [ 1.467550] systemd[1]: Detected virtualization kvm.1777lb # [ 1.468494] systemd[1]: Detected architecture x86-64.1778lb # [ 1.469523] systemd[1]: Running in initrd.1779lb # [ 1.470871] systemd[1]: Initializing machine ID from random generator.1780lb # [ 1.472148] systemd[1]: Hostname set to <lb>.1781worker2 # [ 1.480532] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1782worker2 # [ 1.487447] systemd[1]: Detected virtualization kvm.1783worker2 # [ 1.488666] systemd[1]: Detected architecture x86-64.1784worker2 # [ 1.490019] systemd[1]: Running in initrd.1785worker2 # [ 1.491547] systemd[1]: Initializing machine ID from random generator.1786worker2 # [ 1.493314] systemd[1]: Hostname set to <worker2>.1787client # [ 1.486656] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1788client # [ 1.494115] systemd[1]: Detected virtualization kvm.1789client # [ 1.495401] systemd[1]: Detected architecture x86-64.1790client # [ 1.496890] systemd[1]: Running in initrd.1791client # [ 1.498621] systemd[1]: Initializing machine ID from random generator.1792client # [ 1.500362] systemd[1]: Hostname set to <client>.1793worker1 # [ 1.596061] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1794worker1 # [ 1.601276] systemd[1]: Detected virtualization kvm.1795worker1 # [ 1.602149] systemd[1]: Detected architecture x86-64.1796worker1 # [ 1.603038] systemd[1]: Running in initrd.1797worker1 # [ 1.604137] systemd[1]: Initializing machine ID from random generator.1798worker1 # [ 1.605404] systemd[1]: Hostname set to <worker1>.1799client # [ 1.611328] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800client # [ 1.654333] systemd[1]: Queued start job for default target Initrd Default Target.1801client # [ 1.659058] systemd[1]: Created slice Slice /system/modprobe.1802client # [ 1.660337] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1803client # [ 1.661875] systemd[1]: Expecting device /dev/disk/by-label/nixos...1804client # [ 1.663065] systemd[1]: Reached target Path Units.1805client # [ 1.663979] systemd[1]: Reached target Slice Units.1806client # [ 1.664922] systemd[1]: Reached target Swaps.1807client # [ 1.665840] systemd[1]: Reached target Timer Units.1808client # [ 1.666904] systemd[1]: Listening on D-Bus System Message Bus Socket.1809client # [ 1.668287] systemd[1]: Listening on Journal Socket (/dev/log).1810client # [ 1.669575] systemd[1]: Listening on Journal Sockets.1811lb # [ 1.692759] systemd[1]: bpf-restrict-fs: LSM BPF program attached1812client # [ 1.670600] systemd[1]: Listening on udev Control Socket.1813client # [ 1.671692] systemd[1]: Listening on udev Kernel Socket.1814client # [ 1.672709] systemd[1]: Reached target Socket Units.1815client # [ 1.675067] systemd[1]: Starting Create List of Static Device Nodes...1816client # [ 1.680548] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1817client # [ 1.688700] systemd[1]: Starting Load Kernel Module configfs...1818client # [ 1.710550] systemd[1]: Starting Journal Service...1819lb # [ 1.740248] systemd[1]: Queued start job for default target Initrd Default Target.1820worker2 # [ 1.733526] systemd[1]: bpf-restrict-fs: LSM BPF program attached1821lb # [ 1.744304] systemd[1]: Created slice Slice /system/modprobe.1822lb # [ 1.745543] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1823lb # [ 1.746994] systemd[1]: Expecting device /dev/disk/by-label/nixos...1824lb # [ 1.748132] systemd[1]: Reached target Path Units.1825lb # [ 1.749002] systemd[1]: Reached target Slice Units.1826lb # [ 1.749986] systemd[1]: Reached target Swaps.1827lb # [ 1.750873] systemd[1]: Reached target Timer Units.1828lb # [ 1.751883] systemd[1]: Listening on D-Bus System Message Bus Socket.1829lb # [ 1.753138] systemd[1]: Listening on Journal Socket (/dev/log).1830lb # [ 1.754339] systemd[1]: Listening on Journal Sockets.1831lb # [ 1.755561] systemd[1]: Listening on udev Control Socket.1832lb # [ 1.756875] systemd[1]: Listening on udev Kernel Socket.1833lb # [ 1.757972] systemd[1]: Reached target Socket Units.1834client # [ 1.724572] systemd[1]: Starting Load Kernel Modules...1835lb # [ 1.760656] systemd[1]: Starting Create List of Static Device Nodes...1836client # [ 1.742137] netfs: FS-Cache loaded1837lb # [ 1.766835] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1838client # [ 1.748069] 9pnet: Installing 9P2000 support1839lb # [ 1.772798] systemd[1]: Starting Load Kernel Module configfs...1840client # [ 1.760692] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1841worker2 # [ 1.783204] systemd[1]: Queued start job for default target Initrd Default Target.1842client # [ 1.769761] systemd[1]: Starting Coldplug All udev Devices...1843lb # [ 1.794027] systemd[1]: Starting Journal Service...1844worker2 # [ 1.788099] systemd[1]: Created slice Slice /system/modprobe.1845worker2 # [ 1.789374] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1846worker2 # [ 1.790842] systemd[1]: Expecting device /dev/disk/by-label/nixos...1847worker2 # [ 1.791984] systemd[1]: Reached target Path Units.1848worker2 # [ 1.792957] systemd[1]: Reached target Slice Units.1849worker2 # [ 1.793867] systemd[1]: Reached target Swaps.1850worker2 # [ 1.794707] systemd[1]: Reached target Timer Units.1851worker2 # [ 1.795774] systemd[1]: Listening on D-Bus System Message Bus Socket.1852worker2 # [ 1.797056] systemd[1]: Listening on Journal Socket (/dev/log).1853worker2 # [ 1.798278] systemd[1]: Listening on Journal Sockets.1854worker2 # [ 1.799344] systemd[1]: Listening on udev Control Socket.1855worker2 # [ 1.800420] systemd[1]: Listening on udev Kernel Socket.1856worker2 # [ 1.801395] systemd[1]: Reached target Socket Units.1857client # [ 1.787553] systemd[1]: Finished Create List of Static Device Nodes.1858worker2 # [ 1.803434] systemd[1]: Starting Create List of Static Device Nodes...1859worker1 # [ 1.799427] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker2 # [ 1.807664] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1861client # [ 1.795966] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1862worker2 # [ 1.814797] systemd[1]: Starting Load Kernel Module configfs...1863client # [ 1.805047] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1864client # [ 1.806410] systemd-journald[67]: Collecting audit messages is disabled.1865lb # [ 1.815982] systemd[1]: Starting Load Kernel Modules...1866client # [ 1.813121] systemd[1]: modprobe@configfs.service: Deactivated successfully.1867worker2 # [ 1.831496] systemd[1]: Starting Journal Service...1868client # [ 1.819471] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1869lb # [ 1.844236] netfs: FS-Cache loaded1870client # [ 1.822090] systemd[1]: Finished Load Kernel Module configfs.1871client # [ 1.826167] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1872lb # [ 1.853948] 9pnet: Installing 9P2000 support1873worker1 # [ 1.839427] systemd[1]: Queued start job for default target Initrd Default Target.1874client # [ 1.830782] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1875worker1 # [ 1.843990] systemd[1]: Created slice Slice /system/modprobe.1876worker1 # [ 1.845226] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1877worker1 # [ 1.846713] systemd[1]: Expecting device /dev/disk/by-label/nixos...1878worker1 # [ 1.847896] systemd[1]: Reached target Path Units.1879worker1 # [ 1.848825] systemd[1]: Reached target Slice Units.1880worker1 # [ 1.849782] systemd[1]: Reached target Swaps.1881client # [ 1.840988] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1882worker1 # [ 1.850671] systemd[1]: Reached target Timer Units.1883worker1 # [ 1.851703] systemd[1]: Listening on D-Bus System Message Bus Socket.1884lb # [ 1.865662] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1885worker1 # [ 1.852965] systemd[1]: Listening on Journal Socket (/dev/log).1886worker2 # [ 1.846821] systemd[1]: Starting Load Kernel Modules...1887worker1 # [ 1.854169] systemd[1]: Listening on Journal Sockets.1888worker1 # [ 1.855190] systemd[1]: Listening on udev Control Socket.1889worker1 # [ 1.856287] systemd[1]: Listening on udev Kernel Socket.1890worker1 # [ 1.857321] systemd[1]: Reached target Socket Units.1891worker1 # [ 1.859401] systemd[1]: Starting Create List of Static Device Nodes...1892worker2 # [ 1.867876] netfs: FS-Cache loaded1893lb # [ 1.877144] systemd[1]: Starting Coldplug All udev Devices...1894worker1 # [ 1.863522] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1895worker2 # [ 1.873177] 9pnet: Installing 9P2000 support1896worker1 # [ 1.872814] systemd[1]: Starting Load Kernel Module configfs...1897worker2 # [ 1.880977] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1898client # [ 1.870576] systemd[1]: Finished Load Kernel Modules.1899lb # [ 1.893875] systemd[1]: Finished Create List of Static Device Nodes.1900worker2 # [ 1.890949] systemd[1]: Starting Coldplug All udev Devices...1901worker1 # [ 1.889900] systemd[1]: Starting Journal Service...1902lb # [ 1.905107] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1903client # [ 1.884627] systemd[1]: Starting Apply Kernel Variables...1904lb # [ 1.912838] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1905lb # [ 1.914165] systemd-journald[67]: Collecting audit messages is disabled.1906worker2 # [ 1.908080] systemd[1]: Finished Create List of Static Device Nodes.1907lb # [ 1.919273] systemd[1]: modprobe@configfs.service: Deactivated successfully.1908worker2 # [ 1.913387] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1909lb # [ 1.927826] systemd[1]: Finished Load Kernel Module configfs.1910client # [ 1.906659] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1911worker1 # [ 1.905035] systemd[1]: Starting Load Kernel Modules...1912worker2 # [ 1.922214] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1913lb # [ 1.931417] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1914worker2 # [ 1.926847] systemd-journald[67]: Collecting audit messages is disabled.1915lb # [ 1.935433] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1916worker2 # [ 1.929357] systemd[1]: modprobe@configfs.service: Deactivated successfully.1917worker1 # [ 1.924689] netfs: FS-Cache loaded1918client # [ 1.918663] systemd[1]: Starting Create Static Device Nodes in /dev...1919lb # [ 1.942017] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1920worker1 # [ 1.929580] 9pnet: Installing 9P2000 support1921worker2 # [ 1.937132] systemd[1]: Finished Load Kernel Module configfs.1922lb # [ 1.947072] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1923worker2 # [ 1.939930] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1924worker2 # [ 1.944277] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1925worker1 # [ 1.940782] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1926worker2 # [ 1.948744] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1927client # [ 1.938682] systemd[1]: Finished Apply Kernel Variables.1928worker1 # [ 1.950002] systemd[1]: Starting Coldplug All udev Devices...1929worker2 # [ 1.955974] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1930client # [ 1.949059] systemd[1]: Started Journal Service.1931client # [ 1.810762] systemd-modules-load[68]: Inserted module 'dm_mod'1932client # [ 1.814139] systemd-modules-load[68]: Inserted module 'virtio_balloon'1933worker1 # [ 1.964795] systemd[1]: Finished Create List of Static Device Nodes.1934client # [ 1.817545] systemd-modules-load[68]: Inserted module 'virtio_gpu'1935lb # [ 1.980481] systemd[1]: Finished Load Kernel Modules.1936worker1 # [ 1.969591] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1937worker1 # [ 1.973585] systemd-journald[67]: Collecting audit messages is disabled.1938lb # [ 1.990210] systemd[1]: Starting Apply Kernel Variables...1939worker2 # [ 1.982482] systemd[1]: Finished Load Kernel Modules.1940worker1 # [ 1.979039] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1941client # [ 1.835288] systemd[1]: Finished Create Static Device Nodes in /dev.1942worker1 # [ 1.984104] systemd[1]: modprobe@configfs.service: Deactivated successfully.1943worker2 # [ 1.990952] systemd[1]: Starting Apply Kernel Variables...1944client # [ 1.838325] systemd[1]: Reached target Preparation for Local File Systems.1945client # [ 1.839421] systemd[1]: Reached target Local File Systems.1946worker1 # [ 1.991093] systemd[1]: Finished Load Kernel Module configfs.1947client # [ 1.844098] systemd[1]: Starting Create System Files and Directories...1948worker1 # [ 1.995093] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1949worker1 # [ 1.997758] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1950client # [ 1.851334] systemd[1]: Starting Rule-based Manager for Device Events and Files...1951lb # [ 2.015955] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1952worker1 # [ 2.003645] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1953worker2 # [ 2.011822] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1954worker1 # [ 2.008855] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1955lb # [ 2.028951] systemd[1]: Starting Create Static Device Nodes in /dev...1956worker2 # [ 2.022052] systemd[1]: Starting Create Static Device Nodes in /dev...1957client # [ 1.885895] systemd[1]: Finished Create System Files and Directories.1958worker1 # [ 2.036212] systemd[1]: Finished Load Kernel Modules.1959worker2 # [ 2.042833] systemd[1]: Finished Apply Kernel Variables.1960lb # [ 2.052107] systemd[1]: Finished Apply Kernel Variables.1961lb # [ 2.058363] systemd[1]: Started Journal Service.1962lb # [ 1.916602] systemd-modules-load[68]: Inserted module 'dm_mod'1963worker1 # [ 2.044872] systemd[1]: Starting Apply Kernel Variables...1964lb # [ 1.920583] systemd-modules-load[68]: Inserted module 'virtio_balloon'1965lb # [ 1.921799] systemd-modules-load[68]: Inserted module 'virtio_gpu'1966worker2 # [ 2.055140] systemd[1]: Started Journal Service.1967worker2 # [ 1.914665] systemd-modules-load[68]: Inserted module 'dm_mod'1968worker2 # [ 1.918350] systemd-modules-load[68]: Inserted module 'virtio_balloon'1969worker2 # [ 1.919741] systemd-modules-load[68]: Inserted module 'virtio_gpu'1970worker1 # [ 2.061750] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1971client # [ 1.914490] systemd-udevd[83]: Using default interface naming scheme 'v261'.1972worker2 # [ 1.926963] systemd[1]: Finished Create Static Device Nodes in /dev.1973lb # [ 1.944110] systemd[1]: Finished Create Static Device Nodes in /dev.1974worker2 # [ 1.934140] systemd[1]: Reached target Preparation for Local File Systems.1975worker2 # [ 1.935200] systemd[1]: Reached target Local File Systems.1976lb # [ 1.947089] systemd[1]: Reached target Preparation for Local File Systems.1977lb # [ 1.948188] systemd[1]: Reached target Local File Systems.1978worker1 # [ 2.074276] systemd[1]: Starting Create Static Device Nodes in /dev...1979worker2 # [ 1.939068] systemd[1]: Starting Create System Files and Directories...1980lb # [ 1.953115] systemd[1]: Starting Create System Files and Directories...1981worker2 # [ 1.943382] systemd[1]: Starting Rule-based Manager for Device Events and Files...1982lb # [ 1.957826] systemd[1]: Starting Rule-based Manager for Device Events and Files...1983client # [ 1.941153] systemd[1]: Started Rule-based Manager for Device Events and Files.1984worker1 # [ 2.096723] systemd[1]: Finished Apply Kernel Variables.1985worker1 # [ 2.110802] systemd[1]: Started Journal Service.1986worker2 # [ 1.976730] systemd[1]: Finished Create System Files and Directories.1987worker1 # [ 1.961922] systemd-modules-load[68]: Inserted module 'dm_mod'1988worker1 # [ 1.967072] systemd-modules-load[68]: Inserted module 'virtio_balloon'1989worker1 # [ 1.968286] systemd-modules-load[68]: Inserted module 'virtio_gpu'1990worker1 # [ 1.976975] systemd[1]: Finished Create Static Device Nodes in /dev.1991lb # [ 2.004651] systemd[1]: Finished Create System Files and Directories.1992worker1 # [ 1.984133] systemd[1]: Reached target Preparation for Local File Systems.1993worker1 # [ 1.985196] systemd[1]: Reached target Local File Systems.1994worker1 # [ 1.988068] systemd[1]: Starting Create System Files and Directories...1995worker2 # [ 2.000581] systemd-udevd[83]: Using default interface naming scheme 'v261'.1996worker1 # [ 1.993080] systemd[1]: Starting Rule-based Manager for Device Events and Files...1997client # [ 1.993224] systemd[1]: Finished Coldplug All udev Devices.1998client # [ 1.994989] systemd[1]: Reached target System Initialization.1999client # [ 1.995879] systemd[1]: Reached target Basic System.2000lb # [ 2.031355] systemd-udevd[83]: Using default interface naming scheme 'v261'.
loading 2,074 hidden lines…
4075lb # [ 18.688997] envoy[978]: [2026-09-20 12:20:14.447][978][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4076lb # [ 18.694062] envoy[978]: [2026-09-20 12:20:14.447][978][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4077lb # [ 18.696429] envoy[978]: [2026-09-20 12:20:14.447][978][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4078lb # [ 18.698088] envoy[978]: [2026-09-20 12:20:14.447][978][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4079lb # [ 18.699908] envoy[978]: [2026-09-20 12:20:14.447][978][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4080lb # [ 18.701398] envoy[978]: [2026-09-20 12:20:14.447][978][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4081lb # [ 18.704287] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4082lb # [ 18.706320] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4083lb # [ 18.708697] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4084lb # [ 18.710308] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4085lb # [ 18.712142] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4086lb # [ 18.714456] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4087lb # [ 18.716759] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4088lb # [ 18.719373] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4089lb # [ 18.721324] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4090lb # [ 18.724329] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4091lb # [ 18.726222] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4092lb # [ 18.731286] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4093lb # [ 18.743521] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4094lb # [ 18.745326] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4095lb # [ 18.747269] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4096lb # [ 18.749235] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4097lb # [ 18.751225] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4098lb # [ 18.752952] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4099lb # [ 18.755522] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4100lb # [ 18.761101] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24101lb # [ 18.763565] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4102lb # [ 18.766202] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4103lb # [ 18.768138] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4104lb # [ 18.772525] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4105lb # [ 18.779114] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4106lb # [ 18.781601] envoy[978]: [2026-09-20 12:20:14.448][978][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4107lb # [ 18.807028] envoy[978]: [2026-09-20 12:20:14.582][978][info][main] [source/server/server.cc:531] HTTP header map info:4108lb # [ 18.808411] envoy[978]: [2026-09-20 12:20:14.633][978][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4109lb # [ 18.822193] envoy[978]: [2026-09-20 12:20:14.633][978][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4110lb # [ 18.824103] envoy[978]: [2026-09-20 12:20:14.633][978][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4111lb # [ 18.832122] envoy[978]: [2026-09-20 12:20:14.633][978][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4112lb # [ 18.867299] envoy[978]: [2026-09-20 12:20:14.819][978][info][main] [source/server/server.cc:968] runtime: {}4113lb # [ 18.873212] envoy[978]: [2026-09-20 12:20:14.825][978][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4114lb # [ 18.875473] envoy[978]: [2026-09-20 12:20:14.827][978][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014115lb # [ 18.878151] envoy[978]: [2026-09-20 12:20:14.830][978][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014116lb # [ 18.881298] envoy[978]: [2026-09-20 12:20:14.833][978][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4117lb # [ 18.882946] envoy[978]: [2026-09-20 12:20:14.834][978][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4118lb # [ 18.884626] envoy[978]: [2026-09-20 12:20:14.834][978][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4119lb # [ 18.928318] envoy[978]: [2026-09-20 12:20:14.880][978][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4120worker2 # [ 18.921804] systemd[1]: Reached target Network is Online.4121worker1 # [ 18.910314] systemd[1]: Starting Nix worker-protocol over gRPC...4122worker2 # [ 18.928575] systemd[1]: Starting scheduler-token.service...4123lb # [ 18.948078] envoy[978]: [2026-09-20 12:20:14.899][978][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4124lb # [ 18.950646] envoy[978]: [2026-09-20 12:20:14.902][978][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4125lb # [ 18.953576] envoy[978]: [2026-09-20 12:20:14.902][978][info][main] [source/server/server.cc:1077] starting main dispatch loop4126worker2 # [ 19.025727] systemd[1]: scheduler-token.service: Deactivated successfully.4127worker2 # [ 19.027857] systemd[1]: Finished scheduler-token.service.4128worker2 # [ 19.029528] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 99ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4129worker2 # [ 19.036151] systemd[1]: Starting Nix worker-protocol over gRPC...4130worker1 # [ 19.104491] nix-grpc-daemon[853]: ts=2026-09-20T12:20:14Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4131worker1 # [ 19.113220] nix-grpc-daemon[853]: ts=2026-09-20T12:20:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4132worker1 # [ 19.120698] nix-grpc-daemon[853]: ts=2026-09-20T12:20:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14133worker1 # [ 19.153465] nix-grpc-daemon[853]: ts=2026-09-20T12:20:14Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4134worker1 # [ 19.156763] systemd[1]: Started Nix worker-protocol over gRPC.4135worker2 # [ 19.185131] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4136worker2 # [ 19.193064] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4137worker2 # [ 19.198297] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14138lb # [ 19.221098] envoy[978]: [2026-09-20 12:20:15.172][978][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4139worker1 # [ 19.197994] nix-grpc-daemon[853]: ts=2026-09-20T12:20:14Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4140lb # [ 19.222843] envoy[978]: [2026-09-20 12:20:15.173][978][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4141lb # [ 19.224547] envoy[978]: [2026-09-20 12:20:15.173][978][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4142lb # [ 19.226187] envoy[978]: [2026-09-20 12:20:15.178][978][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4143worker2 # [ 19.235162] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4144worker2 # [ 19.240767] systemd[1]: Started Nix worker-protocol over gRPC.4145worker2 # [ 19.281352] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 19.291433] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker1 # [ 19.398780] nix-grpc-daemon[853]: ts=2026-09-20T12:20:15Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4148worker2 # [ 19.493418] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=scheduler_connected addr=lb:500514149worker2 # [ 19.495432] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4150worker2 # [ 19.696159] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=scheduler_connected addr=lb:500514151worker2 # [ 19.697872] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker1 # [ 19.799328] nix-grpc-daemon[853]: ts=2026-09-20T12:20:15Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4153worker2 # [ 19.899044] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 19.901119] nix-grpc-daemon[857]: ts=2026-09-20T12:20:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker2 # [ 20.102029] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 20.104124] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 20.305354] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 20.307165] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker2 # [ 20.508058] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 20.509993] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 20.602337] nix-grpc-daemon[853]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514162worker1 # [ 20.604833] nix-grpc-daemon[853]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker2 # [ 20.710875] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514164worker2 # [ 20.712568] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker1 # [ 20.806091] nix-grpc-daemon[853]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514166worker1 # [ 20.810284] nix-grpc-daemon[853]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker2 # [ 20.913364] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 20.915135] nix-grpc-daemon[857]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 21.011248] nix-grpc-daemon[853]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514170worker1 # [ 21.015486] nix-grpc-daemon[853]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker2 # [ 21.115858] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514172worker2 # [ 21.117579] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 21.215834] nix-grpc-daemon[853]: ts=2026-09-20T12:20:16Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 21.217619] nix-grpc-daemon[853]: ts=2026-09-20T12:20:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 21.318663] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 21.320340] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 21.418593] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 21.421853] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 21.521332] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 21.523104] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 21.622578] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 21.624411] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 21.723858] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 21.725558] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 21.825290] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 21.828089] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker2 # [ 21.926450] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514188worker2 # [ 21.928230] nix-grpc-daemon[857]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker1 # [ 22.028844] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 22.031141] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 22.128910] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514192worker2 # [ 22.130824] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker1 # [ 22.232206] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 22.235597] nix-grpc-daemon[853]: ts=2026-09-20T12:20:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker2 # [ 22.331885] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514196worker2 # [ 22.333503] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker1 # [ 22.436502] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514198worker1 # [ 22.438588] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker2 # [ 22.534933] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514200worker2 # [ 22.539394] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker1 # [ 22.640968] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514202worker1 # [ 22.644392] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker2 # [ 22.740644] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514204worker2 # [ 22.744664] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 22.842416] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514206worker1 # [ 22.846354] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker2 # [ 22.946170] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 22.950298] nix-grpc-daemon[857]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker1 # [ 23.047443] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514210worker1 # [ 23.051429] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker2 # [ 23.151604] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 23.155574] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker1 # [ 23.251816] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_connected addr=lb:500514214worker1 # [ 23.254043] nix-grpc-daemon[853]: ts=2026-09-20T12:20:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker2 # [ 23.358481] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514216worker2 # [ 23.361801] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker1 # [ 23.454722] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 23.456808] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker2 # [ 23.559942] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514220worker2 # [ 23.564393] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker1 # [ 23.659311] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 23.662707] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker2 # [ 23.765528] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514224worker2 # [ 23.769709] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker1 # [ 23.860281] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514226worker1 # [ 23.864525] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker2 # [ 23.971967] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514228worker2 # [ 23.974221] nix-grpc-daemon[857]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker1 # [ 24.065110] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514230worker1 # [ 24.067339] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker2 # [ 24.175056] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514232worker2 # [ 24.177189] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 24.268270] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_connected addr=lb:500514234worker1 # [ 24.272099] nix-grpc-daemon[853]: ts=2026-09-20T12:20:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker2 # [ 24.379355] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514236worker2 # [ 24.382873] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker1 # [ 24.472845] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514238worker1 # [ 24.475050] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker2 # [ 24.584553] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514240worker2 # [ 24.587105] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 24.676418] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514242worker1 # [ 24.678720] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker2 # [ 24.788588] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514244worker2 # [ 24.791265] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker1 # [ 24.879417] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514246worker1 # [ 24.881709] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker2 # [ 24.991961] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514248worker2 # [ 24.993865] nix-grpc-daemon[857]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker1 # [ 25.082838] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514250worker1 # [ 25.087629] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker2 # [ 25.195194] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514252worker2 # [ 25.199169] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 25.288290] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_connected addr=lb:500514254worker1 # [ 25.290716] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker2 # [ 25.400133] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514256worker2 # [ 25.402134] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker1 # [ 25.491978] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514258worker1 # [ 25.496212] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker2 # [ 25.603596] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514260worker2 # [ 25.607755] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker1 # [ 25.696951] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514262worker1 # [ 25.698800] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker2 # [ 25.808939] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514264worker2 # [ 25.812961] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker1 # [ 25.899369] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514266worker1 # [ 25.901264] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker2 # [ 26.014261] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514268worker2 # [ 26.018750] nix-grpc-daemon[857]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker1 # [ 26.101836] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514270worker1 # [ 26.103601] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker2 # [ 26.219337] nix-grpc-daemon[857]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514272worker2 # [ 26.222241] nix-grpc-daemon[857]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker1 # [ 26.304257] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_connected addr=lb:500514274worker1 # [ 26.306137] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker2 # [ 26.424227] nix-grpc-daemon[857]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514276worker2 # [ 26.428244] nix-grpc-daemon[857]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker1 # [ 26.507143] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514278worker1 # [ 26.511098] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker2 # [ 26.629354] nix-grpc-daemon[857]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 26.631486] nix-grpc-daemon[857]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 26.712090] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514282worker1 # [ 26.714079] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker2 # [ 26.832465] nix-grpc-daemon[857]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514284worker2 # [ 26.834659] nix-grpc-daemon[857]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker1 # [ 26.914602] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514286worker1 # [ 26.916695] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker2 # [ 27.036152] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 27.040222] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 27.117354] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514290worker1 # [ 27.119246] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker2 # [ 27.241299] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514292worker2 # [ 27.243329] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker1 # [ 27.320363] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_connected addr=lb:500514294worker1 # [ 27.324269] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker2 # [ 27.444486] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514296worker2 # [ 27.446590] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker1 # [ 27.525545] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514298worker1 # [ 27.530187] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker2 # [ 27.647373] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514300worker2 # [ 27.648935] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker1 # [ 27.731085] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514302worker1 # [ 27.732855] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker2 # [ 27.850315] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514304worker2 # [ 27.854487] nix-grpc-daemon[857]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker1 # [ 27.934533] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 27.939347] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 28.055639] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 28.059539] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 28.139651] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 28.142078] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 28.260689] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 28.264764] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker1 # [ 28.342829] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_connected addr=lb:500514314worker1 # [ 28.344953] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4315worker2 # [ 28.467698] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514316worker2 # [ 28.470826] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker1 # [ 28.545973] nix-grpc-daemon[853]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514318worker1 # [ 28.550068] nix-grpc-daemon[853]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker2 # [ 28.668734] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514320worker2 # [ 28.670913] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker1 # [ 28.752431] nix-grpc-daemon[853]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514322worker1 # [ 28.756378] nix-grpc-daemon[853]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker2 # [ 28.872375] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514324worker2 # [ 28.876202] nix-grpc-daemon[857]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker1 # [ 28.953272] nix-grpc-daemon[853]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514326worker1 # [ 28.957177] nix-grpc-daemon[853]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker2 # [ 29.077360] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514328worker2 # [ 29.081575] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker1 # [ 29.158163] nix-grpc-daemon[853]: ts=2026-09-20T12:20:24Z level=info event=scheduler_connected addr=lb:500514330worker1 # [ 29.162488] nix-grpc-daemon[853]: ts=2026-09-20T12:20:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker2 # [ 29.282767] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514332worker2 # [ 29.284958] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker1 # [ 29.362874] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514334worker1 # [ 29.365806] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker2 # [ 29.486142] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514336worker2 # [ 29.487931] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker1 # [ 29.567072] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514338worker1 # [ 29.571200] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4339worker2 # [ 29.689255] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514340worker2 # [ 29.691668] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker1 # [ 29.772962] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514342worker1 # [ 29.774721] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker2 # [ 29.892631] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514344worker2 # [ 29.895166] nix-grpc-daemon[857]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4345worker1 # [ 29.974031] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514346worker1 # [ 29.976575] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4347worker2 # [ 30.095870] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514348worker2 # [ 30.098186] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349lb # [ 30.149521] rustfs-bucket-start[1098]: mb s3://farm4350lb # [ 30.155550] systemd[1]: Finished rustfs-bucket.service.4351lb # [ 30.164520] systemd[1]: Starting niks3 server...4352worker1 # [ 30.177273] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_connected addr=lb:500514353worker1 # [ 30.179638] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4354worker2 # [ 30.298984] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514355worker2 # [ 30.301245] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4356lb # [ 30.395362] postgres[1121]: [1121] ERROR: relation "goose_db_version" does not exist at character 364357lb # [ 30.396702] postgres[1121]: [1121] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4358worker1 # [ 30.380564] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514359worker1 # [ 30.383076] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4360lb # [ 30.438814] niks3-server[1114]: 2026/09/20 12:20:26 OK 20241026095416_initial_model.sql (21.63ms)4361lb # [ 30.442427] niks3-server[1114]: 2026/09/20 12:20:26 OK 20251210153512_drop_unused_gin_index.sql (3.84ms)4362lb # [ 30.447665] niks3-server[1114]: 2026/09/20 12:20:26 OK 20251218171726_add_pins.sql (4ms)4363lb # [ 30.452845] niks3-server[1114]: 2026/09/20 12:20:26 OK 20260628120000_add_object_size_and_stats.sql (4.06ms)4364lb # [ 30.457123] niks3-server[1114]: 2026/09/20 12:20:26 OK 20260905000000_add_claims.sql (2.93ms)4365lb # [ 30.461321] niks3-server[1114]: 2026/09/20 12:20:26 OK 20260920000000_drop_claims.sql (3.04ms)4366lb # [ 30.462650] niks3-server[1114]: 2026/09/20 12:20:26 goose: successfully migrated database to version: 202609200000004367lb # [ 30.467934] niks3-server[1114]: 2026/09/20 12:20:26 OK 1_commit_pending_closure.sql (5.23ms)4368lb # [ 30.470192] niks3-server[1114]: 2026/09/20 12:20:26 OK 2_object_stats_trigger.sql (2.24ms)4369lb # [ 30.471368] niks3-server[1114]: 2026/09/20 12:20:26 goose: up to current file version: 24370lb # [ 30.477770] niks3-server[1114]: 2026/09/20 12:20:26 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4371worker2 # [ 30.503092] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514372worker2 # [ 30.504634] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4373lb # [ 30.561619] niks3-server[1114]: 2026/09/20 12:20:26 INFO Read proxy enabled — serving cache objects from S34374lb # [ 30.563588] niks3-server[1114]: 2026/09/20 12:20:26 INFO Using socket-activated listener address=0.0.0.0:57514375lb # [ 30.569596] systemd[1]: Started niks3 server.4376lb # [ 30.570270] niks3-server[1114]: 2026/09/20 12:20:26 INFO systemd watchdog enabled interval=15s4377lb # [ 30.571387] systemd[1]: Reached target Multi-User System.4378lb # [ 30.573904] niks3-server[1114]: 2026/09/20 12:20:26 INFO Starting HTTP server address=0.0.0.0:57514379lb # [ 30.575467] systemd[1]: Startup finished in 914ms (kernel) + 4.364s (initrd) + 25.294s (userspace) = 30.573s.4380lb # [ 30.582192] niks3-server[1114]: 2026/09/20 12:20:26 INFO lead: acquired remote=192.168.1.4:358284381worker2 # [ 30.577496] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_take_over4382worker1 # [ 30.584071] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514383worker1 # [ 30.586624] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4384worker2 # [ 30.703945] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514385worker2 # [ 30.706272] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4386worker1 # [ 30.788120] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514387worker1 # [ 30.790315] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4388worker2 # [ 30.907190] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514389worker2 # [ 30.909440] nix-grpc-daemon[857]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4390worker1 # [ 30.991368] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514391worker1 # [ 30.993892] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4392worker2 # [ 31.110600] nix-grpc-daemon[857]: ts=2026-09-20T12:20:27Z level=info event=scheduler_connected addr=lb:500514393worker2 # [ 31.114572] nix-grpc-daemon[857]: ts=2026-09-20T12:20:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4394worker1 # [ 31.195108] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_connected addr=lb:500514395worker1 # [ 31.199174] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4396lb: (finished: waiting for unit niks3.service, in 18.50 seconds)4397lb: waiting for TCP port 5751 on localhost4398worker2 # [ 31.315359] nix-grpc-daemon[857]: ts=2026-09-20T12:20:27Z level=info event=scheduler_connected addr=lb:500514399worker2 # [ 31.340507] nix-grpc-daemon[857]: ts=2026-09-20T12:20:27Z level=info event=oidc_jwks provider=mock keys=14400worker2 # [ 31.343128] nix-grpc-daemon[857]: ts=2026-09-20T12:20:27Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:381144401worker2 # [ 31.345773] nix-grpc-daemon[857]: ts=2026-09-20T12:20:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04402lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4403lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4404lb: waiting for unit envoy.service4405lb: (finished: waiting for unit envoy.service, in 0.04 seconds)4406lb: waiting for TCP port 50051 on localhost4407worker1 # [ 31.400072] nix-grpc-daemon[853]: ts=2026-09-20T12:20:27Z level=info event=scheduler_connected addr=lb:500514408worker2 # [ 31.413478] nix-grpc-daemon[857]: ts=2026-09-20T12:20:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:381144409lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4410lb # [ 31.431354] envoy[978]: conn peer=[::1]:34774 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24411worker2 # [ 31.417335] nix-grpc-daemon[857]: ts=2026-09-20T12:20:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04412lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4413worker2 # [ 31.528826] systemd[1]: Starting scheduler-token.service...4414worker2 # [ 31.599855] systemd[1]: scheduler-token.service: Deactivated successfully.4415worker2 # [ 31.600974] systemd[1]: Finished scheduler-token.service.4416worker2 # [ 31.602559] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 72ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4417lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4418??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4419 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284420lb: must succeed: curl -sf localhost:9901/clusters4421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4422lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.09 seconds)4423subtest: one node schedules and both builders hold a WorkerSession on it4424lb: must succeed: curl -sf localhost:9901/clusters4425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4426worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4427worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4428lb: must succeed: curl -sf localhost:9901/clusters4429lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4430lb: must succeed: curl -sf localhost:9901/clusters4431lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4432worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4433worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.03 seconds)4434(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.23 seconds)4435subtest: balancer auth: client cert, bearer token, nothing, unknown CN4436worker2 # [ 32.190853] nix-grpc-daemon[857]: ts=2026-09-20T12:20:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:381284437worker2 # [ 32.201476] systemd[1]: Started Nix Daemon.4438worker2 # [ 32.404955] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4439worker2 # [ 32.457515] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5)4440worker2 # [ 32.490679] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)4441worker2 # [ 32.609084] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 241 ms (attempt 3/5)4442worker2 # [ 32.852883] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 453 ms (attempt 4/5)4443worker2 # [ 33.319656] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4444worker1 # [ 33.343364] nix-grpc-daemon[853]: ts=2026-09-20T12:20:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347824445worker1 # [ 33.355565] systemd[1]: Started Nix Daemon.4446worker1 # [ 33.551908] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4447worker1 # [ 33.577855] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 1/5)4448worker1 # [ 33.674702] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 160 ms (attempt 2/5)4449worker1 # [ 33.837449] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 197 ms (attempt 3/5)4450worker1 # [ 34.037888] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 4/5)4451worker1 # [ 34.224569] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4452worker1 # [ 34.243329] nix-grpc-daemon[853]: ts=2026-09-20T12:20:29Z level=debug event=path_info_miss path=f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test error="error: path '/nix/store/f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test' is required, but there is no substituter that can build it"4453worker1 # [ 34.254871] nix-grpc-daemon[853]: ts=2026-09-20T12:20:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347824454worker1 # [ 34.264629] nix-grpc-daemon[853]: ts=2026-09-20T12:20:29Z level=debug event=path_info_miss path=f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test error="error: path '/nix/store/f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test' is required, but there is no substituter that can build it"4455client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4456client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.07 seconds)4457client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4458client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4459worker1 # [ 34.618513] nix-grpc-daemon[853]: ts=2026-09-20T12:20:30Z level=info event=oidc_jwks provider=mock keys=14460worker1 # [ 34.622393] nix-grpc-daemon[853]: ts=2026-09-20T12:20:30Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:347824461worker1 # [ 34.641331] nix-grpc-daemon[853]: ts=2026-09-20T12:20:30Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:347824462worker1 # [ 34.650982] nix-grpc-daemon[853]: ts=2026-09-20T12:20:30Z level=debug event=path_info_miss path=f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test error="error: path '/nix/store/f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test' is required, but there is no substituter that can build it"4463worker1 # [ 34.660798] nix-grpc-daemon[853]: ts=2026-09-20T12:20:30Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:347824464worker1 # [ 34.672673] nix-grpc-daemon[853]: ts=2026-09-20T12:20:30Z level=debug event=path_info_miss path=f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test error="error: path '/nix/store/f3inn73qn76jrc8czkb1dy6mvyjl6giy-nixos-system-client-test' is required, but there is no substituter that can build it"4465client: must succeed: rm -r /var/lib/nix-grpc-store4466client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4467worker2 # [ 34.925906] nix-grpc-daemon[857]: ts=2026-09-20T12:20:30Z level=info event=denied method=QueryMissing cn=- role=none4468worker1 # [ 35.113814] nix-grpc-daemon[853]: ts=2026-09-20T12:20:30Z level=info event=denied method=QueryMissing cn=stranger role=none4469(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.22 seconds)4470subtest: DAG build is scheduled across workers and lands in the cache4471client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24472client # don't know how to build these paths:4473client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4474worker2 # [ 35.338425] nix-grpc-daemon[857]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:381284475worker1 # [ 35.337769] nix-grpc-daemon[853]: ts=2026-09-20T12:20:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347824476worker2 # [ 35.378293] nix-grpc-daemon[857]: ts=2026-09-20T12:20:31Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04477worker2 # [ 35.385645] nix-grpc-daemon[857]: ts=2026-09-20T12:20:31Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14478worker2 # [ 35.436609] nix-grpc-daemon[857]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284479worker2 # [ 35.468451] nix-grpc-daemon[857]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381284480client # copying 1 paths to 'lb:50051'...4481worker2 # [ 35.483176] nix-grpc-daemon[857]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381284482client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4483worker2 # [ 35.512813] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4484worker2 # [ 35.541397] nix-grpc-daemon[857]: ts=2026-09-20T12:20:31Z level=info event=niks3_push_started pid=9384485worker2 # [ 35.819912] nix-grpc-daemon[938]: time=2026-09-20T12:20:31.789Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4486worker2 # [ 36.002356] nix-daemon[926]: accepted connection from pid 950, user nix-grpc-daemon (trusted)4487worker2 # [ 36.019513] nix-daemon[926]: reaped child process 953, status = succeeded4488worker2 # [ 36.157829] nix-daemon[926]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4489worker2 # [ 36.171815] nix-daemon[926]: reaped child process 958, status = succeeded4490lb # [ 36.191567] niks3-server[1114]: 2026/09/20 12:20:32 INFO Received uploads request method=POST path=/api/pending_closures4491worker2 # [ 36.194128] nix-grpc-daemon[938]: time=2026-09-20T12:20:32.167Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4492worker2 # [ 36.196268] nix-grpc-daemon[938]: time=2026-09-20T12:20:32.169Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4493lb # [ 36.234813] niks3-server[1114]: 2026/09/20 12:20:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4494lb # [ 36.238630] niks3-server[1114]: 2026/09/20 12:20:32 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4495lb # [ 36.256469] niks3-server[1114]: 2026/09/20 12:20:32 INFO Signed narinfos id=1 count=14496worker2 # [ 36.247639] nix-grpc-daemon[938]: time=2026-09-20T12:20:32.221Z level=INFO msg="Uploading 1 narinfos"4497lb # [ 36.269105] niks3-server[1114]: 2026/09/20 12:20:32 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4498lb # [ 36.274067] niks3-server[1114]: 2026/09/20 12:20:32 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4499lb # [ 36.284932] niks3-server[1114]: 2026/09/20 12:20:32 INFO Completed upload id=14500worker2 # [ 36.274524] nix-grpc-daemon[938]: time=2026-09-20T12:20:32.247Z level=INFO msg="Upload complete. (432ms)"4501worker2 # [ 36.277334] nix-grpc-daemon[857]: ts=2026-09-20T12:20:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_in=5044502worker2 # [ 36.282528] nix-grpc-daemon[857]: ts=2026-09-20T12:20:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284503lb # [ 36.293683] niks3-server[1114]: 2026/09/20 12:20:32 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4504worker2 # [ 36.290180] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4505worker2 # [ 36.292890] nix-daemon[926]: reaped child process 936, status = succeeded4506client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4507worker2 # [ 36.299888] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4508client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4509client # farm-a> LOG-farm-a-t14510worker2 # [ 36.917462] nix-daemon[926]: reaped child process 962, status = succeeded4511worker2 # [ 37.049196] nix-daemon[926]: accepted connection from pid 978, user nix-grpc-daemon (trusted)4512worker2 # [ 37.064564] nix-daemon[926]: reaped child process 981, status = succeeded4513lb # [ 37.085735] niks3-server[1114]: 2026/09/20 12:20:33 INFO Received uploads request method=POST path=/api/pending_closures4514worker2 # [ 37.083739] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.056Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4515worker2 # [ 37.085561] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.058Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4516lb # [ 37.124109] niks3-server[1114]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4517lb # [ 37.128235] niks3-server[1114]: 2026/09/20 12:20:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4518lb # [ 37.131633] niks3-server[1114]: 2026/09/20 12:20:33 INFO Signed narinfos id=2 count=14519worker2 # [ 37.121679] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.094Z level=INFO msg="Uploading 1 narinfos"4520lb # [ 37.139889] niks3-server[1114]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4521lb # [ 37.144710] niks3-server[1114]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4522lb # [ 37.149883] niks3-server[1114]: 2026/09/20 12:20:33 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4523lb # [ 37.164070] niks3-server[1114]: 2026/09/20 12:20:33 INFO Completed upload id=24524worker2 # [ 37.153533] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.126Z level=INFO msg="Upload complete. (243ms)"4525worker2 # [ 37.157397] nix-grpc-daemon[857]: ts=2026-09-20T12:20:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14526lb # [ 37.174398] niks3-server[1114]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4527worker2 # [ 37.170381] nix-daemon[926]: reaped child process 960, status = succeeded4528worker2 # [ 37.172456] nix-grpc-daemon[857]: ts=2026-09-20T12:20:33Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04529worker2 # [ 37.174646] nix-grpc-daemon[857]: ts=2026-09-20T12:20:33Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14530worker2 # [ 37.219218] nix-grpc-daemon[857]: ts=2026-09-20T12:20:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284531worker2 # [ 37.226295] nix-grpc-daemon[857]: ts=2026-09-20T12:20:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381284532worker2 # [ 37.233961] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.207Z level=INFO msg="All 1 paths already cached"4533client # copying 1 paths to 'lb:50051'...4534worker2 # [ 37.241631] nix-grpc-daemon[857]: ts=2026-09-20T12:20:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381284535client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4536worker2 # [ 37.268350] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4537worker2 # [ 37.432985] nix-daemon[926]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4538worker2 # [ 37.450152] nix-daemon[926]: reaped child process 990, status = succeeded4539worker2 # [ 37.597296] nix-daemon[926]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4540worker2 # [ 37.612447] nix-daemon[926]: reaped child process 996, status = succeeded4541lb # [ 37.633146] niks3-server[1114]: 2026/09/20 12:20:33 INFO Received uploads request method=POST path=/api/pending_closures4542worker2 # [ 37.631524] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.604Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4543worker2 # [ 37.633027] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.604Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4544lb # [ 37.655123] niks3-server[1114]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4545lb # [ 37.662088] niks3-server[1114]: 2026/09/20 12:20:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4546lb # [ 37.666192] niks3-server[1114]: 2026/09/20 12:20:33 INFO Signed narinfos id=3 count=14547worker2 # [ 37.655940] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.629Z level=INFO msg="Uploading 1 narinfos"4548lb # [ 37.670333] niks3-server[1114]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4549lb # [ 37.675515] niks3-server[1114]: 2026/09/20 12:20:33 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4550lb # [ 37.680315] niks3-server[1114]: 2026/09/20 12:20:33 INFO Completed upload id=34551worker2 # [ 37.669309] nix-grpc-daemon[938]: time=2026-09-20T12:20:33.642Z level=INFO msg="Upload complete. (384ms)"4552worker2 # [ 37.671867] nix-grpc-daemon[857]: ts=2026-09-20T12:20:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_in=6244553lb # [ 37.683485] niks3-server[1114]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4554worker2 # [ 37.675259] nix-grpc-daemon[857]: ts=2026-09-20T12:20:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284555worker2 # [ 37.683719] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4556worker2 # [ 37.685688] nix-daemon[926]: reaped child process 985, status = succeeded4557client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4558client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4559worker2 # [ 37.691433] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4560client # farm-b> LOG-farm-b-t14561worker2 # [ 38.506909] nix-daemon[926]: reaped child process 1000, status = succeeded4562worker2 # [ 38.646953] nix-daemon[926]: accepted connection from pid 1015, user nix-grpc-daemon (trusted)4563worker2 # [ 38.662929] nix-daemon[926]: reaped child process 1018, status = succeeded4564lb # [ 38.687461] niks3-server[1114]: 2026/09/20 12:20:34 INFO Received uploads request method=POST path=/api/pending_closures4565worker2 # [ 38.683448] nix-grpc-daemon[938]: time=2026-09-20T12:20:34.656Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4566worker2 # [ 38.685365] nix-grpc-daemon[938]: time=2026-09-20T12:20:34.658Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4567lb # [ 38.712504] niks3-server[1114]: 2026/09/20 12:20:34 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4568lb # [ 38.725825] niks3-server[1114]: 2026/09/20 12:20:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4569lb # [ 38.728924] niks3-server[1114]: 2026/09/20 12:20:34 INFO Signed narinfos id=4 count=14570worker2 # [ 38.718435] nix-grpc-daemon[938]: time=2026-09-20T12:20:34.691Z level=INFO msg="Uploading 1 narinfos"4571lb # [ 38.732691] niks3-server[1114]: 2026/09/20 12:20:34 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4572lb # [ 38.739098] niks3-server[1114]: 2026/09/20 12:20:34 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4573lb # [ 38.742629] niks3-server[1114]: 2026/09/20 12:20:34 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4574lb # [ 38.747283] niks3-server[1114]: 2026/09/20 12:20:34 INFO Completed upload id=44575worker2 # [ 38.736670] nix-grpc-daemon[938]: time=2026-09-20T12:20:34.710Z level=INFO msg="Upload complete. (236ms)"4576lb # [ 38.750084] niks3-server[1114]: 2026/09/20 12:20:34 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4577worker2 # [ 38.739781] nix-grpc-daemon[857]: ts=2026-09-20T12:20:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14578worker2 # [ 38.746465] nix-grpc-daemon[857]: ts=2026-09-20T12:20:34Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04579worker2 # [ 38.748959] nix-grpc-daemon[857]: ts=2026-09-20T12:20:34Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14580worker2 # [ 38.753979] nix-daemon[926]: reaped child process 998, status = succeeded4581worker2 # [ 38.791415] nix-grpc-daemon[857]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284582worker2 # [ 38.800994] nix-grpc-daemon[857]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381284583worker2 # [ 38.809787] nix-grpc-daemon[938]: time=2026-09-20T12:20:34.783Z level=INFO msg="All 2 paths already cached"4584client # copying 1 paths to 'lb:50051'...4585worker2 # [ 38.816400] nix-grpc-daemon[857]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381284586client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4587worker2 # [ 38.826243] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4588lb # [ 38.936238] envoy[978]: conn peer=[2001:db8:1::1]:52400 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=21354589lb # [ 38.938352] envoy[978]: conn peer=[2001:db8:1::1]:46764 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=824590lb # [ 38.940361] envoy[978]: conn peer=[2001:db8:1::1]:46780 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=174591lb # [ 38.942294] envoy[978]: conn peer=[2001:db8:1::1]:46792 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=124592worker2 # [ 38.986056] nix-daemon[926]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4593worker2 # [ 39.001950] nix-daemon[926]: reaped child process 1027, status = succeeded4594worker2 # [ 39.181937] nix-daemon[926]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4595worker2 # [ 39.200328] nix-daemon[926]: reaped child process 1032, status = succeeded4596lb # [ 39.222498] niks3-server[1114]: 2026/09/20 12:20:35 INFO Received uploads request method=POST path=/api/pending_closures4597worker2 # [ 39.220541] nix-grpc-daemon[938]: time=2026-09-20T12:20:35.194Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4598worker2 # [ 39.222379] nix-grpc-daemon[938]: time=2026-09-20T12:20:35.195Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4599lb # [ 39.250841] niks3-server[1114]: 2026/09/20 12:20:35 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4600lb # [ 39.257168] niks3-server[1114]: 2026/09/20 12:20:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4601lb # [ 39.262308] niks3-server[1114]: 2026/09/20 12:20:35 INFO Signed narinfos id=5 count=14602worker2 # [ 39.251756] nix-grpc-daemon[938]: time=2026-09-20T12:20:35.225Z level=INFO msg="Uploading 1 narinfos"4603lb # [ 39.266846] niks3-server[1114]: 2026/09/20 12:20:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4604lb # [ 39.276172] niks3-server[1114]: 2026/09/20 12:20:35 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4605lb # [ 39.281625] niks3-server[1114]: 2026/09/20 12:20:35 INFO Completed upload id=54606worker2 # [ 39.271299] nix-grpc-daemon[938]: time=2026-09-20T12:20:35.244Z level=INFO msg="Upload complete. (436ms)"4607lb # [ 39.284518] niks3-server[1114]: 2026/09/20 12:20:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4608worker2 # [ 39.275120] nix-grpc-daemon[857]: ts=2026-09-20T12:20:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_in=7524609worker2 # [ 39.279573] nix-grpc-daemon[857]: ts=2026-09-20T12:20:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284610worker2 # [ 39.288758] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4611worker2 # [ 39.294526] nix-daemon[926]: reaped child process 1022, status = succeeded4612client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4613client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4614worker2 # [ 39.302110] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4615client # farm-top> LOG-farm-top-t14616worker2 # [ 40.118247] nix-daemon[926]: reaped child process 1036, status = succeeded4617worker2 # [ 40.256100] nix-daemon[926]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4618worker2 # [ 40.271284] nix-daemon[926]: reaped child process 1055, status = succeeded4619lb # [ 40.292951] niks3-server[1114]: 2026/09/20 12:20:36 INFO Received uploads request method=POST path=/api/pending_closures4620worker2 # [ 40.291882] nix-grpc-daemon[938]: time=2026-09-20T12:20:36.265Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4621worker2 # [ 40.295837] nix-grpc-daemon[938]: time=2026-09-20T12:20:36.267Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4622lb # [ 40.323443] niks3-server[1114]: 2026/09/20 12:20:36 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4623lb # [ 40.331854] niks3-server[1114]: 2026/09/20 12:20:36 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4624lb # [ 40.338591] niks3-server[1114]: 2026/09/20 12:20:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4625lb # [ 40.341646] niks3-server[1114]: 2026/09/20 12:20:36 INFO Signed narinfos id=6 count=14626worker2 # [ 40.333473] nix-grpc-daemon[938]: time=2026-09-20T12:20:36.306Z level=INFO msg="Uploading 1 narinfos"4627lb # [ 40.346855] niks3-server[1114]: 2026/09/20 12:20:36 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4628lb # [ 40.356872] niks3-server[1114]: 2026/09/20 12:20:36 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4629lb # [ 40.363829] niks3-server[1114]: 2026/09/20 12:20:36 INFO Completed upload id=64630worker2 # [ 40.355067] nix-grpc-daemon[938]: time=2026-09-20T12:20:36.328Z level=INFO msg="Upload complete. (246ms)"4631lb # [ 40.366636] niks3-server[1114]: 2026/09/20 12:20:36 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4632worker2 # [ 40.361300] nix-grpc-daemon[857]: ts=2026-09-20T12:20:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14633worker2 # [ 40.370147] nix-daemon[926]: reaped child process 1034, status = succeeded4634client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 5.26 seconds)4635client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4636client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.27 seconds)4637client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4638client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4639client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4640client # copying 3 paths...4641client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4642client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4643client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4644client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4645(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.87 seconds)4646subtest: repeat is answered Cached by the scheduler without building4647worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4648worker1 # finding garbage collector roots...4649worker1 # deleting specified paths...4650worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4651worker1 # deleting unused links...4652worker1 # note: hard linking is currently saving 0.0 KiB4653worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.29 seconds)4654worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4655worker2 # finding garbage collector roots...4656worker2 # deleting specified paths...4657worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4658worker2 # deleting unused links...4659worker2 # note: hard linking is currently saving 0.0 KiB4660worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4661client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24662client # don't know how to build these paths:4663client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4664worker1 # [ 41.738459] nix-grpc-daemon[853]: ts=2026-09-20T12:20:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:347824665worker1 # [ 41.746398] nix-grpc-daemon[853]: ts=2026-09-20T12:20:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347824666worker2 # [ 41.784719] nix-grpc-daemon[857]: ts=2026-09-20T12:20:37Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14667client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.24 seconds)4668client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4669client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.17 seconds)4670lb: must succeed: curl -sf localhost:9901/clusters4671lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4672lb: must succeed: curl -sf localhost:9901/clusters4673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4674worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4675worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4676(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.23 seconds)4677subtest: inputs already in the cache are not sent to the scheduler4678lb: must succeed: curl -sf localhost:9901/clusters4679lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4680lb: must succeed: curl -sf localhost:9901/clusters4681lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4682worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4683worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4684client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24685worker1 # [ 42.556430] nix-grpc-daemon[853]: ts=2026-09-20T12:20:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:347824686client # don't know how to build these paths:4687client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4688worker1 # [ 42.566086] nix-grpc-daemon[853]: ts=2026-09-20T12:20:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347824689worker2 # [ 42.591111] nix-grpc-daemon[857]: ts=2026-09-20T12:20:38Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04690worker2 # [ 42.595404] nix-grpc-daemon[857]: ts=2026-09-20T12:20:38Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14691worker2 # [ 42.597917] nix-grpc-daemon[857]: ts=2026-09-20T12:20:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284692worker2 # [ 42.608314] nix-grpc-daemon[857]: ts=2026-09-20T12:20:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381284693worker2 # [ 42.619270] nix-grpc-daemon[938]: time=2026-09-20T12:20:38.592Z level=INFO msg="All 2 paths already cached"4694client # copying 1 paths to 'lb:50051'...4695client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4696worker2 # [ 42.625377] nix-grpc-daemon[857]: ts=2026-09-20T12:20:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381284697worker2 # [ 42.629725] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4698worker2 # [ 42.780270] nix-daemon[926]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4699worker2 # [ 42.794940] nix-daemon[926]: reaped child process 1087, status = succeeded4700worker2 # [ 42.942056] nix-daemon[926]: accepted connection from pid 1089, user nix-grpc-daemon (trusted)4701worker2 # [ 42.956065] nix-daemon[926]: reaped child process 1092, status = succeeded4702lb # [ 42.977282] niks3-server[1114]: 2026/09/20 12:20:38 INFO Received uploads request method=POST path=/api/pending_closures4703worker2 # [ 42.972140] nix-grpc-daemon[938]: time=2026-09-20T12:20:38.945Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4704worker2 # [ 42.973666] nix-grpc-daemon[938]: time=2026-09-20T12:20:38.947Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4705lb # [ 43.001163] niks3-server[1114]: 2026/09/20 12:20:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4706lb # [ 43.004317] niks3-server[1114]: 2026/09/20 12:20:38 INFO Signed narinfos id=7 count=14707worker2 # [ 42.996612] nix-grpc-daemon[938]: time=2026-09-20T12:20:38.970Z level=INFO msg="Uploading 1 narinfos"4708lb # [ 43.011470] niks3-server[1114]: 2026/09/20 12:20:38 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4709lb # [ 43.015332] niks3-server[1114]: 2026/09/20 12:20:38 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4710lb # [ 43.021951] niks3-server[1114]: 2026/09/20 12:20:38 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4711lb # [ 43.026669] niks3-server[1114]: 2026/09/20 12:20:38 INFO Completed upload id=74712worker2 # [ 43.015988] nix-grpc-daemon[938]: time=2026-09-20T12:20:38.989Z level=INFO msg="Upload complete. (379ms)"4713lb # [ 43.030474] niks3-server[1114]: 2026/09/20 12:20:38 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4714worker2 # [ 43.019381] nix-grpc-daemon[857]: ts=2026-09-20T12:20:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_in=7524715worker2 # [ 43.022513] nix-grpc-daemon[857]: ts=2026-09-20T12:20:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284716worker2 # [ 43.029118] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4717worker2 # [ 43.031884] nix-daemon[926]: reaped child process 1082, status = succeeded4718client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4719worker2 # [ 43.037931] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4720client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4721client # farm-top> LOG-farm-top-t1b4722worker2 # [ 44.257687] nix-daemon[926]: reaped child process 1096, status = succeeded4723worker2 # [ 44.418936] nix-daemon[926]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)4724worker2 # [ 44.436986] nix-daemon[926]: reaped child process 1115, status = succeeded4725lb # [ 44.459660] niks3-server[1114]: 2026/09/20 12:20:40 INFO Received uploads request method=POST path=/api/pending_closures4726worker2 # [ 44.457376] nix-grpc-daemon[938]: time=2026-09-20T12:20:40.430Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4727worker2 # [ 44.461298] nix-grpc-daemon[938]: time=2026-09-20T12:20:40.432Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4728lb # [ 44.493500] niks3-server[1114]: 2026/09/20 12:20:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4729lb # [ 44.497940] niks3-server[1114]: 2026/09/20 12:20:40 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4730lb # [ 44.501099] niks3-server[1114]: 2026/09/20 12:20:40 INFO Signed narinfos id=8 count=14731worker2 # [ 44.492270] nix-grpc-daemon[938]: time=2026-09-20T12:20:40.465Z level=INFO msg="Uploading 1 narinfos"4732lb # [ 44.503906] niks3-server[1114]: 2026/09/20 12:20:40 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4733lb # [ 44.508739] niks3-server[1114]: 2026/09/20 12:20:40 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4734lb # [ 44.515836] niks3-server[1114]: 2026/09/20 12:20:40 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4735lb # [ 44.518830] niks3-server[1114]: 2026/09/20 12:20:40 INFO Completed upload id=84736worker2 # [ 44.510264] nix-grpc-daemon[938]: time=2026-09-20T12:20:40.483Z level=INFO msg="Upload complete. (264ms)"4737lb # [ 44.521940] niks3-server[1114]: 2026/09/20 12:20:40 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4738worker2 # [ 44.515467] nix-grpc-daemon[857]: ts=2026-09-20T12:20:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14739worker2 # [ 44.524294] nix-daemon[926]: reaped child process 1094, status = succeeded4740client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.18 seconds)4741client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4742client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.16 seconds)4743lb: must succeed: curl -sf localhost:9901/clusters4744lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4745lb: must succeed: curl -sf localhost:9901/clusters4746lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4747worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4748worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4749(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.63 seconds)4750subtest: two clients wanting the same drv build it once4751client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4752client # [ 44.902862] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4753client # Running as unit: dedup1.service; invocation ID: fd60d2cfa07a4726ab00fa0b0946f74c4754client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.05 seconds)4755client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4756worker2 # [ 45.102461] nix-grpc-daemon[857]: ts=2026-09-20T12:20:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:381284757client # [ 45.092028] nix[892]: don't know how to build these paths:4758client # [ 45.093064] nix[892]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4759worker2 # [ 45.109418] nix-grpc-daemon[857]: ts=2026-09-20T12:20:41Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04760worker2 # [ 45.111986] nix-grpc-daemon[857]: ts=2026-09-20T12:20:41Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14761worker1 # [ 45.100916] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347824762worker1 # [ 45.111990] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347824763client # [ 45.117888] nix[892]: copying 1 paths to 'lb:50051'...4764worker1 # [ 45.119171] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347824765client # [ 45.118968] nix[892]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4766worker1 # [ 45.124841] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4767worker1 # [ 45.141549] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=info event=niks3_push_started pid=9374768worker1 # [ 45.409398] nix-grpc-daemon[937]: time=2026-09-20T12:20:41.047Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4769worker1 # [ 45.609507] nix-daemon[911]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4770worker1 # [ 45.627411] nix-daemon[911]: reaped child process 952, status = succeeded4771worker1 # [ 45.771704] nix-daemon[911]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4772worker1 # [ 45.785568] nix-daemon[911]: reaped child process 957, status = succeeded4773lb # [ 45.818094] niks3-server[1114]: 2026/09/20 12:20:41 INFO Received uploads request method=POST path=/api/pending_closures4774worker1 # [ 45.803028] nix-grpc-daemon[937]: time=2026-09-20T12:20:41.444Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4775worker1 # [ 45.805370] nix-grpc-daemon[937]: time=2026-09-20T12:20:41.446Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4776lb # [ 45.847969] niks3-server[1114]: 2026/09/20 12:20:41 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4777lb # [ 45.852721] niks3-server[1114]: 2026/09/20 12:20:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4778lb # [ 45.855437] niks3-server[1114]: 2026/09/20 12:20:41 INFO Signed narinfos id=9 count=14779worker1 # [ 45.833472] nix-grpc-daemon[937]: time=2026-09-20T12:20:41.475Z level=INFO msg="Uploading 1 narinfos"4780lb # [ 45.863414] niks3-server[1114]: 2026/09/20 12:20:41 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4781lb # [ 45.870562] niks3-server[1114]: 2026/09/20 12:20:41 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4782lb # [ 45.875426] niks3-server[1114]: 2026/09/20 12:20:41 INFO Completed upload id=94783worker1 # [ 45.853447] nix-grpc-daemon[937]: time=2026-09-20T12:20:41.495Z level=INFO msg="Upload complete. (418ms)"4784worker1 # [ 45.855794] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=0 paths=1 nar_bytes_in=4324785lb # [ 45.878418] niks3-server[1114]: 2026/09/20 12:20:41 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4786worker1 # [ 45.862454] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347824787worker1 # [ 45.867611] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4788worker1 # [ 45.870175] nix-daemon[911]: reaped child process 935, status = succeeded4789client # [ 45.874958] nix[892]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4790worker1 # [ 45.876139] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4791client # [ 45.975829] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4792client # Running as unit: dedup2.service; invocation ID: c57693d186784bb3a342840dc1b2bbb64793client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.07 seconds)4794worker2 # [ 46.212780] nix-grpc-daemon[857]: ts=2026-09-20T12:20:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:381284795client # [ 46.205426] nix[908]: don't know how to build these paths:4796client # [ 46.206412] nix[908]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4797worker2 # [ 46.227501] nix-grpc-daemon[857]: ts=2026-09-20T12:20:42Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04798worker1 # [ 46.223639] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347824799client # [ 46.227902] nix[908]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4800client: must succeed: sleep 24801lb # [ 48.936824] envoy[978]: conn peer=[2001:db8:1::1]:46808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9438 tx=3806 ms=50744802lb # [ 48.940722] envoy[978]: conn peer=[2001:db8:1::1]:46818 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=504803lb # [ 48.944324] envoy[978]: conn peer=[2001:db8:1::1]:46828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4152 tx=1684 ms=19584804client: (finished: must succeed: sleep 2, in 2.05 seconds)4805??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4806 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394807client: waiting for success: ! systemctl is-active dedup1 dedup24808??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4809 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394810worker1 # [ 49.496528] nix-daemon[911]: reaped child process 961, status = succeeded4811worker1 # [ 49.637107] nix-daemon[911]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4812worker1 # [ 49.652293] nix-daemon[911]: reaped child process 993, status = succeeded4813lb # [ 49.686207] niks3-server[1114]: 2026/09/20 12:20:45 INFO Received uploads request method=POST path=/api/pending_closures4814worker1 # [ 49.681990] nix-grpc-daemon[937]: time=2026-09-20T12:20:45.323Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4815worker1 # [ 49.684214] nix-grpc-daemon[937]: time=2026-09-20T12:20:45.325Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4816lb # [ 49.739808] niks3-server[1114]: 2026/09/20 12:20:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4817lb # [ 49.743271] niks3-server[1114]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4818lb # [ 49.746072] niks3-server[1114]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4819lb # [ 49.749413] niks3-server[1114]: 2026/09/20 12:20:45 INFO Signed narinfos id=10 count=14820worker1 # [ 49.730039] nix-grpc-daemon[937]: time=2026-09-20T12:20:45.370Z level=INFO msg="Uploading 1 narinfos"4821lb # [ 49.753518] niks3-server[1114]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4822lb # [ 49.759282] niks3-server[1114]: 2026/09/20 12:20:45 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4823worker1 # [ 49.742838] nix-grpc-daemon[937]: time=2026-09-20T12:20:45.384Z level=INFO msg="Upload complete. (255ms)"4824lb # [ 49.765325] niks3-server[1114]: 2026/09/20 12:20:45 INFO Completed upload id=104825lb # [ 49.767571] niks3-server[1114]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4826worker1 # [ 49.746392] nix-grpc-daemon[853]: ts=2026-09-20T12:20:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14827worker1 # [ 49.750353] nix-grpc-daemon[853]: ts=2026-09-20T12:20:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14828worker1 # [ 49.755850] nix-daemon[911]: reaped child process 959, status = succeeded4829client # [ 49.772411] systemd[1]: dedup1.service: Deactivated successfully.4830client # [ 49.774049] systemd[1]: dedup1.service: Consumed 109ms CPU time over 4.871s wall clock time, 39.4M memory peak, 8.4K incoming IP traffic, 8.9K outgoing IP traffic.4831client # [ 49.816132] systemd[1]: dedup2.service: Deactivated successfully.4832client # [ 49.818610] systemd[1]: dedup2.service: Consumed 127ms CPU time over 3.842s wall clock time, 31.2M memory peak, 7.4K incoming IP traffic, 6.3K outgoing IP traffic.4833client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.07 seconds)4834client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4835client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.03 seconds)4836worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4837worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4838worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4839worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4840worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4841worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4842worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4843worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4844(finished: subtest: two clients wanting the same drv build it once, in 5.80 seconds)4845subtest: build hook: nix-daemon with builders = grpc://lb4846client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4847client # [ 50.836442] systemd[1]: Started Nix Daemon.4848client # [ 51.020450] nix-daemon[950]: accepted connection from pid 947, user root (trusted)4849worker1 # [ 51.202643] nix-grpc-daemon[853]: ts=2026-09-20T12:20:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:347824850worker2 # [ 51.225273] nix-grpc-daemon[857]: ts=2026-09-20T12:20:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381284851worker2 # [ 51.234981] nix-grpc-daemon[857]: ts=2026-09-20T12:20:47Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04852worker2 # [ 51.237421] nix-grpc-daemon[857]: ts=2026-09-20T12:20:47Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14853worker2 # [ 51.279849] nix-grpc-daemon[857]: ts=2026-09-20T12:20:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284854worker2 # [ 51.291302] nix-grpc-daemon[857]: ts=2026-09-20T12:20:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381284855worker2 # [ 51.301826] nix-grpc-daemon[857]: ts=2026-09-20T12:20:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381284856worker2 # [ 51.313485] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4857worker2 # [ 51.466775] nix-daemon[926]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)4858worker2 # [ 51.484701] nix-daemon[926]: reaped child process 1154, status = succeeded4859worker2 # [ 51.649739] nix-daemon[926]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)4860worker2 # [ 51.666590] nix-daemon[926]: reaped child process 1159, status = succeeded4861lb # [ 51.687602] niks3-server[1114]: 2026/09/20 12:20:47 INFO Received uploads request method=POST path=/api/pending_closures4862worker2 # [ 51.693350] nix-grpc-daemon[938]: time=2026-09-20T12:20:47.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4863worker2 # [ 51.696036] nix-grpc-daemon[938]: time=2026-09-20T12:20:47.668Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4864lb # [ 51.723219] niks3-server[1114]: 2026/09/20 12:20:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4865lb # [ 51.726049] niks3-server[1114]: 2026/09/20 12:20:47 INFO Signed narinfos id=11 count=14866worker2 # [ 51.717476] nix-grpc-daemon[938]: time=2026-09-20T12:20:47.690Z level=INFO msg="Uploading 1 narinfos"4867lb # [ 51.729993] niks3-server[1114]: 2026/09/20 12:20:47 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4868lb # [ 51.740978] niks3-server[1114]: 2026/09/20 12:20:47 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4869lb # [ 51.742800] niks3-server[1114]: 2026/09/20 12:20:47 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4870lb # [ 51.750555] niks3-server[1114]: 2026/09/20 12:20:47 INFO Completed upload id=114871worker2 # [ 51.740322] nix-grpc-daemon[938]: time=2026-09-20T12:20:47.713Z level=INFO msg="Upload complete. (414ms)"4872worker2 # [ 51.745157] nix-grpc-daemon[857]: ts=2026-09-20T12:20:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_in=5124873lb # [ 51.758908] niks3-server[1114]: 2026/09/20 12:20:47 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4874worker2 # [ 51.748693] nix-grpc-daemon[857]: ts=2026-09-20T12:20:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284875worker2 # [ 51.755920] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4876worker2 # [ 51.770235] nix-daemon[926]: reaped child process 1149, status = succeeded4877worker2 # [ 51.775291] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4878worker2 # [ 52.796135] nix-daemon[926]: reaped child process 1163, status = succeeded4879worker2 # [ 52.934794] nix-daemon[926]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)4880worker2 # [ 52.948944] nix-daemon[926]: reaped child process 1181, status = succeeded4881lb # [ 52.971310] niks3-server[1114]: 2026/09/20 12:20:48 INFO Received uploads request method=POST path=/api/pending_closures4882worker2 # [ 52.965850] nix-grpc-daemon[938]: time=2026-09-20T12:20:48.938Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4883worker2 # [ 52.967635] nix-grpc-daemon[938]: time=2026-09-20T12:20:48.941Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4884lb # [ 52.999267] niks3-server[1114]: 2026/09/20 12:20:48 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4885lb # [ 53.002102] niks3-server[1114]: 2026/09/20 12:20:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4886lb # [ 53.003878] niks3-server[1114]: 2026/09/20 12:20:48 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4887lb # [ 53.006862] niks3-server[1114]: 2026/09/20 12:20:48 INFO Signed narinfos id=12 count=14888worker2 # [ 52.996518] nix-grpc-daemon[938]: time=2026-09-20T12:20:48.969Z level=INFO msg="Uploading 1 narinfos"4889lb # [ 53.009729] niks3-server[1114]: 2026/09/20 12:20:48 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4890lb # [ 53.015057] niks3-server[1114]: 2026/09/20 12:20:48 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4891lb # [ 53.021355] niks3-server[1114]: 2026/09/20 12:20:48 INFO Completed upload id=124892worker2 # [ 53.010669] nix-grpc-daemon[938]: time=2026-09-20T12:20:48.984Z level=INFO msg="Upload complete. (223ms)"4893lb # [ 53.022936] niks3-server[1114]: 2026/09/20 12:20:48 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4894worker2 # [ 53.013864] nix-grpc-daemon[857]: ts=2026-09-20T12:20:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14895worker2 # [ 53.023641] nix-daemon[926]: reaped child process 1161, status = succeeded4896worker1 # [ 53.049899] nix-grpc-daemon[853]: ts=2026-09-20T12:20:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347824897worker1 # [ 53.057791] nix-grpc-daemon[853]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4898worker2 # [ 53.090992] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:381284899worker2 # [ 53.094389] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_out=1284900worker1 # [ 53.310157] nix-grpc-daemon[853]: ts=2026-09-20T12:20:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:347824901worker1 # [ 53.317184] nix-grpc-daemon[853]: ts=2026-09-20T12:20:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347824902worker1 # [ 53.336426] nix-grpc-daemon[937]: time=2026-09-20T12:20:48.977Z level=INFO msg="All 1 paths already cached"4903worker2 # [ 53.360890] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04904worker2 # [ 53.365394] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14905worker2 # [ 53.371431] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284906worker2 # [ 53.383441] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381284907worker2 # [ 53.395700] nix-grpc-daemon[938]: time=2026-09-20T12:20:49.369Z level=INFO msg="All 1 paths already cached"4908worker2 # [ 53.399768] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381284909worker2 # [ 53.403712] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4910worker2 # [ 53.565461] nix-daemon[926]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)4911worker2 # [ 53.582339] nix-daemon[926]: reaped child process 1190, status = succeeded4912worker2 # [ 53.729629] nix-daemon[926]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)4913worker2 # [ 53.747296] nix-daemon[926]: reaped child process 1195, status = succeeded4914lb # [ 53.771923] niks3-server[1114]: 2026/09/20 12:20:49 INFO Received uploads request method=POST path=/api/pending_closures4915worker2 # [ 53.826263] nix-grpc-daemon[938]: time=2026-09-20T12:20:49.799Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4916worker2 # [ 53.828048] nix-grpc-daemon[938]: time=2026-09-20T12:20:49.799Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4917lb # [ 53.861679] niks3-server[1114]: 2026/09/20 12:20:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4918lb # [ 53.866471] niks3-server[1114]: 2026/09/20 12:20:49 INFO Signed narinfos id=13 count=14919worker2 # [ 53.859785] nix-grpc-daemon[938]: time=2026-09-20T12:20:49.833Z level=INFO msg="Uploading 1 narinfos"4920lb # [ 53.880224] niks3-server[1114]: 2026/09/20 12:20:49 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4921lb # [ 53.887623] niks3-server[1114]: 2026/09/20 12:20:49 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4922lb # [ 53.900910] niks3-server[1114]: 2026/09/20 12:20:49 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4923lb # [ 53.903716] niks3-server[1114]: 2026/09/20 12:20:49 INFO Completed upload id=134924worker2 # [ 53.893133] nix-grpc-daemon[938]: time=2026-09-20T12:20:49.866Z level=INFO msg="Upload complete. (481ms)"4925lb # [ 53.906802] niks3-server[1114]: 2026/09/20 12:20:49 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4926worker2 # [ 53.896098] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_in=6244927worker2 # [ 53.898776] nix-grpc-daemon[857]: ts=2026-09-20T12:20:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284928worker2 # [ 53.907383] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4929worker2 # [ 53.913645] nix-daemon[926]: reaped child process 1185, status = succeeded4930worker2 # [ 53.917438] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4931worker2 # [ 55.136796] nix-daemon[926]: reaped child process 1199, status = succeeded4932worker2 # [ 55.290614] nix-daemon[926]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)4933worker2 # [ 55.307684] nix-daemon[926]: reaped child process 1217, status = succeeded4934lb # [ 55.328976] niks3-server[1114]: 2026/09/20 12:20:51 INFO Received uploads request method=POST path=/api/pending_closures4935worker2 # [ 55.330668] nix-grpc-daemon[938]: time=2026-09-20T12:20:51.303Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4936worker2 # [ 55.335222] nix-grpc-daemon[938]: time=2026-09-20T12:20:51.306Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4937lb # [ 55.363867] niks3-server[1114]: 2026/09/20 12:20:51 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4938lb # [ 55.367336] niks3-server[1114]: 2026/09/20 12:20:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4939lb # [ 55.370769] niks3-server[1114]: 2026/09/20 12:20:51 INFO Signed narinfos id=14 count=14940worker2 # [ 55.359959] nix-grpc-daemon[938]: time=2026-09-20T12:20:51.333Z level=INFO msg="Uploading 1 narinfos"4941lb # [ 55.373870] niks3-server[1114]: 2026/09/20 12:20:51 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4942lb # [ 55.376134] niks3-server[1114]: 2026/09/20 12:20:51 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4943lb # [ 55.383254] niks3-server[1114]: 2026/09/20 12:20:51 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4944lb # [ 55.387736] niks3-server[1114]: 2026/09/20 12:20:51 INFO Completed upload id=144945lb # [ 55.389022] niks3-server[1114]: 2026/09/20 12:20:51 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4946worker2 # [ 55.377285] nix-grpc-daemon[938]: time=2026-09-20T12:20:51.350Z level=INFO msg="Upload complete. (250ms)"4947worker2 # [ 55.382160] nix-grpc-daemon[857]: ts=2026-09-20T12:20:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14948worker2 # [ 55.387824] nix-grpc-daemon[857]: ts=2026-09-20T12:20:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:381284949worker2 # [ 55.393705] nix-daemon[926]: reaped child process 1197, status = succeeded4950worker1 # [ 55.393458] nix-grpc-daemon[853]: ts=2026-09-20T12:20:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347824951worker1 # [ 55.401435] nix-grpc-daemon[853]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4952worker1 # [ 55.413138] nix-grpc-daemon[853]: ts=2026-09-20T12:20:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=0 paths=1 nar_bytes_out=1764953worker1 # [ 55.689075] nix-grpc-daemon[853]: ts=2026-09-20T12:20:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:347824954worker1 # [ 55.695205] nix-grpc-daemon[853]: ts=2026-09-20T12:20:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347824955worker1 # [ 55.700873] nix-grpc-daemon[937]: time=2026-09-20T12:20:51.340Z level=INFO msg="All 2 paths already cached"4956worker2 # [ 55.725438] nix-grpc-daemon[857]: ts=2026-09-20T12:20:51Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04957worker2 # [ 55.729583] nix-grpc-daemon[857]: ts=2026-09-20T12:20:51Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14958worker2 # [ 55.780867] nix-grpc-daemon[857]: ts=2026-09-20T12:20:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284959worker2 # [ 55.795281] nix-grpc-daemon[857]: ts=2026-09-20T12:20:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381284960worker2 # [ 55.808172] nix-grpc-daemon[938]: time=2026-09-20T12:20:51.781Z level=INFO msg="All 2 paths already cached"4961worker2 # [ 55.816934] nix-grpc-daemon[857]: ts=2026-09-20T12:20:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381284962worker2 # [ 55.828199] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4963worker2 # [ 56.016579] nix-daemon[926]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)4964worker2 # [ 56.034474] nix-daemon[926]: reaped child process 1226, status = succeeded4965worker2 # [ 56.181293] nix-daemon[926]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)4966worker2 # [ 56.198895] nix-daemon[926]: reaped child process 1231, status = succeeded4967lb # [ 56.218417] niks3-server[1114]: 2026/09/20 12:20:52 INFO Received uploads request method=POST path=/api/pending_closures4968worker2 # [ 56.215692] nix-grpc-daemon[938]: time=2026-09-20T12:20:52.189Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4969worker2 # [ 56.217848] nix-grpc-daemon[938]: time=2026-09-20T12:20:52.190Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4970lb # [ 56.245358] niks3-server[1114]: 2026/09/20 12:20:52 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4971lb # [ 56.254555] niks3-server[1114]: 2026/09/20 12:20:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4972lb # [ 56.260518] niks3-server[1114]: 2026/09/20 12:20:52 INFO Signed narinfos id=15 count=14973worker2 # [ 56.253241] nix-grpc-daemon[938]: time=2026-09-20T12:20:52.226Z level=INFO msg="Uploading 1 narinfos"4974lb # [ 56.265807] niks3-server[1114]: 2026/09/20 12:20:52 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4975lb # [ 56.277614] niks3-server[1114]: 2026/09/20 12:20:52 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4976lb # [ 56.284725] niks3-server[1114]: 2026/09/20 12:20:52 INFO Completed upload id=154977worker2 # [ 56.274481] nix-grpc-daemon[938]: time=2026-09-20T12:20:52.247Z level=INFO msg="Upload complete. (431ms)"4978lb # [ 56.289094] niks3-server[1114]: 2026/09/20 12:20:52 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4979worker2 # [ 56.278119] nix-grpc-daemon[857]: ts=2026-09-20T12:20:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_in=7524980worker2 # [ 56.281183] nix-grpc-daemon[857]: ts=2026-09-20T12:20:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381284981worker2 # [ 56.286438] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4982worker2 # [ 56.296293] nix-daemon[926]: reaped child process 1221, status = succeeded4983worker2 # [ 56.300267] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)4984worker2 # [ 57.320094] nix-daemon[926]: reaped child process 1235, status = succeeded4985worker2 # [ 57.449138] nix-daemon[926]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)4986worker2 # [ 57.466574] nix-daemon[926]: reaped child process 1253, status = succeeded4987lb # [ 57.489448] niks3-server[1114]: 2026/09/20 12:20:53 INFO Received uploads request method=POST path=/api/pending_closures4988worker2 # [ 57.487504] nix-grpc-daemon[938]: time=2026-09-20T12:20:53.460Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4989worker2 # [ 57.494767] nix-grpc-daemon[938]: time=2026-09-20T12:20:53.462Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4990lb # [ 57.537715] niks3-server[1114]: 2026/09/20 12:20:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4991lb # [ 57.541983] niks3-server[1114]: 2026/09/20 12:20:53 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4992lb # [ 57.548834] niks3-server[1114]: 2026/09/20 12:20:53 INFO Signed narinfos id=16 count=14993worker2 # [ 57.541582] nix-grpc-daemon[938]: time=2026-09-20T12:20:53.514Z level=INFO msg="Uploading 1 narinfos"4994lb # [ 57.554421] niks3-server[1114]: 2026/09/20 12:20:53 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4995lb # [ 57.562929] niks3-server[1114]: 2026/09/20 12:20:53 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4996lb # [ 57.570329] niks3-server[1114]: 2026/09/20 12:20:53 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4997lb # [ 57.575147] niks3-server[1114]: 2026/09/20 12:20:53 INFO Completed upload id=164998worker2 # [ 57.566579] nix-grpc-daemon[938]: time=2026-09-20T12:20:53.539Z level=INFO msg="Upload complete. (257ms)"4999lb # [ 57.578492] niks3-server[1114]: 2026/09/20 12:20:53 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5000worker2 # [ 57.574130] nix-grpc-daemon[857]: ts=2026-09-20T12:20:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15001worker2 # [ 57.586852] nix-daemon[926]: reaped child process 1233, status = succeeded5002worker2 # [ 57.618590] nix-grpc-daemon[857]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:381285003worker1 # [ 57.626103] nix-grpc-daemon[853]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347825004worker1 # [ 57.638403] nix-grpc-daemon[853]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5005worker1 # [ 57.653189] nix-grpc-daemon[853]: ts=2026-09-20T12:20:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=0 paths=1 nar_bytes_out=2325006client # [ 57.766327] nix-daemon[950]: reaped child process 952, status = succeeded5007client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.12 seconds)5008client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5009client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5010client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5011client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)5012client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5013client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5014(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.22 seconds)5015subtest: an input only one worker has still reaches the builder5016client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5017client # this derivation will be built:5018client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5019client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5020client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.44 seconds)5021client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5022client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)5023worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5024worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.17 seconds)5025worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5026worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5027client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25028client # [ 58.902028] nix-daemon[950]: accepted connection from pid 1033, user root (trusted)5029client # this derivation will be built:5030client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5031lb # [ 58.936387] envoy[978]: conn peer=[2001:db8:1::1]:35344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1353 ms=46695032lb # [ 58.938124] envoy[978]: conn peer=[2001:db8:1::1]:35348 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=36045033lb # [ 58.939741] envoy[978]: conn peer=[2001:db8:1::1]:35356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3400 tx=1774 ms=18925034lb # [ 58.941381] envoy[978]: conn peer=[2001:db8:1::1]:48622 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=155035lb # [ 58.942919] envoy[978]: conn peer=[2001:db8:1::1]:48638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=725036lb # [ 58.944713] envoy[978]: conn peer=[2001:db8:1::1]:48626 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3867 tx=1880 ms=21535037lb # [ 58.946406] envoy[978]: conn peer=[2001:db8:1::1]:48650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4319 tx=2098 ms=20215038lb # [ 58.948100] envoy[978]: conn peer=[2001:db8:1::1]:48658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=835039worker2 # [ 59.094502] nix-grpc-daemon[857]: ts=2026-09-20T12:20:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:381285040worker1 # [ 59.087182] nix-grpc-daemon[853]: ts=2026-09-20T12:20:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347825041client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5042worker1 # [ 59.230614] nix-daemon[911]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)5043worker1 # [ 59.243047] nix-daemon[911]: reaped child process 1016, status = succeeded5044lb # [ 59.275432] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received uploads request method=POST path=/api/pending_closures5045worker1 # [ 59.260904] nix-grpc-daemon[937]: time=2026-09-20T12:20:54.901Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5046worker1 # [ 59.262405] nix-grpc-daemon[937]: time=2026-09-20T12:20:54.901Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5047lb # [ 59.294813] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5048lb # [ 59.297914] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5049lb # [ 59.300163] niks3-server[1114]: 2026/09/20 12:20:55 INFO Signed narinfos id=17 count=15050worker1 # [ 59.278331] nix-grpc-daemon[937]: time=2026-09-20T12:20:54.919Z level=INFO msg="Uploading 1 narinfos"5051lb # [ 59.303283] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5052lb # [ 59.307178] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5053lb # [ 59.311110] niks3-server[1114]: 2026/09/20 12:20:55 INFO Completed upload id=175054worker1 # [ 59.289080] nix-grpc-daemon[937]: time=2026-09-20T12:20:54.930Z level=INFO msg="Upload complete. (199ms)"5055lb # [ 59.312562] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5056worker2 # [ 59.311577] nix-grpc-daemon[857]: ts=2026-09-20T12:20:55Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05057worker2 # [ 59.313552] nix-grpc-daemon[857]: ts=2026-09-20T12:20:55Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15058worker1 # [ 59.344258] nix-grpc-daemon[853]: ts=2026-09-20T12:20:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347825059worker1 # [ 59.351192] nix-grpc-daemon[853]: ts=2026-09-20T12:20:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347825060client # copying 2 paths to 'lb:50051'...5061worker1 # [ 59.358541] nix-grpc-daemon[853]: ts=2026-09-20T12:20:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347825062client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5063client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5064worker1 # [ 59.368395] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5065worker1 # [ 59.531676] nix-daemon[911]: accepted connection from pid 1020, user nix-grpc-daemon (trusted)5066worker1 # [ 59.547328] nix-daemon[911]: reaped child process 1023, status = succeeded5067worker1 # [ 59.692968] nix-daemon[911]: accepted connection from pid 1025, user nix-grpc-daemon (trusted)5068worker1 # [ 59.707321] nix-daemon[911]: reaped child process 1028, status = succeeded5069lb # [ 59.737456] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received uploads request method=POST path=/api/pending_closures5070lb # [ 59.744339] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received uploads request method=POST path=/api/pending_closures5071worker1 # [ 59.725907] nix-grpc-daemon[937]: time=2026-09-20T12:20:55.367Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5072worker1 # [ 59.727873] nix-grpc-daemon[937]: time=2026-09-20T12:20:55.369Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5073worker1 # [ 59.730805] nix-grpc-daemon[937]: time=2026-09-20T12:20:55.371Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5074lb # [ 59.766311] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5075lb # [ 59.771354] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5076lb # [ 59.774252] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5077lb # [ 59.780856] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5078lb # [ 59.784168] niks3-server[1114]: 2026/09/20 12:20:55 INFO Signed narinfos id=18 count=15079lb # [ 59.786500] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5080lb # [ 59.788968] niks3-server[1114]: 2026/09/20 12:20:55 INFO Signed narinfos id=19 count=25081worker1 # [ 59.767644] nix-grpc-daemon[937]: time=2026-09-20T12:20:55.408Z level=INFO msg="Uploading 3 narinfos"5082lb # [ 59.793196] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5083lb # [ 59.807059] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5084lb # [ 59.812404] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5085lb # [ 59.815818] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5086lb # [ 59.820167] niks3-server[1114]: 2026/09/20 12:20:55 INFO Completed upload id=185087lb # [ 59.821967] niks3-server[1114]: 2026/09/20 12:20:55 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5088lb # [ 59.824690] niks3-server[1114]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5089lb # [ 59.827574] niks3-server[1114]: 2026/09/20 12:20:55 INFO Completed upload id=195090worker1 # [ 59.807158] nix-grpc-daemon[937]: time=2026-09-20T12:20:55.448Z level=INFO msg="Upload complete. (431ms)"5091worker1 # [ 59.812163] nix-grpc-daemon[853]: ts=2026-09-20T12:20:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=0 paths=2 nar_bytes_in=9765092worker1 # [ 59.816920] nix-grpc-daemon[853]: ts=2026-09-20T12:20:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347825093worker1 # [ 59.825867] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5094worker1 # [ 59.832420] nix-daemon[911]: reaped child process 1018, status = succeeded5095worker1 # [ 59.837054] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5096client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5097client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5098worker1 # [ 61.255097] nix-daemon[911]: reaped child process 1032, status = succeeded5099worker1 # [ 61.387078] nix-daemon[911]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5100worker1 # [ 61.401106] nix-daemon[911]: reaped child process 1050, status = succeeded5101lb # [ 61.437342] niks3-server[1114]: 2026/09/20 12:20:57 INFO Received uploads request method=POST path=/api/pending_closures5102worker1 # [ 61.427457] nix-grpc-daemon[937]: time=2026-09-20T12:20:57.067Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5103lb # [ 61.471593] niks3-server[1114]: 2026/09/20 12:20:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5104lb # [ 61.476790] niks3-server[1114]: 2026/09/20 12:20:57 INFO Signed narinfos id=20 count=15105worker1 # [ 61.458950] nix-grpc-daemon[937]: time=2026-09-20T12:20:57.100Z level=INFO msg="Uploading 1 narinfos"5106lb # [ 61.485989] niks3-server[1114]: 2026/09/20 12:20:57 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5107lb # [ 61.500589] niks3-server[1114]: 2026/09/20 12:20:57 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5108lb # [ 61.504759] niks3-server[1114]: 2026/09/20 12:20:57 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5109lb # [ 61.511553] niks3-server[1114]: 2026/09/20 12:20:57 INFO Completed upload id=205110worker1 # [ 61.489541] nix-grpc-daemon[937]: time=2026-09-20T12:20:57.131Z level=INFO msg="Upload complete. (244ms)"5111lb # [ 61.514873] niks3-server[1114]: 2026/09/20 12:20:57 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5112worker1 # [ 61.493220] nix-grpc-daemon[853]: ts=2026-09-20T12:20:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15113worker1 # [ 61.499966] nix-daemon[911]: reaped child process 1030, status = succeeded5114client # copying 1 paths...5115worker1 # [ 61.537572] nix-grpc-daemon[853]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347825116client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5117worker1 # [ 61.547427] nix-grpc-daemon[853]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347825118worker1 # [ 61.549336] nix-grpc-daemon[853]: ts=2026-09-20T12:20:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=0 paths=1 nar_bytes_out=1205119client # [ 61.641158] nix-daemon[950]: reaped child process 1036, status = succeeded5120client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.93 seconds)5121client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25122client # [ 61.800861] nix-daemon[950]: accepted connection from pid 1055, user root (trusted)5123client # this derivation will be built:5124client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5125worker1 # [ 61.967241] nix-grpc-daemon[853]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:347825126worker1 # [ 61.970097] nix-grpc-daemon[853]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347825127client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5128worker1 # [ 61.976446] nix-grpc-daemon[937]: time=2026-09-20T12:20:57.618Z level=INFO msg="All 1 paths already cached"5129worker2 # [ 62.001365] nix-grpc-daemon[857]: ts=2026-09-20T12:20:57Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05130worker2 # [ 62.004809] nix-grpc-daemon[857]: ts=2026-09-20T12:20:57Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15131worker2 # [ 62.007744] nix-grpc-daemon[857]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381285132worker2 # [ 62.017378] nix-grpc-daemon[857]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381285133client # copying 1 paths to 'lb:50051'...5134client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5135worker2 # [ 62.041678] nix-grpc-daemon[857]: ts=2026-09-20T12:20:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:381285136worker2 # [ 62.049351] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)5137worker2 # [ 62.060592] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5)5138worker2 # [ 62.126705] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 2/5)5139worker2 # [ 62.230039] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 347 ms (attempt 3/5)5140worker2 # [ 62.577593] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 558 ms (attempt 4/5)5141worker2 # [ 63.143163] nix-grpc-daemon[857]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5142worker2 # [ 63.148847] nix-grpc-daemon[857]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5143worker2 # [ 63.334783] nix-daemon[926]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5144worker2 # [ 63.349910] nix-daemon[926]: reaped child process 1271, status = succeeded5145worker2 # [ 63.503256] nix-daemon[926]: accepted connection from pid 1273, user nix-grpc-daemon (trusted)5146worker2 # [ 63.516254] nix-daemon[926]: reaped child process 1276, status = succeeded5147lb # [ 63.543341] niks3-server[1114]: 2026/09/20 12:20:59 INFO Received uploads request method=POST path=/api/pending_closures5148worker2 # [ 63.541923] nix-grpc-daemon[938]: time=2026-09-20T12:20:59.514Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5149worker2 # [ 63.543337] nix-grpc-daemon[938]: time=2026-09-20T12:20:59.514Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5150lb # [ 63.574448] niks3-server[1114]: 2026/09/20 12:20:59 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5151lb # [ 63.583625] niks3-server[1114]: 2026/09/20 12:20:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5152lb # [ 63.587394] niks3-server[1114]: 2026/09/20 12:20:59 INFO Signed narinfos id=21 count=15153worker2 # [ 63.578674] nix-grpc-daemon[938]: time=2026-09-20T12:20:59.551Z level=INFO msg="Uploading 1 narinfos"5154lb # [ 63.589163] niks3-server[1114]: 2026/09/20 12:20:59 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5155lb # [ 63.596084] niks3-server[1114]: 2026/09/20 12:20:59 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5156lb # [ 63.610809] niks3-server[1114]: 2026/09/20 12:20:59 INFO Completed upload id=215157worker2 # [ 63.602223] nix-grpc-daemon[938]: time=2026-09-20T12:20:59.575Z level=INFO msg="Upload complete. (431ms)"5158worker2 # [ 63.604885] nix-grpc-daemon[857]: ts=2026-09-20T12:20:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=1 paths=1 nar_bytes_in=5605159lb # [ 63.618302] niks3-server[1114]: 2026/09/20 12:20:59 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5160worker2 # [ 63.608397] nix-grpc-daemon[857]: ts=2026-09-20T12:20:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:381285161worker2 # [ 63.617181] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)5162worker2 # [ 63.624648] nix-daemon[926]: reaped child process 1262, status = succeeded5163worker2 # [ 63.632879] nix-grpc-daemon[857]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5164worker2 # [ 63.641229] nix-daemon[926]: accepted connection from pid 857, user nix-grpc-daemon (trusted)5165client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5166client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5167worker2 # [ 64.464288] nix-daemon[926]: reaped child process 1281, status = succeeded5168worker2 # [ 64.614782] nix-daemon[926]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5169worker2 # [ 64.628160] nix-daemon[926]: reaped child process 1299, status = succeeded5170lb # [ 64.649268] niks3-server[1114]: 2026/09/20 12:21:00 INFO Received uploads request method=POST path=/api/pending_closures5171worker2 # [ 64.645420] nix-grpc-daemon[938]: time=2026-09-20T12:21:00.618Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5172lb # [ 64.675920] niks3-server[1114]: 2026/09/20 12:21:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5173lb # [ 64.681082] niks3-server[1114]: 2026/09/20 12:21:00 INFO Signed narinfos id=22 count=15174worker2 # [ 64.673509] nix-grpc-daemon[938]: time=2026-09-20T12:21:00.646Z level=INFO msg="Uploading 1 narinfos"5175lb # [ 64.689131] niks3-server[1114]: 2026/09/20 12:21:00 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5176lb # [ 64.693263] niks3-server[1114]: 2026/09/20 12:21:00 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5177lb # [ 64.701228] niks3-server[1114]: 2026/09/20 12:21:00 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5178lb # [ 64.708217] niks3-server[1114]: 2026/09/20 12:21:00 INFO Completed upload id=225179worker2 # [ 64.697636] nix-grpc-daemon[938]: time=2026-09-20T12:21:00.671Z level=INFO msg="Upload complete. (248ms)"5180lb # [ 64.711419] niks3-server[1114]: 2026/09/20 12:21:00 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5181worker2 # [ 64.701698] nix-grpc-daemon[857]: ts=2026-09-20T12:21:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15182worker2 # [ 64.710328] nix-daemon[926]: reaped child process 1278, status = succeeded5183client # copying 1 paths...5184worker2 # [ 64.744618] nix-grpc-daemon[857]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:381285185client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5186worker2 # [ 64.754111] nix-grpc-daemon[857]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:381285187worker2 # [ 64.756137] nix-grpc-daemon[857]: ts=2026-09-20T12:21:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38128 duration_s=0 paths=1 nar_bytes_out=1205188client # [ 64.841375] nix-daemon[950]: reaped child process 1058, status = succeeded5189client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.19 seconds)5190client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25191client # [ 64.996761] nix-daemon[950]: accepted connection from pid 1077, user root (trusted)5192client # this derivation will be built:5193client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5194worker2 # [ 65.191229] nix-grpc-daemon[857]: ts=2026-09-20T12:21:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:381285195worker2 # [ 65.194556] nix-grpc-daemon[857]: ts=2026-09-20T12:21:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:381285196client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5197worker2 # [ 65.200696] nix-grpc-daemon[938]: time=2026-09-20T12:21:01.174Z level=INFO msg="All 1 paths already cached"5198worker2 # [ 65.212044] nix-grpc-daemon[857]: ts=2026-09-20T12:21:01Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05199worker2 # [ 65.214356] nix-grpc-daemon[857]: ts=2026-09-20T12:21:01Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15200worker1 # [ 65.244885] nix-grpc-daemon[853]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347825201worker1 # [ 65.253612] nix-grpc-daemon[853]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:347825202worker1 # [ 65.263244] nix-grpc-daemon[937]: time=2026-09-20T12:21:00.904Z level=INFO msg="All 1 paths already cached"5203client # copying 1 paths to 'lb:50051'...5204worker1 # [ 65.271274] nix-grpc-daemon[853]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:347825205client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5206worker1 # [ 65.279870] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5207worker1 # [ 65.441157] nix-daemon[911]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)5208worker1 # [ 65.455989] nix-daemon[911]: reaped child process 1057, status = succeeded5209worker1 # [ 65.594557] nix-daemon[911]: accepted connection from pid 1059, user nix-grpc-daemon (trusted)5210worker1 # [ 65.608708] nix-daemon[911]: reaped child process 1062, status = succeeded5211lb # [ 65.643760] niks3-server[1114]: 2026/09/20 12:21:01 INFO Received uploads request method=POST path=/api/pending_closures5212worker1 # [ 65.631084] nix-grpc-daemon[937]: time=2026-09-20T12:21:01.272Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5213worker1 # [ 65.632596] nix-grpc-daemon[937]: time=2026-09-20T12:21:01.272Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5214lb # [ 65.669466] niks3-server[1114]: 2026/09/20 12:21:01 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5215lb # [ 65.675124] niks3-server[1114]: 2026/09/20 12:21:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5216lb # [ 65.678317] niks3-server[1114]: 2026/09/20 12:21:01 INFO Signed narinfos id=23 count=15217worker1 # [ 65.657668] nix-grpc-daemon[937]: time=2026-09-20T12:21:01.299Z level=INFO msg="Uploading 1 narinfos"5218lb # [ 65.680887] niks3-server[1114]: 2026/09/20 12:21:01 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5219lb # [ 65.686066] niks3-server[1114]: 2026/09/20 12:21:01 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5220lb # [ 65.690504] niks3-server[1114]: 2026/09/20 12:21:01 INFO Completed upload id=235221worker1 # [ 65.668220] nix-grpc-daemon[937]: time=2026-09-20T12:21:01.309Z level=INFO msg="Upload complete. (383ms)"5222worker1 # [ 65.670351] nix-grpc-daemon[853]: ts=2026-09-20T12:21:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=0 paths=1 nar_bytes_in=5605223lb # [ 65.693686] niks3-server[1114]: 2026/09/20 12:21:01 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5224worker1 # [ 65.674051] nix-grpc-daemon[853]: ts=2026-09-20T12:21:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:347825225worker1 # [ 65.682242] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5226worker1 # [ 65.683610] nix-daemon[911]: reaped child process 1052, status = succeeded5227worker1 # [ 65.689070] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5228client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5229client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5230worker1 # [ 66.514845] nix-daemon[911]: reaped child process 1066, status = succeeded5231worker1 # [ 66.656271] nix-daemon[911]: accepted connection from pid 1081, user nix-grpc-daemon (trusted)5232worker1 # [ 66.669551] nix-daemon[911]: reaped child process 1084, status = succeeded5233lb # [ 66.704397] niks3-server[1114]: 2026/09/20 12:21:02 INFO Received uploads request method=POST path=/api/pending_closures5234worker1 # [ 66.690746] nix-grpc-daemon[937]: time=2026-09-20T12:21:02.332Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5235lb # [ 66.736755] niks3-server[1114]: 2026/09/20 12:21:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5236lb # [ 66.742158] niks3-server[1114]: 2026/09/20 12:21:02 INFO Signed narinfos id=24 count=15237worker1 # [ 66.723336] nix-grpc-daemon[937]: time=2026-09-20T12:21:02.364Z level=INFO msg="Uploading 1 narinfos"5238lb # [ 66.751659] niks3-server[1114]: 2026/09/20 12:21:02 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5239lb # [ 66.761147] niks3-server[1114]: 2026/09/20 12:21:02 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5240lb # [ 66.769267] niks3-server[1114]: 2026/09/20 12:21:02 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5241lb # [ 66.776109] niks3-server[1114]: 2026/09/20 12:21:02 INFO Completed upload id=245242worker1 # [ 66.756390] nix-grpc-daemon[937]: time=2026-09-20T12:21:02.398Z level=INFO msg="Upload complete. (260ms)"5243worker1 # [ 66.759279] nix-grpc-daemon[853]: ts=2026-09-20T12:21:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15244lb # [ 66.782388] niks3-server[1114]: 2026/09/20 12:21:02 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5245client # copying 1 paths...5246worker1 # [ 66.765911] nix-grpc-daemon[853]: ts=2026-09-20T12:21:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347825247client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5248worker1 # [ 66.770475] nix-daemon[911]: reaped child process 1064, status = succeeded5249worker1 # [ 66.783234] nix-grpc-daemon[853]: ts=2026-09-20T12:21:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:347825250worker1 # [ 66.785678] nix-grpc-daemon[853]: ts=2026-09-20T12:21:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:34782 duration_s=0 paths=1 nar_bytes_out=1205251client # [ 66.884751] nix-daemon[950]: reaped child process 1080, status = succeeded5252client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.05 seconds)5253(finished: subtest: an input only one worker has still reaches the builder, in 9.00 seconds)5254subtest: upload whose reference the worker lacks is completed from the cache5255client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5256client # this derivation will be built:5257client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5258client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5259client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.35 seconds)5260client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5261client # this derivation will be built:5262client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5263client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5264client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.33 seconds)5265client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5266client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.16 seconds)5267worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5268worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.16 seconds)5269worker2 # [ 67.953391] systemd[1]: Stopping Nix worker-protocol over gRPC...5270worker2 # [ 68.255442] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=signal_exit5271worker2 # [ 68.513675] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=scheduler_yield5272worker1 # [ 68.504104] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_restarting addr=lb:500515273worker2 # [ 68.516100] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=scheduler_restarting addr=lb:500515274lb # [ 68.527580] niks3-server[1114]: 2026/09/20 12:21:04 INFO lead: released remote=192.168.1.4:358285275worker2 # [ 68.518266] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:381145276worker2 # [ 68.519838] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=worker_gone addr=192.168.1.3:500515277worker2 # [ 68.521991] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:381145278worker2 # [ 68.523825] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=worker_gone addr=192.168.1.4:500515279worker1 # [ 68.704451] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_connected addr=lb:500515280worker1 # [ 68.706175] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5281worker2 # [ 68.717368] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=scheduler_connected addr=lb:500515282worker2 # [ 68.721202] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker1 # [ 68.906802] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_connected addr=lb:500515284worker1 # [ 68.908603] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5285worker2 # [ 68.921657] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=scheduler_connected addr=lb:500515286worker2 # [ 68.923444] nix-grpc-daemon[857]: ts=2026-09-20T12:21:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5287lb # [ 68.938150] envoy[978]: conn peer=[2001:db8:1::1]:48664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3852 tx=1757 ms=25175288lb # [ 68.941757] envoy[978]: conn peer=[2001:db8:1::1]:48674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=505289lb # [ 68.945214] envoy[978]: conn peer=[2001:db8:1::1]:48690 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3712 tx=1805 ms=28285290lb # [ 68.948692] envoy[978]: conn peer=[2001:db8:1::1]:56442 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=505291lb # [ 68.952254] envoy[978]: conn peer=[2001:db8:1::1]:56452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545292lb # [ 68.955661] envoy[978]: conn peer=[2001:db8:1::1]:56446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3688 tx=1800 ms=16595293worker1 # [ 69.109275] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_connected addr=lb:500515294worker1 # [ 69.111422] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5295worker2 # [ 69.124132] nix-grpc-daemon[857]: ts=2026-09-20T12:21:05Z level=info event=scheduler_connected addr=lb:500515296worker2 # [ 69.125821] nix-grpc-daemon[857]: ts=2026-09-20T12:21:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5297worker1 # [ 69.311855] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_connected addr=lb:500515298worker1 # [ 69.313536] nix-grpc-daemon[853]: ts=2026-09-20T12:21:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5299worker2 # [ 69.328316] nix-grpc-daemon[857]: ts=2026-09-20T12:21:05Z level=info event=scheduler_connected addr=lb:500515300worker2 # [ 69.331463] nix-grpc-daemon[857]: ts=2026-09-20T12:21:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5301worker2 # [ 69.514598] nix-grpc-daemon[857]: ts=2026-09-20T12:21:05Z level=info event=scheduler_restarting5302worker1 # [ 69.515032] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_connected addr=lb:500515303worker1 # [ 69.516347] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker2 # [ 69.533322] nix-daemon[926]: reaped child process 928, status = succeeded5305worker2 # [ 69.535639] nix-grpc-daemon[857]: ts=2026-09-20T12:21:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5306worker1 # [ 69.715992] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_connected addr=lb:500515307worker1 # [ 69.719905] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5308worker1 # [ 69.920439] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_connected addr=lb:500515309worker1 # [ 69.922114] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5310worker1 # [ 70.122832] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_connected addr=lb:500515311worker1 # [ 70.124517] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5312worker1 # [ 70.324967] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_connected addr=lb:500515313worker1 # [ 70.326614] nix-grpc-daemon[853]: ts=2026-09-20T12:21:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5314worker2 # [ 70.357602] nix-grpc-daemon[857]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5315worker2 # [ 70.359091] nix-grpc-daemon[857]: E0000 00:00:1789906866.330338 900 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5316worker2 # [ 70.361045] nix-grpc-daemon[857]: E0000 00:00:1789906866.330755 900 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5317worker1 # [ 70.527690] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_connected addr=lb:500515318worker1 # [ 70.529385] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5319worker1 # [ 70.560226] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_take_over5320lb # [ 70.583419] niks3-server[1114]: 2026/09/20 12:21:06 INFO lead: acquired remote=192.168.1.3:527765321worker2 # [ 70.725851] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5322worker2 # [ 70.729730] systemd[1]: Stopped Nix worker-protocol over gRPC.5323worker2 # [ 70.732342] systemd[1]: nix-grpc-daemon.service: Consumed 2.791s CPU time over 51.688s wall clock time, 60.8M memory peak, 91.7K incoming IP traffic, 221.4K outgoing IP traffic.5324worker1 # [ 70.730505] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_connected addr=lb:500515325worker2 # [ 70.741283] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5326worker2 # [ 70.744821] systemd[1]: Closed Nix worker-protocol over gRPC.5327worker1 # [ 70.733659] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5328worker2 # [ 70.747752] systemd[1]: nix-grpc-daemon.socket: Consumed 491us CPU time over 1min 819ms wall clock time, 256K memory peak, 85.7K incoming IP traffic, 66.9K outgoing IP traffic.5329lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5330lb: must succeed: curl -sf localhost:9901/clusters5331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5332lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.04 seconds)5333client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25334worker1 # [ 70.934309] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_connected addr=lb:500515335worker1 # [ 70.936077] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5336worker1 # [ 70.968989] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:347825337worker1 # [ 70.974542] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:347825338client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5339client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.22 seconds)5340worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5341worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5342worker2 # [ 71.064725] systemd[1]: Listening on Nix worker-protocol over gRPC.5343worker1 # [ 71.107540] systemd[1]: Stopping Nix worker-protocol over gRPC...5344worker1 # [ 71.136533] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 71.138189] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker1 # [ 71.176383] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=signal_exit5347worker1 # [ 71.339193] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_connected addr=lb:500515348worker1 # [ 71.342979] nix-grpc-daemon[853]: ts=2026-09-20T12:21:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5349worker1 # [ 71.444507] nix-grpc-daemon[853]: ts=2026-09-20T12:21:07Z level=info event=scheduler_yield5350lb # [ 71.469361] niks3-server[1114]: 2026/09/20 12:21:07 INFO lead: released remote=192.168.1.3:527765351worker1 # [ 71.543986] nix-grpc-daemon[853]: ts=2026-09-20T12:21:07Z level=info event=scheduler_connected addr=lb:500515352worker1 # [ 71.545898] nix-grpc-daemon[853]: ts=2026-09-20T12:21:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker2 # [ 71.715537] systemd[1]: Starting scheduler-token.service...5354worker1 # [ 71.746513] nix-grpc-daemon[853]: ts=2026-09-20T12:21:07Z level=info event=scheduler_connected addr=lb:500515355worker1 # [ 71.748441] nix-grpc-daemon[853]: ts=2026-09-20T12:21:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5356worker2 # [ 71.772029] systemd[1]: scheduler-token.service: Deactivated successfully.5357worker2 # [ 71.773594] systemd[1]: Finished scheduler-token.service.5358worker2 # [ 71.774625] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 57ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5359worker2 # [ 71.779704] systemd[1]: Starting Nix worker-protocol over gRPC...5360worker2 # [ 71.923776] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5361worker2 # [ 71.926060] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5362worker2 # [ 71.929788] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15363worker2 # [ 71.937317] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5364worker2 # [ 71.939785] systemd[1]: Started Nix worker-protocol over gRPC.5365worker2 # [ 71.947172] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=scheduler_take_over5366lb # [ 71.958280] niks3-server[1114]: 2026/09/20 12:21:07 INFO lead: acquired remote=192.168.1.4:468865367worker2 # [ 71.954431] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=scheduler_connected addr=lb:500515368worker2 # [ 71.959560] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=oidc_jwks provider=mock keys=15369worker1 # [ 71.948957] nix-grpc-daemon[853]: ts=2026-09-20T12:21:07Z level=info event=scheduler_connected addr=lb:500515370worker2 # [ 71.961153] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:349945371worker2 # [ 71.962680] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05372worker2 # [ 71.964951] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:349945373worker2 # [ 71.966432] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05374worker1 # [ 72.445084] nix-grpc-daemon[853]: ts=2026-09-20T12:21:08Z level=info event=scheduler_restarting5375worker2 # [ 72.462995] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:349945376worker1 # [ 72.456428] nix-grpc-daemon[853]: ts=2026-09-20T12:21:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5377worker2 # [ 72.466763] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:08Z level=info event=worker_gone addr=192.168.1.3:500515378worker1 # [ 72.466315] nix-daemon[911]: reaped child process 913, status = succeeded5379worker1 # [ 72.665142] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5380worker1 # [ 72.666344] systemd[1]: Stopped Nix worker-protocol over gRPC.5381worker1 # [ 72.668529] systemd[1]: nix-grpc-daemon.service: Consumed 1.429s CPU time over 53.755s wall clock time, 47.5M memory peak, 52.4K incoming IP traffic, 98.1K outgoing IP traffic.5382worker1 # [ 72.673265] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5383worker1 # [ 72.675054] systemd[1]: Closed Nix worker-protocol over gRPC.5384worker1 # [ 72.676334] systemd[1]: nix-grpc-daemon.socket: Consumed 537us CPU time over 1min 1.662s wall clock time, 256K memory peak, 58.4K incoming IP traffic, 53K outgoing IP traffic.5385lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5386lb: must succeed: curl -sf localhost:9901/clusters5387lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5388lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5389client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25390client # lb:50051: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, retrying5391worker2 # [ 73.430566] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409425392worker2 # [ 73.442147] nix-daemon[926]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5393worker2 # [ 73.458845] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)5394worker2 # [ 73.500584] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 108 ms (attempt 2/5)5395worker2 # [ 73.614442] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 368 ms (attempt 3/5)5396worker2 # [ 73.987972] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 116 ms (attempt 4/5)5397worker2 # [ 74.109872] nix-grpc-daemon[1325]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5398client # copying 2 paths...5399client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5400client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5401worker2 # [ 74.122314] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409425402worker2 # [ 74.129061] nix-daemon[926]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5403worker2 # [ 74.140173] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:10Z level=info event=niks3_push_started pid=13815404worker2 # [ 74.290870] nix-grpc-daemon[1381]: time=2026-09-20T12:21:10.264Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5405worker2 # [ 74.431755] nix-daemon[926]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5406worker2 # [ 74.443071] nix-daemon[926]: reaped child process 1397, status = succeeded5407lb # [ 74.467255] niks3-server[1114]: 2026/09/20 12:21:10 INFO Received uploads request method=POST path=/api/pending_closures5408lb # [ 74.471551] niks3-server[1114]: 2026/09/20 12:21:10 INFO Received uploads request method=POST path=/api/pending_closures5409worker2 # [ 74.463842] nix-grpc-daemon[1381]: time=2026-09-20T12:21:10.437Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5410worker2 # [ 74.465300] nix-grpc-daemon[1381]: time=2026-09-20T12:21:10.438Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5411worker2 # [ 74.467144] nix-grpc-daemon[1381]: time=2026-09-20T12:21:10.440Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5412lb # [ 74.496281] niks3-server[1114]: 2026/09/20 12:21:10 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5413lb # [ 74.502270] niks3-server[1114]: 2026/09/20 12:21:10 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5414lb # [ 74.509162] niks3-server[1114]: 2026/09/20 12:21:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5415lb # [ 74.511888] niks3-server[1114]: 2026/09/20 12:21:10 INFO Signed narinfos id=25 count=15416lb # [ 74.512899] niks3-server[1114]: 2026/09/20 12:21:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5417lb # [ 74.522633] niks3-server[1114]: 2026/09/20 12:21:10 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5418lb # [ 74.526458] niks3-server[1114]: 2026/09/20 12:21:10 INFO Signed narinfos id=26 count=25419worker2 # [ 74.516421] nix-grpc-daemon[1381]: time=2026-09-20T12:21:10.489Z level=INFO msg="Uploading 3 narinfos"5420lb # [ 74.533187] niks3-server[1114]: 2026/09/20 12:21:10 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5421lb # [ 74.546294] niks3-server[1114]: 2026/09/20 12:21:10 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5422lb # [ 74.553562] niks3-server[1114]: 2026/09/20 12:21:10 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5423lb # [ 74.557282] niks3-server[1114]: 2026/09/20 12:21:10 INFO Completed upload id=255424lb # [ 74.558778] niks3-server[1114]: 2026/09/20 12:21:10 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5425lb # [ 74.560896] niks3-server[1114]: 2026/09/20 12:21:10 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5426lb # [ 74.563574] niks3-server[1114]: 2026/09/20 12:21:10 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5427lb # [ 74.566045] niks3-server[1114]: 2026/09/20 12:21:10 INFO Completed upload id=265428worker2 # [ 74.555574] nix-grpc-daemon[1381]: time=2026-09-20T12:21:10.528Z level=INFO msg="Upload complete. (260ms)"5429worker2 # [ 74.560509] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40942 duration_s=0 paths=2 nar_bytes_in=3125430worker2 # [ 74.570063] nix-daemon[926]: reaped child process 1379, status = succeeded5431client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.85 seconds)5432worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5433worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5434worker1 # [ 74.642526] systemd[1]: Listening on Nix worker-protocol over gRPC.5435worker1 # [ 74.644645] systemd[1]: Starting Nix worker-protocol over gRPC...5436worker1 # [ 74.763962] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5437worker1 # [ 74.766185] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5438worker1 # [ 74.769538] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15439worker1 # [ 74.774849] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5440worker1 # [ 74.776447] systemd[1]: Started Nix worker-protocol over gRPC.5441lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5442lb: must succeed: curl -sf localhost:9901/clusters5443worker1 # [ 74.788386] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:10Z level=info event=scheduler_connected addr=lb:500515444worker2 # [ 74.801586] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:349945445worker2 # [ 74.803326] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05446lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5447lb: must succeed: curl -sf localhost:9901/clusters5448lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5449lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)5450(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.00 seconds)5451subtest: scheduler down: the other node takes the lock and keeps it5452lb: must succeed: curl -sf localhost:9901/clusters5453lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5454lb: must succeed: curl -sf localhost:9901/clusters5455lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5456lb: must succeed: curl -sf localhost:9901/clusters5457lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5458lb: must succeed: curl -sf localhost:9901/clusters5459lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5460worker2 # [ 76.087745] systemd[1]: Stopping Nix worker-protocol over gRPC...5461worker2 # [ 76.942243] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:12Z level=info event=signal_exit5462worker2 # [ 76.948152] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:12Z level=info event=scheduler_yield5463lb # [ 76.961273] niks3-server[1114]: 2026/09/20 12:21:12 INFO lead: released remote=192.168.1.4:468865464worker1 # [ 76.943593] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:12Z level=info event=scheduler_restarting addr=lb:500515465worker2 # [ 76.956515] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:349945466worker2 # [ 76.957908] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:12Z level=info event=worker_gone addr=192.168.1.3:500515467worker2 # [ 76.960338] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:12Z level=info event=scheduler_restarting addr=lb:500515468worker2 # [ 76.962702] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:349945469worker2 # [ 76.964082] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:12Z level=info event=worker_gone addr=192.168.1.4:500515470worker1 # [ 77.144264] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:12Z level=info event=scheduler_connected addr=lb:500515471worker1 # [ 77.147595] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5472worker2 # [ 77.161143] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_connected addr=lb:500515473worker2 # [ 77.162787] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5474worker1 # [ 77.348186] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:12Z level=info event=scheduler_connected addr=lb:500515475worker2 # [ 77.363449] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_connected addr=lb:500515476worker1 # [ 77.351321] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5477worker2 # [ 77.365205] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5478worker1 # [ 77.551788] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:13Z level=info event=scheduler_connected addr=lb:500515479worker1 # [ 77.553603] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5480worker2 # [ 77.565828] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_connected addr=lb:500515481worker2 # [ 77.567493] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker1 # [ 77.754217] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:13Z level=info event=scheduler_connected addr=lb:500515483worker1 # [ 77.756068] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker2 # [ 77.768241] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_connected addr=lb:500515485worker2 # [ 77.769997] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5486worker2 # [ 77.952444] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=scheduler_restarting5487worker2 # [ 77.961910] nix-grpc-daemon[1325]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5488worker2 # [ 77.963523] nix-grpc-daemon[1325]: E0000 00:00:1789906873.935367 1368 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5489worker2 # [ 77.965437] nix-grpc-daemon[1325]: E0000 00:00:1789906873.936988 1368 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5490worker1 # [ 77.957030] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:13Z level=info event=scheduler_connected addr=lb:500515491worker1 # [ 77.958994] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5492worker2 # [ 77.969842] nix-daemon[926]: reaped child process 1374, status = succeeded5493worker2 # [ 77.974927] nix-grpc-daemon[1325]: ts=2026-09-20T12:21:13Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5494worker1 # [ 78.160213] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:13Z level=info event=scheduler_connected addr=lb:500515495worker1 # [ 78.164218] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5496worker1 # [ 78.364434] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_connected addr=lb:500515497worker1 # [ 78.366244] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5498worker1 # [ 78.566705] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_connected addr=lb:500515499worker1 # [ 78.568470] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5500worker1 # [ 78.768891] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_connected addr=lb:500515501worker1 # [ 78.770634] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5502lb # [ 78.937831] envoy[978]: conn peer=[2001:db8:1::4]:45092 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=54734 tx=3054 ms=502645503lb # [ 78.939663] envoy[978]: conn peer=[2001:db8:1::1]:56460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165504lb # [ 78.941243] envoy[978]: conn peer=192.168.1.3:48322 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15123 tx=2765 ms=523065505lb # [ 78.942831] envoy[978]: conn peer=[2001:db8:1::1]:52898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1165 tx=426 ms=16455506lb # [ 78.944402] envoy[978]: conn peer=[2001:db8:1::4]:59918 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4478 tx=349 ms=60205507worker1 # [ 78.971567] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_connected addr=lb:500515508worker1 # [ 78.973467] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5509worker1 # [ 79.174593] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_connected addr=lb:500515510worker1 # [ 79.178467] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5511worker1 # [ 79.378982] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_connected addr=lb:500515512worker1 # [ 79.381081] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5513worker1 # [ 79.581560] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_connected addr=lb:500515514worker1 # [ 79.583377] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5515lb # [ 79.805491] niks3-server[1114]: 2026/09/20 12:21:15 INFO lead: acquired remote=192.168.1.3:497105516worker1 # [ 79.784126] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_connected addr=lb:500515517worker1 # [ 79.788455] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_take_over5518worker1 # [ 79.791278] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5519worker2 # [ 79.962760] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5520worker2 # [ 79.967816] systemd[1]: Stopped Nix worker-protocol over gRPC.5521worker2 # [ 79.971345] systemd[1]: nix-grpc-daemon.service: Consumed 296ms CPU time over 8.181s wall clock time, 30.6M memory peak, 19.3K incoming IP traffic, 26.1K outgoing IP traffic.5522worker2 # [ 79.982433] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5523worker2 # [ 79.986298] systemd[1]: Closed Nix worker-protocol over gRPC.5524worker2 # [ 79.989454] systemd[1]: nix-grpc-daemon.socket: Consumed 329us CPU time over 8.925s wall clock time, 256K memory peak, 19.5K incoming IP traffic, 14.2K outgoing IP traffic.5525worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5526worker1 # [ 79.990495] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_connected addr=lb:500515527worker1 # [ 79.993793] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5528worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5529worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5530worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5531worker1 # [ 80.191098] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_connected addr=lb:500515532worker1 # [ 80.195135] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5533worker1 # [ 80.395798] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:16Z level=info event=scheduler_connected addr=lb:500515534worker1 # [ 80.403494] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:334325535worker1 # [ 80.407744] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05536worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5537worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5538client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25539worker1 # [ 81.356793] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334425540worker1 # [ 81.368204] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5541client # don't know how to build these paths:5542client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5543worker1 # [ 81.382260] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334425544worker1 # [ 81.393290] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5545worker1 # [ 81.402928] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5)5546worker1 # [ 81.502351] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 201 ms (attempt 3/5)5547worker1 # [ 81.709380] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 670 ms (attempt 4/5)5548worker1 # [ 82.386674] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5549worker1 # [ 82.413871] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:18Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05550worker1 # [ 82.420457] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:18Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15551worker1 # [ 82.461949] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334425552worker1 # [ 82.469324] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334425553client # copying 1 paths to 'lb:50051'...5554worker1 # [ 82.477100] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334425555client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5556worker1 # [ 82.485035] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5557worker1 # [ 82.600911] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:18Z level=info event=niks3_push_started pid=11775558worker1 # [ 82.796350] nix-grpc-daemon[1177]: time=2026-09-20T12:21:18.437Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5559worker1 # [ 82.947481] nix-daemon[911]: accepted connection from pid 1190, user nix-grpc-daemon (trusted)5560worker1 # [ 82.958589] nix-daemon[911]: reaped child process 1193, status = succeeded5561worker1 # [ 83.086964] nix-daemon[911]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5562worker1 # [ 83.102607] nix-daemon[911]: reaped child process 1198, status = succeeded5563lb # [ 83.139069] niks3-server[1114]: 2026/09/20 12:21:19 INFO Received uploads request method=POST path=/api/pending_closures5564worker1 # [ 83.125088] nix-grpc-daemon[1177]: time=2026-09-20T12:21:18.765Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5565worker1 # [ 83.126507] nix-grpc-daemon[1177]: time=2026-09-20T12:21:18.765Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5566lb # [ 83.163416] niks3-server[1114]: 2026/09/20 12:21:19 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5567lb # [ 83.167490] niks3-server[1114]: 2026/09/20 12:21:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5568lb # [ 83.169842] niks3-server[1114]: 2026/09/20 12:21:19 INFO Signed narinfos id=27 count=15569worker1 # [ 83.149093] nix-grpc-daemon[1177]: time=2026-09-20T12:21:18.790Z level=INFO msg="Uploading 1 narinfos"5570lb # [ 83.173756] niks3-server[1114]: 2026/09/20 12:21:19 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5571lb # [ 83.178475] niks3-server[1114]: 2026/09/20 12:21:19 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5572lb # [ 83.184256] niks3-server[1114]: 2026/09/20 12:21:19 INFO Completed upload id=275573worker1 # [ 83.162216] nix-grpc-daemon[1177]: time=2026-09-20T12:21:18.803Z level=INFO msg="Upload complete. (362ms)"5574lb # [ 83.185701] niks3-server[1114]: 2026/09/20 12:21:19 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5575worker1 # [ 83.164358] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33442 duration_s=0 paths=1 nar_bytes_in=5205576worker1 # [ 83.167694] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334425577worker1 # [ 83.172586] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5578worker1 # [ 83.178255] nix-daemon[911]: reaped child process 1175, status = succeeded5579client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5580client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5581worker1 # [ 83.182709] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5582client # farm-a> LOG-farm-a-on-standby5583worker1 # [ 84.197602] nix-daemon[911]: reaped child process 1202, status = succeeded5584worker1 # [ 84.337305] nix-daemon[911]: accepted connection from pid 1217, user nix-grpc-daemon (trusted)5585worker1 # [ 84.348339] nix-daemon[911]: reaped child process 1220, status = succeeded5586lb # [ 84.385745] niks3-server[1114]: 2026/09/20 12:21:20 INFO Received uploads request method=POST path=/api/pending_closures5587worker1 # [ 84.371665] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.013Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5588worker1 # [ 84.373281] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.014Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5589lb # [ 84.427516] niks3-server[1114]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5590lb # [ 84.436933] niks3-server[1114]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5591lb # [ 84.445525] niks3-server[1114]: 2026/09/20 12:21:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5592worker1 # [ 84.429803] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.070Z level=INFO msg="Uploading 1 narinfos"5593lb # [ 84.451512] niks3-server[1114]: 2026/09/20 12:21:20 INFO Signed narinfos id=28 count=15594lb # [ 84.455360] niks3-server[1114]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5595lb # [ 84.465738] niks3-server[1114]: 2026/09/20 12:21:20 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5596lb # [ 84.472131] niks3-server[1114]: 2026/09/20 12:21:20 INFO Completed upload id=285597lb # [ 84.473690] niks3-server[1114]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5598worker1 # [ 84.450286] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.091Z level=INFO msg="Upload complete. (260ms)"5599worker1 # [ 84.463320] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33442 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15600worker1 # [ 84.477464] nix-daemon[911]: reaped child process 1200, status = succeeded5601worker1 # [ 84.478897] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:20Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05602worker1 # [ 84.480781] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:20Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15603worker1 # [ 84.525180] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334425604worker1 # [ 84.534103] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334425605worker1 # [ 84.543585] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.185Z level=INFO msg="All 1 paths already cached"5606client # copying 1 paths to 'lb:50051'...5607worker1 # [ 84.549258] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334425608client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5609worker1 # [ 84.559132] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5610worker1 # [ 84.728539] nix-daemon[911]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5611worker1 # [ 84.741708] nix-daemon[911]: reaped child process 1229, status = succeeded5612worker1 # [ 84.895499] nix-daemon[911]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5613worker1 # [ 84.908295] nix-daemon[911]: reaped child process 1234, status = succeeded5614lb # [ 84.945706] niks3-server[1114]: 2026/09/20 12:21:20 INFO Received uploads request method=POST path=/api/pending_closures5615worker1 # [ 84.932848] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.574Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5616worker1 # [ 84.934281] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.574Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5617lb # [ 84.973081] niks3-server[1114]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5618lb # [ 84.983502] niks3-server[1114]: 2026/09/20 12:21:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5619lb # [ 84.986782] niks3-server[1114]: 2026/09/20 12:21:20 INFO Signed narinfos id=29 count=15620worker1 # [ 84.964573] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.605Z level=INFO msg="Uploading 1 narinfos"5621lb # [ 84.989624] niks3-server[1114]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5622lb # [ 84.994581] niks3-server[1114]: 2026/09/20 12:21:20 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5623lb # [ 84.997746] niks3-server[1114]: 2026/09/20 12:21:20 INFO Completed upload id=295624worker1 # [ 84.976887] nix-grpc-daemon[1177]: time=2026-09-20T12:21:20.618Z level=INFO msg="Upload complete. (408ms)"5625lb # [ 84.999429] niks3-server[1114]: 2026/09/20 12:21:20 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5626worker1 # [ 84.979168] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33442 duration_s=0 paths=1 nar_bytes_in=6405627worker1 # [ 84.982300] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334425628worker1 # [ 84.987119] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5629worker1 # [ 84.993059] nix-daemon[911]: reaped child process 1224, status = succeeded5630client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5631worker1 # [ 84.997925] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5632client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5633client # farm-b> LOG-farm-b-on-standby5634worker1 # [ 86.211613] nix-daemon[911]: reaped child process 1238, status = succeeded5635worker1 # [ 86.337615] nix-daemon[911]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5636worker1 # [ 86.349926] nix-daemon[911]: reaped child process 1256, status = succeeded5637lb # [ 86.382397] niks3-server[1114]: 2026/09/20 12:21:22 INFO Received uploads request method=POST path=/api/pending_closures5638worker1 # [ 86.365606] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.007Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5639worker1 # [ 86.367159] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.008Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5640lb # [ 86.411652] niks3-server[1114]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5641lb # [ 86.415052] niks3-server[1114]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5642lb # [ 86.418743] niks3-server[1114]: 2026/09/20 12:21:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5643lb # [ 86.420448] niks3-server[1114]: 2026/09/20 12:21:22 INFO Signed narinfos id=30 count=15644worker1 # [ 86.398662] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.040Z level=INFO msg="Uploading 1 narinfos"5645lb # [ 86.426148] niks3-server[1114]: 2026/09/20 12:21:22 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5646lb # [ 86.429606] niks3-server[1114]: 2026/09/20 12:21:22 INFO Completed upload id=305647worker1 # [ 86.407371] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.049Z level=INFO msg="Upload complete. (203ms)"5648lb # [ 86.432870] niks3-server[1114]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5649worker1 # [ 86.410404] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33442 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15650worker1 # [ 86.415315] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.056Z level=WARN msg="Failed to register uploaded object" key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo error="server returned 404: object is not pending upload\n"5651worker1 # [ 86.419241] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:22Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05652worker1 # [ 86.421200] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:22Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15653worker1 # [ 86.425300] nix-daemon[911]: reaped child process 1236, status = succeeded5654worker1 # [ 86.463422] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334425655worker1 # [ 86.473360] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334425656worker1 # [ 86.485503] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.127Z level=INFO msg="All 2 paths already cached"5657client # copying 1 paths to 'lb:50051'...5658client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5659worker1 # [ 86.491777] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334425660worker1 # [ 86.500129] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5661worker1 # [ 86.632365] nix-daemon[911]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)5662worker1 # [ 86.645184] nix-daemon[911]: reaped child process 1265, status = succeeded5663worker1 # [ 86.770401] nix-daemon[911]: accepted connection from pid 1267, user nix-grpc-daemon (trusted)5664worker1 # [ 86.781278] nix-daemon[911]: reaped child process 1270, status = succeeded5665lb # [ 86.814936] niks3-server[1114]: 2026/09/20 12:21:22 INFO Received uploads request method=POST path=/api/pending_closures5666worker1 # [ 86.798276] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.439Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5667worker1 # [ 86.799851] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.441Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5668lb # [ 86.835234] niks3-server[1114]: 2026/09/20 12:21:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5669lb # [ 86.837564] niks3-server[1114]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5670lb # [ 86.842069] niks3-server[1114]: 2026/09/20 12:21:22 INFO Signed narinfos id=31 count=15671worker1 # [ 86.822355] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.464Z level=INFO msg="Uploading 1 narinfos"5672lb # [ 86.849272] niks3-server[1114]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5673lb # [ 86.855201] niks3-server[1114]: 2026/09/20 12:21:22 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5674lb # [ 86.859173] niks3-server[1114]: 2026/09/20 12:21:22 INFO Completed upload id=315675worker1 # [ 86.837107] nix-grpc-daemon[1177]: time=2026-09-20T12:21:22.478Z level=INFO msg="Upload complete. (329ms)"5676lb # [ 86.861522] niks3-server[1114]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5677worker1 # [ 86.839108] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33442 duration_s=0 paths=1 nar_bytes_in=7685678worker1 # [ 86.841822] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334425679worker1 # [ 86.847965] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5680worker1 # [ 86.851318] nix-daemon[911]: reaped child process 1260, status = succeeded5681worker1 # [ 86.856118] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5682client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5683client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5684client # farm-top> LOG-farm-top-on-standby5685worker1 # [ 88.071145] nix-daemon[911]: reaped child process 1274, status = succeeded5686worker1 # [ 88.221381] nix-daemon[911]: accepted connection from pid 1289, user nix-grpc-daemon (trusted)5687worker1 # [ 88.233587] nix-daemon[911]: reaped child process 1292, status = succeeded5688lb # [ 88.270580] niks3-server[1114]: 2026/09/20 12:21:24 INFO Received uploads request method=POST path=/api/pending_closures5689worker1 # [ 88.255459] nix-grpc-daemon[1177]: time=2026-09-20T12:21:23.896Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5690worker1 # [ 88.256974] nix-grpc-daemon[1177]: time=2026-09-20T12:21:23.898Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5691lb # [ 88.296144] niks3-server[1114]: 2026/09/20 12:21:24 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5692lb # [ 88.298927] niks3-server[1114]: 2026/09/20 12:21:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5693lb # [ 88.301093] niks3-server[1114]: 2026/09/20 12:21:24 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5694lb # [ 88.304094] niks3-server[1114]: 2026/09/20 12:21:24 INFO Signed narinfos id=32 count=15695worker1 # [ 88.283702] nix-grpc-daemon[1177]: time=2026-09-20T12:21:23.925Z level=INFO msg="Uploading 1 narinfos"5696lb # [ 88.307376] niks3-server[1114]: 2026/09/20 12:21:24 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5697lb # [ 88.312110] niks3-server[1114]: 2026/09/20 12:21:24 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5698lb # [ 88.315645] niks3-server[1114]: 2026/09/20 12:21:24 INFO Completed upload id=325699worker1 # [ 88.293543] nix-grpc-daemon[1177]: time=2026-09-20T12:21:23.935Z level=INFO msg="Upload complete. (230ms)"5700lb # [ 88.318433] niks3-server[1114]: 2026/09/20 12:21:24 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5701worker1 # [ 88.296577] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33442 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15702worker1 # [ 88.303369] nix-daemon[911]: reaped child process 1272, status = succeeded5703client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.16 seconds)5704client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5705client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.17 seconds)5706worker2 # [ 88.569721] systemd[1]: Listening on Nix worker-protocol over gRPC.5707worker2 # [ 88.577110] systemd[1]: Starting scheduler-token.service...5708worker2 # [ 88.646785] systemd[1]: scheduler-token.service: Deactivated successfully.5709worker2 # [ 88.647776] systemd[1]: Finished scheduler-token.service.5710worker2 # [ 88.648955] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 70ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5711worker2 # [ 88.654196] systemd[1]: Starting Nix worker-protocol over gRPC...5712worker2 # [ 88.792458] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5713worker2 # [ 88.794219] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5714worker2 # [ 88.797592] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15715worker2 # [ 88.804494] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5716worker2 # [ 88.806148] systemd[1]: Started Nix worker-protocol over gRPC.5717lb: must succeed: curl -sf localhost:9901/clusters5718worker2 # [ 88.825665] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:24Z level=info event=scheduler_connected addr=lb:500515719worker1 # [ 88.824059] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:24Z level=info event=oidc_jwks provider=mock keys=15720worker1 # [ 88.828342] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:334325721worker1 # [ 88.832234] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05722lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5723worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5724lb # [ 88.938326] envoy[978]: conn peer=[2001:db8:1::1]:52906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9590 tx=3867 ms=69585725worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5726lb: must succeed: curl -sf localhost:9901/clusters5727lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5728lb: must succeed: curl -sf localhost:9901/clusters5729lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5730worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5731worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5732(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.20 seconds)5733subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5734client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5735client # [ 89.148476] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5736client # Running as unit: rs.service; invocation ID: f83f342b364d4538bad25c2cbecc7b7d5737client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5738worker2 # [ 89.354569] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:546125739worker2 # [ 89.359805] nix-daemon[926]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5740client # [ 89.359416] nix[1180]: don't know how to build these paths:5741client # [ 89.360372] nix[1180]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5742worker1 # [ 89.368303] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:25Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05743worker1 # [ 89.373819] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:25Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15744worker1 # [ 89.416164] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334425745worker1 # [ 89.422830] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334425746client # [ 89.427173] nix[1180]: copying 1 paths to 'lb:50051'...5747worker1 # [ 89.430805] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334425748client # [ 89.429897] nix[1180]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5749worker1 # [ 89.438315] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5750worker1 # [ 89.583506] nix-daemon[911]: accepted connection from pid 1305, user nix-grpc-daemon (trusted)5751worker1 # [ 89.597076] nix-daemon[911]: reaped child process 1308, status = succeeded5752worker1 # [ 89.747868] nix-daemon[911]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5753worker1 # [ 89.760130] nix-daemon[911]: reaped child process 1313, status = succeeded5754lb # [ 89.798498] niks3-server[1114]: 2026/09/20 12:21:25 INFO Received uploads request method=POST path=/api/pending_closures5755worker1 # [ 89.785182] nix-grpc-daemon[1177]: time=2026-09-20T12:21:25.426Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5756worker1 # [ 89.786650] nix-grpc-daemon[1177]: time=2026-09-20T12:21:25.428Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5757lb # [ 89.831127] niks3-server[1114]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5758lb # [ 89.841643] niks3-server[1114]: 2026/09/20 12:21:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5759worker1 # [ 89.827434] nix-grpc-daemon[1177]: time=2026-09-20T12:21:25.469Z level=INFO msg="Uploading 1 narinfos"5760lb # [ 89.849440] niks3-server[1114]: 2026/09/20 12:21:25 INFO Signed narinfos id=33 count=15761lb # [ 89.856943] niks3-server[1114]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5762lb # [ 89.869139] niks3-server[1114]: 2026/09/20 12:21:25 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5763lb # [ 89.874624] niks3-server[1114]: 2026/09/20 12:21:25 INFO Completed upload id=335764worker1 # [ 89.855453] nix-grpc-daemon[1177]: time=2026-09-20T12:21:25.497Z level=INFO msg="Upload complete. (412ms)"5765lb # [ 89.882785] niks3-server[1114]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5766worker1 # [ 89.859942] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33442 duration_s=0 paths=1 nar_bytes_in=4165767worker1 # [ 89.867334] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334425768worker1 # [ 89.876460] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5769worker1 # [ 89.883540] nix-daemon[911]: reaped child process 1303, status = succeeded5770worker1 # [ 89.888536] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5771client # [ 89.887724] nix[1180]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5772lb: must succeed: curl -sf localhost:9901/clusters5773lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5774lb: must succeed: curl -sf localhost:9901/clusters5775lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5776lb: must succeed: curl -sf localhost:9901/clusters5777lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5778lb: must succeed: curl -sf localhost:9901/clusters5779lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5780worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5781worker2 # [ 91.953164] systemd-journald[287]: Received client request to rotate journal, rotating.5782worker2 # [ 91.965223] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/cc21148ba6df40a18865ee339edef4ce.5783worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.04 seconds)5784worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5785worker1 # [ 91.850086] systemd[1]: Stopping Nix worker-protocol over gRPC...5786worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5787??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5788 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735789client: must succeed: sleep 5.05790worker1 # [ 92.784773] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:28Z level=info event=draining5791client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5792worker1 # [ 97.108406] nix-daemon[911]: reaped child process 1317, status = succeeded5793worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5794worker1 # [ 97.243893] nix-daemon[911]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5795worker1 # [ 97.257441] nix-daemon[911]: reaped child process 1360, status = succeeded5796lb # [ 97.292743] niks3-server[1114]: 2026/09/20 12:21:33 INFO Received uploads request method=POST path=/api/pending_closures5797worker1 # [ 97.275604] nix-grpc-daemon[1177]: time=2026-09-20T12:21:32.917Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5798worker1 # [ 97.277417] nix-grpc-daemon[1177]: time=2026-09-20T12:21:32.918Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5799lb # [ 97.319808] niks3-server[1114]: 2026/09/20 12:21:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5800worker1 # [ 97.302442] nix-grpc-daemon[1177]: time=2026-09-20T12:21:32.943Z level=INFO msg="Uploading 1 narinfos"5801lb # [ 97.325092] niks3-server[1114]: 2026/09/20 12:21:33 INFO Signed narinfos id=34 count=15802lb # [ 97.329119] niks3-server[1114]: 2026/09/20 12:21:33 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5803lb # [ 97.333495] niks3-server[1114]: 2026/09/20 12:21:33 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5804lb # [ 97.338446] niks3-server[1114]: 2026/09/20 12:21:33 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5805lb # [ 97.343304] niks3-server[1114]: 2026/09/20 12:21:33 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5806lb # [ 97.347944] niks3-server[1114]: 2026/09/20 12:21:33 INFO Completed upload id=345807worker1 # [ 97.327352] nix-grpc-daemon[1177]: time=2026-09-20T12:21:32.969Z level=INFO msg="Upload complete. (226ms)"5808lb # [ 97.350675] niks3-server[1114]: 2026/09/20 12:21:33 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5809worker1 # [ 97.330046] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33442 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15810worker1 # [ 97.338432] nix-daemon[911]: reaped child process 1315, status = succeeded5811client # [ 97.364977] systemd[1]: rs.service: Deactivated successfully.5812client # [ 97.369349] systemd[1]: rs.service: Consumed 109ms CPU time over 8.219s wall clock time, 39.1M memory peak, 8.4K incoming IP traffic, 8.9K outgoing IP traffic.5813worker1 # [ 97.785650] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=signal_exit5814worker1 # [ 98.006367] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=scheduler_yield5815lb # [ 98.031238] niks3-server[1114]: 2026/09/20 12:21:33 INFO lead: released remote=192.168.1.3:497105816worker2 # [ 98.022123] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:33Z level=info event=scheduler_restarting addr=lb:500515817worker1 # [ 98.013769] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:334325818worker1 # [ 98.017749] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=worker_gone addr=192.168.1.4:500515819worker1 # [ 98.021049] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=scheduler_restarting addr=lb:500515820worker1 # [ 98.023410] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:334325821worker1 # [ 98.026088] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=worker_gone addr=192.168.1.3:500515822worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.07 seconds)5823client: waiting for success: ! systemctl is-active rs5824client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)5825client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5826worker2 # [ 98.221241] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515827worker2 # [ 98.223900] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5828worker1 # [ 98.217931] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=scheduler_connected addr=lb:500515829worker1 # [ 98.220319] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5830client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5831client: must fail: journalctl -u rs | grep -q 'reconnecting'5832client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5833lb: must succeed: curl -sf localhost:9901/clusters5834lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5835worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5836worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5837worker2 # [ 98.424777] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515838worker2 # [ 98.426567] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5839worker1 # [ 98.421587] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515840worker1 # [ 98.422970] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5841worker2 # [ 98.627521] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515842worker2 # [ 98.629772] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5843worker1 # [ 98.623470] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515844worker1 # [ 98.626854] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5845worker2 # [ 98.815329] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_take_over5846lb # [ 98.826632] niks3-server[1114]: 2026/09/20 12:21:34 INFO lead: acquired remote=192.168.1.4:363665847worker2 # [ 98.830246] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515848worker2 # [ 98.831891] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5849worker1 # [ 98.824649] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:34Z level=info event=scheduler_connected addr=lb:500515850worker1 # [ 98.829557] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5851lb # [ 98.939279] envoy[978]: conn peer=[2001:db8:1::1]:37632 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2917 tx=1331 ms=80035852worker1 # [ 99.007114] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:34Z level=info event=scheduler_restarting5853worker1 # [ 99.019528] nix-daemon[911]: reaped child process 1168, status = succeeded5854worker2 # [ 99.032816] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515855worker2 # [ 99.034436] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5856worker1 # [ 99.034149] nix-grpc-daemon[1107]: ts=2026-09-20T12:21:34Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5857worker2 # [ 99.234345] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515858worker2 # [ 99.237511] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5859worker1 # [ 99.339309] nix-grpc-daemon[1107]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5860worker1 # [ 99.340979] nix-grpc-daemon[1107]: E0000 00:00:1789906894.980527 1150 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5861worker1 # [ 99.343213] nix-grpc-daemon[1107]: E0000 00:00:1789906894.980870 1150 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5862lb: must succeed: curl -sf localhost:9901/clusters5863lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5864worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5865worker2 # [ 99.437927] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515866worker2 # [ 99.439633] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5867worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5868worker2 # [ 99.640891] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515869worker2 # [ 99.644676] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5870worker2 # [ 99.845240] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_connected addr=lb:500515871worker2 # [ 99.847046] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5872worker2 # [ 100.050161] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500515873worker2 # [ 100.061115] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:36Z level=info event=oidc_jwks provider=mock keys=15874worker2 # [ 100.063982] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:443625875worker2 # [ 100.068154] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05876worker1 # [ 100.375348] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5877worker1 # [ 100.376581] systemd[1]: Stopped Nix worker-protocol over gRPC.5878worker1 # [ 100.377555] systemd[1]: nix-grpc-daemon.service: Consumed 1.220s CPU time over 25.729s wall clock time, 33.6M memory peak, 49.7K incoming IP traffic, 101.8K outgoing IP traffic.5879worker1 # [ 100.385814] systemd[1]: Starting Nix worker-protocol over gRPC...5880lb: must succeed: curl -sf localhost:9901/clusters5881lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5882worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5883worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5884worker1 # [ 100.527446] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5885worker1 # [ 100.529269] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5886worker1 # [ 100.532648] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15887worker1 # [ 100.540892] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5888worker1 # [ 100.543476] systemd[1]: Started Nix worker-protocol over gRPC.5889worker1 # [ 100.554780] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:36Z level=info event=scheduler_connected addr=lb:500515890worker2 # [ 100.567579] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:443625891worker2 # [ 100.569288] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05892lb: must succeed: curl -sf localhost:9901/clusters5893lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5894worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5895worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5896(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.54 seconds)5897subtest: scheduler restart mid-build: build finishes, no double build5898client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5899client # [ 101.690152] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5900client # Running as unit: mid.service; invocation ID: 40e3fb01e3de4c429712c3e3735dab945901client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5902worker2 # [ 101.892053] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:546125903client # [ 101.881707] nix[1215]: don't know how to build these paths:5904client # [ 101.882702] nix[1215]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5905worker2 # [ 101.900453] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:37Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05906worker2 # [ 101.902496] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:37Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15907worker2 # [ 101.944271] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546125908worker2 # [ 101.953662] nix-grpc-daemon[1421]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)5909worker2 # [ 102.008174] nix-grpc-daemon[1421]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 2/5)5910worker2 # [ 102.044830] nix-grpc-daemon[1421]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 314 ms (attempt 3/5)5911worker2 # [ 102.364787] nix-grpc-daemon[1421]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 4/5)5912worker2 # [ 102.456864] nix-grpc-daemon[1421]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5913worker2 # [ 102.469221] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546125914client # [ 102.464194] nix[1215]: copying 1 paths to 'lb:50051'...5915worker2 # [ 102.477540] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546125916client # [ 102.469179] nix[1215]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5917worker2 # [ 102.493482] nix-daemon[926]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5918worker2 # [ 102.513304] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:38Z level=info event=niks3_push_started pid=15365919worker2 # [ 102.689501] nix-grpc-daemon[1536]: time=2026-09-20T12:21:38.662Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5920worker2 # [ 102.841332] nix-daemon[926]: accepted connection from pid 1549, user nix-grpc-daemon (trusted)5921worker2 # [ 102.853643] nix-daemon[926]: reaped child process 1552, status = succeeded5922worker2 # [ 103.017668] nix-daemon[926]: accepted connection from pid 1557, user nix-grpc-daemon (trusted)5923worker2 # [ 103.029869] nix-daemon[926]: reaped child process 1561, status = succeeded5924lb # [ 103.054864] niks3-server[1114]: 2026/09/20 12:21:39 INFO Received uploads request method=POST path=/api/pending_closures5925worker2 # [ 103.049525] nix-grpc-daemon[1536]: time=2026-09-20T12:21:39.023Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5926worker2 # [ 103.052394] nix-grpc-daemon[1536]: time=2026-09-20T12:21:39.024Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5927lb # [ 103.075324] niks3-server[1114]: 2026/09/20 12:21:39 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5928lb # [ 103.080128] niks3-server[1114]: 2026/09/20 12:21:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5929worker2 # [ 103.073137] nix-grpc-daemon[1536]: time=2026-09-20T12:21:39.046Z level=INFO msg="Uploading 1 narinfos"5930lb # [ 103.083750] niks3-server[1114]: 2026/09/20 12:21:39 INFO Signed narinfos id=35 count=15931lb # [ 103.090678] niks3-server[1114]: 2026/09/20 12:21:39 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5932lb # [ 103.094110] niks3-server[1114]: 2026/09/20 12:21:39 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5933lb # [ 103.098197] niks3-server[1114]: 2026/09/20 12:21:39 INFO Completed upload id=355934worker2 # [ 103.087552] nix-grpc-daemon[1536]: time=2026-09-20T12:21:39.061Z level=INFO msg="Upload complete. (394ms)"5935lb # [ 103.100314] niks3-server[1114]: 2026/09/20 12:21:39 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5936worker2 # [ 103.089984] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54612 duration_s=0 paths=1 nar_bytes_in=4245937worker2 # [ 103.094623] nix-grpc-daemon[1421]: ts=2026-09-20T12:21:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546125938worker2 # [ 103.101168] nix-daemon[926]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5939worker2 # [ 103.102539] nix-daemon[926]: reaped child process 1534, status = succeeded5940worker2 # [ 103.107825] nix-daemon[926]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5941client # [ 103.096232] nix[1215]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5942lb: must succeed: curl -sf localhost:9901/clusters5943lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5944lb: must succeed: curl -sf localhost:9901/clusters5945lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5946worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5947worker2 # [ 104.262326] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1421 (nix-grpc-daemon) on client request.5948worker2 # [ 104.264540] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5949lb # [ 104.277523] niks3-server[1114]: 2026/09/20 12:21:40 INFO lead: released remote=192.168.1.4:363665950client # [ 104.255262] nix[1215]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5951client # [ 104.257460] nix[1215]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5952worker1 # [ 104.255849] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:39Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5953worker2 # [ 104.284392] systemd[1]: Starting scheduler-token.service...5954worker2 # [ 104.296428] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5955worker2 # [ 104.297893] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5956worker2 # [ 104.299191] systemd[1]: nix-grpc-daemon.service: Consumed 400ms CPU time over 15.637s wall clock time, 31.2M memory peak, 17.4K incoming IP traffic, 32.7K outgoing IP traffic.5957worker2 # [ 104.309465] nix-daemon[926]: reaped child process 1477, status = succeeded5958worker2 # [ 104.310817] nix-daemon[926]: reaped child process 1563, status = succeeded5959worker2 # [ 104.319261] nix-daemon[1565]: unexpected Nix daemon error: error: interrupted by the user5960worker2 # [ 104.324534] nix-daemon[926]: reaped child process 1565, status = failed with exit code 15961worker2 # [ 104.373453] systemd[1]: scheduler-token.service: Deactivated successfully.5962worker2 # [ 104.376187] systemd[1]: Finished scheduler-token.service.5963worker2 # [ 104.376915] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 88ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5964worker2 # [ 104.378825] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5965worker2 # [ 104.382331] systemd[1]: Starting scheduler-token.service...5966worker2 # [ 104.431631] systemd[1]: scheduler-token.service: Deactivated successfully.5967worker2 # [ 104.432717] systemd[1]: Finished scheduler-token.service.5968worker2 # [ 104.434452] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 49ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5969worker2 # [ 104.438911] systemd[1]: Starting Nix worker-protocol over gRPC...5970worker1 # [ 104.456283] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500515971worker2 # [ 104.575866] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5972worker2 # [ 104.578141] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5973worker2 # [ 104.581681] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15974worker2 # [ 104.589062] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5975worker2 # [ 104.593905] systemd[1]: Started Nix worker-protocol over gRPC.5976lb # [ 104.612331] niks3-server[1114]: 2026/09/20 12:21:40 INFO lead: acquired remote=192.168.1.4:573545977worker2 # [ 104.602197] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=scheduler_take_over5978worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.38 seconds)5979worker1 # [ 104.591563] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5980client: must succeed: sleep 5.05981worker2 # [ 104.606939] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500515982worker2 # [ 104.611082] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=oidc_jwks provider=mock keys=15983worker2 # [ 104.612934] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:353825984worker2 # [ 104.614576] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05985worker2 # [ 104.779183] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05986worker2 # [ 104.783204] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15987worker1 # [ 104.791829] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:40Z level=info event=scheduler_connected addr=lb:500515988worker2 # [ 104.804909] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:353825989worker2 # [ 104.812784] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05990worker2 # [ 104.822682] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353985991worker2 # [ 104.832544] nix-daemon[926]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)5992worker2 # [ 104.846265] nix-daemon[926]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)5993client # [ 104.846748] nix[1215]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5994worker2 # [ 104.860338] nix-daemon[926]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)5995lb # [ 108.940700] envoy[978]: conn peer=[2001:db8:1::3]:43388 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5568 tx=1321 ms=242455996lb # [ 108.944918] envoy[978]: conn peer=[2001:db8:1::4]:37258 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=9471 tx=674 ms=154475997client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5998client: waiting for success: ! systemctl is-active mid5999worker2 # [ 110.074812] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:46Z level=info event=niks3_push_started pid=16826000worker2 # [ 110.089115] nix-daemon[926]: reaped child process 1658, status = succeeded6001worker2 # [ 110.237459] nix-grpc-daemon[1682]: time=2026-09-20T12:21:46.210Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6002worker2 # [ 110.377065] nix-daemon[926]: accepted connection from pid 1695, user nix-grpc-daemon (trusted)6003worker2 # [ 110.389550] nix-daemon[926]: reaped child process 1698, status = succeeded6004lb # [ 110.410595] niks3-server[1114]: 2026/09/20 12:21:46 INFO Received uploads request method=POST path=/api/pending_closures6005worker2 # [ 110.405406] nix-grpc-daemon[1682]: time=2026-09-20T12:21:46.378Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6006worker2 # [ 110.406932] nix-grpc-daemon[1682]: time=2026-09-20T12:21:46.380Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6007lb # [ 110.435217] niks3-server[1114]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6008lb # [ 110.441195] niks3-server[1114]: 2026/09/20 12:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6009lb # [ 110.445364] niks3-server[1114]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6010lb # [ 110.455046] niks3-server[1114]: 2026/09/20 12:21:46 INFO Signed narinfos id=36 count=16011worker2 # [ 110.444374] nix-grpc-daemon[1682]: time=2026-09-20T12:21:46.417Z level=INFO msg="Uploading 1 narinfos"6012lb # [ 110.464060] niks3-server[1114]: 2026/09/20 12:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6013lb # [ 110.467317] niks3-server[1114]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6014lb # [ 110.470241] niks3-server[1114]: 2026/09/20 12:21:46 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6015lb # [ 110.475432] niks3-server[1114]: 2026/09/20 12:21:46 INFO Completed upload id=366016worker2 # [ 110.464722] nix-grpc-daemon[1682]: time=2026-09-20T12:21:46.438Z level=INFO msg="Upload complete. (224ms)"6017worker2 # [ 110.468742] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35398 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16018worker2 # [ 110.475506] nix-daemon[926]: reaped child process 1656, status = succeeded6019client # [ 110.503509] systemd[1]: mid.service: Deactivated successfully.6020client # [ 110.508439] systemd[1]: mid.service: Consumed 135ms CPU time over 8.816s wall clock time, 37.3M memory peak, 9.6K incoming IP traffic, 10.7K outgoing IP traffic.6021client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)6022client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6023client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6024(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.34 seconds)6025subtest: interrupting the client stops the build on the worker6026client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6027client # [ 111.005070] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6028client # Running as unit: intr.service; invocation ID: 382c5eadbd3c40cba4d07daa5210e6996029client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.03 seconds)6030worker2 # [ 111.202457] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:353986031client # [ 111.197384] nix[1251]: don't know how to build these paths:6032client # [ 111.198253] nix[1251]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6033worker2 # [ 111.220815] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:47Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06034worker2 # [ 111.224767] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:47Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16035worker1 # [ 111.262426] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906036worker1 # [ 111.274239] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6037worker1 # [ 111.291526] nix-grpc-daemon[1370]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5)6038worker1 # [ 111.360826] nix-grpc-daemon[1370]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 144 ms (attempt 2/5)6039worker1 # [ 111.510925] nix-grpc-daemon[1370]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 3/5)6040worker1 # [ 111.670568] nix-grpc-daemon[1370]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 703 ms (attempt 4/5)6041worker1 # [ 112.380862] nix-grpc-daemon[1370]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6042worker1 # [ 112.398105] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906043client # [ 112.408974] nix[1251]: copying 1 paths to 'lb:50051'...6044worker1 # [ 112.411987] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906045client # [ 112.414231] nix[1251]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6046worker1 # [ 112.427478] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6047worker1 # [ 112.446649] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:48Z level=info event=niks3_push_started pid=14586048worker1 # [ 112.638684] nix-grpc-daemon[1458]: time=2026-09-20T12:21:48.279Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6049worker1 # [ 112.797333] nix-daemon[911]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6050worker1 # [ 112.807950] nix-daemon[911]: reaped child process 1474, status = succeeded6051worker1 # [ 112.965081] nix-daemon[911]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)6052worker1 # [ 112.975947] nix-daemon[911]: reaped child process 1479, status = succeeded6053lb # [ 113.015471] niks3-server[1114]: 2026/09/20 12:21:48 INFO Received uploads request method=POST path=/api/pending_closures6054worker1 # [ 113.008482] nix-grpc-daemon[1458]: time=2026-09-20T12:21:48.650Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6055worker1 # [ 113.010088] nix-grpc-daemon[1458]: time=2026-09-20T12:21:48.651Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6056lb # [ 113.062299] niks3-server[1114]: 2026/09/20 12:21:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6057worker1 # [ 113.046023] nix-grpc-daemon[1458]: time=2026-09-20T12:21:48.687Z level=INFO msg="Uploading 1 narinfos"6058lb # [ 113.068363] niks3-server[1114]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6059lb # [ 113.071741] niks3-server[1114]: 2026/09/20 12:21:49 INFO Signed narinfos id=37 count=16060lb # [ 113.077466] niks3-server[1114]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6061lb # [ 113.085581] niks3-server[1114]: 2026/09/20 12:21:49 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6062lb # [ 113.089326] niks3-server[1114]: 2026/09/20 12:21:49 INFO Completed upload id=376063worker1 # [ 113.067347] nix-grpc-daemon[1458]: time=2026-09-20T12:21:48.709Z level=INFO msg="Upload complete. (425ms)"6064lb # [ 113.091412] niks3-server[1114]: 2026/09/20 12:21:49 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6065worker1 # [ 113.069544] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=4246066worker1 # [ 113.074060] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906067worker1 # [ 113.080114] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6068worker1 # [ 113.082363] nix-daemon[911]: reaped child process 1456, status = succeeded6069worker1 # [ 113.087920] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6070client # [ 113.086682] nix[1251]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6071client: must succeed: systemctl kill -s INT intr6072client # [ 114.487621] systemd[1]: intr.service: Sent signal SIGINT to main process 1251 (nix) on client request.6073client # [ 114.496325] nix[1251]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6074client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6075client # [ 114.502492] nix[1251]: error: interrupted by the user6076worker1 # [ 114.509885] nix-daemon[911]: reaped child process 1481, status = succeeded6077worker1 # [ 114.516273] nix-daemon[1483]: unexpected Nix daemon error: error: interrupted by the user6078worker1 # [ 114.520460] nix-daemon[911]: reaped child process 1483, status = failed with exit code 16079client # [ 114.527251] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6080client # [ 114.528775] systemd[1]: intr.service: Failed with result 'exit-code'.6081client # [ 114.530769] systemd[1]: intr.service: Consumed 120ms CPU time over 3.525s wall clock time, 39.3M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.6082(finished: subtest: interrupting the client stops the build on the worker, in 3.63 seconds)6083subtest: a deploy mid-build drains: the build finishes, the new generation takes over6084client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6085client # [ 114.666687] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6086client # Running as unit: sw.service; invocation ID: c28f60544d5d4ad2b332d6fff471c99f6087client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6088worker2 # [ 114.872992] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:353986089client # [ 114.865669] nix[1272]: don't know how to build these paths:6090client # [ 114.867155] nix[1272]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6091worker2 # [ 114.886281] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:50Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06092worker2 # [ 114.890314] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:50Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16093worker2 # [ 114.940457] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353986094worker2 # [ 114.952384] nix-grpc-daemon[1603]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)6095worker2 # [ 115.040540] nix-grpc-daemon[1603]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 2/5)6096worker2 # [ 115.186940] nix-grpc-daemon[1603]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 3/5)6097worker2 # [ 115.211328] nix-grpc-daemon[1603]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 219 ms (attempt 4/5)6098worker2 # [ 115.436594] nix-grpc-daemon[1603]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6099worker2 # [ 115.452339] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:353986100client # [ 115.449138] nix[1272]: copying 1 paths to 'lb:50051'...6101client # [ 115.453172] nix[1272]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6102worker2 # [ 115.464931] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:353986103worker2 # [ 115.476364] nix-daemon[926]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)6104worker2 # [ 115.648355] nix-daemon[926]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6105worker2 # [ 115.661723] nix-daemon[926]: reaped child process 1736, status = succeeded6106worker2 # [ 115.820175] nix-daemon[926]: accepted connection from pid 1738, user nix-grpc-daemon (trusted)6107worker2 # [ 115.832324] nix-daemon[926]: reaped child process 1741, status = succeeded6108lb # [ 115.859683] niks3-server[1114]: 2026/09/20 12:21:51 INFO Received uploads request method=POST path=/api/pending_closures6109worker2 # [ 115.857389] nix-grpc-daemon[1682]: time=2026-09-20T12:21:51.830Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6110worker2 # [ 115.858819] nix-grpc-daemon[1682]: time=2026-09-20T12:21:51.830Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6111lb # [ 115.885915] niks3-server[1114]: 2026/09/20 12:21:51 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6112lb # [ 115.895313] niks3-server[1114]: 2026/09/20 12:21:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6113lb # [ 115.900776] niks3-server[1114]: 2026/09/20 12:21:51 INFO Signed narinfos id=38 count=16114worker2 # [ 115.893111] nix-grpc-daemon[1682]: time=2026-09-20T12:21:51.866Z level=INFO msg="Uploading 1 narinfos"6115lb # [ 115.907861] niks3-server[1114]: 2026/09/20 12:21:51 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6116lb # [ 115.919359] niks3-server[1114]: 2026/09/20 12:21:51 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6117lb # [ 115.926258] niks3-server[1114]: 2026/09/20 12:21:51 INFO Completed upload id=386118worker2 # [ 115.917092] nix-grpc-daemon[1682]: time=2026-09-20T12:21:51.888Z level=INFO msg="Upload complete. (426ms)"6119worker2 # [ 115.918559] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35398 duration_s=0 paths=1 nar_bytes_in=4166120lb # [ 115.929954] niks3-server[1114]: 2026/09/20 12:21:51 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6121worker2 # [ 115.923306] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353986122worker2 # [ 115.928784] nix-daemon[926]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)6123worker2 # [ 115.931113] nix-daemon[926]: reaped child process 1731, status = succeeded6124worker2 # [ 115.936971] nix-daemon[926]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)6125client # [ 115.924568] nix[1272]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6126worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26127worker2 # Checking switch inhibitors... done6128worker2 # activating the configuration...6129worker2 # [ 117.778857] nixos[1776]: switching to system configuration /nix/store/9yqiiin4142wrambrim8q6llmfvq6ak0-nixos-system-worker2-test6130worker2 # [ 117.782100] systemd[1]: Stopped target Local File Systems.6131worker2 # [ 117.782879] systemd[1]: Stopped target Remote File Systems.6132worker2 # setting up /etc...6133worker2 # [ 118.394849] systemd[1]: Reload requested from client PID 1776 ('.switch-to-conf') (unit backdoor.service)...6134worker2 # [ 118.396423] systemd[1]: Reloading...6135lb # [ 118.940743] envoy[978]: conn peer=[2001:db8:1::1]:55310 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3848 tx=1636 ms=85926136lb # [ 118.944611] envoy[978]: conn peer=[2001:db8:1::1]:44356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=755 ms=33286137worker2 # [ 119.033338] systemd[1]: Reloading finished in 634 ms.6138worker2 # restarting sysinit-reactivation.target6139worker2 # [ 119.085469] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6140worker2 # [ 119.216456] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6141worker2 # [ 119.217707] systemd[1]: Reached target Reactivate sysinit units.6142worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6143worker2 # [ 119.221660] systemd[1]: Reloading D-Bus System Message Bus...6144worker2 # [ 119.224326] dbus-broker-launch[581]: Caught SIGHUP, trigger reload.6145worker2 # [ 119.228340] systemd[1]: Reloading Nix worker-protocol over gRPC...6146worker2 # [ 119.260812] dbus-broker-launch[581]: Looking up NSS user entry for 'systemd-timesync'...6147worker2 # [ 119.261932] dbus-broker-launch[581]: NSS returned no entry for 'systemd-timesync'6148worker2 # [ 119.262941] dbus-broker-launch[581]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6149worker2 # [ 119.275280] systemd[1]: Reloaded Nix worker-protocol over gRPC.6150worker2 # [ 119.291381] systemd[1]: Reloaded D-Bus System Message Bus.6151worker2 # [ 119.297447] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6152worker2 # [ 119.298699] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6153worker2 # [ 119.302061] systemd[1]: Reached target Remote File Systems.6154worker2 # [ 119.306069] systemd[1]: Reached target Local File Systems.6155worker2 # [ 119.306869] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6156worker2 # [ 119.312830] systemd[1]: Starting Create SUID/SGID Wrappers...6157worker2 # [ 119.314065] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6158worker2 # [ 119.315225] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676159worker2 # [ 119.317168] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6160worker2 # [ 119.318602] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6161worker2 # [ 119.319749] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6162worker2 # [ 119.598151] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:55Z level=info event=draining6163worker2 # [ 119.728652] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6164worker2 # [ 119.731239] systemd[1]: Finished Create SUID/SGID Wrappers.6165worker2 # [ 119.733124] systemd[1]: Starting linger-users.service...6166worker2 # [ 119.767068] systemd[1]: linger-users.service: Deactivated successfully.6167worker2 # [ 119.767929] systemd[1]: Finished linger-users.service.6168worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6169worker2 # [ 120.024873] nixos[1776]: finished switching to system configuration /nix/store/9yqiiin4142wrambrim8q6llmfvq6ak0-nixos-system-worker2-test6170worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.90 seconds)6171worker2: must succeed: pgrep -f 'read -t [9]0 x'6172worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6173client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26174worker1 # [ 120.300385] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:496906175client # don't know how to build these paths:6176client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6177worker1 # [ 120.305263] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906178worker2 # [ 120.328739] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:56Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06179worker2 # [ 120.331212] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:56Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16180worker1 # [ 120.362759] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906181worker1 # [ 120.370242] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906182client # copying 1 paths to 'lb:50051'...6183worker1 # [ 120.377401] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906184client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6185worker1 # [ 120.386541] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6186worker1 # [ 120.543556] nix-daemon[911]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)6187worker1 # [ 120.556307] nix-daemon[911]: reaped child process 1532, status = succeeded6188worker1 # [ 120.716980] nix-daemon[911]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6189worker1 # [ 120.729213] nix-daemon[911]: reaped child process 1537, status = succeeded6190lb # [ 120.767515] niks3-server[1114]: 2026/09/20 12:21:56 INFO Received uploads request method=POST path=/api/pending_closures6191worker1 # [ 120.751478] nix-grpc-daemon[1458]: time=2026-09-20T12:21:56.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6192worker1 # [ 120.753047] nix-grpc-daemon[1458]: time=2026-09-20T12:21:56.394Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6193lb # [ 120.784681] niks3-server[1114]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6194lb # [ 120.788571] niks3-server[1114]: 2026/09/20 12:21:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6195lb # [ 120.790384] niks3-server[1114]: 2026/09/20 12:21:56 INFO Signed narinfos id=39 count=16196worker1 # [ 120.768377] nix-grpc-daemon[1458]: time=2026-09-20T12:21:56.409Z level=INFO msg="Uploading 1 narinfos"6197lb # [ 120.794226] niks3-server[1114]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6198lb # [ 120.798056] niks3-server[1114]: 2026/09/20 12:21:56 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6199lb # [ 120.801664] niks3-server[1114]: 2026/09/20 12:21:56 INFO Completed upload id=396200worker1 # [ 120.779648] nix-grpc-daemon[1458]: time=2026-09-20T12:21:56.421Z level=INFO msg="Upload complete. (387ms)"6201lb # [ 120.803464] niks3-server[1114]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6202worker1 # [ 120.781633] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=5286203worker1 # [ 120.785695] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906204worker1 # [ 120.792674] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6205worker1 # [ 120.794113] nix-daemon[911]: reaped child process 1527, status = succeeded6206client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6207client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6208worker1 # [ 120.799927] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6209client # farm-a> LOG-farm-a-during-drain6210worker1 # [ 121.625679] nix-daemon[911]: reaped child process 1541, status = succeeded6211worker1 # [ 121.773805] nix-daemon[911]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)6212worker1 # [ 121.786569] nix-daemon[911]: reaped child process 1559, status = succeeded6213lb # [ 121.819526] niks3-server[1114]: 2026/09/20 12:21:57 INFO Received uploads request method=POST path=/api/pending_closures6214worker1 # [ 121.802520] nix-grpc-daemon[1458]: time=2026-09-20T12:21:57.443Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6215worker1 # [ 121.805060] nix-grpc-daemon[1458]: time=2026-09-20T12:21:57.445Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6216lb # [ 121.840083] niks3-server[1114]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6217lb # [ 121.846591] niks3-server[1114]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6218lb # [ 121.849131] niks3-server[1114]: 2026/09/20 12:21:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6219lb # [ 121.850797] niks3-server[1114]: 2026/09/20 12:21:57 INFO Signed narinfos id=40 count=16220worker1 # [ 121.830610] nix-grpc-daemon[1458]: time=2026-09-20T12:21:57.471Z level=INFO msg="Uploading 1 narinfos"6221lb # [ 121.855249] niks3-server[1114]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6222lb # [ 121.859813] niks3-server[1114]: 2026/09/20 12:21:57 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6223lb # [ 121.862485] niks3-server[1114]: 2026/09/20 12:21:57 INFO Completed upload id=406224worker1 # [ 121.841852] nix-grpc-daemon[1458]: time=2026-09-20T12:21:57.483Z level=INFO msg="Upload complete. (235ms)"6225lb # [ 121.864936] niks3-server[1114]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6226worker1 # [ 121.844250] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16227worker1 # [ 121.851187] nix-daemon[911]: reaped child process 1539, status = succeeded6228worker2 # [ 121.863216] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:57Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06229worker2 # [ 121.866091] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:57Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16230worker1 # [ 121.901179] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906231worker1 # [ 121.913947] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906232worker1 # [ 121.925908] nix-grpc-daemon[1458]: time=2026-09-20T12:21:57.567Z level=INFO msg="All 1 paths already cached"6233client # copying 1 paths to 'lb:50051'...6234worker1 # [ 121.933922] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906235client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6236worker1 # [ 121.960160] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6237worker1 # [ 122.117785] nix-daemon[911]: accepted connection from pid 1563, user nix-grpc-daemon (trusted)6238worker1 # [ 122.130144] nix-daemon[911]: reaped child process 1566, status = succeeded6239worker1 # [ 122.277708] nix-daemon[911]: accepted connection from pid 1568, user nix-grpc-daemon (trusted)6240worker1 # [ 122.290176] nix-daemon[911]: reaped child process 1571, status = succeeded6241lb # [ 122.322070] niks3-server[1114]: 2026/09/20 12:21:58 INFO Received uploads request method=POST path=/api/pending_closures6242worker1 # [ 122.308442] nix-grpc-daemon[1458]: time=2026-09-20T12:21:57.949Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6243worker1 # [ 122.309831] nix-grpc-daemon[1458]: time=2026-09-20T12:21:57.949Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6244lb # [ 122.358233] niks3-server[1114]: 2026/09/20 12:21:58 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6245lb # [ 122.367685] niks3-server[1114]: 2026/09/20 12:21:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6246worker1 # [ 122.351276] nix-grpc-daemon[1458]: time=2026-09-20T12:21:57.992Z level=INFO msg="Uploading 1 narinfos"6247lb # [ 122.373217] niks3-server[1114]: 2026/09/20 12:21:58 INFO Signed narinfos id=41 count=16248lb # [ 122.377522] niks3-server[1114]: 2026/09/20 12:21:58 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6249lb # [ 122.388854] niks3-server[1114]: 2026/09/20 12:21:58 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6250lb # [ 122.394981] niks3-server[1114]: 2026/09/20 12:21:58 INFO Completed upload id=416251worker1 # [ 122.373081] nix-grpc-daemon[1458]: time=2026-09-20T12:21:58.014Z level=INFO msg="Upload complete. (399ms)"6252lb # [ 122.398260] niks3-server[1114]: 2026/09/20 12:21:58 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6253worker1 # [ 122.377122] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=6406254worker1 # [ 122.382055] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906255worker1 # [ 122.392061] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6256worker1 # [ 122.395573] nix-daemon[911]: reaped child process 1561, status = succeeded6257worker1 # [ 122.400581] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6258client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6259client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6260client # farm-b> LOG-farm-b-during-drain6261worker1 # [ 123.630231] nix-daemon[911]: reaped child process 1575, status = succeeded6262worker1 # [ 123.787693] nix-daemon[911]: accepted connection from pid 1590, user nix-grpc-daemon (trusted)6263worker1 # [ 123.799677] nix-daemon[911]: reaped child process 1593, status = succeeded6264lb # [ 123.838667] niks3-server[1114]: 2026/09/20 12:21:59 INFO Received uploads request method=POST path=/api/pending_closures6265worker1 # [ 123.823089] nix-grpc-daemon[1458]: time=2026-09-20T12:21:59.464Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6266worker1 # [ 123.824601] nix-grpc-daemon[1458]: time=2026-09-20T12:21:59.466Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6267lb # [ 123.878203] niks3-server[1114]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6268lb # [ 123.883889] niks3-server[1114]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6269lb # [ 123.890464] niks3-server[1114]: 2026/09/20 12:21:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6270lb # [ 123.897099] niks3-server[1114]: 2026/09/20 12:21:59 INFO Signed narinfos id=42 count=16271worker1 # [ 123.878049] nix-grpc-daemon[1458]: time=2026-09-20T12:21:59.519Z level=INFO msg="Uploading 1 narinfos"6272lb # [ 123.900250] niks3-server[1114]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6273lb # [ 123.910270] niks3-server[1114]: 2026/09/20 12:21:59 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6274lb # [ 123.917705] niks3-server[1114]: 2026/09/20 12:21:59 INFO Completed upload id=426275lb # [ 123.919697] niks3-server[1114]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6276worker1 # [ 123.897407] nix-grpc-daemon[1458]: time=2026-09-20T12:21:59.538Z level=INFO msg="Upload complete. (288ms)"6277worker1 # [ 123.902339] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16278worker2 # [ 123.918084] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:59Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06279worker2 # [ 123.919831] nix-grpc-daemon[1603]: ts=2026-09-20T12:21:59Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16280worker1 # [ 123.916596] nix-daemon[911]: reaped child process 1573, status = succeeded6281worker1 # [ 123.951161] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906282worker1 # [ 123.968198] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906283worker1 # [ 123.982215] nix-grpc-daemon[1458]: time=2026-09-20T12:21:59.623Z level=INFO msg="All 2 paths already cached"6284client # copying 1 paths to 'lb:50051'...6285worker1 # [ 123.991138] nix-grpc-daemon[1370]: ts=2026-09-20T12:21:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906286client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6287worker1 # [ 124.011211] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6288worker1 # [ 124.180898] nix-daemon[911]: accepted connection from pid 1597, user nix-grpc-daemon (trusted)6289worker1 # [ 124.193846] nix-daemon[911]: reaped child process 1600, status = succeeded6290worker1 # [ 124.350302] nix-daemon[911]: accepted connection from pid 1602, user nix-grpc-daemon (trusted)6291worker1 # [ 124.363230] nix-daemon[911]: reaped child process 1605, status = succeeded6292lb # [ 124.395792] niks3-server[1114]: 2026/09/20 12:22:00 INFO Received uploads request method=POST path=/api/pending_closures6293worker1 # [ 124.378099] nix-grpc-daemon[1458]: time=2026-09-20T12:22:00.019Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6294worker1 # [ 124.380070] nix-grpc-daemon[1458]: time=2026-09-20T12:22:00.021Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6295lb # [ 124.409406] niks3-server[1114]: 2026/09/20 12:22:00 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6296lb # [ 124.414627] niks3-server[1114]: 2026/09/20 12:22:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6297lb # [ 124.417464] niks3-server[1114]: 2026/09/20 12:22:00 INFO Signed narinfos id=43 count=16298worker1 # [ 124.395579] nix-grpc-daemon[1458]: time=2026-09-20T12:22:00.037Z level=INFO msg="Uploading 1 narinfos"6299lb # [ 124.420970] niks3-server[1114]: 2026/09/20 12:22:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6300lb # [ 124.426074] niks3-server[1114]: 2026/09/20 12:22:00 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6301lb # [ 124.429157] niks3-server[1114]: 2026/09/20 12:22:00 INFO Completed upload id=436302worker1 # [ 124.408566] nix-grpc-daemon[1458]: time=2026-09-20T12:22:00.050Z level=INFO msg="Upload complete. (385ms)"6303lb # [ 124.431160] niks3-server[1114]: 2026/09/20 12:22:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6304worker1 # [ 124.410793] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=7686305worker1 # [ 124.415115] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906306worker1 # [ 124.419707] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6307worker1 # [ 124.424223] nix-daemon[911]: reaped child process 1595, status = succeeded6308client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6309client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6310worker1 # [ 124.429453] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6311client # farm-top> LOG-farm-top-during-drain6312worker1 # [ 125.257596] nix-daemon[911]: reaped child process 1609, status = succeeded6313worker1 # [ 125.394403] nix-daemon[911]: accepted connection from pid 1624, user nix-grpc-daemon (trusted)6314worker1 # [ 125.405855] nix-daemon[911]: reaped child process 1627, status = succeeded6315lb # [ 125.439665] niks3-server[1114]: 2026/09/20 12:22:01 INFO Received uploads request method=POST path=/api/pending_closures6316worker1 # [ 125.425826] nix-grpc-daemon[1458]: time=2026-09-20T12:22:01.067Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6317worker1 # [ 125.427686] nix-grpc-daemon[1458]: time=2026-09-20T12:22:01.069Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6318lb # [ 125.465407] niks3-server[1114]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6319lb # [ 125.473913] niks3-server[1114]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6320lb # [ 125.484886] niks3-server[1114]: 2026/09/20 12:22:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6321lb # [ 125.489506] niks3-server[1114]: 2026/09/20 12:22:01 INFO Signed narinfos id=44 count=16322worker1 # [ 125.470157] nix-grpc-daemon[1458]: time=2026-09-20T12:22:01.111Z level=INFO msg="Uploading 1 narinfos"6323lb # [ 125.494616] niks3-server[1114]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6324lb # [ 125.503573] niks3-server[1114]: 2026/09/20 12:22:01 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6325lb # [ 125.507032] niks3-server[1114]: 2026/09/20 12:22:01 INFO Completed upload id=446326worker1 # [ 125.484823] nix-grpc-daemon[1458]: time=2026-09-20T12:22:01.126Z level=INFO msg="Upload complete. (247ms)"6327lb # [ 125.508497] niks3-server[1114]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6328worker1 # [ 125.489778] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16329worker1 # [ 125.499924] nix-daemon[911]: reaped child process 1607, status = succeeded6330client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.46 seconds)6331client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6332client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6333worker2: must succeed: pkill -f 'read -t [9]0 x'6334worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6335client: waiting for success: ! systemctl is-active sw6336worker2 # [ 125.970895] nix-daemon[926]: reaped child process 1749, status = succeeded6337worker2 # [ 126.104850] nix-daemon[926]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6338worker2 # [ 126.116873] nix-daemon[926]: reaped child process 1925, status = succeeded6339lb # [ 126.141379] niks3-server[1114]: 2026/09/20 12:22:02 INFO Received uploads request method=POST path=/api/pending_closures6340worker2 # [ 126.138452] nix-grpc-daemon[1682]: time=2026-09-20T12:22:02.111Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6341worker2 # [ 126.140183] nix-grpc-daemon[1682]: time=2026-09-20T12:22:02.113Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6342lb # [ 126.173587] niks3-server[1114]: 2026/09/20 12:22:02 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6343lb # [ 126.187494] niks3-server[1114]: 2026/09/20 12:22:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6344lb # [ 126.197504] niks3-server[1114]: 2026/09/20 12:22:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6345lb # [ 126.203108] niks3-server[1114]: 2026/09/20 12:22:02 INFO Signed narinfos id=45 count=16346worker2 # [ 126.192869] nix-grpc-daemon[1682]: time=2026-09-20T12:22:02.165Z level=INFO msg="Uploading 1 narinfos"6347lb # [ 126.208987] niks3-server[1114]: 2026/09/20 12:22:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6348lb # [ 126.218791] niks3-server[1114]: 2026/09/20 12:22:02 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6349lb # [ 126.223792] niks3-server[1114]: 2026/09/20 12:22:02 INFO Completed upload id=456350worker2 # [ 126.213335] nix-grpc-daemon[1682]: time=2026-09-20T12:22:02.186Z level=INFO msg="Upload complete. (260ms)"6351lb # [ 126.226102] niks3-server[1114]: 2026/09/20 12:22:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6352worker2 # [ 126.221747] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35398 duration_s=10 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16353worker2 # [ 126.234788] nix-daemon[926]: reaped child process 1747, status = succeeded6354client # [ 126.242084] systemd[1]: sw.service: Deactivated successfully.6355client # [ 126.245720] systemd[1]: sw.service: Consumed 117ms CPU time over 11.577s wall clock time, 37.1M memory peak, 8.9K incoming IP traffic, 9.1K outgoing IP traffic.6356worker2 # [ 126.599129] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:02Z level=info event=signal_exit6357client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6358client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6359worker2 # [ 126.822960] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:02Z level=info event=scheduler_yield6360worker1 # [ 126.815305] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:02Z level=info event=scheduler_restarting addr=lb:500516361lb # [ 126.836218] niks3-server[1114]: 2026/09/20 12:22:02 INFO lead: released remote=192.168.1.4:573546362worker2 # [ 126.829584] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:353826363worker2 # [ 126.835283] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:02Z level=info event=worker_gone addr=192.168.1.3:500516364worker2 # [ 126.839631] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:02Z level=info event=scheduler_restarting addr=lb:500516365worker2 # [ 126.843200] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:353826366worker2 # [ 126.846986] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:02Z level=info event=worker_gone addr=192.168.1.4:500516367client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6368worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6369worker2 # [ 126.909423] systemd[1]: Starting scheduler-token.service...6370worker2 # [ 126.965500] systemd[1]: scheduler-token.service: Deactivated successfully.6371worker2 # [ 126.968250] systemd[1]: Finished scheduler-token.service.6372worker2 # [ 126.969328] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 58ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6373worker1 # [ 127.015580] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:02Z level=info event=scheduler_connected addr=lb:500516374worker1 # [ 127.019884] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6375worker2 # [ 127.036198] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516376worker2 # [ 127.041459] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6377worker1 # [ 127.220658] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:02Z level=info event=scheduler_connected addr=lb:500516378worker1 # [ 127.224497] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6379worker2 # [ 127.243619] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516380worker2 # [ 127.247370] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6381worker1 # [ 127.425083] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516382worker1 # [ 127.428876] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6383worker2 # [ 127.444562] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516384worker2 # [ 127.448878] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6385worker1 # [ 127.629763] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516386worker1 # [ 127.633642] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6387worker2 # [ 127.649577] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516388worker2 # [ 127.653561] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6389worker2 # [ 127.828211] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=scheduler_restarting6390worker1 # [ 127.836121] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516391worker2 # [ 127.848156] nix-daemon[926]: reaped child process 1654, status = succeeded6392worker1 # [ 127.839472] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6393worker2 # [ 127.863196] nix-grpc-daemon[1603]: ts=2026-09-20T12:22:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6394worker2 # [ 128.036324] nix-grpc-daemon[1603]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6395worker2 # [ 128.039333] nix-grpc-daemon[1603]: E0000 00:00:1789906924.009494 1646 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6396worker2 # [ 128.043554] nix-grpc-daemon[1603]: E0000 00:00:1789906924.016817 1646 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6397worker1 # [ 128.036720] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516398worker1 # [ 128.040872] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6399worker1 # [ 128.241529] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_connected addr=lb:500516400worker1 # [ 128.245487] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6401worker1 # [ 128.446150] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_connected addr=lb:500516402worker1 # [ 128.450386] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6403worker2 # [ 128.630926] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6404worker2 # [ 128.634184] systemd[1]: nix-grpc-daemon.service: Consumed 649ms CPU time over 24.337s wall clock time, 31.1M memory peak, 25.8K incoming IP traffic, 36.4K outgoing IP traffic.6405worker2 # [ 128.650937] systemd[1]: Starting Nix worker-protocol over gRPC...6406worker1 # [ 128.650839] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_connected addr=lb:500516407worker1 # [ 128.654875] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6408worker2 # [ 128.806869] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6409worker2 # [ 128.808677] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6410worker2 # [ 128.811958] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16411worker2 # [ 128.819057] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6412worker2 # [ 128.822409] systemd[1]: Started Nix worker-protocol over gRPC.6413lb # [ 128.838532] niks3-server[1114]: 2026/09/20 12:22:04 INFO lead: acquired remote=192.168.1.4:511626414worker2 # [ 128.828783] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:04Z level=info event=scheduler_take_over6415worker2 # [ 128.834222] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:04Z level=info event=scheduler_connected addr=lb:500516416worker2 # [ 128.835966] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6417worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.99 seconds)6418lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6419lb: must succeed: curl -sf localhost:9901/clusters6420worker1 # [ 128.857458] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_connected addr=lb:500516421worker1 # [ 128.860822] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6422lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6423lb # [ 128.941004] envoy[978]: conn peer=[2001:db8:1::1]:54698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9598 tx=3823 ms=52056424lb # [ 128.942830] envoy[978]: conn peer=[2001:db8:1::1]:54682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1369 ms=113596425lb # [ 128.944422] envoy[978]: conn peer=[2001:db8:1::4]:37962 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4876 tx=568 ms=232516426worker2 # [ 129.037239] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=scheduler_connected addr=lb:500516427worker2 # [ 129.040890] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6428worker1 # [ 129.057937] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_connected addr=lb:500516429worker1 # [ 129.062386] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6430worker2 # [ 129.242096] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=scheduler_connected addr=lb:500516431worker2 # [ 129.245880] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6432worker1 # [ 129.262961] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_connected addr=lb:500516433worker1 # [ 129.267132] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6434worker2 # [ 129.446979] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=scheduler_connected addr=lb:500516435worker2 # [ 129.450756] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6436worker1 # [ 129.468230] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:05Z level=info event=scheduler_connected addr=lb:500516437worker1 # [ 129.472255] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6438worker2 # [ 129.651627] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=scheduler_connected addr=lb:500516439worker2 # [ 129.664187] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=oidc_jwks provider=mock keys=16440worker2 # [ 129.668871] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:594386441worker2 # [ 129.672860] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06442worker1 # [ 129.673873] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:05Z level=info event=scheduler_connected addr=lb:500516443worker2 # [ 129.686582] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:594386444worker2 # [ 129.690359] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06445lb: must succeed: curl -sf localhost:9901/clusters6446lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6447lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6448lb: must succeed: curl -sf localhost:9901/clusters6449lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6450worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6451worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6452(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.44 seconds)6453subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6454client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16455worker2 # [ 130.238498] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594526456worker2 # [ 130.249796] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6457worker2 # [ 130.263618] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594526458worker2 # [ 130.275603] nix-grpc-daemon[1942]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)6459worker2 # [ 130.331925] nix-grpc-daemon[1942]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 2/5)6460worker2 # [ 130.428464] nix-grpc-daemon[1942]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 315 ms (attempt 3/5)6461worker2 # [ 130.749364] nix-grpc-daemon[1942]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 250 ms (attempt 4/5)6462worker2 # [ 131.005731] nix-grpc-daemon[1942]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6463worker2 # [ 131.051901] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:07Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06464worker2 # [ 131.056139] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:07Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16465worker2 # [ 131.060417] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:07Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06466worker2 # [ 131.064291] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:07Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06467worker1 # [ 131.085940] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906468worker1 # [ 131.097575] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906469worker1 # [ 131.107168] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906470worker1 # [ 131.118356] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6471worker1 # [ 131.308870] nix-daemon[911]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6472worker1 # [ 131.325091] nix-daemon[911]: reaped child process 1634, status = succeeded6473worker1 # [ 131.507780] nix-daemon[911]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6474worker1 # [ 131.520481] nix-daemon[911]: reaped child process 1639, status = succeeded6475lb # [ 131.559815] niks3-server[1114]: 2026/09/20 12:22:07 INFO Received uploads request method=POST path=/api/pending_closures6476worker1 # [ 131.549325] nix-grpc-daemon[1458]: time=2026-09-20T12:22:07.190Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6477worker1 # [ 131.551315] nix-grpc-daemon[1458]: time=2026-09-20T12:22:07.192Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6478lb # [ 131.595923] niks3-server[1114]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6479lb # [ 131.604708] niks3-server[1114]: 2026/09/20 12:22:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6480worker1 # [ 131.587267] nix-grpc-daemon[1458]: time=2026-09-20T12:22:07.228Z level=INFO msg="Uploading 1 narinfos"6481lb # [ 131.610383] niks3-server[1114]: 2026/09/20 12:22:07 INFO Signed narinfos id=46 count=16482lb # [ 131.616505] niks3-server[1114]: 2026/09/20 12:22:07 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6483lb # [ 131.626897] niks3-server[1114]: 2026/09/20 12:22:07 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6484lb # [ 131.630914] niks3-server[1114]: 2026/09/20 12:22:07 INFO Completed upload id=466485worker1 # [ 131.608874] nix-grpc-daemon[1458]: time=2026-09-20T12:22:07.250Z level=INFO msg="Upload complete. (477ms)"6486worker1 # [ 131.614152] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=5046487worker1 # [ 131.619359] nix-grpc-daemon[1458]: time=2026-09-20T12:22:07.254Z level=WARN msg="Failed to register uploaded object" key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo error="server returned 404: object is not pending upload\n"6488worker1 # [ 131.624521] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906489worker1 # [ 131.632163] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6490worker1 # [ 131.634747] nix-daemon[911]: reaped child process 1629, status = succeeded6491worker1 # [ 131.640471] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6492worker1 # [ 133.054104] nix-daemon[911]: reaped child process 1643, status = succeeded6493worker1 # [ 133.202300] nix-daemon[911]: accepted connection from pid 1658, user nix-grpc-daemon (trusted)6494worker1 # [ 133.214869] nix-daemon[911]: reaped child process 1661, status = succeeded6495lb # [ 133.251785] niks3-server[1114]: 2026/09/20 12:22:09 INFO Received uploads request method=POST path=/api/pending_closures6496worker1 # [ 133.237311] nix-grpc-daemon[1458]: time=2026-09-20T12:22:08.877Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6497worker1 # [ 133.238755] nix-grpc-daemon[1458]: time=2026-09-20T12:22:08.877Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6498lb # [ 133.289771] niks3-server[1114]: 2026/09/20 12:22:09 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6499lb # [ 133.292808] niks3-server[1114]: 2026/09/20 12:22:09 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6500lb # [ 133.303096] niks3-server[1114]: 2026/09/20 12:22:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6501lb # [ 133.307577] niks3-server[1114]: 2026/09/20 12:22:09 INFO Signed narinfos id=47 count=16502worker1 # [ 133.288127] nix-grpc-daemon[1458]: time=2026-09-20T12:22:08.929Z level=INFO msg="Uploading 1 narinfos"6503lb # [ 133.310330] niks3-server[1114]: 2026/09/20 12:22:09 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6504lb # [ 133.316360] niks3-server[1114]: 2026/09/20 12:22:09 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6505lb # [ 133.320531] niks3-server[1114]: 2026/09/20 12:22:09 INFO Completed upload id=476506worker1 # [ 133.298460] nix-grpc-daemon[1458]: time=2026-09-20T12:22:08.940Z level=INFO msg="Upload complete. (250ms)"6507lb # [ 133.323027] niks3-server[1114]: 2026/09/20 12:22:09 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6508worker1 # [ 133.301752] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16509worker2 # [ 133.318548] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:09Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16510worker1 # [ 133.313436] nix-daemon[911]: reaped child process 1641, status = succeeded6511worker2 # [ 133.324821] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:09Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06512worker1 # [ 133.349725] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906513worker1 # [ 133.361308] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906514worker1 # [ 133.370249] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906515worker1 # [ 133.380167] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6516worker1 # [ 133.542906] nix-daemon[911]: accepted connection from pid 1665, user nix-grpc-daemon (trusted)6517worker1 # [ 133.555877] nix-daemon[911]: reaped child process 1668, status = succeeded6518worker1 # [ 133.718383] nix-daemon[911]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6519worker1 # [ 133.730505] nix-daemon[911]: reaped child process 1673, status = succeeded6520lb # [ 133.769410] niks3-server[1114]: 2026/09/20 12:22:09 INFO Received uploads request method=POST path=/api/pending_closures6521worker1 # [ 133.756654] nix-grpc-daemon[1458]: time=2026-09-20T12:22:09.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6522worker1 # [ 133.758269] nix-grpc-daemon[1458]: time=2026-09-20T12:22:09.399Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6523lb # [ 133.806092] niks3-server[1114]: 2026/09/20 12:22:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6524worker1 # [ 133.800300] nix-grpc-daemon[1458]: time=2026-09-20T12:22:09.441Z level=INFO msg="Uploading 1 narinfos"6525lb # [ 133.822299] niks3-server[1114]: 2026/09/20 12:22:09 INFO Signed narinfos id=48 count=16526lb # [ 133.831871] niks3-server[1114]: 2026/09/20 12:22:09 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6527lb # [ 134.098757] niks3-server[1114]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6528lb # [ 134.128125] niks3-server[1114]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6529lb # [ 134.142619] niks3-server[1114]: 2026/09/20 12:22:10 INFO Completed upload id=486530worker1 # [ 134.120852] nix-grpc-daemon[1458]: time=2026-09-20T12:22:09.762Z level=INFO msg="Upload complete. (730ms)"6531worker1 # [ 134.125276] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=5046532lb # [ 134.148332] niks3-server[1114]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6533worker1 # [ 134.132399] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906534worker1 # [ 134.142324] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6535worker1 # [ 134.149977] nix-daemon[911]: reaped child process 1663, status = succeeded6536worker1 # [ 134.159685] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6537worker1 # [ 134.791695] nix-daemon[911]: reaped child process 1677, status = succeeded6538worker1 # [ 134.954088] nix-daemon[911]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6539worker1 # [ 134.964992] nix-daemon[911]: reaped child process 1695, status = succeeded6540lb # [ 135.003574] niks3-server[1114]: 2026/09/20 12:22:10 INFO Received uploads request method=POST path=/api/pending_closures6541worker1 # [ 134.989250] nix-grpc-daemon[1458]: time=2026-09-20T12:22:10.630Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6542worker1 # [ 134.992057] nix-grpc-daemon[1458]: time=2026-09-20T12:22:10.632Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6543lb # [ 135.045206] niks3-server[1114]: 2026/09/20 12:22:10 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6544lb # [ 135.052656] niks3-server[1114]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6545lb # [ 135.059584] niks3-server[1114]: 2026/09/20 12:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6546lb # [ 135.064597] niks3-server[1114]: 2026/09/20 12:22:11 INFO Signed narinfos id=49 count=16547worker1 # [ 135.045519] nix-grpc-daemon[1458]: time=2026-09-20T12:22:10.686Z level=INFO msg="Uploading 1 narinfos"6548lb # [ 135.071073] niks3-server[1114]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6549lb # [ 135.080057] niks3-server[1114]: 2026/09/20 12:22:11 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6550lb # [ 135.086271] niks3-server[1114]: 2026/09/20 12:22:11 INFO Completed upload id=496551worker1 # [ 135.064200] nix-grpc-daemon[1458]: time=2026-09-20T12:22:10.705Z level=INFO msg="Upload complete. (292ms)"6552lb # [ 135.088796] niks3-server[1114]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6553worker1 # [ 135.069851] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16554worker2 # [ 135.085464] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:11Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16555worker1 # [ 135.077236] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906556worker2 # [ 135.093340] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:11Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06557worker1 # [ 135.087520] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906558worker1 # [ 135.093496] nix-daemon[911]: reaped child process 1675, status = succeeded6559worker1 # [ 135.098276] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906560worker1 # [ 135.102227] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6561worker1 # [ 135.259367] nix-daemon[911]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)6562worker1 # [ 135.272780] nix-daemon[911]: reaped child process 1702, status = succeeded6563worker1 # [ 135.434278] nix-daemon[911]: accepted connection from pid 1704, user nix-grpc-daemon (trusted)6564worker1 # [ 135.446766] nix-daemon[911]: reaped child process 1707, status = succeeded6565lb # [ 135.483714] niks3-server[1114]: 2026/09/20 12:22:11 INFO Received uploads request method=POST path=/api/pending_closures6566worker1 # [ 135.471656] nix-grpc-daemon[1458]: time=2026-09-20T12:22:11.113Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6567worker1 # [ 135.473254] nix-grpc-daemon[1458]: time=2026-09-20T12:22:11.114Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6568lb # [ 135.514775] niks3-server[1114]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6569lb # [ 135.526242] niks3-server[1114]: 2026/09/20 12:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6570lb # [ 135.530821] niks3-server[1114]: 2026/09/20 12:22:11 INFO Signed narinfos id=50 count=16571worker1 # [ 135.511512] nix-grpc-daemon[1458]: time=2026-09-20T12:22:11.153Z level=INFO msg="Uploading 1 narinfos"6572lb # [ 135.539877] niks3-server[1114]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6573lb # [ 135.547869] niks3-server[1114]: 2026/09/20 12:22:11 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6574lb # [ 135.554900] niks3-server[1114]: 2026/09/20 12:22:11 INFO Completed upload id=506575worker1 # [ 135.532886] nix-grpc-daemon[1458]: time=2026-09-20T12:22:11.174Z level=INFO msg="Upload complete. (424ms)"6576lb # [ 135.557512] niks3-server[1114]: 2026/09/20 12:22:11 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6577worker1 # [ 135.536382] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=5046578worker1 # [ 135.542177] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906579worker1 # [ 135.551127] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6580worker1 # [ 135.558512] nix-daemon[911]: reaped child process 1697, status = succeeded6581worker1 # [ 135.563401] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6582worker1 # [ 136.390542] nix-daemon[911]: reaped child process 1711, status = succeeded6583worker1 # [ 136.554263] nix-daemon[911]: accepted connection from pid 1726, user nix-grpc-daemon (trusted)6584worker1 # [ 136.566907] nix-daemon[911]: reaped child process 1729, status = succeeded6585lb # [ 136.607096] niks3-server[1114]: 2026/09/20 12:22:12 INFO Received uploads request method=POST path=/api/pending_closures6586worker1 # [ 136.645171] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.286Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6587worker1 # [ 136.648667] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.289Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6588lb # [ 136.704800] niks3-server[1114]: 2026/09/20 12:22:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6589lb # [ 136.709742] niks3-server[1114]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6590lb # [ 136.715650] niks3-server[1114]: 2026/09/20 12:22:12 INFO Signed narinfos id=51 count=16591worker1 # [ 136.697154] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.338Z level=INFO msg="Uploading 1 narinfos"6592lb # [ 136.721119] niks3-server[1114]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6593lb # [ 136.731927] niks3-server[1114]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6594lb # [ 136.740279] niks3-server[1114]: 2026/09/20 12:22:12 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6595lb # [ 136.749878] niks3-server[1114]: 2026/09/20 12:22:12 INFO Completed upload id=516596worker1 # [ 136.730542] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.372Z level=INFO msg="Upload complete. (360ms)"6597lb # [ 136.754201] niks3-server[1114]: 2026/09/20 12:22:12 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6598worker2 # [ 136.748316] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:12Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16599worker2 # [ 136.754175] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:12Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06600worker1 # [ 136.738596] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16601worker1 # [ 136.746305] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906602worker1 # [ 136.759741] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906603worker1 # [ 136.766283] nix-daemon[911]: reaped child process 1709, status = succeeded6604worker1 # [ 136.773616] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.415Z level=INFO msg="All 1 paths already cached"6605worker1 # [ 136.780428] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906606worker1 # [ 136.787309] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6607worker1 # [ 136.943598] nix-daemon[911]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6608worker1 # [ 136.956233] nix-daemon[911]: reaped child process 1736, status = succeeded6609worker1 # [ 137.086544] nix-daemon[911]: accepted connection from pid 1738, user nix-grpc-daemon (trusted)6610worker1 # [ 137.098608] nix-daemon[911]: reaped child process 1741, status = succeeded6611lb # [ 137.135180] niks3-server[1114]: 2026/09/20 12:22:13 INFO Received uploads request method=POST path=/api/pending_closures6612worker1 # [ 137.122330] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.763Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6613worker1 # [ 137.124174] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.765Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6614lb # [ 137.165467] niks3-server[1114]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6615lb # [ 137.174195] niks3-server[1114]: 2026/09/20 12:22:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6616lb # [ 137.178731] niks3-server[1114]: 2026/09/20 12:22:13 INFO Signed narinfos id=52 count=16617worker1 # [ 137.160106] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.801Z level=INFO msg="Uploading 1 narinfos"6618lb # [ 137.189402] niks3-server[1114]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6619lb # [ 137.198103] niks3-server[1114]: 2026/09/20 12:22:13 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6620lb # [ 137.202828] niks3-server[1114]: 2026/09/20 12:22:13 INFO Completed upload id=526621worker1 # [ 137.183565] nix-grpc-daemon[1458]: time=2026-09-20T12:22:12.825Z level=INFO msg="Upload complete. (383ms)"6622lb # [ 137.207553] niks3-server[1114]: 2026/09/20 12:22:13 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6623worker1 # [ 137.187354] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=6246624worker1 # [ 137.193280] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906625worker1 # [ 137.202084] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6626worker1 # [ 137.210100] nix-daemon[911]: reaped child process 1731, status = succeeded6627worker1 # [ 137.213415] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6628worker1 # [ 138.039660] nix-daemon[911]: reaped child process 1745, status = succeeded6629worker1 # [ 138.201095] nix-daemon[911]: accepted connection from pid 1760, user nix-grpc-daemon (trusted)6630worker1 # [ 138.214830] nix-daemon[911]: reaped child process 1763, status = succeeded6631lb # [ 138.255162] niks3-server[1114]: 2026/09/20 12:22:14 INFO Received uploads request method=POST path=/api/pending_closures6632worker1 # [ 138.240737] nix-grpc-daemon[1458]: time=2026-09-20T12:22:13.882Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6633worker1 # [ 138.244229] nix-grpc-daemon[1458]: time=2026-09-20T12:22:13.884Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6634lb # [ 138.292184] niks3-server[1114]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6635lb # [ 138.304576] niks3-server[1114]: 2026/09/20 12:22:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6636lb # [ 138.309490] niks3-server[1114]: 2026/09/20 12:22:14 INFO Signed narinfos id=53 count=16637worker1 # [ 138.290472] nix-grpc-daemon[1458]: time=2026-09-20T12:22:13.931Z level=INFO msg="Uploading 1 narinfos"6638lb # [ 138.318737] niks3-server[1114]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6639lb # [ 138.326444] niks3-server[1114]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6640lb # [ 138.332669] niks3-server[1114]: 2026/09/20 12:22:14 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6641lb # [ 138.337647] niks3-server[1114]: 2026/09/20 12:22:14 INFO Completed upload id=536642worker1 # [ 138.315724] nix-grpc-daemon[1458]: time=2026-09-20T12:22:13.957Z level=INFO msg="Upload complete. (294ms)"6643lb # [ 138.348263] niks3-server[1114]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6644worker1 # [ 138.322765] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16645worker2 # [ 138.333568] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:14Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16646worker1 # [ 138.330460] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906647worker2 # [ 138.343293] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:14Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06648worker1 # [ 138.341587] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906649worker1 # [ 138.346716] nix-daemon[911]: reaped child process 1743, status = succeeded6650worker1 # [ 138.352578] nix-grpc-daemon[1458]: time=2026-09-20T12:22:13.994Z level=INFO msg="All 1 paths already cached"6651worker1 # [ 138.358477] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906652worker1 # [ 138.365162] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6653worker1 # [ 138.535259] nix-daemon[911]: accepted connection from pid 1767, user nix-grpc-daemon (trusted)6654worker1 # [ 138.550713] nix-daemon[911]: reaped child process 1770, status = succeeded6655worker1 # [ 138.717836] nix-daemon[911]: accepted connection from pid 1772, user nix-grpc-daemon (trusted)6656worker1 # [ 138.732284] nix-daemon[911]: reaped child process 1775, status = succeeded6657lb # [ 138.769920] niks3-server[1114]: 2026/09/20 12:22:14 INFO Received uploads request method=POST path=/api/pending_closures6658worker1 # [ 138.756080] nix-grpc-daemon[1458]: time=2026-09-20T12:22:14.397Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6659worker1 # [ 138.757584] nix-grpc-daemon[1458]: time=2026-09-20T12:22:14.397Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6660lb # [ 138.796416] niks3-server[1114]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6661lb # [ 138.805873] niks3-server[1114]: 2026/09/20 12:22:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6662lb # [ 138.810833] niks3-server[1114]: 2026/09/20 12:22:14 INFO Signed narinfos id=54 count=16663worker1 # [ 138.791918] nix-grpc-daemon[1458]: time=2026-09-20T12:22:14.433Z level=INFO msg="Uploading 1 narinfos"6664lb # [ 138.818647] niks3-server[1114]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6665lb # [ 138.827123] niks3-server[1114]: 2026/09/20 12:22:14 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6666lb # [ 138.833811] niks3-server[1114]: 2026/09/20 12:22:14 INFO Completed upload id=546667worker1 # [ 138.811997] nix-grpc-daemon[1458]: time=2026-09-20T12:22:14.453Z level=INFO msg="Upload complete. (440ms)"6668lb # [ 138.837275] niks3-server[1114]: 2026/09/20 12:22:14 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6669worker1 # [ 138.816093] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=6246670worker1 # [ 138.821666] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906671worker1 # [ 138.830605] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6672worker1 # [ 138.834643] nix-daemon[911]: reaped child process 1765, status = succeeded6673worker1 # [ 138.839716] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6674worker1 # [ 139.854156] nix-daemon[911]: reaped child process 1779, status = succeeded6675worker1 # [ 140.006417] nix-daemon[911]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6676worker1 # [ 140.019353] nix-daemon[911]: reaped child process 1797, status = succeeded6677lb # [ 140.054434] niks3-server[1114]: 2026/09/20 12:22:16 INFO Received uploads request method=POST path=/api/pending_closures6678worker1 # [ 140.041170] nix-grpc-daemon[1458]: time=2026-09-20T12:22:15.682Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6679worker1 # [ 140.042695] nix-grpc-daemon[1458]: time=2026-09-20T12:22:15.684Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6680lb # [ 140.102272] niks3-server[1114]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6681lb # [ 140.109097] niks3-server[1114]: 2026/09/20 12:22:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6682worker1 # [ 140.094063] nix-grpc-daemon[1458]: time=2026-09-20T12:22:15.735Z level=INFO msg="Uploading 1 narinfos"6683lb # [ 140.116654] niks3-server[1114]: 2026/09/20 12:22:16 INFO Signed narinfos id=55 count=16684lb # [ 140.119088] niks3-server[1114]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6685lb # [ 140.130550] niks3-server[1114]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6686lb # [ 140.135681] niks3-server[1114]: 2026/09/20 12:22:16 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6687lb # [ 140.138401] niks3-server[1114]: 2026/09/20 12:22:16 INFO Completed upload id=556688worker1 # [ 140.116434] nix-grpc-daemon[1458]: time=2026-09-20T12:22:15.758Z level=INFO msg="Upload complete. (269ms)"6689lb # [ 140.141232] niks3-server[1114]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6690worker1 # [ 140.119379] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16691worker2 # [ 140.131348] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:16Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16692worker1 # [ 140.123940] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906693worker2 # [ 140.136912] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:16Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06694worker1 # [ 140.129794] nix-daemon[911]: reaped child process 1777, status = succeeded6695worker1 # [ 140.131344] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906696worker1 # [ 140.137056] nix-grpc-daemon[1458]: time=2026-09-20T12:22:15.778Z level=INFO msg="All 1 paths already cached"6697worker1 # [ 140.140263] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906698worker1 # [ 140.143979] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6699worker1 # [ 140.295591] nix-daemon[911]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6700worker1 # [ 140.308457] nix-daemon[911]: reaped child process 1804, status = succeeded6701worker1 # [ 140.471875] nix-daemon[911]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6702worker1 # [ 140.485043] nix-daemon[911]: reaped child process 1809, status = succeeded6703lb # [ 140.520802] niks3-server[1114]: 2026/09/20 12:22:16 INFO Received uploads request method=POST path=/api/pending_closures6704worker1 # [ 140.508565] nix-grpc-daemon[1458]: time=2026-09-20T12:22:16.150Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6705worker1 # [ 140.510115] nix-grpc-daemon[1458]: time=2026-09-20T12:22:16.151Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6706lb # [ 140.554862] niks3-server[1114]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6707lb # [ 140.560948] niks3-server[1114]: 2026/09/20 12:22:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6708lb # [ 140.564575] niks3-server[1114]: 2026/09/20 12:22:16 INFO Signed narinfos id=56 count=16709worker1 # [ 140.544903] nix-grpc-daemon[1458]: time=2026-09-20T12:22:16.186Z level=INFO msg="Uploading 1 narinfos"6710lb # [ 140.575605] niks3-server[1114]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6711lb # [ 140.584074] niks3-server[1114]: 2026/09/20 12:22:16 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6712lb # [ 140.587579] niks3-server[1114]: 2026/09/20 12:22:16 INFO Completed upload id=566713worker1 # [ 140.565833] nix-grpc-daemon[1458]: time=2026-09-20T12:22:16.207Z level=INFO msg="Upload complete. (415ms)"6714lb # [ 140.589990] niks3-server[1114]: 2026/09/20 12:22:16 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6715worker1 # [ 140.569612] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=6246716worker1 # [ 140.575291] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906717worker1 # [ 140.586230] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6718worker1 # [ 140.588403] nix-daemon[911]: reaped child process 1799, status = succeeded6719worker1 # [ 140.593537] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6720worker1 # [ 141.419449] nix-daemon[911]: reaped child process 1813, status = succeeded6721worker1 # [ 141.565796] nix-daemon[911]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)6722worker1 # [ 141.576330] nix-daemon[911]: reaped child process 1831, status = succeeded6723lb # [ 141.614657] niks3-server[1114]: 2026/09/20 12:22:17 INFO Received uploads request method=POST path=/api/pending_closures6724worker1 # [ 141.597237] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.238Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6725worker1 # [ 141.598771] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.240Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6726lb # [ 141.636675] niks3-server[1114]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6727lb # [ 141.641228] niks3-server[1114]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6728lb # [ 141.643940] niks3-server[1114]: 2026/09/20 12:22:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6729lb # [ 141.645742] niks3-server[1114]: 2026/09/20 12:22:17 INFO Signed narinfos id=57 count=16730worker1 # [ 141.625114] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.266Z level=INFO msg="Uploading 1 narinfos"6731lb # [ 141.650168] niks3-server[1114]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6732lb # [ 141.655261] niks3-server[1114]: 2026/09/20 12:22:17 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6733lb # [ 141.660042] niks3-server[1114]: 2026/09/20 12:22:17 INFO Completed upload id=576734worker1 # [ 141.637914] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.279Z level=INFO msg="Upload complete. (237ms)"6735lb # [ 141.662981] niks3-server[1114]: 2026/09/20 12:22:17 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6736worker1 # [ 141.640345] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16737worker1 # [ 141.646850] nix-daemon[911]: reaped child process 1811, status = succeeded6738worker2 # [ 141.653715] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:17Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16739worker1 # [ 141.649214] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906740worker2 # [ 141.663867] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:17Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06741worker1 # [ 141.656555] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906742worker1 # [ 141.662244] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.303Z level=INFO msg="All 2 paths already cached"6743worker1 # [ 141.666185] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906744worker1 # [ 141.670935] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6745worker1 # [ 141.815535] nix-daemon[911]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)6746worker1 # [ 141.827970] nix-daemon[911]: reaped child process 1838, status = succeeded6747worker1 # [ 141.981551] nix-daemon[911]: accepted connection from pid 1840, user nix-grpc-daemon (trusted)6748worker1 # [ 141.993569] nix-daemon[911]: reaped child process 1843, status = succeeded6749lb # [ 142.031091] niks3-server[1114]: 2026/09/20 12:22:17 INFO Received uploads request method=POST path=/api/pending_closures6750worker1 # [ 142.017429] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.658Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6751worker1 # [ 142.018880] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.658Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6752lb # [ 142.055670] niks3-server[1114]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6753lb # [ 142.067715] niks3-server[1114]: 2026/09/20 12:22:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6754lb # [ 142.074671] niks3-server[1114]: 2026/09/20 12:22:18 INFO Signed narinfos id=58 count=16755worker1 # [ 142.055971] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.697Z level=INFO msg="Uploading 1 narinfos"6756lb # [ 142.079699] niks3-server[1114]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6757lb # [ 142.091717] niks3-server[1114]: 2026/09/20 12:22:18 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6758worker1 # [ 142.076984] nix-grpc-daemon[1458]: time=2026-09-20T12:22:17.718Z level=INFO msg="Upload complete. (401ms)"6759lb # [ 142.099089] niks3-server[1114]: 2026/09/20 12:22:18 INFO Completed upload id=586760worker1 # [ 142.079053] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=7526761lb # [ 142.102214] niks3-server[1114]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6762worker1 # [ 142.084446] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906763worker1 # [ 142.089436] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6764worker1 # [ 142.091774] nix-daemon[911]: reaped child process 1833, status = succeeded6765worker1 # [ 142.096989] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6766worker1 # [ 142.723808] nix-daemon[911]: reaped child process 1847, status = succeeded6767worker1 # [ 142.860343] nix-daemon[911]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6768worker1 # [ 142.872602] nix-daemon[911]: reaped child process 1865, status = succeeded6769lb # [ 142.905519] niks3-server[1114]: 2026/09/20 12:22:18 INFO Received uploads request method=POST path=/api/pending_closures6770worker1 # [ 142.894198] nix-grpc-daemon[1458]: time=2026-09-20T12:22:18.535Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6771worker1 # [ 142.895820] nix-grpc-daemon[1458]: time=2026-09-20T12:22:18.537Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6772lb # [ 142.942258] niks3-server[1114]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6773lb # [ 142.952139] niks3-server[1114]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6774lb # [ 142.961428] niks3-server[1114]: 2026/09/20 12:22:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6775lb # [ 142.966243] niks3-server[1114]: 2026/09/20 12:22:18 INFO Signed narinfos id=59 count=16776worker1 # [ 142.946884] nix-grpc-daemon[1458]: time=2026-09-20T12:22:18.588Z level=INFO msg="Uploading 1 narinfos"6777lb # [ 142.972053] niks3-server[1114]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6778lb # [ 142.980808] niks3-server[1114]: 2026/09/20 12:22:18 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6779lb # [ 142.986183] niks3-server[1114]: 2026/09/20 12:22:18 INFO Completed upload id=596780worker1 # [ 142.963678] nix-grpc-daemon[1458]: time=2026-09-20T12:22:18.605Z level=INFO msg="Upload complete. (259ms)"6781lb # [ 142.989199] niks3-server[1114]: 2026/09/20 12:22:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6782worker1 # [ 142.970192] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16783worker2 # [ 142.982830] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:18Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16784worker1 # [ 142.983076] nix-daemon[911]: reaped child process 1845, status = succeeded6785worker1 # [ 143.020325] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906786worker1 # [ 143.032163] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906787worker1 # [ 143.043804] nix-grpc-daemon[1458]: time=2026-09-20T12:22:18.685Z level=INFO msg="All 2 paths already cached"6788worker1 # [ 143.049242] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906789worker1 # [ 143.058341] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6790worker1 # [ 143.216525] nix-daemon[911]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)6791worker1 # [ 143.230098] nix-daemon[911]: reaped child process 1872, status = succeeded6792worker1 # [ 143.393323] nix-daemon[911]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6793worker1 # [ 143.406519] nix-daemon[911]: reaped child process 1877, status = succeeded6794lb # [ 143.441941] niks3-server[1114]: 2026/09/20 12:22:19 INFO Received uploads request method=POST path=/api/pending_closures6795worker1 # [ 143.424358] nix-grpc-daemon[1458]: time=2026-09-20T12:22:19.066Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6796worker1 # [ 143.425977] nix-grpc-daemon[1458]: time=2026-09-20T12:22:19.067Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6797lb # [ 143.461066] niks3-server[1114]: 2026/09/20 12:22:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6798lb # [ 143.463270] niks3-server[1114]: 2026/09/20 12:22:19 INFO Signed narinfos id=60 count=16799worker1 # [ 143.442473] nix-grpc-daemon[1458]: time=2026-09-20T12:22:19.084Z level=INFO msg="Uploading 1 narinfos"6800lb # [ 143.469838] niks3-server[1114]: 2026/09/20 12:22:19 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6801lb # [ 143.475527] niks3-server[1114]: 2026/09/20 12:22:19 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6802lb # [ 143.479118] niks3-server[1114]: 2026/09/20 12:22:19 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6803lb # [ 143.482919] niks3-server[1114]: 2026/09/20 12:22:19 INFO Completed upload id=606804worker1 # [ 143.462309] nix-grpc-daemon[1458]: time=2026-09-20T12:22:19.103Z level=INFO msg="Upload complete. (392ms)"6805lb # [ 143.484978] niks3-server[1114]: 2026/09/20 12:22:19 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6806worker1 # [ 143.464360] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=7526807worker1 # [ 143.468098] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906808worker1 # [ 143.474633] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6809worker1 # [ 143.477992] nix-daemon[911]: reaped child process 1867, status = succeeded6810worker1 # [ 143.483316] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6811worker1 # [ 144.710769] nix-daemon[911]: reaped child process 1881, status = succeeded6812worker1 # [ 144.840553] nix-daemon[911]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)6813worker1 # [ 144.853506] nix-daemon[911]: reaped child process 1899, status = succeeded6814lb # [ 144.884379] niks3-server[1114]: 2026/09/20 12:22:20 INFO Received uploads request method=POST path=/api/pending_closures6815worker1 # [ 144.866756] nix-grpc-daemon[1458]: time=2026-09-20T12:22:20.508Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6816worker1 # [ 144.868353] nix-grpc-daemon[1458]: time=2026-09-20T12:22:20.509Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6817lb # [ 144.903289] niks3-server[1114]: 2026/09/20 12:22:20 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6818lb # [ 144.906625] niks3-server[1114]: 2026/09/20 12:22:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6819lb # [ 144.909136] niks3-server[1114]: 2026/09/20 12:22:20 INFO Signed narinfos id=61 count=16820worker1 # [ 144.888658] nix-grpc-daemon[1458]: time=2026-09-20T12:22:20.530Z level=INFO msg="Uploading 1 narinfos"6821lb # [ 144.912483] niks3-server[1114]: 2026/09/20 12:22:20 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6822lb # [ 144.915225] niks3-server[1114]: 2026/09/20 12:22:20 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6823lb # [ 144.919467] niks3-server[1114]: 2026/09/20 12:22:20 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6824lb # [ 144.922373] niks3-server[1114]: 2026/09/20 12:22:20 INFO Completed upload id=616825worker1 # [ 144.901495] nix-grpc-daemon[1458]: time=2026-09-20T12:22:20.543Z level=INFO msg="Upload complete. (210ms)"6826lb # [ 144.924036] niks3-server[1114]: 2026/09/20 12:22:20 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6827worker1 # [ 144.904696] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16828worker2 # [ 144.917318] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:20Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16829worker1 # [ 144.912899] nix-daemon[911]: reaped child process 1879, status = succeeded6830worker1 # [ 144.949204] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906831worker1 # [ 144.957409] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:496906832worker1 # [ 144.967096] nix-grpc-daemon[1458]: time=2026-09-20T12:22:20.608Z level=INFO msg="All 2 paths already cached"6833worker1 # [ 144.971754] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:496906834worker1 # [ 144.977870] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6835worker1 # [ 145.134231] nix-daemon[911]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)6836worker1 # [ 145.147366] nix-daemon[911]: reaped child process 1906, status = succeeded6837worker1 # [ 145.317639] nix-daemon[911]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6838worker1 # [ 145.330174] nix-daemon[911]: reaped child process 1911, status = succeeded6839lb # [ 145.367806] niks3-server[1114]: 2026/09/20 12:22:21 INFO Received uploads request method=POST path=/api/pending_closures6840worker1 # [ 145.353700] nix-grpc-daemon[1458]: time=2026-09-20T12:22:20.995Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6841worker1 # [ 145.355641] nix-grpc-daemon[1458]: time=2026-09-20T12:22:20.996Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6842lb # [ 145.396694] niks3-server[1114]: 2026/09/20 12:22:21 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6843lb # [ 145.409507] niks3-server[1114]: 2026/09/20 12:22:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6844lb # [ 145.415913] niks3-server[1114]: 2026/09/20 12:22:21 INFO Signed narinfos id=62 count=16845worker1 # [ 145.396846] nix-grpc-daemon[1458]: time=2026-09-20T12:22:21.038Z level=INFO msg="Uploading 1 narinfos"6846lb # [ 145.420911] niks3-server[1114]: 2026/09/20 12:22:21 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6847lb # [ 145.429941] niks3-server[1114]: 2026/09/20 12:22:21 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6848lb # [ 145.438082] niks3-server[1114]: 2026/09/20 12:22:21 INFO Completed upload id=626849worker1 # [ 145.415853] nix-grpc-daemon[1458]: time=2026-09-20T12:22:21.057Z level=INFO msg="Upload complete. (432ms)"6850lb # [ 145.440734] niks3-server[1114]: 2026/09/20 12:22:21 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6851worker1 # [ 145.419262] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=0 paths=1 nar_bytes_in=7526852worker1 # [ 145.424975] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:496906853worker1 # [ 145.433485] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6854worker1 # [ 145.438121] nix-daemon[911]: reaped child process 1901, status = succeeded6855worker1 # [ 145.443090] nix-daemon[911]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)6856worker1 # [ 146.470400] nix-daemon[911]: reaped child process 1915, status = succeeded6857worker1 # [ 146.624977] nix-daemon[911]: accepted connection from pid 1930, user nix-grpc-daemon (trusted)6858worker1 # [ 146.637384] nix-daemon[911]: reaped child process 1933, status = succeeded6859lb # [ 146.675506] niks3-server[1114]: 2026/09/20 12:22:22 INFO Received uploads request method=POST path=/api/pending_closures6860worker1 # [ 146.662275] nix-grpc-daemon[1458]: time=2026-09-20T12:22:22.303Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6861worker1 # [ 146.663772] nix-grpc-daemon[1458]: time=2026-09-20T12:22:22.305Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6862lb # [ 146.716081] niks3-server[1114]: 2026/09/20 12:22:22 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6863lb # [ 146.722771] niks3-server[1114]: 2026/09/20 12:22:22 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6864lb # [ 146.734720] niks3-server[1114]: 2026/09/20 12:22:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6865lb # [ 146.739461] niks3-server[1114]: 2026/09/20 12:22:22 INFO Signed narinfos id=63 count=16866worker1 # [ 146.720475] nix-grpc-daemon[1458]: time=2026-09-20T12:22:22.361Z level=INFO msg="Uploading 1 narinfos"6867lb # [ 146.742222] niks3-server[1114]: 2026/09/20 12:22:22 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6868lb # [ 146.752093] niks3-server[1114]: 2026/09/20 12:22:22 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6869lb # [ 146.757727] niks3-server[1114]: 2026/09/20 12:22:22 INFO Completed upload id=636870lb # [ 146.759382] niks3-server[1114]: 2026/09/20 12:22:22 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6871worker1 # [ 146.737197] nix-grpc-daemon[1458]: time=2026-09-20T12:22:22.378Z level=INFO msg="Upload complete. (285ms)"6872worker1 # [ 146.744222] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49690 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16873worker1 # [ 146.759666] nix-daemon[911]: reaped child process 1913, status = succeeded6874client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.73 seconds)6875worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6876worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6877client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16878worker2 # [ 147.018066] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594526879worker2 # [ 147.024198] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594526880worker2 # [ 147.038166] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:23Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06881lb # [ 148.942789] envoy[978]: conn peer=[2001:db8:1::1]:58704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27003 tx=10681 ms=165346882client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.22 seconds)6883(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.02 seconds)6884subtest: low disk drains a worker and builds go to the other6885worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6886worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)6887worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6888worker1 # [ 178.569969] nix-grpc-daemon[1370]: ts=2026-09-20T12:22:54Z level=info event=unhealthy reason=min_free6889lb # [ 178.943134] envoy[978]: conn peer=[2001:db8:1::1]:58294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300406890worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6891client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26892client # don't know how to build these paths:6893worker2 # [ 179.410204] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594526894client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6895worker2 # [ 179.413663] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594526896worker2 # [ 179.430241] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:55Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06897worker2 # [ 179.435760] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:55Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16898worker2 # [ 179.475276] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594526899worker2 # [ 179.488237] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594526900client # copying 1 paths to 'lb:50051'...6901client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6902worker2 # [ 179.500715] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594526903worker2 # [ 179.509890] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6904worker2 # [ 179.531115] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:55Z level=info event=niks3_push_started pid=20196905worker2 # [ 179.725389] nix-grpc-daemon[2019]: time=2026-09-20T12:22:55.698Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6906worker2 # [ 179.885096] nix-daemon[926]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)6907worker2 # [ 179.895662] nix-daemon[926]: reaped child process 2035, status = succeeded6908worker2 # [ 180.033604] nix-daemon[926]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)6909worker2 # [ 180.045714] nix-daemon[926]: reaped child process 2040, status = succeeded6910lb # [ 180.067073] niks3-server[1114]: 2026/09/20 12:22:56 INFO Received uploads request method=POST path=/api/pending_closures6911worker2 # [ 180.065614] nix-grpc-daemon[2019]: time=2026-09-20T12:22:56.039Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6912worker2 # [ 180.068733] nix-grpc-daemon[2019]: time=2026-09-20T12:22:56.040Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6913lb # [ 180.103081] niks3-server[1114]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6914lb # [ 180.113085] niks3-server[1114]: 2026/09/20 12:22:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6915lb # [ 180.118378] niks3-server[1114]: 2026/09/20 12:22:56 INFO Signed narinfos id=64 count=16916worker2 # [ 180.110687] nix-grpc-daemon[2019]: time=2026-09-20T12:22:56.083Z level=INFO msg="Uploading 1 narinfos"6917lb # [ 180.132807] niks3-server[1114]: 2026/09/20 12:22:56 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6918lb # [ 180.142355] niks3-server[1114]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6919lb # [ 180.147478] niks3-server[1114]: 2026/09/20 12:22:56 INFO Completed upload id=646920worker2 # [ 180.138604] nix-grpc-daemon[2019]: time=2026-09-20T12:22:56.112Z level=INFO msg="Upload complete. (409ms)"6921lb # [ 180.149824] niks3-server[1114]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6922worker2 # [ 180.140835] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59452 duration_s=0 paths=1 nar_bytes_in=5126923worker2 # [ 180.143811] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594526924worker2 # [ 180.150042] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6925worker2 # [ 180.153406] nix-daemon[926]: reaped child process 2017, status = succeeded6926worker2 # [ 180.158308] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6927client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6928client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6929client # farm-a> LOG-farm-a-drain6930worker2 # [ 180.771545] nix-daemon[926]: reaped child process 2044, status = succeeded6931worker2 # [ 180.916397] nix-daemon[926]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)6932worker2 # [ 180.928597] nix-daemon[926]: reaped child process 2062, status = succeeded6933lb # [ 180.948872] niks3-server[1114]: 2026/09/20 12:22:56 INFO Received uploads request method=POST path=/api/pending_closures6934worker2 # [ 180.942072] nix-grpc-daemon[2019]: time=2026-09-20T12:22:56.915Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6935worker2 # [ 180.943513] nix-grpc-daemon[2019]: time=2026-09-20T12:22:56.917Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6936lb # [ 180.968828] niks3-server[1114]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6937lb # [ 180.971080] niks3-server[1114]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6938lb # [ 180.975864] niks3-server[1114]: 2026/09/20 12:22:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6939lb # [ 180.978199] niks3-server[1114]: 2026/09/20 12:22:56 INFO Signed narinfos id=65 count=16940worker2 # [ 180.967978] nix-grpc-daemon[2019]: time=2026-09-20T12:22:56.941Z level=INFO msg="Uploading 1 narinfos"6941lb # [ 180.982140] niks3-server[1114]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6942lb # [ 180.986184] niks3-server[1114]: 2026/09/20 12:22:56 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6943lb # [ 180.989952] niks3-server[1114]: 2026/09/20 12:22:56 INFO Completed upload id=656944worker2 # [ 180.979086] nix-grpc-daemon[2019]: time=2026-09-20T12:22:56.952Z level=INFO msg="Upload complete. (215ms)"6945lb # [ 180.992371] niks3-server[1114]: 2026/09/20 12:22:56 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6946worker2 # [ 180.981371] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59452 duration_s=0 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16947worker2 # [ 180.987558] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:56Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06948worker2 # [ 180.989551] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:56Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16949worker2 # [ 180.994104] nix-daemon[926]: reaped child process 2042, status = succeeded6950worker2 # [ 181.033880] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594526951worker2 # [ 181.043148] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594526952worker2 # [ 181.080008] nix-grpc-daemon[2019]: time=2026-09-20T12:22:57.053Z level=INFO msg="All 1 paths already cached"6953client # copying 1 paths to 'lb:50051'...6954client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6955worker2 # [ 181.089245] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594526956worker2 # [ 181.097597] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6957worker2 # [ 181.274102] nix-daemon[926]: accepted connection from pid 2068, user nix-grpc-daemon (trusted)6958worker2 # [ 181.285993] nix-daemon[926]: reaped child process 2071, status = succeeded6959worker2 # [ 181.440254] nix-daemon[926]: accepted connection from pid 2073, user nix-grpc-daemon (trusted)6960worker2 # [ 181.452383] nix-daemon[926]: reaped child process 2076, status = succeeded6961lb # [ 181.479065] niks3-server[1114]: 2026/09/20 12:22:57 INFO Received uploads request method=POST path=/api/pending_closures6962worker2 # [ 181.518668] nix-grpc-daemon[2019]: time=2026-09-20T12:22:57.492Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6963worker2 # [ 181.520795] nix-grpc-daemon[2019]: time=2026-09-20T12:22:57.493Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6964lb # [ 181.541080] niks3-server[1114]: 2026/09/20 12:22:57 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6965lb # [ 181.544095] niks3-server[1114]: 2026/09/20 12:22:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6966lb # [ 181.545863] niks3-server[1114]: 2026/09/20 12:22:57 INFO Signed narinfos id=66 count=16967worker2 # [ 181.536889] nix-grpc-daemon[2019]: time=2026-09-20T12:22:57.510Z level=INFO msg="Uploading 1 narinfos"6968lb # [ 181.550505] niks3-server[1114]: 2026/09/20 12:22:57 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6969lb # [ 181.556236] niks3-server[1114]: 2026/09/20 12:22:57 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6970lb # [ 181.560484] niks3-server[1114]: 2026/09/20 12:22:57 INFO Completed upload id=666971worker2 # [ 181.549880] nix-grpc-daemon[2019]: time=2026-09-20T12:22:57.523Z level=INFO msg="Upload complete. (438ms)"6972lb # [ 181.563358] niks3-server[1114]: 2026/09/20 12:22:57 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6973worker2 # [ 181.553849] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59452 duration_s=0 paths=1 nar_bytes_in=6246974worker2 # [ 181.559443] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594526975worker2 # [ 181.570118] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6976worker2 # [ 181.573941] nix-daemon[926]: reaped child process 2066, status = succeeded6977worker2 # [ 181.579119] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6978client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6979client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6980worker2 # [ 182.392034] nix-daemon[926]: reaped child process 2080, status = succeeded6981client # farm-b> LOG-farm-b-drain6982worker2 # [ 182.515513] nix-daemon[926]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)6983worker2 # [ 182.527446] nix-daemon[926]: reaped child process 2098, status = succeeded6984lb # [ 182.552607] niks3-server[1114]: 2026/09/20 12:22:58 INFO Received uploads request method=POST path=/api/pending_closures6985worker2 # [ 182.550672] nix-grpc-daemon[2019]: time=2026-09-20T12:22:58.523Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6986worker2 # [ 182.552432] nix-grpc-daemon[2019]: time=2026-09-20T12:22:58.525Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6987lb # [ 182.593820] niks3-server[1114]: 2026/09/20 12:22:58 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6988lb # [ 182.599243] niks3-server[1114]: 2026/09/20 12:22:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6989worker2 # [ 182.593986] nix-grpc-daemon[2019]: time=2026-09-20T12:22:58.567Z level=INFO msg="Uploading 1 narinfos"6990lb # [ 182.604989] niks3-server[1114]: 2026/09/20 12:22:58 INFO Signed narinfos id=67 count=16991lb # [ 182.612434] niks3-server[1114]: 2026/09/20 12:22:58 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6992lb # [ 182.616543] niks3-server[1114]: 2026/09/20 12:22:58 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6993lb # [ 182.625686] niks3-server[1114]: 2026/09/20 12:22:58 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6994lb # [ 182.628517] niks3-server[1114]: 2026/09/20 12:22:58 INFO Completed upload id=676995worker2 # [ 182.617928] nix-grpc-daemon[2019]: time=2026-09-20T12:22:58.591Z level=INFO msg="Upload complete. (232ms)"6996lb # [ 182.630692] niks3-server[1114]: 2026/09/20 12:22:58 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6997worker2 # [ 182.627904] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59452 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16998worker2 # [ 182.634213] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:58Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06999worker2 # [ 182.637395] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:58Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17000worker2 # [ 182.645438] nix-daemon[926]: reaped child process 2078, status = succeeded7001worker2 # [ 182.671548] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594527002worker2 # [ 182.683414] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594527003worker2 # [ 182.696359] nix-grpc-daemon[2019]: time=2026-09-20T12:22:58.669Z level=INFO msg="All 2 paths already cached"7004client # copying 1 paths to 'lb:50051'...7005client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7006worker2 # [ 182.703445] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594527007worker2 # [ 182.712306] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)7008worker2 # [ 182.873483] nix-daemon[926]: accepted connection from pid 2104, user nix-grpc-daemon (trusted)7009worker2 # [ 182.886071] nix-daemon[926]: reaped child process 2107, status = succeeded7010worker2 # [ 183.015271] nix-daemon[926]: accepted connection from pid 2109, user nix-grpc-daemon (trusted)7011worker2 # [ 183.027701] nix-daemon[926]: reaped child process 2112, status = succeeded7012lb # [ 183.047156] niks3-server[1114]: 2026/09/20 12:22:58 INFO Received uploads request method=POST path=/api/pending_closures7013worker2 # [ 183.245617] nix-grpc-daemon[2019]: time=2026-09-20T12:22:59.219Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7014worker2 # [ 183.247236] nix-grpc-daemon[2019]: time=2026-09-20T12:22:59.220Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7015lb # [ 183.282670] niks3-server[1114]: 2026/09/20 12:22:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7016worker2 # [ 183.277294] nix-grpc-daemon[2019]: time=2026-09-20T12:22:59.250Z level=INFO msg="Uploading 1 narinfos"7017lb # [ 183.288125] niks3-server[1114]: 2026/09/20 12:22:59 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7018lb # [ 183.292288] niks3-server[1114]: 2026/09/20 12:22:59 INFO Signed narinfos id=68 count=17019lb # [ 183.305404] niks3-server[1114]: 2026/09/20 12:22:59 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7020lb # [ 183.320875] niks3-server[1114]: 2026/09/20 12:22:59 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7021lb # [ 183.340160] niks3-server[1114]: 2026/09/20 12:22:59 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7022lb # [ 183.356422] niks3-server[1114]: 2026/09/20 12:22:59 INFO Completed upload id=687023worker2 # [ 183.348581] nix-grpc-daemon[2019]: time=2026-09-20T12:22:59.322Z level=INFO msg="Upload complete. (623ms)"7024worker2 # [ 183.350776] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59452 duration_s=0 paths=1 nar_bytes_in=7527025worker2 # [ 183.353861] nix-grpc-daemon[1942]: ts=2026-09-20T12:22:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594527026worker2 # [ 183.359974] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)7027worker2 # [ 183.363026] nix-daemon[926]: reaped child process 2102, status = succeeded7028worker2 # [ 183.367742] nix-daemon[926]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)7029client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7030client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7031client # farm-top> LOG-farm-top-drain7032worker2 # [ 184.186540] nix-daemon[926]: reaped child process 2116, status = succeeded7033worker2 # [ 184.323777] nix-daemon[926]: accepted connection from pid 2131, user nix-grpc-daemon (trusted)7034worker2 # [ 184.334576] nix-daemon[926]: reaped child process 2134, status = succeeded7035lb # [ 184.361251] niks3-server[1114]: 2026/09/20 12:23:00 INFO Received uploads request method=POST path=/api/pending_closures7036worker2 # [ 184.358959] nix-grpc-daemon[2019]: time=2026-09-20T12:23:00.332Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7037worker2 # [ 184.360458] nix-grpc-daemon[2019]: time=2026-09-20T12:23:00.333Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7038lb # [ 184.401595] niks3-server[1114]: 2026/09/20 12:23:00 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7039lb # [ 184.409820] niks3-server[1114]: 2026/09/20 12:23:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7040lb # [ 184.415072] niks3-server[1114]: 2026/09/20 12:23:00 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7041lb # [ 184.421878] niks3-server[1114]: 2026/09/20 12:23:00 INFO Signed narinfos id=69 count=17042worker2 # [ 184.414023] nix-grpc-daemon[2019]: time=2026-09-20T12:23:00.387Z level=INFO msg="Uploading 1 narinfos"7043lb # [ 184.431680] niks3-server[1114]: 2026/09/20 12:23:00 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7044lb # [ 184.440935] niks3-server[1114]: 2026/09/20 12:23:00 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7045lb # [ 184.445296] niks3-server[1114]: 2026/09/20 12:23:00 INFO Completed upload id=697046lb # [ 184.447935] niks3-server[1114]: 2026/09/20 12:23:00 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7047worker2 # [ 184.435980] nix-grpc-daemon[2019]: time=2026-09-20T12:23:00.409Z level=INFO msg="Upload complete. (262ms)"7048worker2 # [ 184.444162] nix-grpc-daemon[1942]: ts=2026-09-20T12:23:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59452 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17049worker2 # [ 184.456196] nix-daemon[926]: reaped child process 2114, status = succeeded7050client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.28 seconds)7051client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7052client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.17 seconds)7053worker1: must succeed: rm /nix/.rw-store/fill7054worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)7055worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7056worker1 # [ 185.573190] nix-grpc-daemon[1370]: ts=2026-09-20T12:23:01Z level=info event=healthy reason=min_free7057worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)7058(finished: subtest: low disk drains a worker and builds go to the other, in 8.89 seconds)7059(finished: run the VM test script, in 186.98 seconds)7060test script finished in 187.20s7061cleanup7062kill QemuMachine (pid 48)7063client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7064kill QemuMachine (pid 49)7065lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7066kill QemuMachine (pid 51)7067worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7068kill QemuMachine (pid 52)7069worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7070(finished: cleanup, in 0.96 seconds)7071additionally exposed symbols:7072 client, lb, worker1, worker2,7073 vlan1,7074 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh