vm-test-run-nix-grpc-farm log

succeeded `vm-test-run-nix-grpc-farm` `checks.x86_64-linux.farm` · build #279 · raw

1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2: QEMU running (pid 52)13worker2 # Disk image does not exist, creating the virtualisation disk image...14lb # Disk image does not exist, creating the virtualisation disk image...15lb: QEMU running (pid 50)16lb # Formatting '/build/vm-state-lb/tmp.afcAUUKf2o', fmt=raw size=107374182417worker2 # Formatting '/build/vm-state-worker2/tmp.0oFg7hHyQP', fmt=raw size=107374182418worker1: QEMU running (pid 51)19worker1 # Disk image does not exist, creating the virtualisation disk image...20lb # mke2fs 1.47.4 (6-Mar-2025)21worker1 # Formatting '/build/vm-state-worker1/tmp.YGRUXA8PF7', fmt=raw size=107374182422lb # Discarding device blocks: 0/262144 done23worker1 # mke2fs 1.47.4 (6-Mar-2025)24lb # Creating filesystem with 262144 4k blocks and 65536 inodes25worker1 # Discarding device blocks: 0/262144 done26client: QEMU running (pid 49)27client # Disk image does not exist, creating the virtualisation disk image...28worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes29worker2 # mke2fs 1.47.4 (6-Mar-2025)30lb # Filesystem UUID: 74b24c0e-13bd-4711-9292-f51d749a698631(finished: start all VMs, in 0.23 seconds)32worker2 # Discarding device blocks: 0/262144 done33lb # Superblock backups stored on blocks:34client: waiting for unit network-addresses-eth1.service35worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes36worker2 # Filesystem UUID: 8f0a180d-5563-4b51-89db-1d49d7da906e37lb # 32768, 98304, 163840, 22937638client # Formatting '/build/vm-state-client/tmp.4s7zFE6Tlf', fmt=raw size=107374182439worker1 # Filesystem UUID: 5fcd8562-44ec-4660-b948-0cacf3b3534640lb # 41client # mke2fs 1.47.4 (6-Mar-2025)42worker1 # Superblock backups stored on blocks:43lb # Allocating group tables: 0/8 done44client # Discarding device blocks: 0/262144 done45worker1 # 32768, 98304, 163840, 22937646lb # Writing inode tables: 0/8 done47client # Creating filesystem with 262144 4k blocks and 65536 inodes48worker1 # 49lb # Creating journal (8192 blocks): done50client # Filesystem UUID: 1b916dcb-c3ac-49b8-9fa9-59941b94ab4251worker1 # Allocating group tables: 0/8 done52lb # Writing superblocks and filesystem accounting information: 0/8 done53client # Superblock backups stored on blocks:54worker1 # Writing inode tables: 0/8 done55lb # 56client # 32768, 98304, 163840, 22937657worker1 # Creating journal (8192 blocks): done58lb # Virtualisation disk image created.59client # 60worker1 # Writing superblocks and filesystem accounting information: 0/8 done61client: waiting for the VM to finish booting62worker1 # 63worker2 # Superblock backups stored on blocks:64worker1 # Virtualisation disk image created.65worker2 # 32768, 98304, 163840, 22937666client # Allocating group tables: 0/8 done67worker2 # 68client # Writing inode tables: 0/8 done69worker2 # Allocating group tables: 0/8 done70client # Creating journal (8192 blocks): done71worker2 # Writing inode tables: 0/8 done72client # Writing superblocks and filesystem accounting information: 0/8 done73worker2 # Creating journal (8192 blocks): done74client # 75worker2 # Writing superblocks and filesystem accounting information: 0/8 done76client # Virtualisation disk image created.77worker2 # 78worker2 # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker1 # 84worker1 # 85worker2 # 86worker2 # 87worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0088worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90worker1 # 91worker1 # 92worker1 # 93worker1 # 94worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...95worker2 # 96worker2 # 97worker2 # 98worker2 # 99worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00100client # 101client # 102worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00103client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00104worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105worker1 # 106worker1 # 107client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108client # 109client # 110client # 111client # 112worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...113worker2 # 114worker2 # 115client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117client # 118client # 119lb # 120lb # 121lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123lb # 124lb # 125lb # 126lb # 127lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129lb # 130lb # 131worker1 # Booting from ROM...132worker2 # Booting from ROM...133client # Booting from ROM...134lb # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hvv0rrwgpqzhm46a4280mwzmbapcmqq9-nixos-system-worker1-test/init regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration console=ttyS0,115200n8 console=tty0137worker1 # [ 0.000000] BIOS-provided physical RAM map:138worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026141worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable143worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved144worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved145worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5qgax5h37ybfdjp7r7k01yx9abl6wij-nixos-system-worker2-test/init regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration console=ttyS0,115200n8 console=tty0146worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved147worker2 # [ 0.000000] BIOS-provided physical RAM map:148worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable149worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved150client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026151worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved152worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved153worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved154worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved155worker1 # [ 0.000000] NX (Execute Disable) protection: active156worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable157worker1 # [ 0.000000] APIC: Static calls initialized158worker1 # [ 0.000000] SMBIOS 2.8 present.159worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved160worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved161client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test/init regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration console=ttyS0,115200n8 console=tty0162worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014163client # [ 0.000000] BIOS-provided physical RAM map:164worker1 # [ 0.000000] DMI: Memory slots populated: 1/1165worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved166worker1 # [ 0.000000] Hypervisor detected: KVM167client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable168worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved169worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000170client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved171worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved172worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00173client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved174worker1 # [ 0.000000] kvm-clock: using sched offset of 480149910 cycles175worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved176client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable177worker2 # [ 0.000000] NX (Execute Disable) protection: active178worker2 # [ 0.000000] APIC: Static calls initialized179worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns180client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved181worker2 # [ 0.000000] SMBIOS 2.8 present.182worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor183client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved184worker1 # [ 0.000817] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000185worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014186client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved187worker2 # [ 0.000000] DMI: Memory slots populated: 1/1188worker1 # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs189worker2 # [ 0.000000] Hypervisor detected: KVM190client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved191worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000192worker1 # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT193client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved194worker1 # [ 0.002747] found SMP MP-table at [mem 0x000f5450-0x000f545f]195worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00196worker1 # [ 0.002758] Using GB pages for direct mapping197worker2 # [ 0.000000] kvm-clock: using sched offset of 487437573 cycles198client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved199lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026200worker1 # [ 0.002846] RAMDISK: [mem 0x5e367000-0x5ffcffff]201client # [ 0.000000] NX (Execute Disable) protection: active202client # [ 0.000000] APIC: Static calls initialized203worker1 # [ 0.002854] ACPI: Early table checksum verification disabled204worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns205client # [ 0.000000] SMBIOS 2.8 present.206worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor207worker1 # [ 0.002856] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )208worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000209client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014210worker1 # [ 0.002860] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)211client # [ 0.000000] DMI: Memory slots populated: 1/1212worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs213client # [ 0.000000] Hypervisor detected: KVM214worker1 # [ 0.002864] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)215lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r7i2qny6zxqjgksiw8bnyjrpmwmq2adf-nixos-system-lb-test/init regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration console=ttyS0,115200n8 console=tty0216client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000217worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT218lb # [ 0.000000] BIOS-provided physical RAM map:219worker2 # [ 0.002748] found SMP MP-table at [mem 0x000f5450-0x000f545f]220client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00221worker1 # [ 0.002871] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)222lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable223worker2 # [ 0.002760] Using GB pages for direct mapping224client # [ 0.000001] kvm-clock: using sched offset of 457097955 cycles225worker1 # [ 0.002873] ACPI: FACS 0x000000005FFE0000 000040226worker2 # [ 0.002835] RAMDISK: [mem 0x5e367000-0x5ffcffff]227lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved228worker1 # [ 0.002874] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)229worker2 # [ 0.002844] ACPI: Early table checksum verification disabled230client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns231lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved232worker2 # [ 0.002847] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )233client # [ 0.000005] tsc: Detected 2400.012 MHz processor234worker1 # [ 0.002876] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)235lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable236client # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000237worker2 # [ 0.002851] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238worker1 # [ 0.002878] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)239lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved240client # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs241worker2 # [ 0.002856] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)242lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved243worker1 # [ 0.002879] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)244client # [ 0.000835] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT245worker2 # [ 0.002862] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)246lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved247client # [ 0.002725] found SMP MP-table at [mem 0x000f5450-0x000f545f]248worker1 # [ 0.002880] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]249worker2 # [ 0.002865] ACPI: FACS 0x000000005FFE0000 000040250client # [ 0.002736] Using GB pages for direct mapping251lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved252worker1 # [ 0.002881] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]253client # [ 0.002861] RAMDISK: [mem 0x5e366000-0x5ffcffff]254worker2 # [ 0.002866] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)255lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved256worker1 # [ 0.002882] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]257client # [ 0.002871] ACPI: Early table checksum verification disabled258lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved259worker2 # [ 0.002868] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)260worker1 # [ 0.002882] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]261client # [ 0.002874] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )262lb # [ 0.000000] NX (Execute Disable) protection: active263worker1 # [ 0.002883] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]264worker2 # [ 0.002869] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)265lb # [ 0.000000] APIC: Static calls initialized266client # [ 0.002878] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)267lb # [ 0.000000] SMBIOS 2.8 present.268worker1 # [ 0.002883] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]269client # [ 0.002882] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)270worker2 # [ 0.002871] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)271worker1 # [ 0.002884] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]272lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014273worker2 # [ 0.002872] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]274client # [ 0.002889] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)275worker1 # [ 0.003112] No NUMA configuration found276lb # [ 0.000000] DMI: Memory slots populated: 1/1277client # [ 0.002891] ACPI: FACS 0x000000005FFE0000 000040278worker2 # [ 0.002873] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]279lb # [ 0.000000] Hypervisor detected: KVM280worker1 # [ 0.003113] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]281lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000282worker2 # [ 0.002874] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]283client # [ 0.002892] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)284worker1 # [ 0.003116] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]285worker1 # [ 0.005557] Zone ranges:286lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00287worker2 # [ 0.002874] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]288client # [ 0.002894] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)289worker1 # [ 0.005558] DMA [mem 0x0000000000001000-0x0000000000ffffff]290lb # [ 0.000000] kvm-clock: using sched offset of 489455881 cycles291worker2 # [ 0.002875] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]292worker1 # [ 0.005560] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]293client # [ 0.002896] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)294worker1 # [ 0.005562] Normal empty295lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns296worker2 # [ 0.002875] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]297worker1 # [ 0.005562] Device empty298client # [ 0.002897] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)299worker1 # [ 0.005563] Movable zone start for each node300lb # [ 0.000005] tsc: Detected 2400.012 MHz processor301worker2 # [ 0.002875] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]302worker1 # [ 0.005564] Early memory node ranges303lb # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000304worker2 # [ 0.003100] No NUMA configuration found305client # [ 0.002898] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]306worker1 # [ 0.005564] node 0: [mem 0x0000000000001000-0x000000000009efff]307worker2 # [ 0.003100] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]308client # [ 0.002899] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]309lb # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs310worker1 # [ 0.005566] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]311worker2 # [ 0.003103] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]312worker2 # [ 0.005371] Zone ranges:313client # [ 0.002900] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]314lb # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT315worker1 # [ 0.005567] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]316lb # [ 0.002747] found SMP MP-table at [mem 0x000f5450-0x000f545f]317worker2 # [ 0.005372] DMA [mem 0x0000000000001000-0x0000000000ffffff]318client # [ 0.002901] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]319worker1 # [ 0.005587] On node 0, zone DMA: 1 pages in unavailable ranges320lb # [ 0.002758] Using GB pages for direct mapping321worker2 # [ 0.005373] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]322client # [ 0.002901] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]323worker1 # [ 0.005849] On node 0, zone DMA: 97 pages in unavailable ranges324worker2 # [ 0.005374] Normal empty325lb # [ 0.002920] RAMDISK: [mem 0x5e367000-0x5ffcffff]326worker2 # [ 0.005375] Device empty327client # [ 0.002902] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]328worker1 # [ 0.032184] On node 0, zone DMA32: 40 pages in unavailable ranges329lb # [ 0.002928] ACPI: Early table checksum verification disabled330worker1 # [ 0.032651] ACPI: PM-Timer IO Port: 0x608331worker2 # [ 0.005376] Movable zone start for each node332lb # [ 0.002930] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )333client # [ 0.002902] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]334worker2 # [ 0.005377] Early memory node ranges335worker1 # [ 0.032662] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])336worker2 # [ 0.005377] node 0: [mem 0x0000000000001000-0x000000000009efff]337client # [ 0.003139] No NUMA configuration found338lb # [ 0.002934] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)339worker2 # [ 0.005378] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]340worker1 # [ 0.032688] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23341client # [ 0.003139] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]342client # [ 0.003142] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]343worker1 # [ 0.032691] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)344lb # [ 0.002938] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)345client # [ 0.005407] Zone ranges:346worker2 # [ 0.005379] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]347worker2 # [ 0.005401] On node 0, zone DMA: 1 pages in unavailable ranges348worker1 # [ 0.032692] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)349client # [ 0.005408] DMA [mem 0x0000000000001000-0x0000000000ffffff]350lb # [ 0.002944] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)351worker2 # [ 0.005671] On node 0, zone DMA: 97 pages in unavailable ranges352lb # [ 0.002946] ACPI: FACS 0x000000005FFE0000 000040353worker1 # [ 0.032693] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)354client # [ 0.005410] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]355client # [ 0.005411] Normal empty356worker2 # [ 0.032199] On node 0, zone DMA32: 40 pages in unavailable ranges357client # [ 0.005412] Device empty358worker1 # [ 0.032694] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)359worker2 # [ 0.032662] ACPI: PM-Timer IO Port: 0x608360lb # [ 0.002948] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)361client # [ 0.005413] Movable zone start for each node362worker1 # [ 0.032695] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)363worker2 # [ 0.032675] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])364client # [ 0.005414] Early memory node ranges365lb # [ 0.002950] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)366worker1 # [ 0.032697] ACPI: Using ACPI (MADT) for SMP configuration information367client # [ 0.005414] node 0: [mem 0x0000000000001000-0x000000000009efff]368worker2 # [ 0.032700] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23369lb # [ 0.002951] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)370client # [ 0.005415] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]371worker1 # [ 0.032698] ACPI: HPET id: 0x8086a201 base: 0xfed00000372worker2 # [ 0.032703] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)373worker1 # [ 0.032702] TSC deadline timer available374worker2 # [ 0.032705] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)375client # [ 0.005416] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]376lb # [ 0.002953] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)377worker1 # [ 0.032706] CPU topo: Max. logical packages: 1378client # [ 0.005436] On node 0, zone DMA: 1 pages in unavailable ranges379worker1 # [ 0.032706] CPU topo: Max. logical dies: 1380worker2 # [ 0.032706] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)381lb # [ 0.002954] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]382worker1 # [ 0.032707] CPU topo: Max. dies per package: 1383client # [ 0.005704] On node 0, zone DMA: 97 pages in unavailable ranges384worker2 # [ 0.032708] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)385worker1 # [ 0.032711] CPU topo: Max. threads per core: 1386lb # [ 0.002955] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]387client # [ 0.032306] On node 0, zone DMA32: 40 pages in unavailable ranges388worker1 # [ 0.032712] CPU topo: Num. cores per package: 1389worker2 # [ 0.032708] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)390client # [ 0.032758] ACPI: PM-Timer IO Port: 0x608391lb # [ 0.002956] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]392worker1 # [ 0.032712] CPU topo: Num. threads per package: 1393worker2 # [ 0.032711] ACPI: Using ACPI (MADT) for SMP configuration information394client # [ 0.032770] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])395lb # [ 0.002956] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]396worker1 # [ 0.032712] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs397worker2 # [ 0.032712] ACPI: HPET id: 0x8086a201 base: 0xfed00000398client # [ 0.032800] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23399worker2 # [ 0.032716] TSC deadline timer available400lb # [ 0.002956] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]401worker1 # [ 0.032738] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()402worker2 # [ 0.032720] CPU topo: Max. logical packages: 1403client # [ 0.032803] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)404lb # [ 0.002957] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]405worker2 # [ 0.032721] CPU topo: Max. logical dies: 1406worker1 # [ 0.032770] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]407client # [ 0.032804] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)408worker2 # [ 0.032721] CPU topo: Max. dies per package: 1409lb # [ 0.002957] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]410worker1 # [ 0.032771] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]411worker2 # [ 0.032724] CPU topo: Max. threads per core: 1412client # [ 0.032805] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)413lb # [ 0.003203] No NUMA configuration found414worker1 # [ 0.032772] [mem 0x60000000-0xafffffff] available for PCI devices415worker2 # [ 0.032725] CPU topo: Num. cores per package: 1416client # [ 0.032806] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)417worker2 # [ 0.032725] CPU topo: Num. threads per package: 1418worker1 # [ 0.032774] Booting paravirtualized kernel on KVM419lb # [ 0.003204] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]420client # [ 0.032807] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)421lb # [ 0.003206] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]422worker2 # [ 0.032726] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs423lb # [ 0.005440] Zone ranges:424client # [ 0.032809] ACPI: Using ACPI (MADT) for SMP configuration information425worker1 # [ 0.032777] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns426worker2 # [ 0.032747] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()427lb # [ 0.005441] DMA [mem 0x0000000000001000-0x0000000000ffffff]428client # [ 0.032810] ACPI: HPET id: 0x8086a201 base: 0xfed00000429worker1 # [ 0.037249] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1430client # [ 0.032813] TSC deadline timer available431worker2 # [ 0.032782] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]432lb # [ 0.005442] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]433lb # [ 0.005443] Normal empty434client # [ 0.032817] CPU topo: Max. logical packages: 1435worker1 # [ 0.039357] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152436lb # [ 0.005444] Device empty437worker2 # [ 0.032784] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]438client # [ 0.032818] CPU topo: Max. logical dies: 1439worker1 # [ 0.039400] kvm-guest: PV spinlocks disabled, single CPU440lb # [ 0.005445] Movable zone start for each node441client # [ 0.032818] CPU topo: Max. dies per package: 1442worker2 # [ 0.032785] [mem 0x60000000-0xafffffff] available for PCI devices443lb # [ 0.005446] Early memory node ranges444worker2 # [ 0.032787] Booting paravirtualized kernel on KVM445client # [ 0.032822] CPU topo: Max. threads per core: 1446lb # [ 0.005446] node 0: [mem 0x0000000000001000-0x000000000009efff]447client # [ 0.032822] CPU topo: Num. cores per package: 1448client # [ 0.032823] CPU topo: Num. threads per package: 1449lb # [ 0.005447] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]450worker2 # [ 0.032791] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns451client # [ 0.032823] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs452lb # [ 0.005448] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]453worker2 # [ 0.037285] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1454client # [ 0.032841] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()455lb # [ 0.005468] On node 0, zone DMA: 1 pages in unavailable ranges456worker2 # [ 0.039466] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152457worker1 # [ 0.039401] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hvv0rrwgpqzhm46a4280mwzmbapcmqq9-nixos-system-worker1-test/init regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration console=ttyS0,115200n8 console=tty0458lb # [ 0.005731] On node 0, zone DMA: 97 pages in unavailable ranges459client # [ 0.032871] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]460worker2 # [ 0.039514] kvm-guest: PV spinlocks disabled, single CPU461lb # [ 0.032230] On node 0, zone DMA32: 40 pages in unavailable ranges462client # [ 0.032873] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]463lb # [ 0.032685] ACPI: PM-Timer IO Port: 0x608464worker1 # [ 0.039491] Unknown kernel command line parameters "regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration", will be passed to user space.465worker1 # [ 0.039504] random: crng init done466client # [ 0.032874] [mem 0x60000000-0xafffffff] available for PCI devices467lb # [ 0.032696] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])468client # [ 0.032875] Booting paravirtualized kernel on KVM469worker1 # [ 0.039505] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes470lb # [ 0.032721] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23471worker1 # [ 0.041588] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)472client # [ 0.032878] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns473lb # [ 0.032724] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)474worker2 # [ 0.039516] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5qgax5h37ybfdjp7r7k01yx9abl6wij-nixos-system-worker2-test/init regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration console=ttyS0,115200n8 console=tty0475lb # [ 0.032725] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)476client # [ 0.037339] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1477worker1 # [ 0.042653] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)478worker1 # [ 0.042684] Fallback order for Node 0: 0479lb # [ 0.032727] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)480client # [ 0.039518] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152481worker2 # [ 0.039613] Unknown kernel command line parameters "regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration", will be passed to user space.482worker1 # [ 0.042687] Built 1 zonelists, mobility grouping on. Total pages: 393078483client # [ 0.039563] kvm-guest: PV spinlocks disabled, single CPU484lb # [ 0.032728] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)485worker2 # [ 0.039625] random: crng init done486worker1 # [ 0.042688] Policy zone: DMA32487lb # [ 0.032728] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)488worker2 # [ 0.039626] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes489worker1 # [ 0.045333] mem auto-init: stack:all(zero), heap alloc:on, heap free:off490lb # [ 0.032731] ACPI: Using ACPI (MADT) for SMP configuration information491worker1 # [ 0.048392] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1492worker2 # [ 0.041752] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)493worker1 # [ 0.051643] allocated 3145728 bytes of page_ext494lb # [ 0.032732] ACPI: HPET id: 0x8086a201 base: 0xfed00000495worker2 # [ 0.042995] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)496lb # [ 0.032735] TSC deadline timer available497worker1 # [ 0.061456] ftrace: allocating 48731 entries in 192 pages498worker2 # [ 0.043027] Fallback order for Node 0: 0499lb # [ 0.032739] CPU topo: Max. logical packages: 1500worker1 # [ 0.061459] ftrace: allocated 192 pages with 2 groups501client # [ 0.039564] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test/init regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration console=ttyS0,115200n8 console=tty0502lb # [ 0.032740] CPU topo: Max. logical dies: 1503worker2 # [ 0.043031] Built 1 zonelists, mobility grouping on. Total pages: 393078504worker1 # [ 0.062297] Dynamic Preempt: lazy505worker2 # [ 0.043032] Policy zone: DMA32506lb # [ 0.032741] CPU topo: Max. dies per package: 1507worker1 # [ 0.062469] rcu: Preemptible hierarchical RCU implementation.508lb # [ 0.032744] CPU topo: Max. threads per core: 1509worker2 # [ 0.045864] mem auto-init: stack:all(zero), heap alloc:on, heap free:off510client # [ 0.039660] Unknown kernel command line parameters "regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration", will be passed to user space.511worker1 # [ 0.062470] rcu: RCU event tracing is enabled.512lb # [ 0.032745] CPU topo: Num. cores per package: 1513client # [ 0.039672] random: crng init done514worker2 # [ 0.049142] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1515worker1 # [ 0.062470] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.516lb # [ 0.032745] CPU topo: Num. threads per package: 1517worker2 # [ 0.052508] allocated 3145728 bytes of page_ext518client # [ 0.039673] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes519worker1 # [ 0.062472] Trampoline variant of Tasks RCU enabled.520lb # [ 0.032745] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs521worker2 # [ 0.062798] ftrace: allocating 48731 entries in 192 pages522worker1 # [ 0.062472] Rude variant of Tasks RCU enabled.523client # [ 0.041794] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)524worker2 # [ 0.062801] ftrace: allocated 192 pages with 2 groups525worker1 # [ 0.062473] Tracing variant of Tasks RCU enabled.526lb # [ 0.032763] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()527worker2 # [ 0.063658] Dynamic Preempt: lazy528client # [ 0.043049] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)529worker1 # [ 0.062473] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.530worker2 # [ 0.063792] rcu: Preemptible hierarchical RCU implementation.531lb # [ 0.032792] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]532client # [ 0.043080] Fallback order for Node 0: 0533worker2 # [ 0.063792] rcu: RCU event tracing is enabled.534worker1 # [ 0.062474] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1535lb # [ 0.032794] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]536client # [ 0.043083] Built 1 zonelists, mobility grouping on. Total pages: 393078537worker2 # [ 0.063793] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.538client # [ 0.043084] Policy zone: DMA32539lb # [ 0.032795] [mem 0x60000000-0xafffffff] available for PCI devices540worker1 # [ 0.062488] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.541worker2 # [ 0.063795] Trampoline variant of Tasks RCU enabled.542client # [ 0.045566] mem auto-init: stack:all(zero), heap alloc:on, heap free:off543lb # [ 0.032796] Booting paravirtualized kernel on KVM544worker2 # [ 0.063795] Rude variant of Tasks RCU enabled.545client # [ 0.048676] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1546worker1 # [ 0.062490] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.547worker2 # [ 0.063795] Tracing variant of Tasks RCU enabled.548client # [ 0.051999] allocated 3145728 bytes of page_ext549lb # [ 0.032799] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns550worker2 # [ 0.063796] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.551worker1 # [ 0.062491] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.552client # [ 0.062137] ftrace: allocating 48731 entries in 192 pages553lb # [ 0.037244] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1554client # [ 0.062139] ftrace: allocated 192 pages with 2 groups555worker2 # [ 0.063797] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1556worker1 # [ 0.066865] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16557client # [ 0.063013] Dynamic Preempt: lazy558lb # [ 0.039349] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152559worker1 # [ 0.067143] rcu: srcu_init: Setting srcu_struct sizes based on contention.560client # [ 0.063138] rcu: Preemptible hierarchical RCU implementation.561worker2 # [ 0.063851] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.562lb # [ 0.039391] kvm-guest: PV spinlocks disabled, single CPU563client # [ 0.063139] rcu: RCU event tracing is enabled.564worker1 # [ 0.067149] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns565worker2 # [ 0.063853] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.566client # [ 0.063139] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.567client # [ 0.063141] Trampoline variant of Tasks RCU enabled.568worker1 # [ 0.067247] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)569worker2 # [ 0.063854] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.570client # [ 0.063141] Rude variant of Tasks RCU enabled.571worker1 # [ 0.070818] Console: colour VGA+ 80x25572worker2 # [ 0.068484] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16573client # [ 0.063142] Tracing variant of Tasks RCU enabled.574worker1 # [ 0.070821] printk: legacy console [tty0] enabled575worker2 # [ 0.068752] rcu: srcu_init: Setting srcu_struct sizes based on contention.576worker1 # [ 0.100797] printk: legacy console [ttyS0] enabled577client # [ 0.063142] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.578lb # [ 0.039392] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r7i2qny6zxqjgksiw8bnyjrpmwmq2adf-nixos-system-lb-test/init regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration console=ttyS0,115200n8 console=tty0579worker1 # [ 0.199083] ACPI: Core revision 20250807580client # [ 0.063143] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1581worker2 # [ 0.068758] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns582lb # [ 0.039483] Unknown kernel command line parameters "regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration", will be passed to user space.583client # [ 0.063192] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.584worker2 # [ 0.068863] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)585lb # [ 0.039496] random: crng init done586worker1 # [ 0.199959] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns587worker2 # [ 0.072435] Console: colour VGA+ 80x25588client # [ 0.063194] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.589worker2 # [ 0.072438] printk: legacy console [tty0] enabled590lb # [ 0.039497] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes591worker1 # [ 0.201481] APIC: Switch to symmetric I/O mode setup592worker2 # [ 0.101891] printk: legacy console [ttyS0] enabled593client # [ 0.063195] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.594worker1 # [ 0.202450] x2apic enabled595lb # [ 0.041559] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)596worker2 # [ 0.197504] ACPI: Core revision 20250807597client # [ 0.067594] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16598lb # [ 0.042771] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)599worker1 # [ 0.203260] APIC: Switched APIC routing to: physical x2apic600client # [ 0.068046] rcu: srcu_init: Setting srcu_struct sizes based on contention.601lb # [ 0.042802] Fallback order for Node 0: 0602worker2 # [ 0.198365] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns603lb # [ 0.042804] Built 1 zonelists, mobility grouping on. Total pages: 393078604client # [ 0.068053] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns605lb # [ 0.042805] Policy zone: DMA32606worker2 # [ 0.199940] APIC: Switch to symmetric I/O mode setup607worker1 # [ 0.205185] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1608lb # [ 0.045226] mem auto-init: stack:all(zero), heap alloc:on, heap free:off609worker2 # [ 0.200866] x2apic enabled610client # [ 0.068160] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)611client # [ 0.071688] Console: colour VGA+ 80x25612lb # [ 0.048257] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1613worker2 # [ 0.201604] APIC: Switched APIC routing to: physical x2apic614client # [ 0.071691] printk: legacy console [tty0] enabled615lb # [ 0.051476] allocated 3145728 bytes of page_ext616worker1 # [ 0.206187] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns617client # [ 0.101114] printk: legacy console [ttyS0] enabled618lb # [ 0.061195] ftrace: allocating 48731 entries in 192 pages619lb # [ 0.061197] ftrace: allocated 192 pages with 2 groups620client # [ 0.196639] ACPI: Core revision 20250807621worker1 # [ 0.207857] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)622lb # [ 0.061995] Dynamic Preempt: lazy623worker2 # [ 0.203379] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1624lb # [ 0.062166] rcu: Preemptible hierarchical RCU implementation.625worker1 # [ 0.208944] x86/cpu: User Mode Instruction Prevention (UMIP) activated626lb # [ 0.062167] rcu: RCU event tracing is enabled.627worker2 # [ 0.204324] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns628client # [ 0.197495] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns629worker1 # [ 0.210766] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127630lb # [ 0.062167] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.631client # [ 0.199575] APIC: Switch to symmetric I/O mode setup632lb # [ 0.062169] Trampoline variant of Tasks RCU enabled.633worker1 # [ 0.211542] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0634worker2 # [ 0.205917] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)635lb # [ 0.062169] Rude variant of Tasks RCU enabled.636client # [ 0.200519] x2apic enabled637lb # [ 0.062169] Tracing variant of Tasks RCU enabled.638worker2 # [ 0.207002] x86/cpu: User Mode Instruction Prevention (UMIP) activated639client # [ 0.201261] APIC: Switched APIC routing to: physical x2apic640worker1 # [ 0.212613] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto641lb # [ 0.062170] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.642worker2 # [ 0.208049] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127643lb # [ 0.062171] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1644worker1 # [ 0.213854] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl645worker2 # [ 0.208914] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0646client # [ 0.203099] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1647lb # [ 0.062191] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.648worker1 # [ 0.215854] Transient Scheduler Attacks: Vulnerable: No microcode649lb # [ 0.062192] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.650worker2 # [ 0.209917] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto651client # [ 0.204051] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns652worker1 # [ 0.216854] Spectre V2 : Mitigation: Enhanced / Automatic IBRS653lb # [ 0.062193] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.654worker1 # [ 0.217854] Speculative Return Stack Overflow: Mitigation: Safe RET655worker2 # [ 0.210914] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl656client # [ 0.205670] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)657lb # [ 0.066573] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16658worker2 # [ 0.211914] Transient Scheduler Attacks: Vulnerable: No microcode659lb # [ 0.067011] rcu: srcu_init: Setting srcu_struct sizes based on contention.660client # [ 0.206754] x86/cpu: User Mode Instruction Prevention (UMIP) activated661worker2 # [ 0.212913] Spectre V2 : Mitigation: Enhanced / Automatic IBRS662client # [ 0.207797] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127663lb # [ 0.067017] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns664worker2 # [ 0.213913] Speculative Return Stack Overflow: Mitigation: Safe RET665client # [ 0.208667] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0666lb # [ 0.067119] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)667worker1 # [ 0.218854] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization668lb # [ 0.070784] Console: colour VGA+ 80x25669lb # [ 0.070787] printk: legacy console [tty0] enabled670worker1 # [ 0.220861] Spectre V2 : Enabling IBPB for BPF671client # [ 0.209671] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto672lb # [ 0.100729] printk: legacy console [ttyS0] enabled673lb # [ 0.199278] ACPI: Core revision 20250807674worker1 # [ 0.221602] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier675client # [ 0.211667] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl676worker2 # [ 0.214913] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization677worker1 # [ 0.222855] active return thunk: srso_alias_return_thunk678client # [ 0.212667] Transient Scheduler Attacks: Vulnerable: No microcode679lb # [ 0.200215] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns680worker2 # [ 0.216921] Spectre V2 : Enabling IBPB for BPF681client # [ 0.213667] Spectre V2 : Mitigation: Enhanced / Automatic IBRS682worker1 # [ 0.223876] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'683lb # [ 0.201822] APIC: Switch to symmetric I/O mode setup684worker2 # [ 0.217915] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier685lb # [ 0.202780] x2apic enabled686client # [ 0.214667] Speculative Return Stack Overflow: Mitigation: Safe RET687worker1 # [ 0.224854] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'688worker2 # [ 0.218914] active return thunk: srso_alias_return_thunk689lb # [ 0.203538] APIC: Switched APIC routing to: physical x2apic690worker1 # [ 0.225853] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'691worker2 # [ 0.219936] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'692worker1 # [ 0.226854] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'693worker2 # [ 0.220913] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'694lb # [ 0.205343] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1695worker1 # [ 0.227854] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'696client # [ 0.215666] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization697worker2 # [ 0.221913] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'698worker1 # [ 0.228853] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'699client # [ 0.217675] Spectre V2 : Enabling IBPB for BPF700lb # [ 0.206314] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns701worker2 # [ 0.222913] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'702worker1 # [ 0.229853] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'703worker2 # [ 0.223913] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'704client # [ 0.218667] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier705lb # [ 0.207945] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)706worker1 # [ 0.231854] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256707client # [ 0.219667] active return thunk: srso_alias_return_thunk708worker2 # [ 0.224913] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'709lb # [ 0.209028] x86/cpu: User Mode Instruction Prevention (UMIP) activated710worker1 # [ 0.232808] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64711client # [ 0.220688] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'712lb # [ 0.210068] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127713worker2 # [ 0.225913] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'714worker1 # [ 0.233854] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512715client # [ 0.221666] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'716worker2 # [ 0.226913] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256717lb # [ 0.210942] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0718worker1 # [ 0.234854] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024719client # [ 0.222666] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'720worker2 # [ 0.228616] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64721worker1 # [ 0.235853] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8722lb # [ 0.211946] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto723client # [ 0.223666] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'724worker2 # [ 0.229624] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512725worker2 # [ 0.229913] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024726client # [ 0.224666] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'727lb # [ 0.213942] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl728worker1 # [ 0.236854] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.729worker2 # [ 0.230913] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8730client # [ 0.225666] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'731lb # [ 0.214942] Transient Scheduler Attacks: Vulnerable: No microcode732lb # [ 0.215942] Spectre V2 : Mitigation: Enhanced / Automatic IBRS733client # [ 0.226666] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'734worker2 # [ 0.231913] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.735lb # [ 0.216942] Speculative Return Stack Overflow: Mitigation: Safe RET736client # [ 0.227667] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256737client # [ 0.228666] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64738client # [ 0.229667] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512739client # [ 0.230666] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024740lb # [ 0.217942] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization741client # [ 0.231666] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8742lb # [ 0.219949] Spectre V2 : Enabling IBPB for BPF743client # [ 0.233429] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.744lb # [ 0.220943] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier745lb # [ 0.221942] active return thunk: srso_alias_return_thunk746lb # [ 0.222963] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'747lb # [ 0.223942] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748lb # [ 0.224942] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749lb # [ 0.225942] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750lb # [ 0.226942] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751lb # [ 0.227942] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752lb # [ 0.228942] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753lb # [ 0.229942] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754lb # [ 0.230942] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755lb # [ 0.231942] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756lb # [ 0.232942] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757lb # [ 0.233942] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758lb # [ 0.234942] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker1 # [ 0.271533] Freeing SMP alternatives memory: 44K760worker1 # [ 0.271856] pid_max: default: 32768 minimum: 301761worker2 # [ 0.265825] Freeing SMP alternatives memory: 44K762worker1 # [ 0.272943] LSM: initializing lsm=capability,landlock,yama,bpf,ima763worker2 # [ 0.266494] pid_max: default: 32768 minimum: 301764worker1 # [ 0.273959] landlock: Up and running.765worker1 # [ 0.274855] Yama: becoming mindful.766worker2 # [ 0.267022] LSM: initializing lsm=capability,landlock,yama,bpf,ima767worker1 # [ 0.275972] LSM support for eBPF active768worker2 # [ 0.268010] landlock: Up and running.769worker2 # [ 0.268914] Yama: becoming mindful.770worker1 # [ 0.276763] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)771worker2 # [ 0.270133] LSM support for eBPF active772worker1 # [ 0.277893] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)773worker2 # [ 0.271037] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)774client # [ 0.267350] Freeing SMP alternatives memory: 44K775client # [ 0.267670] pid_max: default: 32768 minimum: 301776worker2 # [ 0.271950] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)777worker1 # [ 0.280434] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)778client # [ 0.268771] LSM: initializing lsm=capability,landlock,yama,bpf,ima779client # [ 0.269794] landlock: Up and running.780client # [ 0.270667] Yama: becoming mindful.781worker1 # [ 0.281408] Performance Events: Fam17h+ core perfctr, AMD PMU driver.782worker2 # [ 0.274726] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)783client # [ 0.271696] LSM support for eBPF active784worker1 # [ 0.281859] ... version: 2785worker1 # [ 0.282611] ... bit width: 48786worker2 # [ 0.275487] Performance Events: Fam17h+ core perfctr, AMD PMU driver.787client # [ 0.272470] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)788worker1 # [ 0.282856] ... generic counters: 6789worker2 # [ 0.275918] ... version: 2790worker1 # [ 0.283555] ... generic bitmap: 000000000000003f791worker2 # [ 0.276618] ... bit width: 48792client # [ 0.273704] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)793worker1 # [ 0.283862] ... fixed-purpose counters: 0794worker2 # [ 0.276916] ... generic counters: 6795worker1 # [ 0.284578] ... fixed-purpose bitmap: 0000000000000000796worker2 # [ 0.277651] ... generic bitmap: 000000000000003f797worker2 # [ 0.277927] ... fixed-purpose counters: 0798worker1 # [ 0.284857] ... value mask: 0000ffffffffffff799client # [ 0.276068] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)800worker2 # [ 0.278632] ... fixed-purpose bitmap: 0000000000000000801worker1 # [ 0.285775] ... max period: 00007fffffffffff802worker2 # [ 0.278916] ... value mask: 0000ffffffffffff803client # [ 0.277250] Performance Events: Fam17h+ core perfctr, AMD PMU driver.804worker1 # [ 0.286632] ... global_ctrl mask: 000000000000003f805client # [ 0.277675] ... version: 2806worker2 # [ 0.279811] ... max period: 00007fffffffffff807worker1 # [ 0.286965] signal: max sigframe size: 3376808client # [ 0.278387] ... bit width: 48809worker2 # [ 0.280636] ... global_ctrl mask: 000000000000003f810worker1 # [ 0.287774] rcu: Hierarchical SRCU implementation.811client # [ 0.278669] ... generic counters: 6812worker2 # [ 0.281027] signal: max sigframe size: 3376813worker1 # [ 0.288473] rcu: Max phase no-delay instances is 400.814client # [ 0.279378] ... generic bitmap: 000000000000003f815lb # [ 0.268960] Freeing SMP alternatives memory: 44K816worker2 # [ 0.281867] rcu: Hierarchical SRCU implementation.817client # [ 0.279669] ... fixed-purpose counters: 0818lb # [ 0.269733] pid_max: default: 32768 minimum: 301819worker2 # [ 0.282504] rcu: Max phase no-delay instances is 400.820client # [ 0.280362] ... fixed-purpose bitmap: 0000000000000000821lb # [ 0.270026] LSM: initializing lsm=capability,landlock,yama,bpf,ima822client # [ 0.280669] ... value mask: 0000ffffffffffff823lb # [ 0.271041] landlock: Up and running.824lb # [ 0.271942] Yama: becoming mindful.825client # [ 0.281642] ... max period: 00007fffffffffff826lb # [ 0.273049] LSM support for eBPF active827client # [ 0.282380] ... global_ctrl mask: 000000000000003f828worker1 # [ 0.293400] smp: Bringing up secondary CPUs ...829client # [ 0.282780] signal: max sigframe size: 3376830lb # [ 0.273797] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)831worker1 # [ 0.293871] smp: Brought up 1 node, 1 CPU832client # [ 0.283536] rcu: Hierarchical SRCU implementation.833worker2 # [ 0.287390] smp: Bringing up secondary CPUs ...834worker1 # [ 0.294604] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)835client # [ 0.284270] rcu: Max phase no-delay instances is 400.836lb # [ 0.274984] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)837worker2 # [ 0.287930] smp: Brought up 1 node, 1 CPU838worker2 # [ 0.288581] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)839worker1 # [ 0.295021] Memory: 1454544K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)840lb # [ 0.277233] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)841worker1 # [ 0.296345] devtmpfs: initialized842worker2 # [ 0.289123] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)843lb # [ 0.278494] Performance Events: Fam17h+ core perfctr, AMD PMU driver.844worker1 # [ 0.297039] x86/mm: Memory block size: 128MB845worker2 # [ 0.290180] devtmpfs: initialized846lb # [ 0.278952] ... version: 2847client # [ 0.289108] smp: Bringing up secondary CPUs ...848lb # [ 0.279648] ... bit width: 48849worker2 # [ 0.290977] x86/mm: Memory block size: 128MB850client # [ 0.289681] smp: Brought up 1 node, 1 CPU851lb # [ 0.279944] ... generic counters: 6852worker1 # [ 0.298872] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)853client # [ 0.290353] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)854lb # [ 0.280650] ... generic bitmap: 000000000000003f855lb # [ 0.280945] ... fixed-purpose counters: 0856worker2 # [ 0.292749] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)857worker1 # [ 0.299888] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).858lb # [ 0.281639] ... fixed-purpose bitmap: 0000000000000000859worker1 # [ 0.300936] pinctrl core: initialized pinctrl subsystem860worker2 # [ 0.293819] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).861client # [ 0.290865] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)862lb # [ 0.281944] ... value mask: 0000ffffffffffff863client # [ 0.291938] devtmpfs: initialized864worker2 # [ 0.295014] pinctrl core: initialized pinctrl subsystem865worker1 # [ 0.302110] PM: RTC time: 09:32:24, date: 2026-09-20866lb # [ 0.282801] ... max period: 00007fffffffffff867client # [ 0.292720] x86/mm: Memory block size: 128MB868worker2 # [ 0.296098] PM: RTC time: 09:32:24, date: 2026-09-20869lb # [ 0.283594] ... global_ctrl mask: 000000000000003f870lb # [ 0.284052] signal: max sigframe size: 3376871lb # [ 0.284843] rcu: Hierarchical SRCU implementation.872client # [ 0.294483] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)873lb # [ 0.285532] rcu: Max phase no-delay instances is 400.874worker1 # [ 0.305637] NET: Registered PF_NETLINK/PF_ROUTE protocol family875client # [ 0.295564] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).876worker2 # [ 0.299686] NET: Registered PF_NETLINK/PF_ROUTE protocol family877client # [ 0.296760] pinctrl core: initialized pinctrl subsystem878worker1 # [ 0.307135] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations879worker2 # [ 0.301165] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations880client # [ 0.297916] PM: RTC time: 09:32:24, date: 2026-09-20881worker1 # [ 0.307881] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations882worker2 # [ 0.301938] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations883worker1 # [ 0.309128] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations884lb # [ 0.290492] smp: Bringing up secondary CPUs ...885worker1 # [ 0.309867] audit: initializing netlink subsys (disabled)886worker2 # [ 0.303200] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations887lb # [ 0.290959] smp: Brought up 1 node, 1 CPU888worker2 # [ 0.303925] audit: initializing netlink subsys (disabled)889client # [ 0.301407] NET: Registered PF_NETLINK/PF_ROUTE protocol family890worker1 # [ 0.311030] thermal_sys: Registered thermal governor 'fair_share'891lb # [ 0.291629] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)892worker2 # [ 0.305065] thermal_sys: Registered thermal governor 'fair_share'893worker1 # [ 0.311033] thermal_sys: Registered thermal governor 'bang_bang'894client # [ 0.302145] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations895worker2 # [ 0.305067] thermal_sys: Registered thermal governor 'bang_bang'896worker1 # [ 0.311858] thermal_sys: Registered thermal governor 'step_wise'897lb # [ 0.292157] Memory: 1454592K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)898client # [ 0.302693] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations899worker2 # [ 0.305917] thermal_sys: Registered thermal governor 'step_wise'900worker1 # [ 0.312832] thermal_sys: Registered thermal governor 'user_space'901lb # [ 0.293162] devtmpfs: initialized902worker2 # [ 0.306851] thermal_sys: Registered thermal governor 'user_space'903client # [ 0.303939] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations904lb # [ 0.293935] x86/mm: Memory block size: 128MB905worker1 # [ 0.313596] audit: type=2000 audit(1789896744.722:1): state=initialized audit_enabled=0 res=1906client # [ 0.304679] audit: initializing netlink subsys (disabled)907worker2 # [ 0.307659] audit: type=2000 audit(1789896744.725:1): state=initialized audit_enabled=0 res=1908worker1 # [ 0.314860] thermal_sys: Registered thermal governor 'power_allocator'909client # [ 0.305866] thermal_sys: Registered thermal governor 'fair_share'910worker1 # [ 0.314875] cpuidle: using governor menu911worker2 # [ 0.308927] thermal_sys: Registered thermal governor 'power_allocator'912lb # [ 0.295555] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)913worker2 # [ 0.308941] cpuidle: using governor menu914client # [ 0.305869] thermal_sys: Registered thermal governor 'bang_bang'915worker1 # [ 0.317105] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5916lb # [ 0.295968] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).917client # [ 0.306670] thermal_sys: Registered thermal governor 'step_wise'918worker2 # [ 0.311182] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5919lb # [ 0.297035] pinctrl core: initialized pinctrl subsystem920client # [ 0.307615] thermal_sys: Registered thermal governor 'user_space'921worker1 # [ 0.318140] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]922lb # [ 0.298148] PM: RTC time: 09:32:24, date: 2026-09-20923worker2 # [ 0.312162] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]924client # [ 0.308378] audit: type=2000 audit(1789896744.731:1): state=initialized audit_enabled=0 res=1925worker1 # [ 0.318861] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry926worker2 # [ 0.312919] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry927client # [ 0.309671] thermal_sys: Registered thermal governor 'power_allocator'928worker1 # [ 0.319869] PCI: Using configuration type 1 for base access929client # [ 0.309707] cpuidle: using governor menu930worker2 # [ 0.313928] PCI: Using configuration type 1 for base access931worker1 # [ 0.320977] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.932lb # [ 0.301565] NET: Registered PF_NETLINK/PF_ROUTE protocol family933client # [ 0.311887] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5934worker2 # [ 0.315021] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.935lb # [ 0.302407] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations936client # [ 0.312878] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]937lb # [ 0.302968] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations938client # [ 0.313672] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry939client # [ 0.314681] PCI: Using configuration type 1 for base access940lb # [ 0.304218] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations941lb # [ 0.304954] audit: initializing netlink subsys (disabled)942worker1 # [ 0.326117] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages943client # [ 0.315798] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.944worker1 # [ 0.326858] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page945lb # [ 0.306084] thermal_sys: Registered thermal governor 'fair_share'946lb # [ 0.306086] thermal_sys: Registered thermal governor 'bang_bang'947worker2 # [ 0.322137] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages948lb # [ 0.306945] thermal_sys: Registered thermal governor 'step_wise'949worker2 # [ 0.322917] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page950lb # [ 0.307861] thermal_sys: Registered thermal governor 'user_space'951lb # [ 0.308663] audit: type=2000 audit(1789896744.741:1): state=initialized audit_enabled=0 res=1952client # [ 0.320897] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages953worker1 # [ 0.331858] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages954lb # [ 0.309947] thermal_sys: Registered thermal governor 'power_allocator'955client # [ 0.321671] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page956lb # [ 0.309961] cpuidle: using governor menu957worker1 # [ 0.332857] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page958lb # [ 0.312150] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5959worker2 # [ 0.327916] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages960worker2 # [ 0.328916] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page961lb # [ 0.313171] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]962lb # [ 0.313948] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry963client # [ 0.326671] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages964lb # [ 0.314956] PCI: Using configuration type 1 for base access965client # [ 0.327669] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page966lb # [ 0.316057] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.967worker1 # [ 0.343256] ACPI: Added _OSI(Module Device)968worker2 # [ 0.337525] ACPI: Added _OSI(Module Device)969worker1 # [ 0.343858] ACPI: Added _OSI(Processor Device)970worker2 # [ 0.337917] ACPI: Added _OSI(Processor Device)971lb # [ 0.321064] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages972worker1 # [ 0.344641] ACPI: Added _OSI(Processor Aggregator Device)973worker2 # [ 0.338653] ACPI: Added _OSI(Processor Aggregator Device)974lb # [ 0.321946] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page975client # [ 0.338104] ACPI: Added _OSI(Module Device)976client # [ 0.338671] ACPI: Added _OSI(Processor Device)977client # [ 0.339438] ACPI: Added _OSI(Processor Aggregator Device)978lb # [ 0.326946] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages979lb # [ 0.327944] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page980worker2 # [ 0.345474] ACPI: 1 ACPI AML tables successfully acquired and loaded981worker1 # [ 0.353014] ACPI: 1 ACPI AML tables successfully acquired and loaded982worker2 # [ 0.348878] ACPI: Interpreter enabled983worker2 # [ 0.349391] ACPI: PM: (supports S0 S3 S4 S5)984worker1 # [ 0.357299] ACPI: Interpreter enabled985worker2 # [ 0.349917] ACPI: Using IOAPIC for interrupt routing986worker1 # [ 0.357872] ACPI: PM: (supports S0 S3 S4 S5)987worker1 # [ 0.358600] ACPI: Using IOAPIC for interrupt routing988worker2 # [ 0.350828] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug989client # [ 0.348306] ACPI: 1 ACPI AML tables successfully acquired and loaded990lb # [ 0.338283] ACPI: Added _OSI(Module Device)991lb # [ 0.338946] ACPI: Added _OSI(Processor Device)992worker1 # [ 0.360919] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug993lb # [ 0.339722] ACPI: Added _OSI(Processor Aggregator Device)994client # [ 0.352081] ACPI: Interpreter enabled995worker1 # [ 0.361857] PCI: Using E820 reservations for host bridge windows996client # [ 0.352684] ACPI: PM: (supports S0 S3 S4 S5)997worker2 # [ 0.355916] PCI: Using E820 reservations for host bridge windows998client # [ 0.353392] ACPI: Using IOAPIC for interrupt routing999worker2 # [ 0.357014] ACPI: Enabled 2 GPEs in block 00 to 3F1000client # [ 0.353745] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1001worker1 # [ 0.365009] ACPI: Enabled 2 GPEs in block 00 to 3F1002client # [ 0.354669] PCI: Using E820 reservations for host bridge windows1003client # [ 0.357806] ACPI: Enabled 2 GPEs in block 00 to 3F1004lb # [ 0.348161] ACPI: 1 ACPI AML tables successfully acquired and loaded1005worker2 # [ 0.365600] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1006worker1 # [ 0.372715] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1007worker2 # [ 0.366534] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1008worker1 # [ 0.373647] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1009lb # [ 0.352356] ACPI: Interpreter enabled1010lb # [ 0.352960] ACPI: PM: (supports S0 S3 S4 S5)1011worker2 # [ 0.366993] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1012worker1 # [ 0.374936] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1013lb # [ 0.353667] ACPI: Using IOAPIC for interrupt routing1014worker2 # [ 0.368038] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1015worker1 # [ 0.375983] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1016client # [ 0.366487] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1017worker2 # [ 0.369270] PCI host bridge to bus 0000:001018worker1 # [ 0.377210] PCI host bridge to bus 0000:001019worker2 # [ 0.369921] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1020client # [ 0.367476] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1021worker1 # [ 0.377862] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1022lb # [ 0.355998] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1023worker2 # [ 0.370916] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1024worker1 # [ 0.378857] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1025lb # [ 0.356945] PCI: Using E820 reservations for host bridge windows1026client # [ 0.369121] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1027worker2 # [ 0.371916] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1028worker1 # [ 0.379857] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1029client # [ 0.369791] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1030worker2 # [ 0.372916] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1031worker1 # [ 0.380856] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1032client # [ 0.371026] PCI host bridge to bus 0000:001033lb # [ 0.360014] ACPI: Enabled 2 GPEs in block 00 to 3F1034worker2 # [ 0.373916] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1035worker1 # [ 0.381856] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1036client # [ 0.371674] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1037worker2 # [ 0.374916] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1038client # [ 0.372670] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1039worker1 # [ 0.382856] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1040worker2 # [ 0.375929] pci_bus 0000:00: root bus resource [bus 00-ff]1041worker1 # [ 0.383857] pci_bus 0000:00: root bus resource [bus 00-ff]1042client # [ 0.373669] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1043worker2 # [ 0.376918] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1044client # [ 0.374669] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1045worker1 # [ 0.384858] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1046client # [ 0.375669] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1047worker2 # [ 0.378380] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1048worker1 # [ 0.386364] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1049client # [ 0.376669] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1050client # [ 0.377675] pci_bus 0000:00: root bus resource [bus 00-ff]1051lb # [ 0.367578] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1052client # [ 0.378686] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1053lb # [ 0.367951] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1054lb # [ 0.369023] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1055client # [ 0.380181] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1056lb # [ 0.370067] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1057lb # [ 0.371289] PCI host bridge to bus 0000:001058worker2 # [ 0.381007] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1059lb # [ 0.371949] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1060worker1 # [ 0.389008] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1061worker2 # [ 0.381928] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1062worker1 # [ 0.389870] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1063lb # [ 0.372945] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1064worker2 # [ 0.382892] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1065worker1 # [ 0.390841] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1066lb # [ 0.373945] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1067worker2 # [ 0.383902] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1068lb # [ 0.374945] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1069worker1 # [ 0.391966] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1070client # [ 0.382743] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1071lb # [ 0.375945] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1072worker2 # [ 0.385710] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1073client # [ 0.383682] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1074lb # [ 0.376944] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1075worker1 # [ 0.393611] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1076client # [ 0.384638] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1077lb # [ 0.377945] pci_bus 0000:00: root bus resource [bus 00-ff]1078lb # [ 0.378911] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1079worker2 # [ 0.388949] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1080client # [ 0.385662] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1081worker1 # [ 0.395894] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1082worker2 # [ 0.389770] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1083worker1 # [ 0.396746] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1084lb # [ 0.380397] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1085client # [ 0.387408] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1086worker2 # [ 0.390674] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1087worker1 # [ 0.397620] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1088worker2 # [ 0.390922] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1089worker1 # [ 0.398764] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1090client # [ 0.390721] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1091worker1 # [ 0.400211] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1092worker2 # [ 0.392620] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1093client # [ 0.391559] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1094client # [ 0.392419] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1095worker1 # [ 0.401891] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1096client # [ 0.392675] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1097worker2 # [ 0.395951] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1098worker1 # [ 0.402835] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1099lb # [ 0.383006] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1100worker2 # [ 0.396874] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1101worker1 # [ 0.403634] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1102lb # [ 0.383958] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1103worker2 # [ 0.397656] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1104client # [ 0.394210] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1105lb # [ 0.384920] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1106worker1 # [ 0.405307] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1107worker2 # [ 0.398505] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1108lb # [ 0.386137] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1109client # [ 0.396568] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1110client # [ 0.397327] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1111lb # [ 0.387658] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1112client # [ 0.397691] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1113worker1 # [ 0.408057] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1114worker2 # [ 0.401222] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1115worker1 # [ 0.408864] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1116client # [ 0.399227] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1117worker2 # [ 0.402548] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1118lb # [ 0.389982] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1119worker1 # [ 0.409837] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1120worker2 # [ 0.402938] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1121lb # [ 0.390868] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1122worker1 # [ 0.411293] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1123lb # [ 0.391710] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1124worker2 # [ 0.404489] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125lb # [ 0.392824] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1126client # [ 0.402002] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1127worker1 # [ 0.412887] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1128client # [ 0.403356] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1129lb # [ 0.394302] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1130worker2 # [ 0.406740] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1131worker1 # [ 0.413787] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1132client # [ 0.403691] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1133worker2 # [ 0.407560] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1134worker1 # [ 0.414592] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1135worker2 # [ 0.407938] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1136lb # [ 0.395982] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1137client # [ 0.405248] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1138lb # [ 0.396837] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1139worker1 # [ 0.415407] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1140lb # [ 0.397709] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1141worker2 # [ 0.410025] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1142client # [ 0.407493] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1143worker1 # [ 0.417864] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1144client # [ 0.408296] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1145lb # [ 0.399483] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1146worker1 # [ 0.418744] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1147client # [ 0.408691] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1148worker2 # [ 0.412639] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1149worker1 # [ 0.419593] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1150worker2 # [ 0.412923] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1151worker1 # [ 0.421334] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1152worker2 # [ 0.413937] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1153client # [ 0.411120] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1154lb # [ 0.402519] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1155lb # [ 0.402959] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1156worker2 # [ 0.415492] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1157lb # [ 0.403950] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1158client # [ 0.413497] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1159worker1 # [ 0.423797] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1160client # [ 0.414319] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1161worker1 # [ 0.424888] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1162lb # [ 0.405525] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1163client # [ 0.414691] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1164worker1 # [ 0.425862] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1165worker2 # [ 0.418251] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1166lb # [ 0.406952] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1167client # [ 0.416277] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1168worker2 # [ 0.419602] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1169worker1 # [ 0.427439] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1170lb # [ 0.407808] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1171worker2 # [ 0.419939] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1172lb # [ 0.408660] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1173worker1 # [ 0.428865] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1174worker2 # [ 0.421560] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1175worker1 # [ 0.429803] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1176client # [ 0.419045] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1177lb # [ 0.409538] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1178worker1 # [ 0.430640] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1179client # [ 0.420304] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1180worker1 # [ 0.431733] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1181worker2 # [ 0.423640] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1182client # [ 0.420693] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1183lb # [ 0.410952] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1184worker2 # [ 0.423924] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1185lb # [ 0.411846] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1186worker2 # [ 0.424903] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1187worker1 # [ 0.433221] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1188client # [ 0.422219] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1189lb # [ 0.412720] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1190worker2 # [ 0.425738] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1191worker1 # [ 0.435592] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1192lb # [ 0.414369] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1193client # [ 0.424402] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1194worker1 # [ 0.435881] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1195worker2 # [ 0.427472] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1196client # [ 0.424677] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1197client # [ 0.425630] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1198worker1 # [ 0.437448] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1199lb # [ 0.416794] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1200worker2 # [ 0.428962] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1201lb # [ 0.417606] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1202client # [ 0.426505] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1203worker2 # [ 0.429964] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1204worker1 # [ 0.438899] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1205lb # [ 0.418688] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1206worker1 # [ 0.439778] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1207client # [ 0.428261] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1208worker2 # [ 0.431454] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1209worker1 # [ 0.440627] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1210lb # [ 0.419575] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1211client # [ 0.429682] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1212worker1 # [ 0.442455] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1213client # [ 0.430674] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1214lb # [ 0.420979] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1215worker2 # [ 0.433752] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1216lb # [ 0.421818] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1217worker2 # [ 0.434923] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1218worker1 # [ 0.443864] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1219client # [ 0.432207] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1220lb # [ 0.422699] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1221worker2 # [ 0.435843] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1222worker1 # [ 0.444716] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1223lb # [ 0.423803] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1224worker1 # [ 0.445621] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1225worker2 # [ 0.437310] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1226client # [ 0.433702] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1227lb # [ 0.425324] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1228worker1 # [ 0.447380] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1229client # [ 0.434676] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1230worker2 # [ 0.439610] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1231worker1 # [ 0.448473] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1232lb # [ 0.426773] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1233client # [ 0.435680] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1234worker2 # [ 0.439923] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1235lb # [ 0.427700] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1236worker2 # [ 0.440899] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1237worker1 # [ 0.449061] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1238client # [ 0.437242] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1239lb # [ 0.428520] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1240worker1 # [ 0.450371] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1241worker2 # [ 0.442304] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1242client # [ 0.438697] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1243lb # [ 0.429984] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1244client # [ 0.439563] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1245worker2 # [ 0.443735] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1246lb # [ 0.430839] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1247client # [ 0.440432] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1248worker2 # [ 0.444719] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1249lb # [ 0.431685] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1250worker2 # [ 0.446651] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1251client # [ 0.441235] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1252lb # [ 0.432556] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1253client # [ 0.442491] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1254lb # [ 0.433952] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1255client # [ 0.443551] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1256lb # [ 0.434870] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1257lb # [ 0.435751] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1258client # [ 0.445383] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1259lb # [ 0.437464] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1260worker1 # [ 0.451059] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1261worker1 # [ 0.453440] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1262lb # [ 0.438707] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1263worker1 # [ 0.454161] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1264lb # [ 0.439176] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1265worker1 # [ 0.455411] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1266lb # [ 0.440488] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1267worker2 # [ 0.447147] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1268worker1 # [ 0.456399] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1269worker2 # [ 0.449681] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1270worker1 # [ 0.457159] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1271worker2 # [ 0.450801] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1272worker1 # [ 0.458128] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1273worker2 # [ 0.452573] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1274client # [ 0.445858] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1275worker1 # [ 0.459741] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1276worker2 # [ 0.453177] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1277worker1 # [ 0.460562] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1278client # [ 0.448470] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1279worker2 # [ 0.454233] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1280worker1 # [ 0.461179] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1281client # [ 0.449752] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1282worker2 # [ 0.455175] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1283worker1 # [ 0.462391] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1284client # [ 0.451340] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1285lb # [ 0.441168] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1286client # [ 0.451898] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1287worker2 # [ 0.456970] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1288lb # [ 0.443505] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1289worker2 # [ 0.457821] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1290client # [ 0.452960] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1291worker1 # [ 0.466589] ACPI: PCI: Interrupt link LNKA configured for IRQ 101292lb # [ 0.444217] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1293worker2 # [ 0.458980] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1294client # [ 0.453977] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1295worker1 # [ 0.466977] ACPI: PCI: Interrupt link LNKB configured for IRQ 101296worker1 # [ 0.467958] ACPI: PCI: Interrupt link LNKC configured for IRQ 111297lb # [ 0.445536] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1298worker2 # [ 0.460698] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1299client # [ 0.455725] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1300worker1 # [ 0.468956] ACPI: PCI: Interrupt link LNKD configured for IRQ 111301lb # [ 0.446258] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1302client # [ 0.456605] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1303worker1 # [ 0.469961] ACPI: PCI: Interrupt link LNKE configured for IRQ 101304worker1 # [ 0.470957] ACPI: PCI: Interrupt link LNKF configured for IRQ 101305lb # [ 0.447243] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1306client # [ 0.457759] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1307worker2 # [ 0.463852] ACPI: PCI: Interrupt link LNKA configured for IRQ 101308worker1 # [ 0.471961] ACPI: PCI: Interrupt link LNKG configured for IRQ 111309lb # [ 0.448232] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1310client # [ 0.459233] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1311worker2 # [ 0.464729] ACPI: PCI: Interrupt link LNKB configured for IRQ 101312worker1 # [ 0.473015] ACPI: PCI: Interrupt link LNKH configured for IRQ 111313worker2 # [ 0.465695] ACPI: PCI: Interrupt link LNKC configured for IRQ 111314worker1 # [ 0.473899] ACPI: PCI: Interrupt link GSIA configured for IRQ 161315lb # [ 0.449744] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1316worker2 # [ 0.466012] ACPI: PCI: Interrupt link LNKD configured for IRQ 111317worker1 # [ 0.474874] ACPI: PCI: Interrupt link GSIB configured for IRQ 171318lb # [ 0.450622] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1319worker2 # [ 0.467017] ACPI: PCI: Interrupt link LNKE configured for IRQ 101320worker1 # [ 0.475846] ACPI: PCI: Interrupt link GSIC configured for IRQ 181321client # [ 0.462808] ACPI: PCI: Interrupt link LNKA configured for IRQ 101322lb # [ 0.451277] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1323worker2 # [ 0.468015] ACPI: PCI: Interrupt link LNKF configured for IRQ 101324worker1 # [ 0.476617] ACPI: PCI: Interrupt link GSID configured for IRQ 191325client # [ 0.463779] ACPI: PCI: Interrupt link LNKB configured for IRQ 101326worker2 # [ 0.469016] ACPI: PCI: Interrupt link LNKG configured for IRQ 111327worker1 # [ 0.476870] ACPI: PCI: Interrupt link GSIE configured for IRQ 201328lb # [ 0.452527] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1329client # [ 0.464769] ACPI: PCI: Interrupt link LNKC configured for IRQ 111330worker2 # [ 0.470023] ACPI: PCI: Interrupt link LNKH configured for IRQ 111331worker1 # [ 0.477839] ACPI: PCI: Interrupt link GSIF configured for IRQ 211332client # [ 0.465941] ACPI: PCI: Interrupt link LNKD configured for IRQ 111333worker2 # [ 0.470952] ACPI: PCI: Interrupt link GSIA configured for IRQ 161334worker1 # [ 0.478604] ACPI: PCI: Interrupt link GSIG configured for IRQ 221335worker2 # [ 0.471883] ACPI: PCI: Interrupt link GSIB configured for IRQ 171336client # [ 0.466767] ACPI: PCI: Interrupt link LNKE configured for IRQ 101337worker1 # [ 0.478874] ACPI: PCI: Interrupt link GSIH configured for IRQ 231338worker2 # [ 0.472631] ACPI: PCI: Interrupt link GSIC configured for IRQ 181339client # [ 0.467771] ACPI: PCI: Interrupt link LNKF configured for IRQ 101340lb # [ 0.456051] ACPI: PCI: Interrupt link LNKA configured for IRQ 101341worker2 # [ 0.472930] ACPI: PCI: Interrupt link GSID configured for IRQ 191342worker1 # [ 0.480940] iommu: Default domain type: Translated1343client # [ 0.468771] ACPI: PCI: Interrupt link LNKG configured for IRQ 111344worker2 # [ 0.473856] ACPI: PCI: Interrupt link GSIE configured for IRQ 201345lb # [ 0.457052] ACPI: PCI: Interrupt link LNKB configured for IRQ 101346worker1 # [ 0.481757] iommu: DMA domain TLB invalidation policy: lazy mode1347client # [ 0.469767] ACPI: PCI: Interrupt link LNKH configured for IRQ 111348lb # [ 0.458046] ACPI: PCI: Interrupt link LNKC configured for IRQ 111349worker2 # [ 0.474622] ACPI: PCI: Interrupt link GSIF configured for IRQ 211350worker1 # [ 0.482830] ACPI: bus type USB registered1351client # [ 0.470708] ACPI: PCI: Interrupt link GSIA configured for IRQ 161352worker1 # [ 0.483379] usbcore: registered new interface driver usbfs1353worker2 # [ 0.474932] ACPI: PCI: Interrupt link GSIG configured for IRQ 221354lb # [ 0.459045] ACPI: PCI: Interrupt link LNKD configured for IRQ 111355client # [ 0.471681] ACPI: PCI: Interrupt link GSIB configured for IRQ 171356worker2 # [ 0.475853] ACPI: PCI: Interrupt link GSIH configured for IRQ 231357worker1 # [ 0.483870] usbcore: registered new interface driver hub1358lb # [ 0.460074] ACPI: PCI: Interrupt link LNKE configured for IRQ 101359client # [ 0.472636] ACPI: PCI: Interrupt link GSIC configured for IRQ 181360worker1 # [ 0.484722] usbcore: registered new device driver usb1361lb # [ 0.461042] ACPI: PCI: Interrupt link LNKF configured for IRQ 101362client # [ 0.473402] ACPI: PCI: Interrupt link GSID configured for IRQ 191363worker2 # [ 0.477633] iommu: Default domain type: Translated1364client # [ 0.473683] ACPI: PCI: Interrupt link GSIE configured for IRQ 201365worker1 # [ 0.486408] NetLabel: Initializing1366lb # [ 0.462046] ACPI: PCI: Interrupt link LNKG configured for IRQ 111367worker2 # [ 0.477923] iommu: DMA domain TLB invalidation policy: lazy mode1368worker1 # [ 0.486858] NetLabel: domain hash size = 1281369client # [ 0.474651] ACPI: PCI: Interrupt link GSIF configured for IRQ 211370worker2 # [ 0.479073] ACPI: bus type USB registered1371lb # [ 0.463048] ACPI: PCI: Interrupt link LNKH configured for IRQ 111372worker1 # [ 0.487580] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1373client # [ 0.475413] ACPI: PCI: Interrupt link GSIG configured for IRQ 221374worker2 # [ 0.479775] usbcore: registered new interface driver usbfs1375lb # [ 0.463986] ACPI: PCI: Interrupt link GSIA configured for IRQ 161376worker1 # [ 0.487895] NetLabel: unlabeled traffic allowed by default1377client # [ 0.475684] ACPI: PCI: Interrupt link GSIH configured for IRQ 231378worker2 # [ 0.480576] usbcore: registered new interface driver hub1379worker1 # [ 0.488777] PCI: Using ACPI for IRQ routing1380lb # [ 0.464960] ACPI: PCI: Interrupt link GSIB configured for IRQ 171381worker2 # [ 0.480925] usbcore: registered new device driver usb1382lb # [ 0.465956] ACPI: PCI: Interrupt link GSIC configured for IRQ 181383client # [ 0.477867] iommu: Default domain type: Translated1384lb # [ 0.466959] ACPI: PCI: Interrupt link GSID configured for IRQ 191385worker2 # [ 0.482679] NetLabel: Initializing1386client # [ 0.478683] iommu: DMA domain TLB invalidation policy: lazy mode1387worker2 # [ 0.482922] NetLabel: domain hash size = 1281388lb # [ 0.467957] ACPI: PCI: Interrupt link GSIE configured for IRQ 201389client # [ 0.479962] ACPI: bus type USB registered1390worker2 # [ 0.483774] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1391lb # [ 0.468946] ACPI: PCI: Interrupt link GSIF configured for IRQ 211392client # [ 0.480716] usbcore: registered new interface driver usbfs1393worker2 # [ 0.484630] NetLabel: unlabeled traffic allowed by default1394lb # [ 0.469924] ACPI: PCI: Interrupt link GSIG configured for IRQ 221395client # [ 0.481589] usbcore: registered new interface driver hub1396worker2 # [ 0.484930] PCI: Using ACPI for IRQ routing1397lb # [ 0.470711] ACPI: PCI: Interrupt link GSIH configured for IRQ 231398client # [ 0.482347] usbcore: registered new device driver usb1399lb # [ 0.471955] iommu: Default domain type: Translated1400client # [ 0.483846] NetLabel: Initializing1401client # [ 0.484455] NetLabel: domain hash size = 1281402lb # [ 0.472816] iommu: DMA domain TLB invalidation policy: lazy mode1403client # [ 0.484670] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1404lb # [ 0.473968] ACPI: bus type USB registered1405client # [ 0.485684] NetLabel: unlabeled traffic allowed by default1406lb # [ 0.474750] usbcore: registered new interface driver usbfs1407client # [ 0.486630] PCI: Using ACPI for IRQ routing1408lb # [ 0.475681] usbcore: registered new interface driver hub1409lb # [ 0.475955] usbcore: registered new device driver usb1410lb # [ 0.477724] NetLabel: Initializing1411lb # [ 0.477951] NetLabel: domain hash size = 1281412lb # [ 0.478721] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413lb # [ 0.478988] NetLabel: unlabeled traffic allowed by default1414lb # [ 0.479956] PCI: Using ACPI for IRQ routing1415worker1 # [ 0.535308] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker1 # [ 0.535852] pci 0000:00:01.0: vgaarb: bridge control possible1417worker1 # [ 0.535852] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker1 # [ 0.535862] vgaarb: loaded1419worker2 # [ 0.528620] pci 0000:00:01.0: vgaarb: setting as boot VGA device1420worker2 # [ 0.528912] pci 0000:00:01.0: vgaarb: bridge control possible1421worker1 # [ 0.536647] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01422worker1 # [ 0.537475] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1423worker2 # [ 0.528912] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1424worker2 # [ 0.528921] vgaarb: loaded1425worker2 # [ 0.529673] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01426worker2 # [ 0.529918] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1427client # [ 0.530465] pci 0000:00:01.0: vgaarb: setting as boot VGA device1428client # [ 0.530665] pci 0000:00:01.0: vgaarb: bridge control possible1429worker1 # [ 0.541950] clocksource: Switched to clocksource kvm-clock1430client # [ 0.530665] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1431client # [ 0.530674] vgaarb: loaded1432worker2 # [ 0.533002] clocksource: Switched to clocksource kvm-clock1433worker1 # [ 0.543689] VFS: Disk quotas dquot_6.6.01434client # [ 0.531429] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01435client # [ 0.531672] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1436worker2 # [ 0.536432] VFS: Disk quotas dquot_6.6.01437worker1 # [ 0.546638] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1438worker1 # [ 0.548078] pnp: PnP ACPI init1439worker2 # [ 0.537133] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1440worker2 # [ 0.538479] pnp: PnP ACPI init1441worker1 # [ 0.548945] ACPI: IRQ 4 override to edge(!), high(!)1442lb # [ 0.526535] pci 0000:00:01.0: vgaarb: setting as boot VGA device1443worker2 # [ 0.539297] ACPI: IRQ 4 override to edge(!), high(!)1444worker1 # [ 0.549937] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1445lb # [ 0.526941] pci 0000:00:01.0: vgaarb: bridge control possible1446client # [ 0.535779] clocksource: Switched to clocksource kvm-clock1447worker2 # [ 0.540260] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1448worker1 # [ 0.551487] pnp: PnP ACPI: found 6 devices1449lb # [ 0.526941] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1450worker2 # [ 0.541738] pnp: PnP ACPI: found 6 devices1451lb # [ 0.526950] vgaarb: loaded1452lb # [ 0.527691] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01453client # [ 0.539512] VFS: Disk quotas dquot_6.6.01454lb # [ 0.527947] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1455client # [ 0.540258] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1456client # [ 0.541674] pnp: PnP ACPI init1457client # [ 0.542519] ACPI: IRQ 4 override to edge(!), high(!)1458client # [ 0.543526] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1459worker1 # [ 0.559145] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1460lb # [ 0.532027] clocksource: Switched to clocksource kvm-clock1461client # [ 0.545151] pnp: PnP ACPI: found 6 devices1462worker2 # [ 0.549356] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1463worker1 # [ 0.560628] clocksource: Switched to clocksource acpi_pm1464worker2 # [ 0.550772] clocksource: Switched to clocksource acpi_pm1465worker1 # [ 0.561638] NET: Registered PF_INET protocol family1466worker2 # [ 0.551749] NET: Registered PF_INET protocol family1467lb # [ 0.535466] VFS: Disk quotas dquot_6.6.01468worker1 # [ 0.562854] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1469worker2 # [ 0.552921] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1470lb # [ 0.536213] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1471lb # [ 0.537566] pnp: PnP ACPI init1472lb # [ 0.538414] ACPI: IRQ 4 override to edge(!), high(!)1473lb # [ 0.539430] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1474client # [ 0.552822] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1475lb # [ 0.541014] pnp: PnP ACPI: found 6 devices1476client # [ 0.554282] clocksource: Switched to clocksource acpi_pm1477client # [ 0.555298] NET: Registered PF_INET protocol family1478client # [ 0.556493] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1479lb # [ 0.548588] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1480lb # [ 0.550119] clocksource: Switched to clocksource acpi_pm1481lb # [ 0.551134] NET: Registered PF_INET protocol family1482worker1 # [ 0.579187] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1483lb # [ 0.552325] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1484worker2 # [ 0.569511] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1485worker1 # [ 0.580794] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1486worker2 # [ 0.571011] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1487worker1 # [ 0.582153] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1488worker2 # [ 0.572360] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1489worker1 # [ 0.583523] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1490worker2 # [ 0.573733] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1491worker1 # [ 0.584835] TCP: Hash tables configured (established 16384 bind 16384)1492worker2 # [ 0.575017] TCP: Hash tables configured (established 16384 bind 16384)1493worker1 # [ 0.586108] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1494worker2 # [ 0.576212] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1495worker1 # [ 0.587500] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1496worker2 # [ 0.577516] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1497worker1 # [ 0.588688] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1498client # [ 0.573781] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1499worker2 # [ 0.578634] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1500worker1 # [ 0.589913] NET: Registered PF_UNIX/PF_LOCAL protocol family1501worker2 # [ 0.579808] NET: Registered PF_UNIX/PF_LOCAL protocol family1502client # [ 0.575206] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1503worker1 # [ 0.590859] NET: Registered PF_XDP protocol family1504worker2 # [ 0.580765] NET: Registered PF_XDP protocol family1505worker1 # [ 0.591679] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1506client # [ 0.576492] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1507worker2 # [ 0.581588] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1508worker1 # [ 0.592654] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1509client # [ 0.577817] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1510worker2 # [ 0.582601] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1511worker1 # [ 0.593668] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1512client # [ 0.579052] TCP: Hash tables configured (established 16384 bind 16384)1513worker2 # [ 0.583587] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1514worker1 # [ 0.594768] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1515worker2 # [ 0.584660] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1516client # [ 0.580194] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1517worker1 # [ 0.595853] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1518worker2 # [ 0.585736] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1519client # [ 0.581426] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1520lb # [ 0.568424] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1521worker1 # [ 0.596956] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1522worker2 # [ 0.586759] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1523client # [ 0.582450] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1524lb # [ 0.569918] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1525worker1 # [ 0.598663] ACPI: \_SB_.GSIA: Enabled at IRQ 161526client # [ 0.583663] NET: Registered PF_UNIX/PF_LOCAL protocol family1527worker2 # [ 0.588435] ACPI: \_SB_.GSIA: Enabled at IRQ 161528lb # [ 0.571255] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1529client # [ 0.584608] NET: Registered PF_XDP protocol family1530client # [ 0.585447] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1531lb # [ 0.572593] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1532worker1 # [ 0.601241] ACPI: \_SB_.GSIB: Enabled at IRQ 171533worker2 # [ 0.590922] ACPI: \_SB_.GSIB: Enabled at IRQ 171534client # [ 0.586479] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1535lb # [ 0.573857] TCP: Hash tables configured (established 16384 bind 16384)1536client # [ 0.587453] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1537lb # [ 0.574979] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1538worker1 # [ 0.603235] ACPI: \_SB_.GSIC: Enabled at IRQ 181539worker2 # [ 0.592847] ACPI: \_SB_.GSIC: Enabled at IRQ 181540client # [ 0.588568] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1541lb # [ 0.576355] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1542client # [ 0.589659] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1543worker1 # [ 0.605309] ACPI: \_SB_.GSID: Enabled at IRQ 191544worker2 # [ 0.594830] ACPI: \_SB_.GSID: Enabled at IRQ 191545lb # [ 0.577534] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1546client # [ 0.590789] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1547lb # [ 0.578728] NET: Registered PF_UNIX/PF_LOCAL protocol family1548worker1 # [ 0.606996] PCI: CLS 0 bytes, default 641549worker2 # [ 0.596476] PCI: CLS 0 bytes, default 641550lb # [ 0.579708] NET: Registered PF_XDP protocol family1551client # [ 0.592644] ACPI: \_SB_.GSIA: Enabled at IRQ 161552lb # [ 0.580605] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1553worker2 # [ 0.597352] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1554worker1 # [ 0.607869] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1555lb # [ 0.581614] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1556worker2 # [ 0.599012] Trying to unpack rootfs image as initramfs...1557worker1 # [ 0.609723] Trying to unpack rootfs image as initramfs...1558client # [ 0.595412] ACPI: \_SB_.GSIB: Enabled at IRQ 171559lb # [ 0.582634] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1560lb # [ 0.583791] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1561client # [ 0.597307] ACPI: \_SB_.GSIC: Enabled at IRQ 181562lb # [ 0.584980] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1563lb # [ 0.586150] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1564client # [ 0.599267] ACPI: \_SB_.GSID: Enabled at IRQ 191565lb # [ 0.587913] ACPI: \_SB_.GSIA: Enabled at IRQ 161566client # [ 0.600936] PCI: CLS 0 bytes, default 641567client # [ 0.601869] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1568lb # [ 0.590548] ACPI: \_SB_.GSIB: Enabled at IRQ 171569client # [ 0.603712] Trying to unpack rootfs image as initramfs...1570lb # [ 0.592494] ACPI: \_SB_.GSIC: Enabled at IRQ 181571lb # [ 0.594471] ACPI: \_SB_.GSID: Enabled at IRQ 191572lb # [ 0.596310] PCI: CLS 0 bytes, default 641573lb # [ 0.597245] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574lb # [ 0.599050] Trying to unpack rootfs image as initramfs...1575worker2 # [ 0.643514] Initialise system trusted keyrings1576worker1 # [ 0.656152] Initialise system trusted keyrings1577worker1 # [ 0.657096] workingset: timestamp_bits=40 max_order=19 bucket_order=01578worker2 # [ 0.647219] workingset: timestamp_bits=40 max_order=19 bucket_order=01579client # [ 0.649892] Initialise system trusted keyrings1580client # [ 0.652688] workingset: timestamp_bits=40 max_order=19 bucket_order=01581lb # [ 0.644355] Initialise system trusted keyrings1582lb # [ 0.648001] workingset: timestamp_bits=40 max_order=19 bucket_order=01583worker2 # [ 0.668602] Key type asymmetric registered1584worker2 # [ 0.669370] Asymmetric key parser 'x509' registered1585worker1 # [ 0.680883] Key type asymmetric registered1586worker1 # [ 0.681615] Asymmetric key parser 'x509' registered1587worker1 # [ 0.682511] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1588worker2 # [ 0.672249] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1589worker1 # [ 0.685865] io scheduler mq-deadline registered1590worker2 # [ 0.675244] io scheduler mq-deadline registered1591worker1 # [ 0.686625] io scheduler kyber registered1592worker2 # [ 0.676019] io scheduler kyber registered1593client # [ 0.674424] Key type asymmetric registered1594worker2 # [ 0.678766] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1595client # [ 0.675159] Asymmetric key parser 'x509' registered1596worker1 # [ 0.690325] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1597worker2 # [ 0.680008] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1598worker1 # [ 0.691606] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1599lb # [ 0.667280] Key type asymmetric registered1600client # [ 0.679721] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1601client # [ 0.681098] io scheduler mq-deadline registered1602worker2 # [ 0.685781] Linux agpgart interface v0.1031603worker1 # [ 0.696801] Linux agpgart interface v0.1031604worker2 # [ 0.686553] ACPI: bus type drm_connector registered1605worker1 # [ 0.697555] ACPI: bus type drm_connector registered1606lb # [ 0.669980] Asymmetric key parser 'x509' registered1607client # [ 0.683653] io scheduler kyber registered1608lb # [ 0.670832] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1609client # [ 0.685104] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1610worker2 # [ 0.689622] usbcore: registered new interface driver usbserial_generic1611worker1 # [ 0.700874] usbcore: registered new interface driver usbserial_generic1612worker2 # [ 0.690670] usbserial: USB Serial support registered for generic1613client # [ 0.686402] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1614worker1 # [ 0.701992] usbserial: USB Serial support registered for generic1615lb # [ 0.676077] io scheduler mq-deadline registered1616lb # [ 0.676851] io scheduler kyber registered1617worker2 # [ 0.694186] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1618worker2 # [ 0.694186] Please enable it if your BIOS has the CPPC option.1619worker1 # [ 0.705745] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1620worker1 # [ 0.705745] Please enable it if your BIOS has the CPPC option.1621client # [ 0.692459] Linux agpgart interface v0.1031622worker2 # [ 0.696447] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1623client # [ 0.693211] ACPI: bus type drm_connector registered1624lb # [ 0.680475] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1625worker1 # [ 0.708045] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1626lb # [ 0.681765] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1627worker2 # [ 0.700297] drop_monitor: Initializing network drop monitor service1628client # [ 0.696095] usbcore: registered new interface driver usbserial_generic1629worker2 # [ 0.701446] NET: Registered PF_INET6 protocol family1630worker1 # [ 0.711854] drop_monitor: Initializing network drop monitor service1631client # [ 0.697185] usbserial: USB Serial support registered for generic1632worker1 # [ 0.713089] NET: Registered PF_INET6 protocol family1633lb # [ 0.685742] Linux agpgart interface v0.1031634lb # [ 0.686525] ACPI: bus type drm_connector registered1635worker2 # [ 0.704637] Segment Routing with IPv61636worker2 # [ 0.705332] In-situ OAM (IOAM) with IPv61637worker1 # [ 0.716249] Segment Routing with IPv61638client # [ 0.700652] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1639client # [ 0.700652] Please enable it if your BIOS has the CPPC option.1640worker1 # [ 0.717750] In-situ OAM (IOAM) with IPv61641worker2 # [ 0.707469] IPI shorthand broadcast: enabled1642client # [ 0.702958] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1643worker1 # [ 0.720017] IPI shorthand broadcast: enabled1644lb # [ 0.692080] usbcore: registered new interface driver usbserial_generic1645lb # [ 0.693148] usbserial: USB Serial support registered for generic1646client # [ 0.706781] drop_monitor: Initializing network drop monitor service1647lb # [ 0.694144] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1648client # [ 0.707988] NET: Registered PF_INET6 protocol family1649lb # [ 0.694144] Please enable it if your BIOS has the CPPC option.1650lb # [ 0.696354] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1651client # [ 0.711112] Segment Routing with IPv61652worker2 # [ 0.716672] sched_clock: Marking stable (584015160, 132154684)->(794177979, -78008135)1653client # [ 0.713674] In-situ OAM (IOAM) with IPv61654worker1 # [ 0.728327] sched_clock: Marking stable (592016852, 135703337)->(800321268, -72601079)1655lb # [ 0.701092] drop_monitor: Initializing network drop monitor service1656client # [ 0.714746] IPI shorthand broadcast: enabled1657lb # [ 0.702274] NET: Registered PF_INET6 protocol family1658worker2 # [ 0.721277] registered taskstats version 11659lb # [ 0.704427] Segment Routing with IPv61660worker1 # [ 0.732925] registered taskstats version 11661worker2 # [ 0.722256] Loading compiled-in X.509 certificates1662worker1 # [ 0.733864] Loading compiled-in X.509 certificates1663lb # [ 0.706970] In-situ OAM (IOAM) with IPv61664lb # [ 0.708018] IPI shorthand broadcast: enabled1665client # [ 0.722960] sched_clock: Marking stable (589013581, 133613152)->(798510987, -75884254)1666client # [ 0.726905] registered taskstats version 11667client # [ 0.727876] Loading compiled-in X.509 certificates1668lb # [ 0.716149] sched_clock: Marking stable (579014278, 136924406)->(791727132, -75788448)1669lb # [ 0.721188] registered taskstats version 11670worker2 # [ 0.738781] Demotion targets for Node 0: null1671lb # [ 0.722142] Loading compiled-in X.509 certificates1672worker1 # [ 0.752733] Demotion targets for Node 0: null1673worker2 # [ 0.742224] Key type .fscrypt registered1674worker2 # [ 0.742873] Key type fscrypt-provisioning registered1675worker1 # [ 0.753639] Key type .fscrypt registered1676worker1 # [ 0.754385] Key type fscrypt-provisioning registered1677worker2 # [ 0.743807] ima: No TPM chip found, activating TPM-bypass!1678worker1 # [ 0.755355] ima: No TPM chip found, activating TPM-bypass!1679worker2 # [ 0.746189] ima: Allocated hash algorithm: sha11680worker2 # [ 0.746970] ima: No architecture policies found1681worker1 # [ 0.759739] ima: Allocated hash algorithm: sha11682worker2 # [ 0.749184] PM: Magic number: 10:951:5251683worker1 # [ 0.760546] ima: No architecture policies found1684worker2 # [ 0.749869] PM: hash matches drivers/base/power/main.c:8931685worker1 # [ 0.762746] PM: Magic number: 10:951:5251686client # [ 0.747646] Demotion targets for Node 0: null1687client # [ 0.748542] Key type .fscrypt registered1688worker1 # [ 0.763462] PM: hash matches drivers/base/power/main.c:8931689client # [ 0.749241] Key type fscrypt-provisioning registered1690worker2 # [ 0.753716] RAS: Correctable Errors collector initialized.1691client # [ 0.750225] ima: No TPM chip found, activating TPM-bypass!1692worker1 # [ 0.767326] RAS: Correctable Errors collector initialized.1693lb # [ 0.739954] Demotion targets for Node 0: null1694lb # [ 0.740826] Key type .fscrypt registered1695lb # [ 0.741534] Key type fscrypt-provisioning registered1696client # [ 0.754656] ima: Allocated hash algorithm: sha11697lb # [ 0.742477] ima: No TPM chip found, activating TPM-bypass!1698client # [ 0.755471] ima: No architecture policies found1699client # [ 0.757640] PM: Magic number: 10:951:5251700worker2 # [ 0.762331] clk: Disabling unused clocks1701client # [ 0.758398] PM: hash matches drivers/base/power/main.c:8931702worker2 # [ 0.763025] PM: genpd: Disabling unused power domains1703lb # [ 0.746953] ima: Allocated hash algorithm: sha11704lb # [ 0.747710] ima: No architecture policies found1705worker1 # [ 0.776160] clk: Disabling unused clocks1706lb # [ 0.748665] PM: Magic number: 10:951:5251707lb # [ 0.749396] PM: hash matches drivers/base/power/main.c:8931708client # [ 0.762351] RAS: Correctable Errors collector initialized.1709worker1 # [ 0.778738] PM: genpd: Disabling unused power domains1710lb # [ 0.753780] RAS: Correctable Errors collector initialized.1711client # [ 0.771079] clk: Disabling unused clocks1712client # [ 0.773643] PM: genpd: Disabling unused power domains1713lb # [ 0.762405] clk: Disabling unused clocks1714lb # [ 0.764952] PM: genpd: Disabling unused power domains1715worker2 # [ 0.927821] Freeing initrd memory: 29092K1716worker2 # [ 0.930938] Freeing unused decrypted memory: 2028K1717worker1 # [ 0.943803] Freeing initrd memory: 29092K1718worker2 # [ 0.933519] Freeing unused kernel image (initmem) memory: 3644K1719worker2 # [ 0.934582] Write protecting the kernel read-only data: 32768k1720worker1 # [ 0.946982] Freeing unused decrypted memory: 2028K1721worker2 # [ 0.936415] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker2 # [ 0.937895] Freeing unused kernel image (rodata/data gap) memory: 752K1723worker1 # [ 0.949536] Freeing unused kernel image (initmem) memory: 3644K1724worker1 # [ 0.950664] Write protecting the kernel read-only data: 32768k1725lb # [ 0.924502] Freeing initrd memory: 29092K1726client # [ 0.937602] Freeing initrd memory: 29096K1727worker1 # [ 0.952591] Freeing unused kernel image (text/rodata gap) memory: 1208K1728worker1 # [ 0.954136] Freeing unused kernel image (rodata/data gap) memory: 752K1729lb # [ 0.927640] Freeing unused decrypted memory: 2028K1730client # [ 0.940982] Freeing unused decrypted memory: 2028K1731lb # [ 0.930229] Freeing unused kernel image (initmem) memory: 3644K1732client # [ 0.943880] Freeing unused kernel image (initmem) memory: 3644K1733lb # [ 0.931294] Write protecting the kernel read-only data: 32768k1734client # [ 0.944995] Write protecting the kernel read-only data: 32768k1735lb # [ 0.933120] Freeing unused kernel image (text/rodata gap) memory: 1208K1736client # [ 0.946993] Freeing unused kernel image (text/rodata gap) memory: 1208K1737lb # [ 0.934603] Freeing unused kernel image (rodata/data gap) memory: 752K1738client # [ 0.948584] Freeing unused kernel image (rodata/data gap) memory: 752K1739worker2 # [ 0.989392] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740worker2 # [ 0.990523] Run /init as init process1741worker1 # [ 1.005211] x86/mm: Checked W+X mappings: passed, no W+X pages found.1742worker1 # [ 1.006329] Run /init as init process1743worker2 # [ 1.000862] systemd[1]: Inserted module 'autofs4'1744lb # [ 0.985848] x86/mm: Checked W+X mappings: passed, no W+X pages found.1745lb # [ 0.986918] Run /init as init process1746client # [ 0.999943] x86/mm: Checked W+X mappings: passed, no W+X pages found.1747client # [ 1.001061] Run /init as init process1748worker1 # [ 1.016572] systemd[1]: Inserted module 'autofs4'1749lb # [ 0.997014] systemd[1]: Inserted module 'autofs4'1750client # [ 1.011796] systemd[1]: Inserted module 'autofs4'1751worker2 # [ 1.017199] fuse: init (API version 7.45)1752worker1 # [ 1.033079] fuse: init (API version 7.45)1753worker2 # [ 1.022998] ACPI: \_SB_.GSIG: Enabled at IRQ 221754worker2 # [ 1.025570] ACPI: \_SB_.GSIH: Enabled at IRQ 231755worker1 # [ 1.038987] ACPI: \_SB_.GSIG: Enabled at IRQ 221756worker2 # [ 1.028809] ACPI: \_SB_.GSIE: Enabled at IRQ 201757lb # [ 1.013247] fuse: init (API version 7.45)1758worker1 # [ 1.041602] ACPI: \_SB_.GSIH: Enabled at IRQ 231759worker2 # [ 1.031353] ACPI: \_SB_.GSIF: Enabled at IRQ 211760client # [ 1.028569] fuse: init (API version 7.45)1761worker1 # [ 1.044980] ACPI: \_SB_.GSIE: Enabled at IRQ 201762lb # [ 1.019016] ACPI: \_SB_.GSIG: Enabled at IRQ 221763worker1 # [ 1.047471] ACPI: \_SB_.GSIF: Enabled at IRQ 211764client # [ 1.034571] ACPI: \_SB_.GSIG: Enabled at IRQ 221765lb # [ 1.021869] ACPI: \_SB_.GSIH: Enabled at IRQ 231766client # [ 1.037195] ACPI: \_SB_.GSIH: Enabled at IRQ 231767lb # [ 1.025723] ACPI: \_SB_.GSIE: Enabled at IRQ 201768client # [ 1.040335] ACPI: \_SB_.GSIE: Enabled at IRQ 201769lb # [ 1.028246] ACPI: \_SB_.GSIF: Enabled at IRQ 211770client # [ 1.042766] ACPI: \_SB_.GSIF: Enabled at IRQ 211771worker2 # [ 1.056843] systemd[1]: Successfully made /usr/ read-only.1772worker1 # [ 1.074835] systemd[1]: Successfully made /usr/ read-only.1773lb # [ 1.056083] systemd[1]: Successfully made /usr/ read-only.1774client # [ 1.069760] systemd[1]: Successfully made /usr/ read-only.1775worker1 # [ 1.410526] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776worker2 # [ 1.393245] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1777worker1 # [ 1.415606] systemd[1]: Detected virtualization kvm.1778worker1 # [ 1.416517] systemd[1]: Detected architecture x86-64.1779worker2 # [ 1.405141] systemd[1]: Detected virtualization kvm.1780worker1 # [ 1.417468] systemd[1]: Running in initrd.1781worker1 # [ 1.418659] systemd[1]: Initializing machine ID from random generator.1782worker2 # [ 1.407303] systemd[1]: Detected architecture x86-64.1783worker1 # [ 1.419942] systemd[1]: Hostname set to <worker1>.1784worker2 # [ 1.409453] systemd[1]: Running in initrd.1785worker2 # [ 1.411891] systemd[1]: Initializing machine ID from random generator.1786client # [ 1.405421] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1787client # [ 1.410242] systemd[1]: Detected virtualization kvm.1788client # [ 1.411139] systemd[1]: Detected architecture x86-64.1789worker2 # [ 1.414782] systemd[1]: Hostname set to <worker2>.1790client # [ 1.412047] systemd[1]: Running in initrd.1791client # [ 1.413183] systemd[1]: Initializing machine ID from random generator.1792client # [ 1.414418] systemd[1]: Hostname set to <client>.1793lb # [ 1.392809] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1794lb # [ 1.404829] systemd[1]: Detected virtualization kvm.1795lb # [ 1.406996] systemd[1]: Detected architecture x86-64.1796lb # [ 1.409201] systemd[1]: Running in initrd.1797lb # [ 1.411733] systemd[1]: Initializing machine ID from random generator.1798lb # [ 1.414610] systemd[1]: Hostname set to <lb>.1799worker1 # [ 1.614329] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker2 # [ 1.619290] systemd[1]: bpf-restrict-fs: LSM BPF program attached1801client # [ 1.634954] systemd[1]: bpf-restrict-fs: LSM BPF program attached1802worker1 # [ 1.655203] systemd[1]: Queued start job for default target Initrd Default Target.1803worker1 # [ 1.659136] systemd[1]: Created slice Slice /system/modprobe.1804worker1 # [ 1.660359] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1805worker1 # [ 1.661844] systemd[1]: Expecting device /dev/disk/by-label/nixos...1806worker1 # [ 1.663019] systemd[1]: Reached target Path Units.1807worker1 # [ 1.663902] systemd[1]: Reached target Slice Units.1808worker1 # [ 1.664770] systemd[1]: Reached target Swaps.1809worker1 # [ 1.665614] systemd[1]: Reached target Timer Units.1810lb # [ 1.637062] systemd[1]: bpf-restrict-fs: LSM BPF program attached1811worker1 # [ 1.666650] systemd[1]: Listening on D-Bus System Message Bus Socket.1812worker1 # [ 1.667925] systemd[1]: Listening on Journal Socket (/dev/log).1813worker2 # [ 1.657486] systemd[1]: Queued start job for default target Initrd Default Target.1814worker1 # [ 1.669110] systemd[1]: Listening on Journal Sockets.1815worker1 # [ 1.670125] systemd[1]: Listening on udev Control Socket.1816worker1 # [ 1.671197] systemd[1]: Listening on udev Kernel Socket.1817worker1 # [ 1.672177] systemd[1]: Reached target Socket Units.1818worker2 # [ 1.661474] systemd[1]: Created slice Slice /system/modprobe.1819worker2 # [ 1.662646] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1820worker1 # [ 1.674164] systemd[1]: Starting Create List of Static Device Nodes...1821worker2 # [ 1.664132] systemd[1]: Expecting device /dev/disk/by-label/nixos...1822worker2 # [ 1.665280] systemd[1]: Reached target Path Units.1823worker2 # [ 1.666197] systemd[1]: Reached target Slice Units.1824worker2 # [ 1.667095] systemd[1]: Reached target Swaps.1825worker2 # [ 1.667922] systemd[1]: Reached target Timer Units.1826worker1 # [ 1.678785] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1827worker2 # [ 1.668913] systemd[1]: Listening on D-Bus System Message Bus Socket.1828worker2 # [ 1.670222] systemd[1]: Listening on Journal Socket (/dev/log).1829worker2 # [ 1.671434] systemd[1]: Listening on Journal Sockets.1830worker2 # [ 1.672482] systemd[1]: Listening on udev Control Socket.1831worker2 # [ 1.673534] systemd[1]: Listening on udev Kernel Socket.1832worker2 # [ 1.674554] systemd[1]: Reached target Socket Units.1833worker1 # [ 1.685933] systemd[1]: Starting Load Kernel Module configfs...1834worker2 # [ 1.676355] systemd[1]: Starting Create List of Static Device Nodes...1835worker2 # [ 1.680238] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1836worker2 # [ 1.688257] systemd[1]: Starting Load Kernel Module configfs...1837worker1 # [ 1.700794] systemd[1]: Starting Journal Service...1838client # [ 1.689303] systemd[1]: Queued start job for default target Initrd Default Target.1839client # [ 1.693965] systemd[1]: Created slice Slice /system/modprobe.1840client # [ 1.695408] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1841client # [ 1.696853] systemd[1]: Expecting device /dev/disk/by-label/nixos...1842client # [ 1.697997] systemd[1]: Reached target Path Units.1843worker2 # [ 1.702790] systemd[1]: Starting Journal Service...1844client # [ 1.698907] systemd[1]: Reached target Slice Units.1845client # [ 1.699791] systemd[1]: Reached target Swaps.1846client # [ 1.700591] systemd[1]: Reached target Timer Units.1847client # [ 1.701575] systemd[1]: Listening on D-Bus System Message Bus Socket.1848client # [ 1.702889] systemd[1]: Listening on Journal Socket (/dev/log).1849client # [ 1.704102] systemd[1]: Listening on Journal Sockets.1850client # [ 1.705143] systemd[1]: Listening on udev Control Socket.1851lb # [ 1.692158] systemd[1]: Queued start job for default target Initrd Default Target.1852client # [ 1.706156] systemd[1]: Listening on udev Kernel Socket.1853client # [ 1.707154] systemd[1]: Reached target Socket Units.1854client # [ 1.708995] systemd[1]: Starting Create List of Static Device Nodes...1855lb # [ 1.696404] systemd[1]: Created slice Slice /system/modprobe.1856lb # [ 1.697709] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1857worker1 # [ 1.714797] systemd[1]: Starting Load Kernel Modules...1858lb # [ 1.699228] systemd[1]: Expecting device /dev/disk/by-label/nixos...1859lb # [ 1.700380] systemd[1]: Reached target Path Units.1860client # [ 1.713764] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1861lb # [ 1.701294] systemd[1]: Reached target Slice Units.1862lb # [ 1.702181] systemd[1]: Reached target Swaps.1863lb # [ 1.703025] systemd[1]: Reached target Timer Units.1864lb # [ 1.704025] systemd[1]: Listening on D-Bus System Message Bus Socket.1865lb # [ 1.705280] systemd[1]: Listening on Journal Socket (/dev/log).1866lb # [ 1.706466] systemd[1]: Listening on Journal Sockets.1867worker1 # [ 1.734532] netfs: FS-Cache loaded1868lb # [ 1.707523] systemd[1]: Listening on udev Control Socket.1869worker2 # [ 1.714267] systemd[1]: Starting Load Kernel Modules...1870lb # [ 1.708642] systemd[1]: Listening on udev Kernel Socket.1871client # [ 1.721698] systemd[1]: Starting Load Kernel Module configfs...1872lb # [ 1.709604] systemd[1]: Reached target Socket Units.1873worker1 # [ 1.739379] 9pnet: Installing 9P2000 support1874lb # [ 1.711528] systemd[1]: Starting Create List of Static Device Nodes...1875lb # [ 1.716010] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1876worker2 # [ 1.733677] netfs: FS-Cache loaded1877worker1 # [ 1.747364] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1878worker2 # [ 1.738451] 9pnet: Installing 9P2000 support1879lb # [ 1.722658] systemd[1]: Starting Load Kernel Module configfs...1880client # [ 1.735702] systemd[1]: Starting Journal Service...1881worker1 # [ 1.755938] systemd[1]: Starting Coldplug All udev Devices...1882worker2 # [ 1.746096] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1883worker2 # [ 1.755415] systemd[1]: Starting Coldplug All udev Devices...1884lb # [ 1.739634] systemd[1]: Starting Journal Service...1885worker1 # [ 1.770842] systemd[1]: Finished Create List of Static Device Nodes.1886client # [ 1.749736] systemd[1]: Starting Load Kernel Modules...1887worker1 # [ 1.778417] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1888worker2 # [ 1.771312] systemd[1]: Finished Create List of Static Device Nodes.1889worker1 # [ 1.782485] systemd-journald[67]: Collecting audit messages is disabled.1890client # [ 1.770085] netfs: FS-Cache loaded1891worker2 # [ 1.776082] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1892worker1 # [ 1.787150] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1893lb # [ 1.753014] systemd[1]: Starting Load Kernel Modules...1894worker1 # [ 1.792183] systemd[1]: modprobe@configfs.service: Deactivated successfully.1895worker2 # [ 1.783738] systemd-journald[67]: Collecting audit messages is disabled.1896client # [ 1.779944] 9pnet: Installing 9P2000 support1897worker2 # [ 1.784886] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1898worker1 # [ 1.798100] systemd[1]: Finished Load Kernel Module configfs.1899lb # [ 1.771405] netfs: FS-Cache loaded1900worker1 # [ 1.802196] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1901worker2 # [ 1.792487] systemd[1]: modprobe@configfs.service: Deactivated successfully.1902lb # [ 1.776317] 9pnet: Installing 9P2000 support1903client # [ 1.789795] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1904worker1 # [ 1.804875] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1905worker2 # [ 1.797228] systemd[1]: Finished Load Kernel Module configfs.1906worker1 # [ 1.810747] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1907lb # [ 1.784832] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1908client # [ 1.798074] systemd-journald[67]: Collecting audit messages is disabled.1909worker2 # [ 1.802621] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1910client # [ 1.800821] systemd[1]: Starting Coldplug All udev Devices...1911worker1 # [ 1.815769] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1912worker2 # [ 1.804822] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1913lb # [ 1.791747] systemd[1]: Starting Coldplug All udev Devices...1914worker2 # [ 1.812195] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1915worker2 # [ 1.815295] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1916client # [ 1.813732] systemd[1]: Finished Create List of Static Device Nodes.1917lb # [ 1.807042] systemd[1]: Finished Create List of Static Device Nodes.1918client # [ 1.820242] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1919lb # [ 1.812595] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1920client # [ 1.828018] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1921worker1 # [ 1.843307] systemd[1]: Finished Load Kernel Modules.1922lb # [ 1.820399] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1923client # [ 1.834028] systemd[1]: modprobe@configfs.service: Deactivated successfully.1924worker1 # [ 1.851995] systemd[1]: Starting Apply Kernel Variables...1925lb # [ 1.825757] systemd-journald[67]: Collecting audit messages is disabled.1926lb # [ 1.827392] systemd[1]: modprobe@configfs.service: Deactivated successfully.1927client # [ 1.841024] systemd[1]: Finished Load Kernel Module configfs.1928worker2 # [ 1.845719] systemd[1]: Finished Load Kernel Modules.1929client # [ 1.844164] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1930lb # [ 1.834276] systemd[1]: Finished Load Kernel Module configfs.1931client # [ 1.848088] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1932worker2 # [ 1.854381] systemd[1]: Starting Apply Kernel Variables...1933lb # [ 1.837605] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1934lb # [ 1.839875] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1935client # [ 1.853650] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1936worker1 # [ 1.869054] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1937client # [ 1.860714] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1938worker2 # [ 1.865305] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1939lb # [ 1.848122] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1940worker1 # [ 1.877933] systemd[1]: Starting Create Static Device Nodes in /dev...1941lb # [ 1.852173] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1942worker2 # [ 1.874559] systemd[1]: Starting Create Static Device Nodes in /dev...1943worker1 # [ 1.892825] systemd[1]: Finished Apply Kernel Variables.1944client # [ 1.884337] systemd[1]: Finished Load Kernel Modules.1945worker1 # [ 1.905843] systemd[1]: Started Journal Service.1946client # [ 1.892832] systemd[1]: Starting Apply Kernel Variables...1947lb # [ 1.880506] systemd[1]: Finished Load Kernel Modules.1948worker1 # [ 1.768901] systemd-modules-load[68]: Inserted module 'dm_mod'1949worker2 # [ 1.898300] systemd[1]: Finished Apply Kernel Variables.1950worker1 # [ 1.773989] systemd-modules-load[68]: Inserted module 'virtio_balloon'1951worker1 # [ 1.775811] systemd-modules-load[68]: Inserted module 'virtio_gpu'1952lb # [ 1.887014] systemd[1]: Starting Apply Kernel Variables...1953worker2 # [ 1.773192] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.906405] systemd[1]: Started Journal Service.1954worker2 # 1955worker2 # [ 1.776297] systemd-modules-load[68]: Inserted module 'virtio_balloon'1956worker2 # [ 1.779522] systemd-modules-load[68]: Inserted module 'virtio_gpu'1957worker1 # [ 1.789204] systemd[1]: Finished Create Static Device Nodes in /dev.1958client # [ 1.909258] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1959worker1 # [ 1.792092] systemd[1]: Reached target Preparation for Local File Systems.1960worker1 # [ 1.793114] systemd[1]: Reached target Local File Systems.1961worker1 # [ 1.795761] systemd[1]: Starting Create System Files and Directories...1962lb # [ 1.904088] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1963worker2 # [ 1.789730] systemd[1]: Finished Create Static Device Nodes in /dev.1964worker2 # [ 1.792125] systemd[1]: Reached target Preparation for Local File Systems.1965client # [ 1.919855] systemd[1]: Starting Create Static Device Nodes in /dev...1966worker1 # [ 1.801040] systemd[1]: Starting Rule-based Manager for Device Events and Files...1967worker2 # [ 1.794944] systemd[1]: Reached target Local File Systems.1968client # [ 1.784839] systemd-modules-load[68]: Inserted module 'dm_mod'1969worker2 # [ 1.796263] systemd[1]: Starting Create System Files and Directories...1970client # [ 1.790359] systemd-modules-load[68]: Inserted module 'virtio_balloon'1971client # [ 1.791558] systemd-modules-load[68]: Inserted module 'virtio_gpu'1972lb # [ 1.913381] systemd[1]: Starting Create Static Device Nodes in /dev...1973client # [ 1.930851] systemd[1]: Started Journal Service.1974worker2 # [ 1.804071] systemd[1]: Starting Rule-based Manager for Device Events and Files...1975lb # [ 1.927087] systemd[1]: Finished Apply Kernel Variables.1976client # [ 1.810120] systemd[1]: Finished Apply Kernel Variables.1977worker1 # [ 1.833096] systemd[1]: Finished Create System Files and Directories.1978lb # [ 1.943821] systemd[1]: Started Journal Service.1979client # [ 1.823251] systemd[1]: Finished Create Static Device Nodes in /dev.1980worker2 # [ 1.829533] systemd[1]: Finished Create System Files and Directories.1981client # [ 1.824387] systemd[1]: Reached target Preparation for Local File Systems.1982client # [ 1.825506] systemd[1]: Reached target Local File Systems.1983lb # [ 1.810251] systemd-modules-load[68]: Inserted module 'dm_mod'1984lb # [ 1.811655] systemd-modules-load[68]: Inserted module 'virtio_balloon'1985client # [ 1.829132] systemd[1]: Starting Create System Files and Directories...1986lb # [ 1.815310] systemd-modules-load[68]: Inserted module 'virtio_gpu'1987client # [ 1.834265] systemd[1]: Starting Rule-based Manager for Device Events and Files...1988lb # [ 1.819082] systemd[1]: Finished Create Static Device Nodes in /dev.1989lb # [ 1.826163] systemd[1]: Reached target Preparation for Local File Systems.1990lb # [ 1.827184] systemd[1]: Reached target Local File Systems.1991worker1 # [ 1.856876] systemd-udevd[83]: Using default interface naming scheme 'v261'.1992worker2 # [ 1.851248] systemd-udevd[83]: Using default interface naming scheme 'v261'.1993lb # [ 1.830067] systemd[1]: Starting Create System Files and Directories...1994lb # [ 1.834533] systemd[1]: Starting Rule-based Manager for Device Events and Files...1995worker2 # [ 1.874597] systemd[1]: Started Rule-based Manager for Device Events and Files.1996client # [ 1.869096] systemd[1]: Finished Create System Files and Directories.1997worker1 # [ 1.883087] systemd[1]: Started Rule-based Manager for Device Events and Files.1998lb # [ 1.862469] systemd[1]: Finished Create System Files and Directories.1999client # [ 1.887248] systemd-udevd[83]: Using default interface naming scheme 'v261'.2000lb # [ 1.885934] systemd-udevd[83]: Using default interface naming scheme 'v261'.

loading 1,813 hidden lines…

3814worker2 # [ 10.633282] systemd[1]: Starting Extra networking commands....3815client # [ 10.692540] dhcpcd[555]: dhcpcd-10.3.2 starting3816client # [ 10.701102] dhcpcd[562]: dev: loaded udev3817client # [ 10.856954] 8021q: 802.1Q VLAN Support v1.83818worker2 # [ 10.731436] network-addresses-eth1-start[629]: adding address 192.168.1.4/24... done3819worker2 # [ 10.748681] network-addresses-eth1-start[629]: adding address 2001:db8:1::4/64... done3820client # [ 10.753062] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.3821worker2 # [ 10.768079] systemd[1]: Finished Address configuration of eth1.3822worker2 # [ 10.794900] dhcpcd[637]: dhcpcd-10.3.2 starting3823worker2 # [ 10.804302] dhcpcd[691]: dev: loaded udev3824worker2 # [ 10.813608] systemd[1]: Finished Extra networking commands..3825worker2 # [ 10.815136] systemd[1]: Reached target Network.3826worker2 # [ 10.818434] systemd[1]: Starting Permit User Sessions...3827client # [ 10.946131] cfg80211: Loading compiled-in X.509 certificates for regulatory database3828worker2 # [ 10.961801] 8021q: 802.1Q VLAN Support v1.83829worker2 # [ 10.962510] 8021q: adding VLAN 0 to HW filter on device eth13830client # [ 10.977762] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'3831client # [ 10.978430] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'3832worker2 # [ 10.852093] systemd[1]: Finished Permit User Sessions.3833client # [ 10.982673] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -23834worker2 # [ 10.855491] systemd[1]: Started Getty on tty1.3835client # [ 10.983478] cfg80211: failed to load regulatory.db3836worker2 # [ 10.858136] systemd[1]: Reached target Login Prompts.3837worker2 # [ 10.868757] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.3838client # [ 10.893276] systemd[1]: Finished nftables firewall.3839client # [ 10.894713] systemd[1]: Reached target Preparation for Network.3840client # [ 10.900597] systemd[1]: Starting Address configuration of eth1...3841client # [ 10.903780] systemd[1]: Starting Extra networking commands....3842worker2 # [ 11.054889] cfg80211: Loading compiled-in X.509 certificates for regulatory database3843client # [ 11.055719] 8021q: adding VLAN 0 to HW filter on device eth03844client # [ 10.923466] dhcpcd[562]: eth0: waiting for carrier3845client # [ 10.924678] dhcpcd[562]: eth0: carrier acquired3846client # [ 10.937332] dhcpcd[562]: DUID 00:01:00:01:32:42:64:b3:52:54:00:12:34:563847client # [ 10.938572] dhcpcd[562]: eth0: IAID 00:12:34:563848worker2 # [ 11.076867] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'3849worker2 # [ 11.077649] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'3850client # [ 10.939478] dhcpcd[562]: eth0: adding address fe80::5054:ff:fe12:34563851worker2 # [ 11.080503] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -23852worker2 # [ 11.081326] cfg80211: failed to load regulatory.db3853client # [ 11.109340] 8021q: adding VLAN 0 to HW filter on device eth13854worker2 # [ 11.122959] 8021q: adding VLAN 0 to HW filter on device eth03855worker2 # [ 10.992040] dhcpcd[691]: eth0: waiting for carrier3856worker2 # [ 10.992767] dhcpcd[691]: eth0: carrier acquired3857client # [ 10.990381] network-addresses-eth1-start[585]: adding address 192.168.1.1/24... done3858worker2 # [ 11.000573] dhcpcd[691]: DUID 00:01:00:01:32:42:64:b3:52:54:00:12:34:563859worker2 # [ 11.001559] dhcpcd[691]: eth0: IAID 00:12:34:563860worker2 # [ 11.002266] dhcpcd[691]: eth0: adding address fe80::5054:ff:fe12:34563861client # [ 11.002773] network-addresses-eth1-start[585]: adding address 2001:db8:1::1/64... done3862client # [ 11.019156] systemd[1]: Finished Address configuration of eth1.3863client # [ 11.041086] systemd[1]: Finished Extra networking commands..3864client # [ 11.043141] systemd[1]: Reached target Network.3865client # [ 11.046094] systemd[1]: Starting Permit User Sessions...3866client # [ 11.072797] systemd[1]: Finished Permit User Sessions.3867client # [ 11.076790] systemd[1]: Started Getty on tty1.3868client # [ 11.077872] systemd[1]: Reached target Login Prompts.3869client: (finished: waiting for unit network-addresses-eth1.service, in 12.24 seconds)3870client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }3871client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3872lb: waiting for unit network-addresses-eth1.service3873lb: waiting for the VM to finish booting3874lb: Guest shell says: b'Spawning backdoor root shell...\n'3875lb: connected to guest root shell3876lb: (connecting took 0.00 seconds)3877lb: (finished: waiting for the VM to finish booting, in 0.00 seconds)3878lb: (finished: waiting for unit network-addresses-eth1.service, in 0.06 seconds)3879lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }3880lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3881worker1: waiting for unit network-addresses-eth1.service3882worker1: waiting for the VM to finish booting3883worker1: Guest shell says: b'Spawning backdoor root shell...\n'3884worker1: connected to guest root shell3885worker1: (connecting took 0.00 seconds)3886worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds)3887worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.06 seconds)3888worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }3889worker2 # [ 11.767970] dhcpcd[691]: eth0: soliciting a DHCP lease3890worker2 # [ 11.917278] NET: Registered PF_PACKET protocol family3891worker2 # [ 11.789578] dhcpcd[691]: eth0: offered 10.0.2.15 from 10.0.2.23892worker2 # [ 11.793128] dhcpcd[691]: eth0: probing address 10.0.2.15/243893worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3894worker2: waiting for unit network-addresses-eth1.service3895worker2: waiting for the VM to finish booting3896worker2: Guest shell says: b'Spawning backdoor root shell...\n'3897worker2: connected to guest root shell3898worker2: (connecting took 0.00 seconds)3899worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds)3900lb # [ 11.787075] dhcpcd[712]: eth0: soliciting a DHCP lease3901lb # [ 11.938191] NET: Registered PF_PACKET protocol family3902lb # [ 11.806478] dhcpcd[712]: eth0: offered 10.0.2.15 from 10.0.2.23903lb # [ 11.809147] dhcpcd[712]: eth0: probing address 10.0.2.15/243904worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.07 seconds)3905worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }3906worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.03 seconds)3907lb: waiting for unit niks3.service3908lb # [ 12.221964] postgresql-pre-start[727]: selecting default time zone ... UTC3909lb # [ 12.226621] postgresql-pre-start[727]: creating configuration files ... ok3910lb # [ 12.366491] dhcpcd[712]: eth0: soliciting an IPv6 router3911lb # [ 12.367789] dhcpcd[712]: eth0: Router Advertisement from fe80::23912lb # [ 12.369429] dhcpcd[712]: eth0: adding address fec0::5054:ff:fe12:3456/643913lb # [ 12.370334] dhcpcd[712]: eth0: adding route to fec0::/643914lb # [ 12.371282] dhcpcd[712]: eth0: adding default route via fe80::23915worker1 # [ 12.422400] dhcpcd[691]: eth0: soliciting an IPv6 router3916worker1 # [ 12.425457] dhcpcd[691]: eth0: Router Advertisement from fe80::23917worker1 # [ 12.427582] dhcpcd[691]: eth0: adding address fec0::5054:ff:fe12:3456/643918worker1 # [ 12.429840] dhcpcd[691]: eth0: adding route to fec0::/643919worker1 # [ 12.431754] dhcpcd[691]: eth0: adding default route via fe80::23920lb # [ 12.466832] postgresql-pre-start[727]: running bootstrap script ... ok3921client # [ 12.668470] dhcpcd[562]: eth0: soliciting a DHCP lease3922worker2 # [ 12.688822] dhcpcd[691]: eth0: soliciting an IPv6 router3923worker2 # [ 12.691328] dhcpcd[691]: eth0: Router Advertisement from fe80::23924worker2 # [ 12.693460] dhcpcd[691]: eth0: adding address fec0::5054:ff:fe12:3456/643925worker2 # [ 12.695618] dhcpcd[691]: eth0: adding route to fec0::/643926worker2 # [ 12.697524] dhcpcd[691]: eth0: adding default route via fe80::23927client # [ 12.825875] NET: Registered PF_PACKET protocol family3928client # [ 12.699202] dhcpcd[562]: eth0: soliciting an IPv6 router3929client # [ 12.702162] dhcpcd[562]: eth0: offered 10.0.2.15 from 10.0.2.23930client # [ 12.705178] dhcpcd[562]: eth0: probing address 10.0.2.15/243931client # [ 12.707128] dhcpcd[562]: eth0: Router Advertisement from fe80::23932client # [ 12.709438] dhcpcd[562]: eth0: adding address fec0::5054:ff:fe12:3456/643933client # [ 12.711598] dhcpcd[562]: eth0: adding route to fec0::/643934client # [ 12.713480] dhcpcd[562]: eth0: adding default route via fe80::23935lb # [ 13.030148] postgresql-pre-start[727]: performing post-bootstrap initialization ... ok3936lb # [ 13.169022] postgresql-pre-start[727]: syncing data to disk ... ok3937lb # [ 13.170142] postgresql-pre-start[727]: initdb: warning: enabling "trust" authentication for local connections3938lb # [ 13.171504] postgresql-pre-start[727]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3939lb # [ 13.173571] postgresql-pre-start[727]: Success. You can now start the database server using:3940lb # [ 13.174728] postgresql-pre-start[727]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3941lb # [ 13.297311] postgres[792]: [792] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3942lb # [ 13.302702] postgres[792]: [792] LOG: listening on IPv6 address "::1", port 54323943lb # [ 13.303744] postgres[792]: [792] LOG: listening on IPv4 address "127.0.0.1", port 54323944lb # [ 13.306229] postgres[792]: [792] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3945lb # [ 13.319379] postgres[801]: [801] LOG: database system was shut down at 2026-09-20 09:32:37 GMT3946lb # [ 13.323574] postgres[792]: [792] LOG: database system is ready to accept connections3947lb # [ 13.329994] systemd[1]: Started PostgreSQL Server.3948lb # [ 13.336344] systemd[1]: Starting PostgreSQL Setup Scripts...3949lb # [ 13.538125] postgresql-setup-start[818]: CREATE DATABASE3950lb # [ 13.585776] postgresql-setup-start[823]: CREATE ROLE3951lb # [ 13.608340] postgresql-setup-start[825]: ALTER DATABASE3952lb # [ 13.614276] systemd[1]: Finished PostgreSQL Setup Scripts.3953lb # [ 13.616284] systemd[1]: Reached target PostgreSQL.3954worker1 # [ 15.113692] dhcpcd[691]: eth0: leased 10.0.2.15 for 86400 seconds3955worker1 # [ 15.116707] dhcpcd[691]: eth0: adding route to 10.0.2.0/243956worker1 # [ 15.118914] dhcpcd[691]: eth0: adding default route via 10.0.2.23957worker1 # [ 15.240860] systemd[1]: Started DHCP Client.3958worker1 # [ 15.243047] systemd[1]: Reached target Multi-User System.3959worker1 # [ 15.243841] systemd[1]: Startup finished in 874ms (kernel) + 4.084s (initrd) + 10.284s (userspace) = 15.242s.3960worker2 # [ 16.242687] dhcpcd[691]: eth0: leased 10.0.2.15 for 86400 seconds3961worker2 # [ 16.245454] dhcpcd[691]: eth0: adding route to 10.0.2.0/243962worker2 # [ 16.247652] dhcpcd[691]: eth0: adding default route via 10.0.2.23963worker2 # [ 16.364857] systemd[1]: Started DHCP Client.3964worker2 # [ 16.366885] systemd[1]: Reached target Multi-User System.3965worker2 # [ 16.367805] systemd[1]: Startup finished in 861ms (kernel) + 4.955s (initrd) + 10.549s (userspace) = 16.366s.3966lb # [ 16.462253] dhcpcd[712]: eth0: leased 10.0.2.15 for 86400 seconds3967lb # [ 16.465387] dhcpcd[712]: eth0: adding route to 10.0.2.0/243968lb # [ 16.467738] dhcpcd[712]: eth0: adding default route via 10.0.2.23969lb # [ 16.608593] systemd[1]: Started DHCP Client.3970lb # [ 16.610940] systemd[1]: Reached target Network is Online.3971lb # [ 16.615547] systemd[1]: Started Envoy reverse proxy.3972lb # [ 16.619821] systemd[1]: Started mock-oidc.service.3973lb # [ 16.727031] mock-oidc-server[911]: Mock OIDC Server running3974lb # [ 16.728125] mock-oidc-server[911]: OIDC Address: 192.168.1.2:80803975lb # [ 16.729243] mock-oidc-server[911]: Issue Address: 0.0.0.0:80813976lb # [ 16.730208] mock-oidc-server[911]: Issuer: http://192.168.1.2:8080/oidc3977lb # [ 16.731672] mock-oidc-server[911]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3978lb # [ 16.732915] mock-oidc-server[911]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3979lb # [ 16.734501] mock-oidc-server[911]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3980lb # [ 17.202118] envoy[910]: [2026-09-20 09:32:42.162][910][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3981lb # [ 17.203925] envoy[910]: [2026-09-20 09:32:42.162][910][info][main] [source/server/server.cc:469] statically linked extensions:3982lb # [ 17.205682] envoy[910]: [2026-09-20 09:32:42.162][910][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3983lb # [ 17.207832] envoy[910]: [2026-09-20 09:32:42.162][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13984lb # [ 17.209945] envoy[910]: [2026-09-20 09:32:42.162][910][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3985lb # [ 17.226615] envoy[910]: [2026-09-20 09:32:42.162][910][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3986lb # [ 17.228887] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3987lb # [ 17.231277] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3988lb # [ 17.235114] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3989lb # [ 17.239321] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3990lb # [ 17.242056] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3991lb # [ 17.243776] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3992lb # [ 17.245784] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3993lb # [ 17.247385] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3994lb # [ 17.249172] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3995lb # [ 17.251743] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3996lb # [ 17.254260] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3997lb # [ 17.257603] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3998client # [ 17.275526] dhcpcd[562]: eth0: leased 10.0.2.15 for 86400 seconds3999lb # [ 17.260089] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4000client # [ 17.277777] dhcpcd[562]: eth0: adding route to 10.0.2.0/244001lb # [ 17.262223] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4002lb # [ 17.263868] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4003client # [ 17.280927] dhcpcd[562]: eth0: adding default route via 10.0.2.24004lb # [ 17.265424] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4005lb # [ 17.269486] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4006lb # [ 17.273285] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4007lb # [ 17.275360] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4008lb # [ 17.279406] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4009lb # [ 17.281099] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4010lb # [ 17.284300] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4011lb # [ 17.286628] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4012lb # [ 17.288906] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4013lb # [ 17.290912] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4014lb # [ 17.292684] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4015lb # [ 17.297482] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4016lb # [ 17.299776] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4017lb # [ 17.304405] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4018lb # [ 17.306199] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4019lb # [ 17.308275] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4020lb # [ 17.310823] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4021lb # [ 17.316666] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4022lb # [ 17.318376] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4023lb # [ 17.321291] envoy[910]: [2026-09-20 09:32:42.163][910][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4024lb # [ 17.323919] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4025lb # [ 17.326452] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4026lb # [ 17.352551] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4027lb # [ 17.354411] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4028lb # [ 17.356396] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4029lb # [ 17.359361] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4030lb # [ 17.370695] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4031lb # [ 17.373378] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4032client # [ 17.394652] systemd[1]: Started DHCP Client.4033lb # [ 17.376392] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4034client # [ 17.396945] systemd[1]: Reached target Multi-User System.4035lb # [ 17.380509] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4036client # [ 17.397939] systemd[1]: Startup finished in 871ms (kernel) + 4.996s (initrd) + 11.530s (userspace) = 17.397s.4037lb # [ 17.382454] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24038lb # [ 17.384962] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4039lb # [ 17.387108] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4040lb # [ 17.388849] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4041lb # [ 17.391483] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4042lb # [ 17.393531] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4043lb # [ 17.395901] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4044lb # [ 17.397399] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4045lb # [ 17.399994] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4046lb # [ 17.402297] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24047lb # [ 17.404181] envoy[910]: [2026-09-20 09:32:42.167][910][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4048lb # [ 17.406041] envoy[910]: [2026-09-20 09:32:42.169][910][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4049lb # [ 17.408881] envoy[910]: [2026-09-20 09:32:42.169][910][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4050lb # [ 17.411390] envoy[910]: [2026-09-20 09:32:42.169][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24051lb # [ 17.413095] envoy[910]: [2026-09-20 09:32:42.169][910][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4052lb # [ 17.415375] envoy[910]: [2026-09-20 09:32:42.169][910][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84053lb # [ 17.417406] envoy[910]: [2026-09-20 09:32:42.171][910][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4054lb # [ 17.420398] envoy[910]: [2026-09-20 09:32:42.171][910][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4055lb # [ 17.423233] envoy[910]: [2026-09-20 09:32:42.171][910][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4056lb # [ 17.429131] envoy[910]: [2026-09-20 09:32:42.171][910][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4057lb # [ 17.431087] envoy[910]: [2026-09-20 09:32:42.177][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4058lb # [ 17.432854] envoy[910]: [2026-09-20 09:32:42.177][910][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4059lb # [ 17.435284] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4060lb # [ 17.441183] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4061lb # [ 17.443461] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4062lb # [ 17.445571] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4063lb # [ 17.447850] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4064lb # [ 17.449729] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4065lb # [ 17.455052] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4066lb # [ 17.461093] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4067lb # [ 17.462741] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4068lb # [ 17.464931] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4069lb # [ 17.467560] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4070lb # [ 17.469393] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4071lb # [ 17.472241] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4072lb # [ 17.480986] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4073lb # [ 17.483191] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4074lb # [ 17.485560] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4075lb # [ 17.487474] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4076lb # [ 17.489954] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4077lb # [ 17.491982] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4078lb # [ 17.494517] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4079lb # [ 17.500093] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4080lb # [ 17.501607] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4081lb # [ 17.503392] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4082lb # [ 17.505996] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4083lb # [ 17.512794] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4084lb # [ 17.515490] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4085lb # [ 17.518112] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4086lb # [ 17.520634] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4087lb # [ 17.523873] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4088lb # [ 17.526082] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4089lb # [ 17.528522] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4090lb # [ 17.531093] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4091lb # [ 17.533032] envoy[910]: [2026-09-20 09:32:42.178][910][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4092lb # [ 17.534782] envoy[910]: [2026-09-20 09:32:42.267][910][info][main] [source/server/server.cc:531] HTTP header map info:4093lb # [ 17.536040] envoy[910]: [2026-09-20 09:32:42.343][910][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4094lb # [ 17.548623] envoy[910]: [2026-09-20 09:32:42.344][910][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4095lb # [ 17.549965] envoy[910]: [2026-09-20 09:32:42.344][910][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4096lb # [ 17.557268] envoy[910]: [2026-09-20 09:32:42.344][910][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4097lb # [ 17.597052] envoy[910]: [2026-09-20 09:32:42.558][910][info][main] [source/server/server.cc:968] runtime: {}4098lb # [ 17.604124] envoy[910]: [2026-09-20 09:32:42.565][910][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4099lb # [ 17.606399] envoy[910]: [2026-09-20 09:32:42.568][910][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014100lb # [ 17.609126] envoy[910]: [2026-09-20 09:32:42.570][910][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014101lb # [ 17.613237] envoy[910]: [2026-09-20 09:32:42.574][910][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4102lb # [ 17.614830] envoy[910]: [2026-09-20 09:32:42.576][910][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4103lb # [ 17.616348] envoy[910]: [2026-09-20 09:32:42.576][910][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4104lb # [ 17.667300] envoy[910]: [2026-09-20 09:32:42.628][910][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4105worker1 # [ 17.701878] systemd[1]: Starting Nix worker-protocol over gRPC...4106worker2 # [ 17.696588] systemd[1]: Starting Nix worker-protocol over gRPC...4107lb # [ 17.688845] envoy[910]: [2026-09-20 09:32:42.650][910][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4108lb # [ 17.691369] envoy[910]: [2026-09-20 09:32:42.653][910][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4109lb # [ 17.694480] envoy[910]: [2026-09-20 09:32:42.653][910][info][main] [source/server/server.cc:1077] starting main dispatch loop4110worker2 # [ 17.906763] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4111worker2 # [ 17.918620] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14112worker1 # [ 17.927288] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4113worker1 # [ 17.939860] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14114worker2 # [ 17.954649] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4115worker2 # [ 17.957667] systemd[1]: Started Nix worker-protocol over gRPC.4116worker1 # [ 17.979915] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4117worker1 # [ 17.982969] systemd[1]: Started Nix worker-protocol over gRPC.4118lb # [ 17.986577] envoy[910]: [2026-09-20 09:32:42.947][910][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4119lb # [ 17.991887] envoy[910]: [2026-09-20 09:32:42.948][910][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4120lb # [ 17.996176] envoy[910]: [2026-09-20 09:32:42.948][910][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4121lb # [ 18.000232] envoy[910]: [2026-09-20 09:32:42.948][910][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4122worker2 # [ 18.045526] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=scheduler_connected addr=lb:500514123worker1 # [ 18.055745] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514124worker1 # [ 18.063251] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4125worker2 # [ 18.056324] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4126worker2 # [ 18.257059] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=scheduler_connected addr=lb:500514127worker1 # [ 18.264348] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514128worker2 # [ 18.259034] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4129worker1 # [ 18.266356] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4130worker2 # [ 18.460133] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=scheduler_connected addr=lb:500514131worker1 # [ 18.467630] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514132worker2 # [ 18.464157] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker1 # [ 18.471794] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4134worker2 # [ 18.664560] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=scheduler_connected addr=lb:500514135worker2 # [ 18.666511] nix-grpc-daemon[783]: ts=2026-09-20T09:32:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4136worker1 # [ 18.673254] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514137worker1 # [ 18.677267] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker2 # [ 18.867221] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 18.868996] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker1 # [ 18.878461] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514141worker1 # [ 18.882414] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 19.070211] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 19.071890] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 19.084868] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514145worker1 # [ 19.088230] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4146worker2 # [ 19.272786] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514147worker1 # [ 19.285446] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 19.276637] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker1 # [ 19.287779] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4150worker2 # [ 19.477178] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514151worker1 # [ 19.488598] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514152worker2 # [ 19.480365] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker1 # [ 19.490804] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4154worker2 # [ 19.681385] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_connected addr=lb:500514155worker1 # [ 19.691756] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 19.685296] nix-grpc-daemon[783]: ts=2026-09-20T09:32:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker1 # [ 19.695161] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4158worker2 # [ 19.886227] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514159worker1 # [ 19.897378] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 19.890264] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 19.900685] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker1 # [ 20.098781] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514163worker2 # [ 20.091800] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 20.102564] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 20.095573] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4166worker2 # [ 20.296180] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514167worker1 # [ 20.303707] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 20.298318] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 20.307823] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4170worker2 # [ 20.498854] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514171worker2 # [ 20.500576] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker1 # [ 20.508865] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514173worker1 # [ 20.512943] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4174worker2 # [ 20.701285] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_connected addr=lb:500514175worker2 # [ 20.703195] nix-grpc-daemon[783]: ts=2026-09-20T09:32:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker1 # [ 20.714263] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514177worker1 # [ 20.718382] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4178worker2 # [ 20.903820] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 20.905639] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 20.919446] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514181worker1 # [ 20.923547] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker2 # [ 21.108575] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514183worker2 # [ 21.111949] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker1 # [ 21.124875] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514185worker1 # [ 21.126276] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4186worker2 # [ 21.309074] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514187worker2 # [ 21.311086] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker1 # [ 21.326335] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514189worker1 # [ 21.330257] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4190worker2 # [ 21.512144] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514191worker2 # [ 21.516114] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker1 # [ 21.529142] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514193worker1 # [ 21.532319] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4194worker2 # [ 21.716916] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_connected addr=lb:500514195worker2 # [ 21.721066] nix-grpc-daemon[783]: ts=2026-09-20T09:32:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 21.731141] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514197worker1 # [ 21.734466] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198worker2 # [ 21.921431] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514199worker2 # [ 21.923469] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker1 # [ 21.932502] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514201worker1 # [ 21.935834] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker2 # [ 22.125754] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514203worker2 # [ 22.128887] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker1 # [ 22.138322] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514205worker1 # [ 22.141642] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4206worker2 # [ 22.326664] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514207worker2 # [ 22.330569] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker1 # [ 22.340466] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514209worker1 # [ 22.343635] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker2 # [ 22.531441] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514211worker1 # [ 22.541363] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 22.543639] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 22.535433] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker1 # [ 22.744527] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514215worker2 # [ 22.736361] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 22.746464] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 22.740453] nix-grpc-daemon[783]: ts=2026-09-20T09:32:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker1 # [ 22.947273] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514219worker1 # [ 22.949262] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker2 # [ 22.941499] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514221worker2 # [ 22.945509] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4222worker1 # [ 23.150963] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514223worker1 # [ 23.152369] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker2 # [ 23.146644] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514225worker2 # [ 23.147991] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker1 # [ 23.352228] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 23.347509] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 23.356253] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 23.351418] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker2 # [ 23.551846] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514231worker1 # [ 23.557530] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514232worker2 # [ 23.553582] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 23.561616] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker2 # [ 23.755395] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_connected addr=lb:500514235worker1 # [ 23.762620] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 23.766841] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 23.760202] nix-grpc-daemon[783]: ts=2026-09-20T09:32:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker2 # [ 23.960823] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 23.967895] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_connected addr=lb:500514240worker2 # [ 23.964686] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 23.971955] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker1 # [ 24.173183] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_connected addr=lb:500514243worker1 # [ 24.175193] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker2 # [ 24.166214] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514245worker2 # [ 24.170126] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246lb # [ 24.332753] rustfs-bucket-start[993]: mb s3://farm4247lb # [ 24.338716] systemd[1]: Finished rustfs-bucket.service.4248lb # [ 24.345504] systemd[1]: Starting niks3 server...4249worker1 # [ 24.376116] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_connected addr=lb:500514250worker1 # [ 24.378087] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker2 # [ 24.370845] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514252worker2 # [ 24.374699] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 24.579360] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 24.575882] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514255worker1 # [ 24.583361] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker2 # [ 24.579122] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257lb # [ 24.578410] postgres[1009]: [1009] ERROR: relation "goose_db_version" does not exist at character 364258lb # [ 24.579715] postgres[1009]: [1009] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4259lb # [ 24.619442] niks3-server[1003]: 2026/09/20 09:32:49 OK 20241026095416_initial_model.sql (22.06ms)4260lb # [ 24.622755] niks3-server[1003]: 2026/09/20 09:32:49 OK 20251210153512_drop_unused_gin_index.sql (3.46ms)4261lb # [ 24.627728] niks3-server[1003]: 2026/09/20 09:32:49 OK 20251218171726_add_pins.sql (3.14ms)4262lb # [ 24.632870] niks3-server[1003]: 2026/09/20 09:32:49 OK 20260628120000_add_object_size_and_stats.sql (5.55ms)4263lb # [ 24.637191] niks3-server[1003]: 2026/09/20 09:32:49 OK 20260905000000_add_claims.sql (2.96ms)4264lb # [ 24.641395] niks3-server[1003]: 2026/09/20 09:32:49 OK 20260920000000_drop_claims.sql (3.02ms)4265lb # [ 24.642828] niks3-server[1003]: 2026/09/20 09:32:49 goose: successfully migrated database to version: 202609200000004266lb # [ 24.648580] niks3-server[1003]: 2026/09/20 09:32:49 OK 1_commit_pending_closure.sql (5.82ms)4267lb # [ 24.651215] niks3-server[1003]: 2026/09/20 09:32:49 OK 2_object_stats_trigger.sql (1.47ms)4268lb # [ 24.652363] niks3-server[1003]: 2026/09/20 09:32:49 goose: up to current file version: 24269lb # [ 24.659217] niks3-server[1003]: 2026/09/20 09:32:49 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4270lb # [ 24.735515] niks3-server[1003]: 2026/09/20 09:32:49 INFO Read proxy enabled — serving cache objects from S34271lb # [ 24.737385] niks3-server[1003]: 2026/09/20 09:32:49 INFO Using socket-activated listener address=0.0.0.0:57514272lb # [ 24.743245] niks3-server[1003]: 2026/09/20 09:32:49 INFO systemd watchdog enabled interval=15s4273lb # [ 24.744435] systemd[1]: Started niks3 server.4274lb # [ 24.746612] niks3-server[1003]: 2026/09/20 09:32:49 INFO Starting HTTP server address=0.0.0.0:57514275lb # [ 24.748603] systemd[1]: Reached target Multi-User System.4276lb # [ 24.749880] systemd[1]: Startup finished in 853ms (kernel) + 4.113s (initrd) + 19.782s (userspace) = 24.749s.4277lb # [ 24.756211] niks3-server[1003]: 2026/09/20 09:32:49 INFO lead: acquired remote=192.168.1.3:355444278worker2 # [ 24.777117] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_connected addr=lb:500514279worker1 # [ 24.784283] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 24.780830] nix-grpc-daemon[783]: ts=2026-09-20T09:32:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 24.788684] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker1 # [ 24.795342] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_take_over4283worker2 # [ 24.981650] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 24.989446] nix-grpc-daemon[783]: ts=2026-09-20T09:32:50Z level=info event=scheduler_connected addr=lb:500514285worker2 # [ 24.985623] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4286worker1 # [ 24.993464] nix-grpc-daemon[783]: ts=2026-09-20T09:32:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker2 # [ 25.187289] nix-grpc-daemon[783]: ts=2026-09-20T09:32:49Z level=info event=scheduler_connected addr=lb:500514288worker1 # [ 25.195327] nix-grpc-daemon[783]: ts=2026-09-20T09:32:50Z level=info event=scheduler_connected addr=lb:500514289worker1 # [ 25.213297] nix-grpc-daemon[783]: ts=2026-09-20T09:32:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04290worker1 # [ 25.219688] nix-grpc-daemon[783]: ts=2026-09-20T09:32:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04291lb: (finished: waiting for unit niks3.service, in 13.39 seconds)4292lb: waiting for TCP port 5751 on localhost4293lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4294lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4295lb: waiting for unit envoy.service4296lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4297lb: waiting for TCP port 50051 on localhost4298lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4299lb # [ 25.403651] envoy[910]: conn peer=[::1]:43740 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24300lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4301lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4302??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4303 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284304lb: must succeed: curl -sf localhost:9901/clusters4305lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4306lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4307subtest: one node schedules and both builders hold a WorkerSession on it4308lb: must succeed: curl -sf localhost:9901/clusters4309lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4310worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4311worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)4312(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.13 seconds)4313subtest: balancer auth: client cert, bearer token, nothing, unknown CN4314worker1 # [ 26.083198] nix-grpc-daemon[783]: ts=2026-09-20T09:32:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:436264315worker1 # [ 26.096499] systemd[1]: Started Nix Daemon.4316worker1 # [ 26.322557] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4317worker1 # [ 26.368397] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)4318worker1 # [ 26.380165] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 2/5)4319worker1 # [ 26.466930] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 353 ms (attempt 3/5)4320worker1 # [ 26.825849] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 320 ms (attempt 4/5)4321worker1 # [ 27.158180] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4322worker1 # [ 27.188509] nix-grpc-daemon[783]: ts=2026-09-20T09:32:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:436264323worker1 # [ 27.207768] nix-grpc-daemon[783]: ts=2026-09-20T09:32:52Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4324worker2 # [ 27.219350] nix-grpc-daemon[783]: ts=2026-09-20T09:32:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:501924325worker2 # [ 27.231484] systemd[1]: Started Nix Daemon.4326worker2 # [ 27.433711] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4327worker2 # [ 27.460503] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)4328worker2 # [ 27.474899] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 2/5)4329worker2 # [ 27.540429] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 186 ms (attempt 3/5)4330worker2 # [ 27.729450] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 4/5)4331worker2 # [ 27.839514] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4332worker2 # [ 27.853144] nix-grpc-daemon[783]: ts=2026-09-20T09:32:52Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4333client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4334client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4335client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4336client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4337worker1 # [ 28.244578] nix-grpc-daemon[783]: ts=2026-09-20T09:32:53Z level=info event=oidc_jwks provider=mock keys=14338worker1 # [ 28.246437] nix-grpc-daemon[783]: ts=2026-09-20T09:32:53Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:436264339worker2 # [ 28.264813] nix-grpc-daemon[783]: ts=2026-09-20T09:32:52Z level=info event=oidc_jwks provider=mock keys=14340worker2 # [ 28.268535] nix-grpc-daemon[783]: ts=2026-09-20T09:32:52Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:501924341worker1 # [ 28.287596] nix-grpc-daemon[783]: ts=2026-09-20T09:32:53Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:436264342worker2 # [ 28.276801] nix-grpc-daemon[783]: ts=2026-09-20T09:32:52Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4343worker1 # [ 28.293441] nix-grpc-daemon[783]: ts=2026-09-20T09:32:53Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4344client: must succeed: rm -r /var/lib/nix-grpc-store4345client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4346worker2 # [ 30.543978] nix-grpc-daemon[783]: ts=2026-09-20T09:32:54Z level=info event=denied method=QueryMissing cn=- role=none4347worker1 # [ 30.751920] nix-grpc-daemon[783]: ts=2026-09-20T09:32:55Z level=info event=denied method=QueryMissing cn=stranger role=none4348(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.99 seconds)4349subtest: DAG build is scheduled across workers and lands in the cache4350client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24351worker1 # [ 30.980103] nix-grpc-daemon[783]: ts=2026-09-20T09:32:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:436264352client # don't know how to build these paths:4353client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4354worker1 # [ 30.993308] nix-grpc-daemon[783]: ts=2026-09-20T09:32:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436264355worker1 # [ 31.019562] nix-grpc-daemon[783]: ts=2026-09-20T09:32:56Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04356worker1 # [ 31.024151] nix-grpc-daemon[783]: ts=2026-09-20T09:32:56Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14357worker2 # [ 31.067326] nix-grpc-daemon[783]: ts=2026-09-20T09:32:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924358worker2 # [ 31.086147] nix-grpc-daemon[783]: ts=2026-09-20T09:32:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924359client # copying 1 paths to 'lb:50051'...4360worker2 # [ 31.095143] nix-grpc-daemon[783]: ts=2026-09-20T09:32:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501924361client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4362worker2 # [ 31.110743] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4363worker2 # [ 31.125234] nix-grpc-daemon[783]: ts=2026-09-20T09:32:55Z level=info event=niks3_push_started pid=8504364worker2 # [ 31.575816] nix-daemon[840]: accepted connection from pid 861, user nix-grpc-daemon (trusted)4365worker2 # [ 31.592669] nix-daemon[840]: reaped child process 865, status = succeeded4366worker2 # [ 31.729295] nix-daemon[840]: accepted connection from pid 867, user nix-grpc-daemon (trusted)4367worker2 # [ 31.741398] nix-daemon[840]: reaped child process 870, status = succeeded4368lb # [ 31.734854] niks3-server[1003]: 2026/09/20 09:32:56 INFO Received uploads request method=POST path=/api/pending_closures4369worker2 # [ 31.787703] nix-grpc-daemon[850]: time=2026-09-20T09:32:55.991Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4370worker2 # [ 31.791225] nix-grpc-daemon[850]: time=2026-09-20T09:32:55.995Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4371lb # [ 31.811102] niks3-server[1003]: 2026/09/20 09:32:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4372lb # [ 31.813780] niks3-server[1003]: 2026/09/20 09:32:56 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4373lb # [ 31.829801] niks3-server[1003]: 2026/09/20 09:32:56 INFO Signed narinfos id=1 count=14374worker2 # [ 31.853375] nix-grpc-daemon[850]: time=2026-09-20T09:32:56.058Z level=INFO msg="Uploading 1 narinfos"4375lb # [ 31.839883] niks3-server[1003]: 2026/09/20 09:32:56 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4376lb # [ 31.843425] niks3-server[1003]: 2026/09/20 09:32:56 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4377lb # [ 31.850839] niks3-server[1003]: 2026/09/20 09:32:56 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4378lb # [ 31.858406] niks3-server[1003]: 2026/09/20 09:32:56 INFO Completed upload id=14379worker2 # [ 31.880529] nix-grpc-daemon[850]: time=2026-09-20T09:32:56.085Z level=INFO msg="Upload complete. (475ms)"4380worker2 # [ 31.882982] nix-grpc-daemon[783]: ts=2026-09-20T09:32:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_in=5044381worker2 # [ 31.887829] nix-grpc-daemon[783]: ts=2026-09-20T09:32:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924382worker2 # [ 31.898457] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4383worker2 # [ 31.903663] nix-daemon[840]: reaped child process 848, status = succeeded4384client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4385worker2 # [ 31.913491] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4386client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4387client # farm-a> LOG-farm-a-t14388worker2 # [ 33.129558] nix-daemon[840]: reaped child process 874, status = succeeded4389worker2 # [ 33.266809] nix-daemon[840]: accepted connection from pid 889, user nix-grpc-daemon (trusted)4390worker2 # [ 33.278495] nix-daemon[840]: reaped child process 892, status = succeeded4391lb # [ 33.272744] niks3-server[1003]: 2026/09/20 09:32:58 INFO Received uploads request method=POST path=/api/pending_closures4392worker2 # [ 33.305406] nix-grpc-daemon[850]: time=2026-09-20T09:32:57.510Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4393worker2 # [ 33.306995] nix-grpc-daemon[850]: time=2026-09-20T09:32:57.511Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4394lb # [ 33.320570] niks3-server[1003]: 2026/09/20 09:32:58 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4395lb # [ 33.328829] niks3-server[1003]: 2026/09/20 09:32:58 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4396lb # [ 33.338573] niks3-server[1003]: 2026/09/20 09:32:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4397worker2 # [ 33.366455] nix-grpc-daemon[850]: time=2026-09-20T09:32:57.570Z level=INFO msg="Uploading 1 narinfos"4398lb # [ 33.344395] niks3-server[1003]: 2026/09/20 09:32:58 INFO Signed narinfos id=2 count=14399lb # [ 33.350092] niks3-server[1003]: 2026/09/20 09:32:58 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4400lb # [ 33.361755] niks3-server[1003]: 2026/09/20 09:32:58 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4401lb # [ 33.366308] niks3-server[1003]: 2026/09/20 09:32:58 INFO Completed upload id=24402worker2 # [ 33.388824] nix-grpc-daemon[850]: time=2026-09-20T09:32:57.593Z level=INFO msg="Upload complete. (266ms)"4403lb # [ 33.368086] niks3-server[1003]: 2026/09/20 09:32:58 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4404worker2 # [ 33.391592] nix-grpc-daemon[783]: ts=2026-09-20T09:32:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14405worker2 # [ 33.401481] nix-daemon[840]: reaped child process 872, status = succeeded4406worker1 # [ 33.411884] nix-grpc-daemon[783]: ts=2026-09-20T09:32:58Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04407worker1 # [ 33.413871] nix-grpc-daemon[783]: ts=2026-09-20T09:32:58Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14408worker2 # [ 33.449287] nix-grpc-daemon[783]: ts=2026-09-20T09:32:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924409worker2 # [ 33.457817] nix-grpc-daemon[783]: ts=2026-09-20T09:32:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924410worker2 # [ 33.468152] nix-grpc-daemon[850]: time=2026-09-20T09:32:57.673Z level=INFO msg="All 1 paths already cached"4411client # copying 1 paths to 'lb:50051'...4412client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4413worker2 # [ 33.474324] nix-grpc-daemon[783]: ts=2026-09-20T09:32:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501924414worker2 # [ 33.483129] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4415worker2 # [ 33.636049] nix-daemon[840]: accepted connection from pid 897, user nix-grpc-daemon (trusted)4416worker2 # [ 33.649104] nix-daemon[840]: reaped child process 900, status = succeeded4417worker2 # [ 33.802622] nix-daemon[840]: accepted connection from pid 903, user nix-grpc-daemon (trusted)4418worker2 # [ 33.814410] nix-daemon[840]: reaped child process 906, status = succeeded4419lb # [ 33.806254] niks3-server[1003]: 2026/09/20 09:32:58 INFO Received uploads request method=POST path=/api/pending_closures4420worker2 # [ 33.834918] nix-grpc-daemon[850]: time=2026-09-20T09:32:58.039Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4421worker2 # [ 33.836403] nix-grpc-daemon[850]: time=2026-09-20T09:32:58.041Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4422lb # [ 33.825848] niks3-server[1003]: 2026/09/20 09:32:58 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4423lb # [ 33.830180] niks3-server[1003]: 2026/09/20 09:32:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4424lb # [ 33.832071] niks3-server[1003]: 2026/09/20 09:32:58 INFO Signed narinfos id=3 count=14425worker2 # [ 33.855400] nix-grpc-daemon[850]: time=2026-09-20T09:32:58.060Z level=INFO msg="Uploading 1 narinfos"4426lb # [ 33.834311] niks3-server[1003]: 2026/09/20 09:32:58 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4427lb # [ 33.840062] niks3-server[1003]: 2026/09/20 09:32:58 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4428lb # [ 33.845890] niks3-server[1003]: 2026/09/20 09:32:58 INFO Completed upload id=34429worker2 # [ 33.867969] nix-grpc-daemon[850]: time=2026-09-20T09:32:58.072Z level=INFO msg="Upload complete. (378ms)"4430lb # [ 33.847450] niks3-server[1003]: 2026/09/20 09:32:58 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4431worker2 # [ 33.870542] nix-grpc-daemon[783]: ts=2026-09-20T09:32:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_in=6244432worker2 # [ 33.873561] nix-grpc-daemon[783]: ts=2026-09-20T09:32:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924433worker2 # [ 33.880389] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4434worker2 # [ 33.883285] nix-daemon[840]: reaped child process 895, status = succeeded4435worker2 # [ 33.888411] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4436client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4437client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4438client # farm-b> LOG-farm-b-t14439worker2 # [ 35.114802] nix-daemon[840]: reaped child process 910, status = succeeded4440worker2 # [ 35.233300] nix-daemon[840]: accepted connection from pid 925, user nix-grpc-daemon (trusted)4441worker2 # [ 35.244446] nix-daemon[840]: reaped child process 928, status = succeeded4442lb # [ 35.238689] niks3-server[1003]: 2026/09/20 09:33:00 INFO Received uploads request method=POST path=/api/pending_closures4443worker2 # [ 35.274670] nix-grpc-daemon[850]: time=2026-09-20T09:32:59.479Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4444worker2 # [ 35.276185] nix-grpc-daemon[850]: time=2026-09-20T09:32:59.481Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4445lb # [ 35.284676] niks3-server[1003]: 2026/09/20 09:33:00 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4446lb # [ 35.297602] niks3-server[1003]: 2026/09/20 09:33:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4447lb # [ 35.305623] niks3-server[1003]: 2026/09/20 09:33:00 INFO Signed narinfos id=4 count=14448worker2 # [ 35.330873] nix-grpc-daemon[850]: time=2026-09-20T09:32:59.535Z level=INFO msg="Uploading 1 narinfos"4449lb # [ 35.310538] niks3-server[1003]: 2026/09/20 09:33:00 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4450lb # [ 35.317406] niks3-server[1003]: 2026/09/20 09:33:00 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4451lb # [ 35.326029] niks3-server[1003]: 2026/09/20 09:33:00 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4452lb # [ 35.332212] niks3-server[1003]: 2026/09/20 09:33:00 INFO Completed upload id=44453worker2 # [ 35.354491] nix-grpc-daemon[850]: time=2026-09-20T09:32:59.559Z level=INFO msg="Upload complete. (258ms)"4454lb # [ 35.334277] niks3-server[1003]: 2026/09/20 09:33:00 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4455worker2 # [ 35.361158] nix-grpc-daemon[783]: ts=2026-09-20T09:32:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14456worker1 # [ 35.374542] nix-grpc-daemon[783]: ts=2026-09-20T09:33:00Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04457worker1 # [ 35.378620] nix-grpc-daemon[783]: ts=2026-09-20T09:33:00Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14458worker2 # [ 35.374869] nix-daemon[840]: reaped child process 908, status = succeeded4459worker2 # [ 35.411661] nix-grpc-daemon[783]: ts=2026-09-20T09:32:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924460worker2 # [ 35.422073] nix-grpc-daemon[783]: ts=2026-09-20T09:32:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924461worker2 # [ 35.432517] nix-grpc-daemon[850]: time=2026-09-20T09:32:59.637Z level=INFO msg="All 2 paths already cached"4462client # copying 1 paths to 'lb:50051'...4463worker2 # [ 35.439355] nix-grpc-daemon[783]: ts=2026-09-20T09:32:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501924464client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4465worker2 # [ 35.459166] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4466worker2 # [ 35.622946] nix-daemon[840]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4467worker2 # [ 35.635237] nix-daemon[840]: reaped child process 935, status = succeeded4468worker2 # [ 35.790480] nix-daemon[840]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4469worker2 # [ 35.803025] nix-daemon[840]: reaped child process 940, status = succeeded4470lb # [ 35.796382] niks3-server[1003]: 2026/09/20 09:33:00 INFO Received uploads request method=POST path=/api/pending_closures4471worker2 # [ 35.831153] nix-grpc-daemon[850]: time=2026-09-20T09:33:00.036Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4472worker2 # [ 35.832809] nix-grpc-daemon[850]: time=2026-09-20T09:33:00.037Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4473lb # [ 35.836067] niks3-server[1003]: 2026/09/20 09:33:00 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4474lb # [ 35.839449] niks3-server[1003]: 2026/09/20 09:33:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4475lb # [ 35.842063] niks3-server[1003]: 2026/09/20 09:33:00 INFO Signed narinfos id=5 count=14476worker2 # [ 35.863774] nix-grpc-daemon[850]: time=2026-09-20T09:33:00.068Z level=INFO msg="Uploading 1 narinfos"4477lb # [ 35.846188] niks3-server[1003]: 2026/09/20 09:33:00 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4478lb # [ 35.853312] niks3-server[1003]: 2026/09/20 09:33:00 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4479lb # [ 35.856895] niks3-server[1003]: 2026/09/20 09:33:00 INFO Completed upload id=54480worker2 # [ 35.878863] nix-grpc-daemon[850]: time=2026-09-20T09:33:00.083Z level=INFO msg="Upload complete. (407ms)"4481lb # [ 35.858352] niks3-server[1003]: 2026/09/20 09:33:00 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4482worker2 # [ 35.880878] nix-grpc-daemon[783]: ts=2026-09-20T09:33:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_in=7524483worker2 # [ 35.885506] nix-grpc-daemon[783]: ts=2026-09-20T09:33:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924484worker2 # [ 35.892266] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4485worker2 # [ 35.895422] nix-daemon[840]: reaped child process 930, status = succeeded4486worker2 # [ 35.900395] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4487client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4488client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4489client # farm-top> LOG-farm-top-t14490worker2 # [ 36.920182] nix-daemon[840]: reaped child process 944, status = succeeded4491worker2 # [ 37.076709] nix-daemon[840]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4492worker2 # [ 37.089370] nix-daemon[840]: reaped child process 963, status = succeeded4493lb # [ 37.081126] niks3-server[1003]: 2026/09/20 09:33:02 INFO Received uploads request method=POST path=/api/pending_closures4494worker2 # [ 37.134609] nix-grpc-daemon[850]: time=2026-09-20T09:33:01.339Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4495worker2 # [ 37.137663] nix-grpc-daemon[850]: time=2026-09-20T09:33:01.341Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4496lb # [ 37.151652] niks3-server[1003]: 2026/09/20 09:33:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4497lb # [ 37.158105] niks3-server[1003]: 2026/09/20 09:33:02 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4498lb # [ 37.162249] niks3-server[1003]: 2026/09/20 09:33:02 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4499worker2 # [ 37.184499] nix-grpc-daemon[850]: time=2026-09-20T09:33:01.388Z level=INFO msg="Uploading 1 narinfos"4500lb # [ 37.164426] niks3-server[1003]: 2026/09/20 09:33:02 INFO Signed narinfos id=6 count=14501lb # [ 37.170976] niks3-server[1003]: 2026/09/20 09:33:02 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4502lb # [ 37.198799] niks3-server[1003]: 2026/09/20 09:33:02 INFO Completed upload id=64503worker2 # [ 37.220963] nix-grpc-daemon[850]: time=2026-09-20T09:33:01.425Z level=INFO msg="Upload complete. (312ms)"4504lb # [ 37.200465] niks3-server[1003]: 2026/09/20 09:33:02 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4505lb # [ 37.205105] niks3-server[1003]: 2026/09/20 09:33:02 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4506worker2 # [ 37.226695] nix-grpc-daemon[783]: ts=2026-09-20T09:33:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14507worker2 # [ 37.241550] nix-daemon[840]: reaped child process 942, status = succeeded4508client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.50 seconds)4509client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4510client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4511client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4512client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4513client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4514lb # [ 37.676895] envoy[910]: conn peer=[2001:db8:1::1]:57956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=18194515lb # [ 37.680707] envoy[910]: conn peer=[2001:db8:1::1]:57972 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=363 ms=764516lb # [ 37.684185] envoy[910]: conn peer=[2001:db8:1::1]:57984 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114517lb # [ 37.687515] envoy[910]: conn peer=[2001:db8:1::1]:57986 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=94518lb # [ 37.691087] envoy[910]: conn peer=[2001:db8:1::1]:57994 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=3794 ms=62914519client # copying 3 paths...4520client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4521client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4522client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4523client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4524(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.03 seconds)4525subtest: repeat is answered Cached by the scheduler without building4526worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4527worker1 # finding garbage collector roots...4528worker1 # deleting specified paths...4529worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4530worker1 # deleting unused links...4531worker1 # note: hard linking is currently saving 0.0 KiB4532worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4533worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4534worker2 # finding garbage collector roots...4535worker2 # deleting specified paths...4536worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4537worker2 # deleting unused links...4538worker2 # note: hard linking is currently saving 0.0 KiB4539worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4540client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24541worker1 # [ 38.511866] nix-grpc-daemon[783]: ts=2026-09-20T09:33:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:436264542client # don't know how to build these paths:4543client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4544worker1 # [ 38.520699] nix-grpc-daemon[783]: ts=2026-09-20T09:33:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436264545worker1 # [ 38.549760] nix-grpc-daemon[783]: ts=2026-09-20T09:33:03Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14546client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.25 seconds)4547client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4548client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4549lb: must succeed: curl -sf localhost:9901/clusters4550lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4551lb: must succeed: curl -sf localhost:9901/clusters4552lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4553worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4554worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.07 seconds)4555(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.22 seconds)4556subtest: inputs already in the cache are not sent to the scheduler4557lb: must succeed: curl -sf localhost:9901/clusters4558lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4559lb: must succeed: curl -sf localhost:9901/clusters4560lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4561worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4562worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4563client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24564worker2 # [ 39.363807] nix-grpc-daemon[783]: ts=2026-09-20T09:33:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:501924565client # don't know how to build these paths:4566client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4567worker2 # [ 39.372727] nix-grpc-daemon[783]: ts=2026-09-20T09:33:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924568worker2 # [ 39.383419] nix-grpc-daemon[850]: time=2026-09-20T09:33:03.587Z level=INFO msg="All 2 paths already cached"4569worker1 # [ 39.404894] nix-grpc-daemon[783]: ts=2026-09-20T09:33:04Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04570worker1 # [ 39.409418] nix-grpc-daemon[783]: ts=2026-09-20T09:33:04Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14571worker2 # [ 39.442406] nix-grpc-daemon[783]: ts=2026-09-20T09:33:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924572worker2 # [ 39.455237] nix-grpc-daemon[783]: ts=2026-09-20T09:33:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924573worker2 # [ 39.468627] nix-grpc-daemon[850]: time=2026-09-20T09:33:03.673Z level=INFO msg="All 2 paths already cached"4574client # copying 1 paths to 'lb:50051'...4575client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4576worker2 # [ 39.476942] nix-grpc-daemon[783]: ts=2026-09-20T09:33:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501924577worker2 # [ 39.488183] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4578worker2 # [ 39.658877] nix-daemon[840]: accepted connection from pid 980, user nix-grpc-daemon (trusted)4579worker2 # [ 39.672173] nix-daemon[840]: reaped child process 983, status = succeeded4580worker2 # [ 39.841760] nix-daemon[840]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4581worker2 # [ 39.853590] nix-daemon[840]: reaped child process 989, status = succeeded4582lb # [ 39.846381] niks3-server[1003]: 2026/09/20 09:33:04 INFO Received uploads request method=POST path=/api/pending_closures4583worker2 # [ 39.877957] nix-grpc-daemon[850]: time=2026-09-20T09:33:04.082Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4584worker2 # [ 39.879474] nix-grpc-daemon[850]: time=2026-09-20T09:33:04.084Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4585lb # [ 39.876141] niks3-server[1003]: 2026/09/20 09:33:04 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4586lb # [ 39.888729] niks3-server[1003]: 2026/09/20 09:33:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4587lb # [ 39.894917] niks3-server[1003]: 2026/09/20 09:33:04 INFO Signed narinfos id=7 count=14588worker2 # [ 39.919988] nix-grpc-daemon[850]: time=2026-09-20T09:33:04.124Z level=INFO msg="Uploading 1 narinfos"4589lb # [ 39.899934] niks3-server[1003]: 2026/09/20 09:33:04 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4590lb # [ 39.912868] niks3-server[1003]: 2026/09/20 09:33:04 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4591worker2 # [ 39.941236] nix-grpc-daemon[850]: time=2026-09-20T09:33:04.146Z level=INFO msg="Upload complete. (441ms)"4592lb # [ 39.919276] niks3-server[1003]: 2026/09/20 09:33:04 INFO Completed upload id=74593lb # [ 39.921549] niks3-server[1003]: 2026/09/20 09:33:04 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4594worker2 # [ 39.943912] nix-grpc-daemon[783]: ts=2026-09-20T09:33:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_in=7524595worker2 # [ 39.950104] nix-grpc-daemon[783]: ts=2026-09-20T09:33:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924596worker2 # [ 39.960254] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4597worker2 # [ 39.965047] nix-daemon[840]: reaped child process 978, status = succeeded4598worker2 # [ 39.968203] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4599client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4600client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4601client # farm-top> LOG-farm-top-t1b4602worker2 # [ 40.993823] nix-daemon[840]: reaped child process 993, status = succeeded4603worker2 # [ 41.133921] nix-daemon[840]: accepted connection from pid 1008, user nix-grpc-daemon (trusted)4604worker2 # [ 41.145734] nix-daemon[840]: reaped child process 1011, status = succeeded4605lb # [ 41.142090] niks3-server[1003]: 2026/09/20 09:33:06 INFO Received uploads request method=POST path=/api/pending_closures4606worker2 # [ 41.171377] nix-grpc-daemon[850]: time=2026-09-20T09:33:05.376Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4607worker2 # [ 41.172901] nix-grpc-daemon[850]: time=2026-09-20T09:33:05.377Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4608lb # [ 41.178189] niks3-server[1003]: 2026/09/20 09:33:06 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4609lb # [ 41.190169] niks3-server[1003]: 2026/09/20 09:33:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4610lb # [ 41.197343] niks3-server[1003]: 2026/09/20 09:33:06 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4611worker2 # [ 41.224041] nix-grpc-daemon[850]: time=2026-09-20T09:33:05.428Z level=INFO msg="Uploading 1 narinfos"4612lb # [ 41.202117] niks3-server[1003]: 2026/09/20 09:33:06 INFO Signed narinfos id=8 count=14613lb # [ 41.209620] niks3-server[1003]: 2026/09/20 09:33:06 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4614lb # [ 41.218075] niks3-server[1003]: 2026/09/20 09:33:06 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4615lb # [ 41.223159] niks3-server[1003]: 2026/09/20 09:33:06 INFO Completed upload id=84616lb # [ 41.226192] niks3-server[1003]: 2026/09/20 09:33:06 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4617worker2 # [ 41.247146] nix-grpc-daemon[850]: time=2026-09-20T09:33:05.451Z level=INFO msg="Upload complete. (272ms)"4618worker2 # [ 41.252607] nix-grpc-daemon[783]: ts=2026-09-20T09:33:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14619worker2 # [ 41.266211] nix-daemon[840]: reaped child process 991, status = succeeded4620client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.13 seconds)4621client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4622client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4623lb: must succeed: curl -sf localhost:9901/clusters4624lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4625lb: must succeed: curl -sf localhost:9901/clusters4626lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4627worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4628worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4629(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.59 seconds)4630subtest: two clients wanting the same drv build it once4631client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4632client # [ 41.638699] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4633client # Running as unit: dedup1.service; invocation ID: b68af0ab6d434ee09731653e989656774634client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.04 seconds)4635client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4636worker1 # [ 41.849396] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:436264637client # [ 41.843162] nix[892]: don't know how to build these paths:4638client # [ 41.844083] nix[892]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4639worker1 # [ 41.866215] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04640worker1 # [ 41.872983] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14641worker1 # [ 41.878461] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436264642worker1 # [ 41.893591] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436264643client # [ 41.892124] nix[892]: copying 1 paths to 'lb:50051'...4644client # [ 41.892862] nix[892]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4645worker1 # [ 41.904413] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:436264646worker1 # [ 41.916128] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4647worker1 # [ 41.936663] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=info event=niks3_push_started pid=8984648worker1 # [ 42.370091] nix-daemon[845]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4649worker1 # [ 42.386573] nix-daemon[845]: reaped child process 914, status = succeeded4650worker1 # [ 42.529467] nix-daemon[845]: accepted connection from pid 916, user nix-grpc-daemon (trusted)4651worker1 # [ 42.544188] nix-daemon[845]: reaped child process 919, status = succeeded4652lb # [ 42.526798] niks3-server[1003]: 2026/09/20 09:33:07 INFO Received uploads request method=POST path=/api/pending_closures4653worker1 # [ 42.568626] nix-grpc-daemon[898]: time=2026-09-20T09:33:07.737Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4654worker1 # [ 42.570558] nix-grpc-daemon[898]: time=2026-09-20T09:33:07.739Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4655lb # [ 42.564122] niks3-server[1003]: 2026/09/20 09:33:07 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4656lb # [ 42.574084] niks3-server[1003]: 2026/09/20 09:33:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4657lb # [ 42.579410] niks3-server[1003]: 2026/09/20 09:33:07 INFO Signed narinfos id=9 count=14658worker1 # [ 42.611137] nix-grpc-daemon[898]: time=2026-09-20T09:33:07.781Z level=INFO msg="Uploading 1 narinfos"4659lb # [ 42.592201] niks3-server[1003]: 2026/09/20 09:33:07 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4660lb # [ 42.597508] niks3-server[1003]: 2026/09/20 09:33:07 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4661lb # [ 42.602627] niks3-server[1003]: 2026/09/20 09:33:07 INFO Completed upload id=94662worker1 # [ 42.634080] nix-grpc-daemon[898]: time=2026-09-20T09:33:07.804Z level=INFO msg="Upload complete. (425ms)"4663lb # [ 42.606865] niks3-server[1003]: 2026/09/20 09:33:07 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4664worker1 # [ 42.637079] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=0 paths=1 nar_bytes_in=4324665worker1 # [ 42.643440] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436264666worker1 # [ 42.648148] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4667worker1 # [ 42.650647] nix-daemon[845]: reaped child process 896, status = succeeded4668worker1 # [ 42.656212] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4669client # [ 42.642437] nix[892]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4670client # [ 42.708610] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4671client # Running as unit: dedup2.service; invocation ID: b11e1bf0ec824efe8763f0ff0ffd5ab14672client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4673client # [ 42.996280] nix[908]: don't know how to build these paths:4674worker2 # [ 43.001251] nix-grpc-daemon[783]: ts=2026-09-20T09:33:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:501924675client # [ 42.997587] nix[908]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4676worker1 # [ 43.015678] nix-grpc-daemon[783]: ts=2026-09-20T09:33:08Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04677worker1 # [ 43.021156] nix-grpc-daemon[783]: ts=2026-09-20T09:33:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436264678client # [ 43.010560] nix[908]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4679client: must succeed: sleep 24680client: (finished: must succeed: sleep 2, in 2.03 seconds)4681??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4682 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394683client: waiting for success: ! systemctl is-active dedup1 dedup24684??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4685 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394686worker1 # [ 47.285577] nix-daemon[845]: reaped child process 923, status = succeeded4687worker1 # [ 47.415205] nix-daemon[845]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4688worker1 # [ 47.426696] nix-daemon[845]: reaped child process 961, status = succeeded4689lb # [ 47.407125] niks3-server[1003]: 2026/09/20 09:33:12 INFO Received uploads request method=POST path=/api/pending_closures4690worker1 # [ 47.440744] nix-grpc-daemon[898]: time=2026-09-20T09:33:12.611Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4691worker1 # [ 47.442263] nix-grpc-daemon[898]: time=2026-09-20T09:33:12.612Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4692lb # [ 47.433712] niks3-server[1003]: 2026/09/20 09:33:12 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4693lb # [ 47.437278] niks3-server[1003]: 2026/09/20 09:33:12 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4694lb # [ 47.441058] niks3-server[1003]: 2026/09/20 09:33:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4695lb # [ 47.443569] niks3-server[1003]: 2026/09/20 09:33:12 INFO Signed narinfos id=10 count=14696worker1 # [ 47.473459] nix-grpc-daemon[898]: time=2026-09-20T09:33:12.643Z level=INFO msg="Uploading 1 narinfos"4697lb # [ 47.448605] niks3-server[1003]: 2026/09/20 09:33:12 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4698lb # [ 47.451928] niks3-server[1003]: 2026/09/20 09:33:12 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4699lb # [ 47.456217] niks3-server[1003]: 2026/09/20 09:33:12 INFO Completed upload id=104700worker1 # [ 47.486666] nix-grpc-daemon[898]: time=2026-09-20T09:33:12.657Z level=INFO msg="Upload complete. (215ms)"4701lb # [ 47.458295] niks3-server[1003]: 2026/09/20 09:33:12 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4702worker1 # [ 47.491096] nix-grpc-daemon[783]: ts=2026-09-20T09:33:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14703worker1 # [ 47.494141] nix-grpc-daemon[783]: ts=2026-09-20T09:33:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14704worker1 # [ 47.499881] nix-daemon[845]: reaped child process 921, status = succeeded4705client # [ 47.518757] systemd[1]: dedup1.service: Deactivated successfully.4706client # [ 47.523927] systemd[1]: dedup1.service: Consumed 110ms CPU time over 5.883s wall clock time, 39.4M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.4707client # [ 47.560528] systemd[1]: dedup2.service: Deactivated successfully.4708client # [ 47.562390] systemd[1]: dedup2.service: Consumed 133ms CPU time over 4.853s wall clock time, 33.2M memory peak, 7.6K incoming IP traffic, 6.4K outgoing IP traffic.4709lb # [ 47.676797] envoy[910]: conn peer=[2001:db8:1::1]:36060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=554710lb # [ 47.678314] envoy[910]: conn peer=[2001:db8:1::1]:36064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4126 tx=1689 ms=19014711lb # [ 47.679797] envoy[910]: conn peer=[2001:db8:1::1]:36066 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1353 ms=56554712lb # [ 47.681357] envoy[910]: conn peer=[2001:db8:1::1]:37416 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1085 ms=45464713client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.11 seconds)4714client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4715client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4716worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4717worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4718worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4719worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4720worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4721worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.07 seconds)4722worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4723worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.08 seconds)4724(finished: subtest: two clients wanting the same drv build it once, in 6.99 seconds)4725subtest: build hook: nix-daemon with builders = grpc://lb4726client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4727client # [ 48.779617] systemd[1]: Started Nix Daemon.4728client # [ 48.944025] nix-daemon[951]: accepted connection from pid 948, user root (trusted)4729worker1 # [ 49.127811] nix-grpc-daemon[783]: ts=2026-09-20T09:33:14Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:436264730worker1 # [ 49.136561] nix-grpc-daemon[783]: ts=2026-09-20T09:33:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436264731worker1 # [ 49.152150] nix-grpc-daemon[783]: ts=2026-09-20T09:33:14Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04732worker1 # [ 49.156160] nix-grpc-daemon[783]: ts=2026-09-20T09:33:14Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14733worker2 # [ 49.192350] nix-grpc-daemon[783]: ts=2026-09-20T09:33:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924734worker2 # [ 49.203676] nix-grpc-daemon[783]: ts=2026-09-20T09:33:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924735worker2 # [ 49.215784] nix-grpc-daemon[783]: ts=2026-09-20T09:33:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501924736worker2 # [ 49.224960] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4737worker2 # [ 49.410047] nix-daemon[840]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)4738worker2 # [ 49.422880] nix-daemon[840]: reaped child process 1046, status = succeeded4739worker2 # [ 49.584359] nix-daemon[840]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)4740worker2 # [ 49.596798] nix-daemon[840]: reaped child process 1051, status = succeeded4741lb # [ 49.590587] niks3-server[1003]: 2026/09/20 09:33:14 INFO Received uploads request method=POST path=/api/pending_closures4742worker2 # [ 49.616835] nix-grpc-daemon[850]: time=2026-09-20T09:33:13.821Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4743worker2 # [ 49.618392] nix-grpc-daemon[850]: time=2026-09-20T09:33:13.823Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4744lb # [ 49.606548] niks3-server[1003]: 2026/09/20 09:33:14 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4745lb # [ 49.609587] niks3-server[1003]: 2026/09/20 09:33:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4746lb # [ 49.611251] niks3-server[1003]: 2026/09/20 09:33:14 INFO Signed narinfos id=11 count=14747worker2 # [ 49.633520] nix-grpc-daemon[850]: time=2026-09-20T09:33:13.838Z level=INFO msg="Uploading 1 narinfos"4748lb # [ 49.615335] niks3-server[1003]: 2026/09/20 09:33:14 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4749lb # [ 49.619871] niks3-server[1003]: 2026/09/20 09:33:14 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4750lb # [ 49.623387] niks3-server[1003]: 2026/09/20 09:33:14 INFO Completed upload id=114751worker2 # [ 49.645485] nix-grpc-daemon[850]: time=2026-09-20T09:33:13.850Z level=INFO msg="Upload complete. (406ms)"4752lb # [ 49.624871] niks3-server[1003]: 2026/09/20 09:33:14 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4753worker2 # [ 49.647446] nix-grpc-daemon[783]: ts=2026-09-20T09:33:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_in=5124754worker2 # [ 49.651522] nix-grpc-daemon[783]: ts=2026-09-20T09:33:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924755worker2 # [ 49.657366] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4756worker2 # [ 49.659609] nix-daemon[840]: reaped child process 1041, status = succeeded4757worker2 # [ 49.664796] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4758worker2 # [ 50.890853] nix-daemon[840]: reaped child process 1055, status = succeeded4759worker2 # [ 51.047978] nix-daemon[840]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4760worker2 # [ 51.060975] nix-daemon[840]: reaped child process 1073, status = succeeded4761lb # [ 51.054481] niks3-server[1003]: 2026/09/20 09:33:16 INFO Received uploads request method=POST path=/api/pending_closures4762worker2 # [ 51.084620] nix-grpc-daemon[850]: time=2026-09-20T09:33:15.289Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4763worker2 # [ 51.087060] nix-grpc-daemon[850]: time=2026-09-20T09:33:15.291Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4764lb # [ 51.098395] niks3-server[1003]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4765lb # [ 51.108102] niks3-server[1003]: 2026/09/20 09:33:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4766lb # [ 51.115101] niks3-server[1003]: 2026/09/20 09:33:16 INFO Signed narinfos id=12 count=14767worker2 # [ 51.140550] nix-grpc-daemon[850]: time=2026-09-20T09:33:15.345Z level=INFO msg="Uploading 1 narinfos"4768lb # [ 51.120183] niks3-server[1003]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4769lb # [ 51.123573] niks3-server[1003]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4770lb # [ 51.135281] niks3-server[1003]: 2026/09/20 09:33:16 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4771lb # [ 51.137793] niks3-server[1003]: 2026/09/20 09:33:16 INFO Completed upload id=124772worker2 # [ 51.159889] nix-grpc-daemon[850]: time=2026-09-20T09:33:15.364Z level=INFO msg="Upload complete. (287ms)"4773lb # [ 51.139327] niks3-server[1003]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4774worker2 # [ 51.164271] nix-grpc-daemon[783]: ts=2026-09-20T09:33:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14775worker2 # [ 51.171670] nix-daemon[840]: reaped child process 1053, status = succeeded4776worker1 # [ 51.175369] nix-grpc-daemon[783]: ts=2026-09-20T09:33:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:436264777worker1 # [ 51.187316] nix-grpc-daemon[783]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4778worker1 # [ 51.215914] nix-grpc-daemon[783]: ts=2026-09-20T09:33:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:436264779worker1 # [ 51.222946] nix-grpc-daemon[783]: ts=2026-09-20T09:33:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=0 paths=1 nar_bytes_out=1284780worker2 # [ 51.478364] nix-grpc-daemon[783]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:501924781worker2 # [ 51.480889] nix-grpc-daemon[783]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924782worker2 # [ 51.485940] nix-grpc-daemon[850]: time=2026-09-20T09:33:15.690Z level=INFO msg="All 1 paths already cached"4783worker1 # [ 51.504245] nix-grpc-daemon[783]: ts=2026-09-20T09:33:16Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04784worker1 # [ 51.506603] nix-grpc-daemon[783]: ts=2026-09-20T09:33:16Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14785worker2 # [ 51.501552] nix-grpc-daemon[783]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924786worker2 # [ 51.509818] nix-grpc-daemon[783]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924787worker2 # [ 51.517698] nix-grpc-daemon[850]: time=2026-09-20T09:33:15.722Z level=INFO msg="All 1 paths already cached"4788worker2 # [ 51.521953] nix-grpc-daemon[783]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501924789worker2 # [ 51.527773] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4790worker2 # [ 51.675421] nix-daemon[840]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4791worker2 # [ 51.689039] nix-daemon[840]: reaped child process 1080, status = succeeded4792worker2 # [ 51.848119] nix-daemon[840]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)4793worker2 # [ 51.859920] nix-daemon[840]: reaped child process 1085, status = succeeded4794lb # [ 51.854380] niks3-server[1003]: 2026/09/20 09:33:16 INFO Received uploads request method=POST path=/api/pending_closures4795worker2 # [ 51.884036] nix-grpc-daemon[850]: time=2026-09-20T09:33:16.088Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4796worker2 # [ 51.885848] nix-grpc-daemon[850]: time=2026-09-20T09:33:16.090Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4797lb # [ 51.879950] niks3-server[1003]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4798lb # [ 51.892856] niks3-server[1003]: 2026/09/20 09:33:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4799worker2 # [ 51.919960] nix-grpc-daemon[850]: time=2026-09-20T09:33:16.124Z level=INFO msg="Uploading 1 narinfos"4800lb # [ 51.898158] niks3-server[1003]: 2026/09/20 09:33:16 INFO Signed narinfos id=13 count=14801lb # [ 51.903365] niks3-server[1003]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4802lb # [ 51.914585] niks3-server[1003]: 2026/09/20 09:33:16 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4803lb # [ 51.921206] niks3-server[1003]: 2026/09/20 09:33:16 INFO Completed upload id=134804worker2 # [ 51.943107] nix-grpc-daemon[850]: time=2026-09-20T09:33:16.148Z level=INFO msg="Upload complete. (409ms)"4805lb # [ 51.925204] niks3-server[1003]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4806worker2 # [ 51.946798] nix-grpc-daemon[783]: ts=2026-09-20T09:33:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_in=6244807worker2 # [ 51.951664] nix-grpc-daemon[783]: ts=2026-09-20T09:33:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924808worker2 # [ 51.962554] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4809worker2 # [ 51.965759] nix-daemon[840]: reaped child process 1075, status = succeeded4810worker2 # [ 51.970623] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4811worker2 # [ 53.189435] nix-daemon[840]: reaped child process 1089, status = succeeded4812worker2 # [ 53.343269] nix-daemon[840]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4813worker2 # [ 53.354254] nix-daemon[840]: reaped child process 1107, status = succeeded4814lb # [ 53.348723] niks3-server[1003]: 2026/09/20 09:33:18 INFO Received uploads request method=POST path=/api/pending_closures4815worker2 # [ 53.377193] nix-grpc-daemon[850]: time=2026-09-20T09:33:17.581Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4816worker2 # [ 53.378831] nix-grpc-daemon[850]: time=2026-09-20T09:33:17.583Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4817lb # [ 53.385393] niks3-server[1003]: 2026/09/20 09:33:18 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4818lb # [ 53.398097] niks3-server[1003]: 2026/09/20 09:33:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4819lb # [ 53.404231] niks3-server[1003]: 2026/09/20 09:33:18 INFO Signed narinfos id=14 count=14820worker2 # [ 53.429585] nix-grpc-daemon[850]: time=2026-09-20T09:33:17.634Z level=INFO msg="Uploading 1 narinfos"4821lb # [ 53.411956] niks3-server[1003]: 2026/09/20 09:33:18 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4822lb # [ 53.421233] niks3-server[1003]: 2026/09/20 09:33:18 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4823lb # [ 53.427205] niks3-server[1003]: 2026/09/20 09:33:18 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4824lb # [ 53.431542] niks3-server[1003]: 2026/09/20 09:33:18 INFO Completed upload id=144825lb # [ 53.433543] niks3-server[1003]: 2026/09/20 09:33:18 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4826worker2 # [ 53.455298] nix-grpc-daemon[850]: time=2026-09-20T09:33:17.660Z level=INFO msg="Upload complete. (277ms)"4827worker1 # [ 53.473511] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:436264828worker2 # [ 53.461623] nix-grpc-daemon[783]: ts=2026-09-20T09:33:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14829worker1 # [ 53.478979] nix-grpc-daemon[783]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4830worker2 # [ 53.474726] nix-daemon[840]: reaped child process 1087, status = succeeded4831worker1 # [ 53.499160] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:436264832worker1 # [ 53.501578] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=0 paths=1 nar_bytes_out=1764833worker2 # [ 53.743073] nix-grpc-daemon[783]: ts=2026-09-20T09:33:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:501924834worker2 # [ 53.750858] nix-grpc-daemon[783]: ts=2026-09-20T09:33:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924835worker2 # [ 53.761700] nix-grpc-daemon[850]: time=2026-09-20T09:33:17.966Z level=INFO msg="All 2 paths already cached"4836worker1 # [ 53.784281] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04837worker1 # [ 53.789173] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14838worker2 # [ 53.784461] nix-grpc-daemon[783]: ts=2026-09-20T09:33:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924839worker2 # [ 53.795559] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924840worker2 # [ 53.808459] nix-grpc-daemon[850]: time=2026-09-20T09:33:18.013Z level=INFO msg="All 2 paths already cached"4841worker2 # [ 53.815792] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501924842worker2 # [ 53.824396] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4843worker2 # [ 53.989230] nix-daemon[840]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)4844worker2 # [ 54.002169] nix-daemon[840]: reaped child process 1114, status = succeeded4845worker2 # [ 54.145239] nix-daemon[840]: accepted connection from pid 1116, user nix-grpc-daemon (trusted)4846worker2 # [ 54.157080] nix-daemon[840]: reaped child process 1119, status = succeeded4847lb # [ 54.150710] niks3-server[1003]: 2026/09/20 09:33:19 INFO Received uploads request method=POST path=/api/pending_closures4848worker2 # [ 54.236743] nix-grpc-daemon[850]: time=2026-09-20T09:33:18.441Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4849worker2 # [ 54.238191] nix-grpc-daemon[850]: time=2026-09-20T09:33:18.441Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4850lb # [ 54.235820] niks3-server[1003]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4851lb # [ 54.244439] niks3-server[1003]: 2026/09/20 09:33:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4852lb # [ 54.249460] niks3-server[1003]: 2026/09/20 09:33:19 INFO Signed narinfos id=15 count=14853worker2 # [ 54.275265] nix-grpc-daemon[850]: time=2026-09-20T09:33:18.479Z level=INFO msg="Uploading 1 narinfos"4854lb # [ 54.262562] niks3-server[1003]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4855lb # [ 54.271997] niks3-server[1003]: 2026/09/20 09:33:19 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4856lb # [ 54.278452] niks3-server[1003]: 2026/09/20 09:33:19 INFO Completed upload id=154857worker2 # [ 54.300555] nix-grpc-daemon[850]: time=2026-09-20T09:33:18.505Z level=WARN msg="Failed to register uploaded object" key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo error="server returned 404: object is not pending upload\n"4858worker2 # [ 54.306061] nix-grpc-daemon[850]: time=2026-09-20T09:33:18.505Z level=INFO msg="Upload complete. (462ms)"4859worker2 # [ 54.310486] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_in=7524860worker2 # [ 54.316742] nix-grpc-daemon[783]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501924861worker2 # [ 54.332373] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4862worker2 # [ 54.337858] nix-daemon[840]: reaped child process 1109, status = succeeded4863worker2 # [ 54.346532] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4864worker2 # [ 55.764569] nix-daemon[840]: reaped child process 1123, status = succeeded4865worker2 # [ 55.889623] nix-daemon[840]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)4866worker2 # [ 55.902247] nix-daemon[840]: reaped child process 1141, status = succeeded4867lb # [ 55.893357] niks3-server[1003]: 2026/09/20 09:33:20 INFO Received uploads request method=POST path=/api/pending_closures4868worker2 # [ 55.923492] nix-grpc-daemon[850]: time=2026-09-20T09:33:20.128Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4869worker2 # [ 55.925444] nix-grpc-daemon[850]: time=2026-09-20T09:33:20.129Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4870lb # [ 55.929809] niks3-server[1003]: 2026/09/20 09:33:20 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4871lb # [ 55.936239] niks3-server[1003]: 2026/09/20 09:33:20 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4872lb # [ 55.948891] niks3-server[1003]: 2026/09/20 09:33:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4873lb # [ 55.953742] niks3-server[1003]: 2026/09/20 09:33:20 INFO Signed narinfos id=16 count=14874worker2 # [ 55.978708] nix-grpc-daemon[850]: time=2026-09-20T09:33:20.183Z level=INFO msg="Uploading 1 narinfos"4875lb # [ 55.963482] niks3-server[1003]: 2026/09/20 09:33:20 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4876lb # [ 55.970349] niks3-server[1003]: 2026/09/20 09:33:20 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4877lb # [ 55.977585] niks3-server[1003]: 2026/09/20 09:33:20 INFO Completed upload id=164878worker2 # [ 55.999860] nix-grpc-daemon[850]: time=2026-09-20T09:33:20.204Z level=INFO msg="Upload complete. (240ms)"4879lb # [ 55.979793] niks3-server[1003]: 2026/09/20 09:33:20 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4880worker2 # [ 56.005564] nix-grpc-daemon[783]: ts=2026-09-20T09:33:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14881worker2 # [ 56.013555] nix-grpc-daemon[783]: ts=2026-09-20T09:33:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:501924882worker2 # [ 56.022368] nix-daemon[840]: reaped child process 1121, status = succeeded4883worker2 # [ 56.026127] nix-grpc-daemon[783]: ts=2026-09-20T09:33:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:501924884worker2 # [ 56.030451] nix-grpc-daemon[783]: ts=2026-09-20T09:33:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_out=2324885client # [ 56.139809] nix-daemon[951]: reaped child process 953, status = succeeded4886client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.56 seconds)4887client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4888client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4889client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4890client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4891client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4892client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4893(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.64 seconds)4894subtest: an input only one worker has still reaches the builder4895client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4896client # this derivation will be built:4897client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4898client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4899client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.40 seconds)4900client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4901client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.19 seconds)4902worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4903worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)4904worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4905worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)4906client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24907client # [ 57.279320] nix-daemon[951]: accepted connection from pid 1033, user root (trusted)4908client # this derivation will be built:4909client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4910worker1 # [ 57.466760] nix-grpc-daemon[783]: ts=2026-09-20T09:33:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:436264911client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...4912worker2 # [ 57.463374] nix-grpc-daemon[783]: ts=2026-09-20T09:33:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501924913client # copying 1 paths...4914client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4915worker2 # [ 57.476301] nix-grpc-daemon[783]: ts=2026-09-20T09:33:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501924916worker2 # [ 57.487189] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4917worker2 # [ 57.657057] nix-daemon[840]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)4918worker2 # [ 57.671363] nix-daemon[840]: reaped child process 1151, status = succeeded4919lb # [ 57.659461] niks3-server[1003]: 2026/09/20 09:33:22 INFO Received uploads request method=POST path=/api/pending_closures4920worker2 # [ 57.689386] nix-grpc-daemon[850]: time=2026-09-20T09:33:21.894Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4921worker2 # [ 57.691152] nix-grpc-daemon[850]: time=2026-09-20T09:33:21.895Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4922lb # [ 57.677721] envoy[910]: conn peer=[2001:db8:1::1]:37420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3423 tx=1768 ms=21474923lb # [ 57.683265] envoy[910]: conn peer=[2001:db8:1::1]:37432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=594924lb # [ 57.686925] envoy[910]: conn peer=[2001:db8:1::1]:37434 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3854 tx=1965 ms=20694925lb # [ 57.692566] envoy[910]: conn peer=[2001:db8:1::1]:38628 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=554926lb # [ 57.696217] envoy[910]: conn peer=[2001:db8:1::1]:38642 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4298 tx=2093 ms=23424927lb # [ 57.699828] envoy[910]: conn peer=[2001:db8:1::1]:38656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=554928lb # [ 57.707637] niks3-server[1003]: 2026/09/20 09:33:22 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4929lb # [ 57.720380] niks3-server[1003]: 2026/09/20 09:33:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4930worker2 # [ 57.746325] nix-grpc-daemon[850]: time=2026-09-20T09:33:21.951Z level=INFO msg="Uploading 1 narinfos"4931lb # [ 57.724481] niks3-server[1003]: 2026/09/20 09:33:22 INFO Signed narinfos id=17 count=14932lb # [ 57.729350] niks3-server[1003]: 2026/09/20 09:33:22 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4933lb # [ 57.736593] niks3-server[1003]: 2026/09/20 09:33:22 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4934lb # [ 57.739379] niks3-server[1003]: 2026/09/20 09:33:22 INFO Completed upload id=174935worker2 # [ 57.761874] nix-grpc-daemon[850]: time=2026-09-20T09:33:21.966Z level=INFO msg="Upload complete. (263ms)"4936lb # [ 57.741854] niks3-server[1003]: 2026/09/20 09:33:22 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4937worker2 # [ 57.766211] nix-grpc-daemon[783]: ts=2026-09-20T09:33:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_in=1204938worker2 # [ 57.778126] nix-daemon[840]: reaped child process 1146, status = succeeded4939worker1 # [ 57.786635] nix-grpc-daemon[783]: ts=2026-09-20T09:33:22Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04940worker1 # [ 57.789655] nix-grpc-daemon[783]: ts=2026-09-20T09:33:22Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14941worker1 # [ 57.830729] nix-grpc-daemon[783]: ts=2026-09-20T09:33:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436264942worker1 # [ 57.840361] nix-grpc-daemon[783]: ts=2026-09-20T09:33:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436264943client # copying 2 paths to 'lb:50051'...4944worker1 # [ 57.852518] nix-grpc-daemon[783]: ts=2026-09-20T09:33:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:436264945client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4946client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4947worker1 # [ 57.871943] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4948worker1 # [ 58.031678] nix-daemon[845]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4949worker1 # [ 58.046636] nix-daemon[845]: reaped child process 994, status = succeeded4950worker1 # [ 58.211841] nix-daemon[845]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4951worker1 # [ 58.225582] nix-daemon[845]: reaped child process 1000, status = succeeded4952lb # [ 58.210539] niks3-server[1003]: 2026/09/20 09:33:23 INFO Received uploads request method=POST path=/api/pending_closures4953lb # [ 58.219153] niks3-server[1003]: 2026/09/20 09:33:23 INFO Received uploads request method=POST path=/api/pending_closures4954worker1 # [ 58.255655] nix-grpc-daemon[898]: time=2026-09-20T09:33:23.426Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4955worker1 # [ 58.257291] nix-grpc-daemon[898]: time=2026-09-20T09:33:23.428Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4956worker1 # [ 58.259185] nix-grpc-daemon[898]: time=2026-09-20T09:33:23.429Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4957lb # [ 58.262507] niks3-server[1003]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst4958lb # [ 58.267085] niks3-server[1003]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4959lb # [ 58.278109] niks3-server[1003]: 2026/09/20 09:33:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4960lb # [ 58.285554] niks3-server[1003]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls4961lb # [ 58.289204] niks3-server[1003]: 2026/09/20 09:33:23 INFO Signed narinfos id=18 count=14962lb # [ 58.291889] niks3-server[1003]: 2026/09/20 09:33:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4963lb # [ 58.294114] niks3-server[1003]: 2026/09/20 09:33:23 INFO Signed narinfos id=19 count=24964lb # [ 58.296183] niks3-server[1003]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls4965worker1 # [ 58.324286] nix-grpc-daemon[898]: time=2026-09-20T09:33:23.494Z level=INFO msg="Uploading 3 narinfos"4966lb # [ 58.310570] niks3-server[1003]: 2026/09/20 09:33:23 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4967lb # [ 58.313719] niks3-server[1003]: 2026/09/20 09:33:23 INFO Completed upload id=184968lb # [ 58.316278] niks3-server[1003]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4969lb # [ 58.318395] niks3-server[1003]: 2026/09/20 09:33:23 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4970lb # [ 58.320643] niks3-server[1003]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4971lb # [ 58.323113] niks3-server[1003]: 2026/09/20 09:33:23 INFO Completed upload id=194972lb # [ 58.324646] niks3-server[1003]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo4973worker1 # [ 58.352372] nix-grpc-daemon[898]: time=2026-09-20T09:33:23.522Z level=INFO msg="Upload complete. (472ms)"4974worker1 # [ 58.358217] nix-grpc-daemon[783]: ts=2026-09-20T09:33:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=0 paths=2 nar_bytes_in=9764975worker1 # [ 58.363699] nix-grpc-daemon[783]: ts=2026-09-20T09:33:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436264976worker1 # [ 58.374268] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4977worker1 # [ 58.385670] nix-daemon[845]: reaped child process 989, status = succeeded4978worker1 # [ 58.388665] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4979client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...4980client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4981worker1 # [ 59.212284] nix-daemon[845]: reaped child process 1004, status = succeeded4982worker1 # [ 59.369123] nix-daemon[845]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4983worker1 # [ 59.381328] nix-daemon[845]: reaped child process 1022, status = succeeded4984lb # [ 59.363489] niks3-server[1003]: 2026/09/20 09:33:24 INFO Received uploads request method=POST path=/api/pending_closures4985worker1 # [ 59.400828] nix-grpc-daemon[898]: time=2026-09-20T09:33:24.571Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4986lb # [ 59.397154] niks3-server[1003]: 2026/09/20 09:33:24 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4987lb # [ 59.404642] niks3-server[1003]: 2026/09/20 09:33:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4988lb # [ 59.410060] niks3-server[1003]: 2026/09/20 09:33:24 INFO Signed narinfos id=20 count=14989worker1 # [ 59.443039] nix-grpc-daemon[898]: time=2026-09-20T09:33:24.613Z level=INFO msg="Uploading 1 narinfos"4990lb # [ 59.419676] niks3-server[1003]: 2026/09/20 09:33:24 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls4991lb # [ 59.431614] niks3-server[1003]: 2026/09/20 09:33:24 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4992lb # [ 59.438620] niks3-server[1003]: 2026/09/20 09:33:24 INFO Completed upload id=204993worker1 # [ 59.467373] nix-grpc-daemon[898]: time=2026-09-20T09:33:24.638Z level=INFO msg="Upload complete. (270ms)"4994lb # [ 59.442270] niks3-server[1003]: 2026/09/20 09:33:24 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo4995worker1 # [ 59.473123] nix-grpc-daemon[783]: ts=2026-09-20T09:33:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=14996worker1 # [ 59.484530] nix-daemon[845]: reaped child process 1002, status = succeeded4997client # copying 1 paths...4998worker1 # [ 59.517479] nix-grpc-daemon[783]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:436264999client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5000worker2 # [ 59.529302] nix-grpc-daemon[783]: ts=2026-09-20T09:33:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:501925001worker2 # [ 59.544820] nix-grpc-daemon[783]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5002worker2 # [ 59.563260] nix-grpc-daemon[783]: ts=2026-09-20T09:33:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=0 paths=1 nar_bytes_out=1205003client # [ 59.664858] nix-daemon[951]: reaped child process 1036, status = succeeded5004client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.60 seconds)5005client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25006client # [ 59.859537] nix-daemon[951]: accepted connection from pid 1055, user root (trusted)5007client # this derivation will be built:5008client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5009worker2 # [ 60.043120] nix-grpc-daemon[783]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:501925010client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5011worker2 # [ 60.049288] nix-grpc-daemon[783]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501925012worker2 # [ 60.055196] nix-grpc-daemon[850]: time=2026-09-20T09:33:24.258Z level=INFO msg="All 1 paths already cached"5013worker1 # [ 60.113854] nix-grpc-daemon[783]: ts=2026-09-20T09:33:25Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05014worker1 # [ 60.118667] nix-grpc-daemon[783]: ts=2026-09-20T09:33:25Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15015worker2 # [ 60.113615] nix-grpc-daemon[783]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501925016worker2 # [ 60.124232] nix-grpc-daemon[783]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501925017client # copying 1 paths to 'lb:50051'...5018client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5019worker2 # [ 60.142767] nix-grpc-daemon[783]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501925020worker2 # [ 60.155171] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5021worker2 # [ 60.171378] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 1/5)5022worker2 # [ 60.220228] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 2/5)5023worker2 # [ 60.291885] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 245 ms (attempt 3/5)5024worker2 # [ 60.543373] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 472 ms (attempt 4/5)5025worker2 # [ 61.021459] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5026worker2 # [ 61.026961] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5027worker2 # [ 61.225318] nix-daemon[840]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5028worker2 # [ 61.238563] nix-daemon[840]: reaped child process 1163, status = succeeded5029worker2 # [ 61.395042] nix-daemon[840]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)5030worker2 # [ 61.406876] nix-daemon[840]: reaped child process 1168, status = succeeded5031lb # [ 61.401805] niks3-server[1003]: 2026/09/20 09:33:26 INFO Received uploads request method=POST path=/api/pending_closures5032worker2 # [ 61.434439] nix-grpc-daemon[850]: time=2026-09-20T09:33:25.639Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5033worker2 # [ 61.436345] nix-grpc-daemon[850]: time=2026-09-20T09:33:25.640Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5034lb # [ 61.432959] niks3-server[1003]: 2026/09/20 09:33:26 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5035lb # [ 61.447780] niks3-server[1003]: 2026/09/20 09:33:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5036lb # [ 61.453519] niks3-server[1003]: 2026/09/20 09:33:26 INFO Signed narinfos id=21 count=15037worker2 # [ 61.479408] nix-grpc-daemon[850]: time=2026-09-20T09:33:25.684Z level=INFO msg="Uploading 1 narinfos"5038lb # [ 61.461332] niks3-server[1003]: 2026/09/20 09:33:26 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5039lb # [ 61.470226] niks3-server[1003]: 2026/09/20 09:33:26 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5040lb # [ 61.477212] niks3-server[1003]: 2026/09/20 09:33:26 INFO Completed upload id=215041worker2 # [ 61.499498] nix-grpc-daemon[850]: time=2026-09-20T09:33:25.704Z level=INFO msg="Upload complete. (450ms)"5042lb # [ 61.479945] niks3-server[1003]: 2026/09/20 09:33:26 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5043worker2 # [ 61.503511] nix-grpc-daemon[783]: ts=2026-09-20T09:33:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 paths=1 nar_bytes_in=5605044worker2 # [ 61.509958] nix-grpc-daemon[783]: ts=2026-09-20T09:33:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501925045worker2 # [ 61.519913] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5046worker2 # [ 61.535365] nix-daemon[840]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5047client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5048worker2 # [ 61.536673] nix-daemon[840]: reaped child process 1154, status = succeeded5049client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5050worker2 # [ 62.348238] nix-daemon[840]: reaped child process 1172, status = succeeded5051worker2 # [ 62.494344] nix-daemon[840]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)5052worker2 # [ 62.506317] nix-daemon[840]: reaped child process 1190, status = succeeded5053lb # [ 62.495888] niks3-server[1003]: 2026/09/20 09:33:27 INFO Received uploads request method=POST path=/api/pending_closures5054worker2 # [ 62.525713] nix-grpc-daemon[850]: time=2026-09-20T09:33:26.730Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5055lb # [ 62.532562] niks3-server[1003]: 2026/09/20 09:33:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5056lb # [ 62.537509] niks3-server[1003]: 2026/09/20 09:33:27 INFO Signed narinfos id=22 count=15057worker2 # [ 62.562939] nix-grpc-daemon[850]: time=2026-09-20T09:33:26.767Z level=INFO msg="Uploading 1 narinfos"5058lb # [ 62.545893] niks3-server[1003]: 2026/09/20 09:33:27 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5059lb # [ 62.553743] niks3-server[1003]: 2026/09/20 09:33:27 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5060lb # [ 62.562475] niks3-server[1003]: 2026/09/20 09:33:27 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5061lb # [ 62.567574] niks3-server[1003]: 2026/09/20 09:33:27 INFO Completed upload id=225062worker2 # [ 62.591808] nix-grpc-daemon[850]: time=2026-09-20T09:33:26.796Z level=INFO msg="Upload complete. (250ms)"5063lb # [ 62.571965] niks3-server[1003]: 2026/09/20 09:33:27 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5064client # copying 1 paths...5065worker2 # [ 62.597495] nix-grpc-daemon[783]: ts=2026-09-20T09:33:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50192 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15066worker2 # [ 62.604156] nix-grpc-daemon[783]: ts=2026-09-20T09:33:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:501925067client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5068worker2 # [ 62.613632] nix-daemon[840]: reaped child process 1170, status = succeeded5069worker1 # [ 62.629559] nix-grpc-daemon[783]: ts=2026-09-20T09:33:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:436265070worker1 # [ 62.640379] nix-grpc-daemon[783]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5071worker1 # [ 62.653745] nix-grpc-daemon[783]: ts=2026-09-20T09:33:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=0 paths=1 nar_bytes_out=1205072client # [ 62.744305] nix-daemon[951]: reaped child process 1058, status = succeeded5073client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.08 seconds)5074client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25075client # [ 62.919112] nix-daemon[951]: accepted connection from pid 1076, user root (trusted)5076client # this derivation will be built:5077client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5078worker2 # [ 63.108249] nix-grpc-daemon[783]: ts=2026-09-20T09:33:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:501925079client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5080worker2 # [ 63.113946] nix-grpc-daemon[783]: ts=2026-09-20T09:33:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501925081worker2 # [ 63.118787] nix-grpc-daemon[850]: time=2026-09-20T09:33:27.322Z level=INFO msg="All 1 paths already cached"5082worker1 # [ 63.135890] nix-grpc-daemon[783]: ts=2026-09-20T09:33:28Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05083worker1 # [ 63.141633] nix-grpc-daemon[783]: ts=2026-09-20T09:33:28Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15084worker1 # [ 63.179687] nix-grpc-daemon[783]: ts=2026-09-20T09:33:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436265085worker1 # [ 63.192214] nix-grpc-daemon[783]: ts=2026-09-20T09:33:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:436265086worker1 # [ 63.205823] nix-grpc-daemon[898]: time=2026-09-20T09:33:28.376Z level=INFO msg="All 1 paths already cached"5087client # copying 1 paths to 'lb:50051'...5088worker1 # [ 63.214086] nix-grpc-daemon[783]: ts=2026-09-20T09:33:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:436265089client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5090worker1 # [ 63.224305] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5091worker1 # [ 63.388275] nix-daemon[845]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)5092worker1 # [ 63.402990] nix-daemon[845]: reaped child process 1034, status = succeeded5093worker1 # [ 63.563466] nix-daemon[845]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)5094worker1 # [ 63.577648] nix-daemon[845]: reaped child process 1039, status = succeeded5095lb # [ 63.562668] niks3-server[1003]: 2026/09/20 09:33:28 INFO Received uploads request method=POST path=/api/pending_closures5096worker1 # [ 63.596858] nix-grpc-daemon[898]: time=2026-09-20T09:33:28.767Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5097worker1 # [ 63.598463] nix-grpc-daemon[898]: time=2026-09-20T09:33:28.769Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5098lb # [ 63.581922] niks3-server[1003]: 2026/09/20 09:33:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5099lb # [ 63.584457] niks3-server[1003]: 2026/09/20 09:33:28 INFO Signed narinfos id=23 count=15100worker1 # [ 63.615148] nix-grpc-daemon[898]: time=2026-09-20T09:33:28.785Z level=INFO msg="Uploading 1 narinfos"5101lb # [ 63.588840] niks3-server[1003]: 2026/09/20 09:33:28 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5102lb # [ 63.591649] niks3-server[1003]: 2026/09/20 09:33:28 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5103lb # [ 63.597958] niks3-server[1003]: 2026/09/20 09:33:28 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5104lb # [ 63.601867] niks3-server[1003]: 2026/09/20 09:33:28 INFO Completed upload id=235105worker1 # [ 63.631068] nix-grpc-daemon[898]: time=2026-09-20T09:33:28.801Z level=INFO msg="Upload complete. (394ms)"5106lb # [ 63.603729] niks3-server[1003]: 2026/09/20 09:33:28 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5107worker1 # [ 63.632946] nix-grpc-daemon[783]: ts=2026-09-20T09:33:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=0 paths=1 nar_bytes_in=5605108worker1 # [ 63.639549] nix-grpc-daemon[783]: ts=2026-09-20T09:33:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:436265109worker1 # [ 63.644878] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5110worker1 # [ 63.647405] nix-daemon[845]: reaped child process 1029, status = succeeded5111worker1 # [ 63.653131] nix-daemon[845]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5112client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5113client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5114worker1 # [ 64.467568] nix-daemon[845]: reaped child process 1043, status = succeeded5115worker1 # [ 64.612815] nix-daemon[845]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)5116worker1 # [ 64.626377] nix-daemon[845]: reaped child process 1061, status = succeeded5117lb # [ 64.612241] niks3-server[1003]: 2026/09/20 09:33:29 INFO Received uploads request method=POST path=/api/pending_closures5118worker1 # [ 64.645869] nix-grpc-daemon[898]: time=2026-09-20T09:33:29.816Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5119lb # [ 64.627413] niks3-server[1003]: 2026/09/20 09:33:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5120lb # [ 64.629317] niks3-server[1003]: 2026/09/20 09:33:29 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5121lb # [ 64.632237] niks3-server[1003]: 2026/09/20 09:33:29 INFO Signed narinfos id=24 count=15122worker1 # [ 64.661306] nix-grpc-daemon[898]: time=2026-09-20T09:33:29.831Z level=INFO msg="Uploading 1 narinfos"5123lb # [ 64.634775] niks3-server[1003]: 2026/09/20 09:33:29 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5124lb # [ 64.639151] niks3-server[1003]: 2026/09/20 09:33:29 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5125lb # [ 64.643270] niks3-server[1003]: 2026/09/20 09:33:29 INFO Completed upload id=245126worker1 # [ 64.673738] nix-grpc-daemon[898]: time=2026-09-20T09:33:29.844Z level=INFO msg="Upload complete. (214ms)"5127lb # [ 64.644939] niks3-server[1003]: 2026/09/20 09:33:29 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5128worker1 # [ 64.676776] nix-grpc-daemon[783]: ts=2026-09-20T09:33:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15129worker1 # [ 64.683622] nix-daemon[845]: reaped child process 1041, status = succeeded5130client # copying 1 paths...5131worker1 # [ 64.720467] nix-grpc-daemon[783]: ts=2026-09-20T09:33:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:436265132client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5133worker1 # [ 64.736125] nix-grpc-daemon[783]: ts=2026-09-20T09:33:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:436265134worker1 # [ 64.739657] nix-grpc-daemon[783]: ts=2026-09-20T09:33:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43626 duration_s=0 paths=1 nar_bytes_out=1205135client # [ 64.817592] nix-daemon[951]: reaped child process 1079, status = succeeded5136client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.08 seconds)5137(finished: subtest: an input only one worker has still reaches the builder, in 8.59 seconds)5138subtest: upload whose reference the worker lacks is completed from the cache5139client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5140client # this derivation will be built:5141client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5142client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5143client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.53 seconds)5144client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5145client # this derivation will be built:5146client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5147client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5148client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.40 seconds)5149client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5150client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5151worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5152worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5153worker2 # [ 66.212772] systemd[1]: Stopping Nix worker-protocol over gRPC...5154worker2 # [ 66.977272] nix-grpc-daemon[783]: ts=2026-09-20T09:33:31Z level=info event=signal_exit5155lb # [ 67.678402] envoy[910]: conn peer=[2001:db8:1::1]:38660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4257 tx=1766 ms=21575156lb # [ 67.682351] envoy[910]: conn peer=[2001:db8:1::1]:38676 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=885157lb # [ 67.685961] envoy[910]: conn peer=[2001:db8:1::1]:38692 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3684 tx=1803 ms=26565158lb # [ 67.689594] envoy[910]: conn peer=[2001:db8:1::1]:38694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=785159lb # [ 67.693659] envoy[910]: conn peer=[2001:db8:1::1]:53748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3712 tx=1803 ms=16765160lb # [ 67.697182] envoy[910]: conn peer=[2001:db8:1::1]:53762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545161worker2 # [ 68.236577] nix-grpc-daemon[783]: ts=2026-09-20T09:33:32Z level=info event=scheduler_restarting5162worker1 # [ 68.255742] nix-grpc-daemon[783]: ts=2026-09-20T09:33:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:436105163worker1 # [ 68.259548] nix-grpc-daemon[783]: ts=2026-09-20T09:33:33Z level=info event=worker_gone addr=192.168.1.4:500515164worker2 # [ 68.259599] nix-grpc-daemon[783]: ts=2026-09-20T09:33:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5165worker2 # [ 68.277535] nix-daemon[840]: reaped child process 842, status = succeeded5166worker2 # [ 68.643844] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5167worker2 # [ 68.647203] systemd[1]: Stopped Nix worker-protocol over gRPC.5168worker2 # [ 68.651672] systemd[1]: nix-grpc-daemon.service: Consumed 2.505s CPU time over 50.947s wall clock time, 60.9M memory peak, 84.8K incoming IP traffic, 154.8K outgoing IP traffic.5169worker2 # [ 68.663609] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5170worker2 # [ 68.666777] systemd[1]: Closed Nix worker-protocol over gRPC.5171worker2 # [ 68.669351] systemd[1]: nix-grpc-daemon.socket: Consumed 298us CPU time over 58.412s wall clock time, 256K memory peak, 70.2K incoming IP traffic, 58.6K outgoing IP traffic.5172lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5173lb: must succeed: curl -sf localhost:9901/clusters5174lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5175lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5176client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25177worker1 # [ 68.969589] nix-grpc-daemon[783]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:436265178worker1 # [ 68.976579] nix-grpc-daemon[783]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:436265179client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5180client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.25 seconds)5181worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5182worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5183worker2 # [ 69.081209] systemd[1]: Listening on Nix worker-protocol over gRPC.5184worker2 # [ 69.094515] systemd[1]: Starting Nix worker-protocol over gRPC...5185worker1 # [ 69.130966] systemd[1]: Stopping Nix worker-protocol over gRPC...5186worker2 # [ 69.237972] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5187worker2 # [ 69.240348] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15188worker2 # [ 69.246678] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5189worker2 # [ 69.250154] systemd[1]: Started Nix worker-protocol over gRPC.5190worker2 # [ 69.277060] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500515191worker1 # [ 69.286296] nix-grpc-daemon[783]: ts=2026-09-20T09:33:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05192worker1 # [ 70.004149] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=signal_exit5193worker1 # [ 70.276645] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=scheduler_yield5194worker1 # [ 70.280883] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:436105195lb # [ 70.252497] niks3-server[1003]: 2026/09/20 09:33:35 INFO lead: released remote=192.168.1.3:355445196worker2 # [ 70.274124] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:34Z level=info event=scheduler_restarting addr=lb:500515197worker1 # [ 70.283129] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=worker_gone addr=192.168.1.4:500515198worker1 # [ 70.284656] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=scheduler_restarting addr=lb:500515199worker1 # [ 70.287648] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:436105200worker1 # [ 70.289405] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=worker_gone addr=192.168.1.3:500515201worker2 # [ 70.473621] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500515202worker1 # [ 70.482490] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500515203worker1 # [ 70.484572] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5204worker2 # [ 70.477768] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5205worker2 # [ 70.678462] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500515206worker1 # [ 70.685509] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500515207worker2 # [ 70.681902] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5208worker1 # [ 70.689610] nix-grpc-daemon[783]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5209worker2 # [ 70.882809] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500515210worker1 # [ 70.890353] nix-grpc-daemon[783]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500515211worker2 # [ 70.886971] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5212worker1 # [ 70.894314] nix-grpc-daemon[783]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5213worker2 # [ 71.087866] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500515214worker1 # [ 71.095122] nix-grpc-daemon[783]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500515215worker1 # [ 71.098939] nix-grpc-daemon[783]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5216worker2 # [ 71.091844] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5217worker1 # [ 71.277460] nix-grpc-daemon[783]: ts=2026-09-20T09:33:36Z level=info event=scheduler_restarting5218worker2 # [ 71.293297] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500515219worker2 # [ 71.297350] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5220worker1 # [ 71.305958] nix-daemon[845]: reaped child process 847, status = succeeded5221worker1 # [ 71.316240] nix-grpc-daemon[783]: ts=2026-09-20T09:33:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5222worker2 # [ 71.497932] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500515223worker2 # [ 71.501791] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5224worker2 # [ 71.702574] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500515225worker2 # [ 71.706521] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5226worker2 # [ 71.907624] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500515227worker2 # [ 71.911580] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5228worker2 # [ 72.112452] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500515229worker2 # [ 72.116439] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5230worker1 # [ 72.146530] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5231worker1 # [ 72.150246] nix-grpc-daemon[783]: E0000 00:00:1789896817.316439 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5232worker1 # [ 72.154921] nix-grpc-daemon[783]: E0000 00:00:1789896817.317527 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5233worker2 # [ 72.318450] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500515234worker2 # [ 72.321550] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5235worker2 # [ 72.519466] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500515236worker2 # [ 72.523381] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5237worker1 # [ 72.571557] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5238worker1 # [ 72.574835] systemd[1]: Stopped Nix worker-protocol over gRPC.5239worker1 # [ 72.579164] systemd[1]: nix-grpc-daemon.service: Consumed 1.459s CPU time over 54.870s wall clock time, 52.8M memory peak, 51.2K incoming IP traffic, 97.1K outgoing IP traffic.5240worker1 # [ 72.590570] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5241worker1 # [ 72.594724] systemd[1]: Closed Nix worker-protocol over gRPC.5242worker1 # [ 72.598315] systemd[1]: nix-grpc-daemon.socket: Consumed 297us CPU time over 1min 3.313s wall clock time, 256K memory peak, 73.5K incoming IP traffic, 60.6K outgoing IP traffic.5243lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5244lb: must succeed: curl -sf localhost:9901/clusters5245lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5246lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.04 seconds)5247client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25248worker2 # [ 72.724125] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500515249worker2 # [ 72.727930] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5250worker2 # [ 72.854938] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:465745251worker2 # [ 72.866510] nix-daemon[840]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5252worker2 # [ 72.883331] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)5253worker2 # [ 72.928496] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500515254worker2 # [ 72.932346] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5255worker2 # [ 72.979282] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 2/5)5256worker2 # [ 73.133388] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500515257worker2 # [ 73.137394] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258worker2 # [ 73.153956] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 218 ms (attempt 3/5)5259worker2 # [ 73.339406] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500515260worker2 # [ 73.342986] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5261worker2 # [ 73.377688] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 118 ms (attempt 4/5)5262worker2 # [ 73.502258] nix-grpc-daemon[1211]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5263client # copying 2 paths...5264client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5265worker2 # [ 73.524694] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:465745266client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5267worker2 # [ 73.535295] nix-daemon[840]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5268worker2 # [ 73.543648] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500515269worker2 # [ 73.547479] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5270worker2 # [ 73.565349] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=niks3_push_started pid=12675271worker2 # [ 73.751840] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500515272worker2 # [ 73.753574] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5273worker2 # [ 73.907881] nix-daemon[840]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5274worker2 # [ 73.918566] nix-daemon[840]: reaped child process 1283, status = succeeded5275lb # [ 73.916579] niks3-server[1003]: 2026/09/20 09:33:38 INFO Received uploads request method=POST path=/api/pending_closures5276lb # [ 73.924193] niks3-server[1003]: 2026/09/20 09:33:38 INFO Received uploads request method=POST path=/api/pending_closures5277worker2 # [ 73.953988] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500515278worker2 # [ 73.955862] nix-grpc-daemon[1267]: time=2026-09-20T09:33:38.160Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5279worker2 # [ 73.960375] nix-grpc-daemon[1267]: time=2026-09-20T09:33:38.162Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5280worker2 # [ 73.962062] nix-grpc-daemon[1267]: time=2026-09-20T09:33:38.162Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5281worker2 # [ 73.963702] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5282lb # [ 73.977564] niks3-server[1003]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5283lb # [ 73.987176] niks3-server[1003]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5284lb # [ 73.994352] niks3-server[1003]: 2026/09/20 09:33:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5285lb # [ 74.002276] niks3-server[1003]: 2026/09/20 09:33:38 INFO Signed narinfos id=25 count=15286lb # [ 74.004280] niks3-server[1003]: 2026/09/20 09:33:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5287lb # [ 74.006259] niks3-server[1003]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5288lb # [ 74.008022] niks3-server[1003]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5289lb # [ 74.010109] niks3-server[1003]: 2026/09/20 09:33:38 INFO Signed narinfos id=26 count=25290worker2 # [ 74.034257] nix-grpc-daemon[1267]: time=2026-09-20T09:33:38.238Z level=INFO msg="Uploading 3 narinfos"5291lb # [ 74.026861] niks3-server[1003]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5292lb # [ 74.029414] niks3-server[1003]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5293lb # [ 74.033250] niks3-server[1003]: 2026/09/20 09:33:38 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5294lb # [ 74.036450] niks3-server[1003]: 2026/09/20 09:33:38 INFO Completed upload id=255295lb # [ 74.037819] niks3-server[1003]: 2026/09/20 09:33:38 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5296lb # [ 74.039855] niks3-server[1003]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5297lb # [ 74.042146] niks3-server[1003]: 2026/09/20 09:33:39 INFO Completed upload id=265298worker2 # [ 74.064529] nix-grpc-daemon[1267]: time=2026-09-20T09:33:38.269Z level=INFO msg="Upload complete. (319ms)"5299worker2 # [ 74.070158] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46574 duration_s=0 paths=2 nar_bytes_in=3125300worker2 # [ 74.081889] nix-daemon[840]: reaped child process 1265, status = succeeded5301client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.46 seconds)5302worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5303worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5304worker2 # [ 74.159283] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500515305worker1 # [ 74.171402] systemd[1]: Listening on Nix worker-protocol over gRPC.5306worker2 # [ 74.163065] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5307worker1 # [ 74.176572] systemd[1]: Starting Nix worker-protocol over gRPC...5308lb # [ 74.234943] niks3-server[1003]: 2026/09/20 09:33:39 INFO lead: acquired remote=192.168.1.4:445665309worker2 # [ 74.257544] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=scheduler_take_over5310worker1 # [ 74.336414] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5311worker1 # [ 74.341059] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15312worker1 # [ 74.347990] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5313worker1 # [ 74.352790] systemd[1]: Started Nix worker-protocol over gRPC.5314lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5315lb: must succeed: curl -sf localhost:9901/clusters5316worker2 # [ 74.364719] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500515317worker2 # [ 74.372948] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05318worker1 # [ 74.387723] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500515319worker2 # [ 74.382085] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05320lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5321lb: must succeed: curl -sf localhost:9901/clusters5322lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5323lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.13 seconds)5324(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.65 seconds)5325subtest: scheduler down: the other node takes the lock and keeps it5326lb: must succeed: curl -sf localhost:9901/clusters5327lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5328lb: must succeed: curl -sf localhost:9901/clusters5329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5330lb: must succeed: curl -sf localhost:9901/clusters5331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5332lb: must succeed: curl -sf localhost:9901/clusters5333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5334worker2 # [ 75.706451] systemd[1]: Stopping Nix worker-protocol over gRPC...5335worker2 # [ 76.252050] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=signal_exit5336worker2 # [ 76.462817] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=scheduler_yield5337worker2 # [ 76.467988] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:465825338lb # [ 76.445665] niks3-server[1003]: 2026/09/20 09:33:41 INFO lead: released remote=192.168.1.4:445665339worker2 # [ 76.469358] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=worker_gone addr=192.168.1.3:500515340worker1 # [ 76.474788] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:41Z level=info event=scheduler_restarting addr=lb:500515341worker2 # [ 76.471146] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=scheduler_restarting addr=lb:500515342worker2 # [ 76.473118] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:465825343worker2 # [ 76.474435] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=worker_gone addr=192.168.1.4:500515344worker1 # [ 76.674668] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 76.676939] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker2 # [ 76.672479] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500515347worker2 # [ 76.676502] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5348worker1 # [ 76.877709] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500515349worker1 # [ 76.879719] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5350worker2 # [ 76.877369] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500515351worker2 # [ 76.881297] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5352worker1 # [ 77.080779] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500515353worker1 # [ 77.084949] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5354worker2 # [ 77.081664] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500515355worker2 # [ 77.083463] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5356worker1 # [ 77.286332] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500515357worker2 # [ 77.284178] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500515358worker1 # [ 77.290306] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5359worker2 # [ 77.285799] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5360worker2 # [ 77.464079] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:41Z level=info event=scheduler_restarting5361worker2 # [ 77.483402] nix-daemon[840]: reaped child process 1260, status = succeeded5362worker1 # [ 77.492810] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500515363worker1 # [ 77.496171] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker2 # [ 77.497183] nix-grpc-daemon[1211]: ts=2026-09-20T09:33:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5365worker1 # [ 77.693876] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500515366worker1 # [ 77.697980] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367lb # [ 77.679179] envoy[910]: conn peer=[2001:db8:1::4]:46612 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=8858 tx=2017 ms=502165368lb # [ 77.683211] envoy[910]: conn peer=[2001:db8:1::1]:53764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215369lb # [ 77.686688] envoy[910]: conn peer=[2001:db8:1::3]:56064 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=8941 tx=1787 ms=532595370lb # [ 77.690345] envoy[910]: conn peer=[2001:db8:1::1]:57994 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1004 tx=282 ms=12285371lb # [ 77.693857] envoy[910]: conn peer=[2001:db8:1::4]:45266 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5259 tx=1022 ms=82145372worker1 # [ 77.898946] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_connected addr=lb:500515373worker1 # [ 77.903144] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374worker1 # [ 78.103724] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_connected addr=lb:500515375worker1 # [ 78.107758] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker2 # [ 78.280907] nix-grpc-daemon[1211]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5377worker2 # [ 78.285129] nix-grpc-daemon[1211]: E0000 00:00:1789896822.485282 1254 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5378worker2 # [ 78.289903] nix-grpc-daemon[1211]: E0000 00:00:1789896822.485799 1254 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5379worker1 # [ 78.308773] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_connected addr=lb:500515380worker1 # [ 78.312892] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5381worker1 # [ 78.514412] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_connected addr=lb:500515382worker1 # [ 78.516201] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5383worker1 # [ 78.716973] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_connected addr=lb:500515384worker1 # [ 78.719070] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5385worker1 # [ 78.919752] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_connected addr=lb:500515386worker1 # [ 78.921727] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker1 # [ 79.122933] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_connected addr=lb:500515388worker1 # [ 79.127673] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5389worker2 # [ 79.280747] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5390worker2 # [ 79.283582] systemd[1]: Stopped Nix worker-protocol over gRPC.5391worker2 # [ 79.286277] systemd[1]: nix-grpc-daemon.service: Consumed 362ms CPU time over 10.183s wall clock time, 32.5M memory peak, 21.1K incoming IP traffic, 31.1K outgoing IP traffic.5392worker2 # [ 79.296847] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5393worker2 # [ 79.301165] systemd[1]: Closed Nix worker-protocol over gRPC.5394worker2 # [ 79.304522] systemd[1]: nix-grpc-daemon.socket: Consumed 294us CPU time over 10.225s wall clock time, 256K memory peak, 20.1K incoming IP traffic, 15.2K outgoing IP traffic.5395worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5396worker1 # [ 79.328556] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_connected addr=lb:500515397worker1 # [ 79.333838] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5398lb # [ 79.331414] niks3-server[1003]: 2026/09/20 09:33:44 INFO lead: acquired remote=192.168.1.3:597785399worker1 # [ 79.363789] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_take_over5400worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5401worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5402worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.05 seconds)5403worker1 # [ 79.534274] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_connected addr=lb:500515404worker1 # [ 79.537487] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5405worker1 # [ 79.738992] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=scheduler_connected addr=lb:500515406worker1 # [ 79.747214] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05407worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5408worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5409client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25410worker1 # [ 80.742913] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:599885411worker1 # [ 80.755619] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5412client # don't know how to build these paths:5413client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5414worker1 # [ 80.770647] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599885415worker1 # [ 80.781977] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)5416worker1 # [ 80.803581] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 2/5)5417worker1 # [ 80.885537] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 129 ms (attempt 3/5)5418worker1 # [ 81.020897] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 4/5)5419worker1 # [ 81.095555] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5420worker1 # [ 81.124724] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:46Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05421worker1 # [ 81.129661] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:46Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15422worker1 # [ 81.174575] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599885423worker1 # [ 81.188664] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599885424client # copying 1 paths to 'lb:50051'...5425worker1 # [ 81.197535] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599885426client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5427worker1 # [ 81.202853] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5428worker1 # [ 81.215249] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:46Z level=info event=niks3_push_started pid=11585429worker1 # [ 81.549582] nix-daemon[845]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5430worker1 # [ 81.563938] nix-daemon[845]: reaped child process 1174, status = succeeded5431worker1 # [ 81.727418] nix-daemon[845]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5432worker1 # [ 81.741528] nix-daemon[845]: reaped child process 1179, status = succeeded5433lb # [ 81.727887] niks3-server[1003]: 2026/09/20 09:33:46 INFO Received uploads request method=POST path=/api/pending_closures5434worker1 # [ 81.766399] nix-grpc-daemon[1158]: time=2026-09-20T09:33:46.935Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5435worker1 # [ 81.767832] nix-grpc-daemon[1158]: time=2026-09-20T09:33:46.935Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5436lb # [ 81.770174] niks3-server[1003]: 2026/09/20 09:33:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5437lb # [ 81.781305] niks3-server[1003]: 2026/09/20 09:33:46 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5438lb # [ 81.800716] niks3-server[1003]: 2026/09/20 09:33:46 INFO Signed narinfos id=27 count=15439worker1 # [ 81.833712] nix-grpc-daemon[1158]: time=2026-09-20T09:33:47.004Z level=INFO msg="Uploading 1 narinfos"5440lb # [ 81.812293] niks3-server[1003]: 2026/09/20 09:33:46 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5441lb # [ 81.820079] niks3-server[1003]: 2026/09/20 09:33:46 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5442lb # [ 81.823110] niks3-server[1003]: 2026/09/20 09:33:46 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5443lb # [ 81.826328] niks3-server[1003]: 2026/09/20 09:33:46 INFO Completed upload id=275444worker1 # [ 81.857144] nix-grpc-daemon[1158]: time=2026-09-20T09:33:47.027Z level=INFO msg="Upload complete. (462ms)"5445worker1 # [ 81.863460] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59988 duration_s=0 paths=1 nar_bytes_in=5205446worker1 # [ 81.869798] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599885447worker1 # [ 81.885918] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5448worker1 # [ 81.892187] nix-daemon[845]: reaped child process 1156, status = succeeded5449worker1 # [ 81.899536] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5450client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5451client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5452client # farm-a> LOG-farm-a-on-standby5453worker1 # [ 83.315889] nix-daemon[845]: reaped child process 1183, status = succeeded5454worker1 # [ 83.466087] nix-daemon[845]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)5455worker1 # [ 83.480656] nix-daemon[845]: reaped child process 1201, status = succeeded5456lb # [ 83.466803] niks3-server[1003]: 2026/09/20 09:33:48 INFO Received uploads request method=POST path=/api/pending_closures5457worker1 # [ 83.518441] nix-grpc-daemon[1158]: time=2026-09-20T09:33:48.688Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5458worker1 # [ 83.521063] nix-grpc-daemon[1158]: time=2026-09-20T09:33:48.690Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5459lb # [ 83.525516] niks3-server[1003]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5460lb # [ 83.533129] niks3-server[1003]: 2026/09/20 09:33:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5461lb # [ 83.536811] niks3-server[1003]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5462lb # [ 83.541532] niks3-server[1003]: 2026/09/20 09:33:48 INFO Signed narinfos id=28 count=15463worker1 # [ 83.571244] nix-grpc-daemon[1158]: time=2026-09-20T09:33:48.741Z level=INFO msg="Uploading 1 narinfos"5464lb # [ 83.548489] niks3-server[1003]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5465lb # [ 83.556152] niks3-server[1003]: 2026/09/20 09:33:48 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5466lb # [ 83.581483] niks3-server[1003]: 2026/09/20 09:33:48 INFO Completed upload id=285467worker1 # [ 83.610946] nix-grpc-daemon[1158]: time=2026-09-20T09:33:48.781Z level=INFO msg="Upload complete. (303ms)"5468worker1 # [ 83.622922] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59988 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15469worker1 # [ 83.632891] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:48Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05470worker1 # [ 83.638299] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:48Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15471lb # [ 83.613875] niks3-server[1003]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5472worker1 # [ 83.648515] nix-daemon[845]: reaped child process 1181, status = succeeded5473worker1 # [ 83.674780] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599885474worker1 # [ 83.686260] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599885475worker1 # [ 83.697489] nix-grpc-daemon[1158]: time=2026-09-20T09:33:48.868Z level=INFO msg="All 1 paths already cached"5476client # copying 1 paths to 'lb:50051'...5477client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5478worker1 # [ 83.705148] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599885479worker1 # [ 83.714182] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5480worker1 # [ 83.877243] nix-daemon[845]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)5481worker1 # [ 83.891072] nix-daemon[845]: reaped child process 1210, status = succeeded5482worker1 # [ 84.044465] nix-daemon[845]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)5483worker1 # [ 84.058384] nix-daemon[845]: reaped child process 1215, status = succeeded5484lb # [ 84.040094] niks3-server[1003]: 2026/09/20 09:33:49 INFO Received uploads request method=POST path=/api/pending_closures5485worker1 # [ 84.076106] nix-grpc-daemon[1158]: time=2026-09-20T09:33:49.246Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5486worker1 # [ 84.078105] nix-grpc-daemon[1158]: time=2026-09-20T09:33:49.248Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5487lb # [ 84.057456] niks3-server[1003]: 2026/09/20 09:33:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5488lb # [ 84.060628] niks3-server[1003]: 2026/09/20 09:33:49 INFO Signed narinfos id=29 count=15489worker1 # [ 84.091513] nix-grpc-daemon[1158]: time=2026-09-20T09:33:49.262Z level=INFO msg="Uploading 1 narinfos"5490lb # [ 84.063372] niks3-server[1003]: 2026/09/20 09:33:49 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5491lb # [ 84.066242] niks3-server[1003]: 2026/09/20 09:33:49 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5492lb # [ 84.070598] niks3-server[1003]: 2026/09/20 09:33:49 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5493lb # [ 84.075927] niks3-server[1003]: 2026/09/20 09:33:49 INFO Completed upload id=295494worker1 # [ 84.105357] nix-grpc-daemon[1158]: time=2026-09-20T09:33:49.275Z level=INFO msg="Upload complete. (378ms)"5495lb # [ 84.077724] niks3-server[1003]: 2026/09/20 09:33:49 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5496worker1 # [ 84.106598] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59988 duration_s=0 paths=1 nar_bytes_in=6405497worker1 # [ 84.112203] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599885498worker1 # [ 84.118199] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5499worker1 # [ 84.120656] nix-daemon[845]: reaped child process 1205, status = succeeded5500worker1 # [ 84.126150] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5501client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5502client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5503worker1 # [ 85.140863] nix-daemon[845]: reaped child process 1219, status = succeeded5504client # farm-b> LOG-farm-b-on-standby5505worker1 # [ 85.280194] nix-daemon[845]: accepted connection from pid 1234, user nix-grpc-daemon (trusted)5506worker1 # [ 85.295708] nix-daemon[845]: reaped child process 1237, status = succeeded5507lb # [ 85.278614] niks3-server[1003]: 2026/09/20 09:33:50 INFO Received uploads request method=POST path=/api/pending_closures5508worker1 # [ 85.312584] nix-grpc-daemon[1158]: time=2026-09-20T09:33:50.483Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5509worker1 # [ 85.314170] nix-grpc-daemon[1158]: time=2026-09-20T09:33:50.484Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5510lb # [ 85.295596] niks3-server[1003]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5511lb # [ 85.301568] niks3-server[1003]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5512lb # [ 85.309200] niks3-server[1003]: 2026/09/20 09:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5513worker1 # [ 85.342320] nix-grpc-daemon[1158]: time=2026-09-20T09:33:50.512Z level=INFO msg="Uploading 1 narinfos"5514lb # [ 85.313106] niks3-server[1003]: 2026/09/20 09:33:50 INFO Signed narinfos id=30 count=15515lb # [ 85.317191] niks3-server[1003]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5516lb # [ 85.322321] niks3-server[1003]: 2026/09/20 09:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5517lb # [ 85.324899] niks3-server[1003]: 2026/09/20 09:33:50 INFO Completed upload id=305518worker1 # [ 85.355319] nix-grpc-daemon[1158]: time=2026-09-20T09:33:50.526Z level=INFO msg="Upload complete. (222ms)"5519lb # [ 85.327688] niks3-server[1003]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5520worker1 # [ 85.359057] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59988 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15521worker1 # [ 85.365299] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:50Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05522worker1 # [ 85.367147] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:50Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15523worker1 # [ 85.371925] nix-daemon[845]: reaped child process 1217, status = succeeded5524worker1 # [ 85.410565] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599885525worker1 # [ 85.421625] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599885526worker1 # [ 85.432973] nix-grpc-daemon[1158]: time=2026-09-20T09:33:50.603Z level=INFO msg="All 2 paths already cached"5527client # copying 1 paths to 'lb:50051'...5528client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5529worker1 # [ 85.440380] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599885530worker1 # [ 85.447314] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5531worker1 # [ 85.606082] nix-daemon[845]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5532worker1 # [ 85.620231] nix-daemon[845]: reaped child process 1246, status = succeeded5533worker1 # [ 85.770618] nix-daemon[845]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)5534worker1 # [ 85.785108] nix-daemon[845]: reaped child process 1251, status = succeeded5535lb # [ 85.767351] niks3-server[1003]: 2026/09/20 09:33:50 INFO Received uploads request method=POST path=/api/pending_closures5536worker1 # [ 85.805934] nix-grpc-daemon[1158]: time=2026-09-20T09:33:50.976Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5537worker1 # [ 85.807499] nix-grpc-daemon[1158]: time=2026-09-20T09:33:50.978Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5538lb # [ 85.798125] niks3-server[1003]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5539lb # [ 85.809125] niks3-server[1003]: 2026/09/20 09:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5540lb # [ 85.814527] niks3-server[1003]: 2026/09/20 09:33:50 INFO Signed narinfos id=31 count=15541worker1 # [ 85.846800] nix-grpc-daemon[1158]: time=2026-09-20T09:33:51.017Z level=INFO msg="Uploading 1 narinfos"5542lb # [ 85.819542] niks3-server[1003]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5543lb # [ 85.831697] niks3-server[1003]: 2026/09/20 09:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5544worker1 # [ 85.867234] nix-grpc-daemon[1158]: time=2026-09-20T09:33:51.037Z level=INFO msg="Upload complete. (408ms)"5545lb # [ 85.838080] niks3-server[1003]: 2026/09/20 09:33:50 INFO Completed upload id=315546worker1 # [ 85.869938] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59988 duration_s=0 paths=1 nar_bytes_in=7685547lb # [ 85.842615] niks3-server[1003]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5548worker1 # [ 85.875951] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599885549worker1 # [ 85.886161] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5550worker1 # [ 85.890166] nix-daemon[845]: reaped child process 1241, status = succeeded5551worker1 # [ 85.895525] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5552client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5553client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5554client # farm-top> LOG-farm-top-on-standby5555worker1 # [ 87.124640] nix-daemon[845]: reaped child process 1255, status = succeeded5556worker1 # [ 87.287164] nix-daemon[845]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5557worker1 # [ 87.301115] nix-daemon[845]: reaped child process 1273, status = succeeded5558lb # [ 87.287659] niks3-server[1003]: 2026/09/20 09:33:52 INFO Received uploads request method=POST path=/api/pending_closures5559worker1 # [ 87.327903] nix-grpc-daemon[1158]: time=2026-09-20T09:33:52.498Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5560worker1 # [ 87.329555] nix-grpc-daemon[1158]: time=2026-09-20T09:33:52.500Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5561lb # [ 87.330234] niks3-server[1003]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5562lb # [ 87.342757] niks3-server[1003]: 2026/09/20 09:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5563worker1 # [ 87.378642] nix-grpc-daemon[1158]: time=2026-09-20T09:33:52.548Z level=INFO msg="Uploading 1 narinfos"5564lb # [ 87.348817] niks3-server[1003]: 2026/09/20 09:33:52 INFO Signed narinfos id=32 count=15565lb # [ 87.352262] niks3-server[1003]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5566lb # [ 87.360968] niks3-server[1003]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5567lb # [ 87.368805] niks3-server[1003]: 2026/09/20 09:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5568lb # [ 87.373135] niks3-server[1003]: 2026/09/20 09:33:52 INFO Completed upload id=325569lb # [ 87.374746] niks3-server[1003]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5570worker1 # [ 87.402355] nix-grpc-daemon[1158]: time=2026-09-20T09:33:52.573Z level=INFO msg="Upload complete. (297ms)"5571worker1 # [ 87.411134] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59988 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15572worker1 # [ 87.421915] nix-daemon[845]: reaped child process 1253, status = succeeded5573client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.92 seconds)5574client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5575client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5576worker2 # [ 87.700700] systemd[1]: Listening on Nix worker-protocol over gRPC.5577lb # [ 87.678817] envoy[910]: conn peer=[2001:db8:1::1]:57998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9590 tx=3818 ms=66825578worker2 # [ 87.708876] systemd[1]: Starting Nix worker-protocol over gRPC...5579worker2 # [ 87.868970] nix-grpc-daemon[1304]: ts=2026-09-20T09:33:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5580worker2 # [ 87.872177] nix-grpc-daemon[1304]: ts=2026-09-20T09:33:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15581worker2 # [ 87.878058] nix-grpc-daemon[1304]: ts=2026-09-20T09:33:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5582worker2 # [ 87.882238] systemd[1]: Started Nix worker-protocol over gRPC.5583lb: must succeed: curl -sf localhost:9901/clusters5584worker2 # [ 87.910230] nix-grpc-daemon[1304]: ts=2026-09-20T09:33:52Z level=info event=scheduler_connected addr=lb:500515585worker1 # [ 87.919909] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5587worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5588worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5589lb: must succeed: curl -sf localhost:9901/clusters5590lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5591lb: must succeed: curl -sf localhost:9901/clusters5592lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5593worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5594worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5595(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.73 seconds)5596subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5597client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5598client # [ 88.242101] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5599client # Running as unit: rs.service; invocation ID: 820cc460e63a405982ed38b9cc3c49dd5600client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.04 seconds)5601worker1 # [ 88.460472] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:599885602client # [ 88.453800] nix[1179]: don't know how to build these paths:5603client # [ 88.454790] nix[1179]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5604worker1 # [ 88.473737] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:53Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05605worker1 # [ 88.479230] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:53Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15606worker1 # [ 88.517312] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599885607worker1 # [ 88.528705] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599885608client # [ 88.525685] nix[1179]: copying 1 paths to 'lb:50051'...5609client # [ 88.526603] nix[1179]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5610worker1 # [ 88.539439] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599885611worker1 # [ 88.550851] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5612worker1 # [ 88.695100] nix-daemon[845]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)5613worker1 # [ 88.707853] nix-daemon[845]: reaped child process 1291, status = succeeded5614worker1 # [ 88.849288] nix-daemon[845]: accepted connection from pid 1293, user nix-grpc-daemon (trusted)5615worker1 # [ 88.860300] nix-daemon[845]: reaped child process 1296, status = succeeded5616lb # [ 88.847669] niks3-server[1003]: 2026/09/20 09:33:53 INFO Received uploads request method=POST path=/api/pending_closures5617worker1 # [ 88.885523] nix-grpc-daemon[1158]: time=2026-09-20T09:33:54.056Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5618worker1 # [ 88.887627] nix-grpc-daemon[1158]: time=2026-09-20T09:33:54.058Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5619lb # [ 88.875914] niks3-server[1003]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5620lb # [ 88.885458] niks3-server[1003]: 2026/09/20 09:33:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5621lb # [ 88.889659] niks3-server[1003]: 2026/09/20 09:33:53 INFO Signed narinfos id=33 count=15622worker1 # [ 88.921923] nix-grpc-daemon[1158]: time=2026-09-20T09:33:54.092Z level=INFO msg="Uploading 1 narinfos"5623lb # [ 88.904599] niks3-server[1003]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5624lb # [ 88.907441] niks3-server[1003]: 2026/09/20 09:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5625worker1 # [ 88.942823] nix-grpc-daemon[1158]: time=2026-09-20T09:33:54.113Z level=INFO msg="Upload complete. (380ms)"5626lb # [ 88.913674] niks3-server[1003]: 2026/09/20 09:33:53 INFO Completed upload id=335627worker1 # [ 88.944907] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59988 duration_s=0 paths=1 nar_bytes_in=4165628lb # [ 88.919248] niks3-server[1003]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5629worker1 # [ 88.950182] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599885630worker1 # [ 88.956869] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5631worker1 # [ 88.961875] nix-daemon[845]: reaped child process 1286, status = succeeded5632client # [ 88.953187] nix[1179]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5633worker1 # [ 88.966960] nix-daemon[845]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5634lb: must succeed: curl -sf localhost:9901/clusters5635lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5636lb: must succeed: curl -sf localhost:9901/clusters5637lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5638lb: must succeed: curl -sf localhost:9901/clusters5639lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5640lb: must succeed: curl -sf localhost:9901/clusters5641lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5642worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5643worker2 # [ 91.095902] systemd-journald[286]: Received client request to rotate journal, rotating.5644worker2 # [ 91.113347] systemd-journald[286]: Vacuuming done, freed 0B of archived journals from /var/log/journal/5f2bab52de4a4ddc99f8d1cd976a516c.5645worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5646worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5647worker1 # [ 91.041092] systemd[1]: Stopping Nix worker-protocol over gRPC...5648worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5649??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5650 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735651client: must succeed: sleep 5.05652worker1 # [ 91.356648] nix-grpc-daemon[1084]: ts=2026-09-20T09:33:56Z level=info event=draining5653client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5654worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5655worker1 # [ 96.397743] nix-daemon[845]: reaped child process 1300, status = succeeded5656worker1 # [ 96.561621] nix-daemon[845]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5657worker1 # [ 96.572326] nix-daemon[845]: reaped child process 1343, status = succeeded5658lb # [ 96.561390] niks3-server[1003]: 2026/09/20 09:34:01 INFO Received uploads request method=POST path=/api/pending_closures5659worker1 # [ 96.601411] nix-grpc-daemon[1158]: time=2026-09-20T09:34:01.771Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5660worker1 # [ 96.602898] nix-grpc-daemon[1158]: time=2026-09-20T09:34:01.773Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5661lb # [ 96.603650] niks3-server[1003]: 2026/09/20 09:34:01 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5662lb # [ 96.616388] niks3-server[1003]: 2026/09/20 09:34:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5663lb # [ 96.621645] niks3-server[1003]: 2026/09/20 09:34:01 INFO Signed narinfos id=34 count=15664worker1 # [ 96.654109] nix-grpc-daemon[1158]: time=2026-09-20T09:34:01.824Z level=INFO msg="Uploading 1 narinfos"5665lb # [ 96.627632] niks3-server[1003]: 2026/09/20 09:34:01 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5666lb # [ 96.636632] niks3-server[1003]: 2026/09/20 09:34:01 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5667lb # [ 96.645352] niks3-server[1003]: 2026/09/20 09:34:01 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5668lb # [ 96.649879] niks3-server[1003]: 2026/09/20 09:34:01 INFO Completed upload id=345669lb # [ 96.651786] niks3-server[1003]: 2026/09/20 09:34:01 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5670worker1 # [ 96.679312] nix-grpc-daemon[1158]: time=2026-09-20T09:34:01.849Z level=INFO msg="Upload complete. (298ms)"5671worker1 # [ 96.686895] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59988 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15672worker1 # [ 96.700534] nix-daemon[845]: reaped child process 1298, status = succeeded5673client # [ 96.702281] systemd[1]: rs.service: Deactivated successfully.5674client # [ 96.706211] systemd[1]: rs.service: Consumed 102ms CPU time over 8.462s wall clock time, 39.1M memory peak, 8.4K incoming IP traffic, 9.1K outgoing IP traffic.5675worker1 # [ 97.357987] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=signal_exit5676worker1 # [ 97.581104] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=scheduler_yield5677lb # [ 97.554261] niks3-server[1003]: 2026/09/20 09:34:02 INFO lead: released remote=192.168.1.3:597785678worker2 # [ 97.577181] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:01Z level=info event=scheduler_restarting addr=lb:500515679worker1 # [ 97.587571] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:599805680worker1 # [ 97.590945] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=worker_gone addr=192.168.1.4:500515681worker1 # [ 97.595979] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=scheduler_restarting addr=lb:500515682worker1 # [ 97.599159] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:599805683worker1 # [ 97.602482] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=worker_gone addr=192.168.1.3:500515684lb # [ 97.680388] envoy[910]: conn peer=[2001:db8:1::1]:33822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1324 ms=82345685worker2 # [ 97.777424] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:01Z level=info event=scheduler_connected addr=lb:500515686worker2 # [ 97.781681] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5687worker1 # [ 97.797224] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515688worker1 # [ 97.800417] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5689worker2 # [ 97.890207] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_take_over5690lb # [ 97.868388] niks3-server[1003]: 2026/09/20 09:34:02 INFO lead: acquired remote=192.168.1.4:455865691worker2 # [ 97.982246] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515692worker2 # [ 97.986127] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5693worker1 # [ 97.997790] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515694worker1 # [ 98.001652] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5695worker2 # [ 98.186913] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515696worker2 # [ 98.190832] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5697worker1 # [ 98.202581] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515698worker1 # [ 98.206665] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5699worker2 # [ 98.391610] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515700worker2 # [ 98.395677] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5701worker1 # [ 98.407307] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515702worker1 # [ 98.411131] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5703worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.14 seconds)5704client: waiting for success: ! systemctl is-active rs5705client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5706client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5707client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5708client: must fail: journalctl -u rs | grep -q 'reconnecting'5709worker1 # [ 98.585246] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:03Z level=info event=scheduler_restarting5710client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5711lb: must succeed: curl -sf localhost:9901/clusters5712worker2 # [ 98.597795] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515713worker1 # [ 98.606507] nix-daemon[845]: reaped child process 1149, status = succeeded5714worker2 # [ 98.601250] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5715worker1 # [ 98.623507] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5716worker1 # [ 98.628557] nix-grpc-daemon[1084]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5717worker1 # [ 98.631975] nix-grpc-daemon[1084]: E0000 00:00:1789896843.798952 1127 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5718worker1 # [ 98.636894] nix-grpc-daemon[1084]: E0000 00:00:1789896843.799059 1127 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5719lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5720worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5721worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5722worker2 # [ 98.798857] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515723worker2 # [ 98.802930] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5724worker2 # [ 99.003654] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515725worker2 # [ 99.007689] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5726worker2 # [ 99.208500] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515727worker2 # [ 99.212384] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5728worker2 # [ 99.413269] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515729worker2 # [ 99.420135] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05730lb: must succeed: curl -sf localhost:9901/clusters5731lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5732worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5733worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5734worker1 # [ 100.083846] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5735worker1 # [ 100.086521] systemd[1]: Stopped Nix worker-protocol over gRPC.5736worker1 # [ 100.089341] systemd[1]: nix-grpc-daemon.service: Consumed 1.383s CPU time over 25.905s wall clock time, 35.1M memory peak, 46.9K incoming IP traffic, 88.8K outgoing IP traffic.5737worker1 # [ 100.104554] systemd[1]: Starting Nix worker-protocol over gRPC...5738worker1 # [ 100.267128] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5739worker1 # [ 100.271144] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15740worker1 # [ 100.276898] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5741worker1 # [ 100.280470] systemd[1]: Started Nix worker-protocol over gRPC.5742worker1 # [ 100.303363] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:05Z level=info event=scheduler_connected addr=lb:500515743worker2 # [ 100.299302] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05744lb: must succeed: curl -sf localhost:9901/clusters5745lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5746worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5747worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5748(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.77 seconds)5749subtest: scheduler restart mid-build: build finishes, no double build5750client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5751client # [ 101.038353] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5752client # Running as unit: mid.service; invocation ID: b12f55aebe3d4f7e88fa8d43a37d44ff5753client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5754worker1 # [ 101.265766] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334745755worker1 # [ 101.277689] nix-daemon[845]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5756client # [ 101.278278] nix[1215]: don't know how to build these paths:5757client # [ 101.279202] nix[1215]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5758worker2 # [ 101.291378] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:05Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05759worker2 # [ 101.296810] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:05Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15760worker2 # [ 101.342691] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501645761worker2 # [ 101.355422] nix-daemon[840]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5762worker2 # [ 101.372611] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5)5763worker2 # [ 101.399812] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 2/5)5764worker2 # [ 101.466748] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 329 ms (attempt 3/5)5765worker2 # [ 101.802214] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 534 ms (attempt 4/5)5766worker2 # [ 102.344141] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5767worker2 # [ 102.361812] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501645768client # [ 102.366696] nix[1215]: copying 1 paths to 'lb:50051'...5769client # [ 102.371080] nix[1215]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5770worker2 # [ 102.376453] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501645771worker2 # [ 102.387302] nix-daemon[840]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5772worker2 # [ 102.404461] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:06Z level=info event=niks3_push_started pid=14375773worker2 # [ 102.736444] nix-daemon[840]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5774worker2 # [ 102.746922] nix-daemon[840]: reaped child process 1453, status = succeeded5775worker2 # [ 102.907447] nix-daemon[840]: accepted connection from pid 1455, user nix-grpc-daemon (trusted)5776worker2 # [ 102.919373] nix-daemon[840]: reaped child process 1458, status = succeeded5777lb # [ 102.911215] niks3-server[1003]: 2026/09/20 09:34:07 INFO Received uploads request method=POST path=/api/pending_closures5778worker2 # [ 102.946954] nix-grpc-daemon[1437]: time=2026-09-20T09:34:07.150Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5779worker2 # [ 102.948383] nix-grpc-daemon[1437]: time=2026-09-20T09:34:07.150Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5780lb # [ 102.940821] niks3-server[1003]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5781lb # [ 102.945077] niks3-server[1003]: 2026/09/20 09:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5782lb # [ 102.947634] niks3-server[1003]: 2026/09/20 09:34:07 INFO Signed narinfos id=35 count=15783worker2 # [ 102.969894] nix-grpc-daemon[1437]: time=2026-09-20T09:34:07.174Z level=INFO msg="Uploading 1 narinfos"5784lb # [ 102.954582] niks3-server[1003]: 2026/09/20 09:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5785lb # [ 102.961986] niks3-server[1003]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5786lb # [ 102.974228] niks3-server[1003]: 2026/09/20 09:34:07 INFO Completed upload id=355787worker2 # [ 102.996467] nix-grpc-daemon[1437]: time=2026-09-20T09:34:07.201Z level=INFO msg="Upload complete. (420ms)"5788worker2 # [ 102.999067] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50164 duration_s=0 paths=1 nar_bytes_in=4245789lb # [ 102.979251] niks3-server[1003]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5790worker2 # [ 103.003611] nix-grpc-daemon[1304]: ts=2026-09-20T09:34:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501645791worker2 # [ 103.010227] nix-daemon[840]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5792worker2 # [ 103.011558] nix-daemon[840]: reaped child process 1435, status = succeeded5793worker2 # [ 103.017110] nix-daemon[840]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5794client # [ 103.010716] nix[1215]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5795lb: must succeed: curl -sf localhost:9901/clusters5796lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5797lb: must succeed: curl -sf localhost:9901/clusters5798lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5799worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5800worker2 # [ 104.818344] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1304 (nix-grpc-daemon) on client request.5801worker2 # [ 104.821171] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5802lb # [ 104.802073] niks3-server[1003]: 2026/09/20 09:34:09 INFO lead: released remote=192.168.1.4:455865803client # [ 104.826763] nix[1215]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5804worker1 # [ 104.838795] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5805client # [ 104.830775] nix[1215]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5806worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5807client: must succeed: sleep 5.05808worker2 # [ 104.848249] nix-daemon[840]: reaped child process 1426, status = succeeded5809worker2 # [ 104.849586] nix-daemon[840]: reaped child process 1460, status = succeeded5810worker2 # [ 104.855382] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5811worker2 # [ 104.856901] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5812worker2 # [ 104.858568] systemd[1]: nix-grpc-daemon.service: Consumed 425ms CPU time over 17.146s wall clock time, 33.1M memory peak, 15.2K incoming IP traffic, 24.4K outgoing IP traffic.5813worker2 # [ 104.867252] nix-daemon[1462]: unexpected Nix daemon error: error: interrupted by the user5814worker2 # [ 104.870765] nix-daemon[840]: reaped child process 1462, status = failed with exit code 15815worker1 # [ 105.039082] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:10Z level=info event=scheduler_connected addr=lb:500515816worker2 # [ 105.035251] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5817worker2 # [ 105.044623] systemd[1]: Starting Nix worker-protocol over gRPC...5818worker2 # [ 105.185814] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5819worker2 # [ 105.189474] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15820worker2 # [ 105.195495] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5821worker2 # [ 105.199849] systemd[1]: Started Nix worker-protocol over gRPC.5822worker2 # [ 105.207187] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=info event=scheduler_take_over5823lb # [ 105.184595] niks3-server[1003]: 2026/09/20 09:34:10 INFO lead: acquired remote=192.168.1.4:485445824worker2 # [ 105.210669] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05825worker2 # [ 105.225995] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=info event=scheduler_connected addr=lb:500515826worker2 # [ 105.227701] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05827worker2 # [ 105.338794] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05828worker2 # [ 105.340570] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:09Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15829worker1 # [ 105.390345] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334745830worker1 # [ 105.403263] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)5831worker1 # [ 105.456060] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 2/5)5832worker1 # [ 105.584357] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 3/5)5833worker1 # [ 105.724657] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 630 ms (attempt 4/5)5834worker1 # [ 106.356907] nix-grpc-daemon[1350]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5835worker1 # [ 106.371522] nix-grpc-daemon[1350]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5836worker1 # [ 106.394804] nix-daemon[845]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5837client # [ 106.394283] nix[1215]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5838worker1 # [ 106.408467] nix-daemon[845]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5839lb # [ 107.680727] envoy[910]: conn peer=[2001:db8:1::3]:53374 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5369 tx=1264 ms=242355840lb # [ 107.684760] envoy[910]: conn peer=[2001:db8:1::4]:52182 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2336 tx=595 ms=169275841client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5842client: waiting for success: ! systemctl is-active mid5843worker1 # [ 110.229884] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:15Z level=info event=niks3_push_started pid=14515844worker1 # [ 110.261453] nix-daemon[845]: reaped child process 1427, status = succeeded5845worker1 # [ 110.557547] nix-daemon[845]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)5846worker1 # [ 110.569829] nix-daemon[845]: reaped child process 1467, status = succeeded5847lb # [ 110.557738] niks3-server[1003]: 2026/09/20 09:34:15 INFO Received uploads request method=POST path=/api/pending_closures5848worker1 # [ 110.596536] nix-grpc-daemon[1451]: time=2026-09-20T09:34:15.766Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5849worker1 # [ 110.599058] nix-grpc-daemon[1451]: time=2026-09-20T09:34:15.768Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5850lb # [ 110.602454] niks3-server[1003]: 2026/09/20 09:34:15 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5851lb # [ 110.620817] niks3-server[1003]: 2026/09/20 09:34:15 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5852lb # [ 110.626634] niks3-server[1003]: 2026/09/20 09:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5853lb # [ 110.632518] niks3-server[1003]: 2026/09/20 09:34:15 INFO Signed narinfos id=36 count=15854worker1 # [ 110.664775] nix-grpc-daemon[1451]: time=2026-09-20T09:34:15.834Z level=INFO msg="Uploading 1 narinfos"5855lb # [ 110.642343] niks3-server[1003]: 2026/09/20 09:34:15 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5856lb # [ 110.645637] niks3-server[1003]: 2026/09/20 09:34:15 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5857lb # [ 110.650070] niks3-server[1003]: 2026/09/20 09:34:15 INFO Completed upload id=365858worker1 # [ 110.679498] nix-grpc-daemon[1451]: time=2026-09-20T09:34:15.850Z level=INFO msg="Upload complete. (254ms)"5859lb # [ 110.651781] niks3-server[1003]: 2026/09/20 09:34:15 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5860worker1 # [ 110.684299] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33474 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15861worker1 # [ 110.695828] nix-daemon[845]: reaped child process 1425, status = succeeded5862client # [ 110.720314] systemd[1]: mid.service: Deactivated successfully.5863client # [ 110.724139] systemd[1]: mid.service: Consumed 129ms CPU time over 9.684s wall clock time, 39.3M memory peak, 9.6K incoming IP traffic, 10.6K outgoing IP traffic.5864client: (finished: waiting for success: ! systemctl is-active mid, in 1.11 seconds)5865client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5866client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5867(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.30 seconds)5868subtest: interrupting the client stops the build on the worker5869client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5870client # [ 111.308791] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5871client # Running as unit: intr.service; invocation ID: 3521a917763e4f698968b3a4650981105872client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5873worker1 # [ 111.527153] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334745874client # [ 111.520496] nix[1251]: don't know how to build these paths:5875client # [ 111.521391] nix[1251]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5876worker2 # [ 111.534666] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:15Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05877worker2 # [ 111.538677] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:15Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=15878worker2 # [ 111.579917] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585805879worker2 # [ 111.588863] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)5880worker2 # [ 111.607841] nix-grpc-daemon[1498]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)5881worker2 # [ 111.673658] nix-grpc-daemon[1498]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5)5882worker2 # [ 111.700346] nix-grpc-daemon[1498]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 159 ms (attempt 3/5)5883worker2 # [ 111.864989] nix-grpc-daemon[1498]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 137 ms (attempt 4/5)5884worker2 # [ 112.008369] nix-grpc-daemon[1498]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5885worker2 # [ 112.024139] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585805886client # [ 112.028198] nix[1251]: copying 1 paths to 'lb:50051'...5887worker2 # [ 112.034690] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585805888client # [ 112.031143] nix[1251]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5889worker2 # [ 112.047163] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)5890worker2 # [ 112.066380] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:16Z level=info event=niks3_push_started pid=15705891worker2 # [ 112.422797] nix-daemon[840]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)5892worker2 # [ 112.433764] nix-daemon[840]: reaped child process 1586, status = succeeded5893worker2 # [ 112.584694] nix-daemon[840]: accepted connection from pid 1588, user nix-grpc-daemon (trusted)5894worker2 # [ 112.596926] nix-daemon[840]: reaped child process 1595, status = succeeded5895lb # [ 112.587691] niks3-server[1003]: 2026/09/20 09:34:17 INFO Received uploads request method=POST path=/api/pending_closures5896worker2 # [ 112.619257] nix-grpc-daemon[1570]: time=2026-09-20T09:34:16.822Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5897worker2 # [ 112.620579] nix-grpc-daemon[1570]: time=2026-09-20T09:34:16.823Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5898lb # [ 112.621243] niks3-server[1003]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5899lb # [ 112.627751] niks3-server[1003]: 2026/09/20 09:34:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5900lb # [ 112.632451] niks3-server[1003]: 2026/09/20 09:34:17 INFO Signed narinfos id=37 count=15901worker2 # [ 112.657521] nix-grpc-daemon[1570]: time=2026-09-20T09:34:16.862Z level=INFO msg="Uploading 1 narinfos"5902lb # [ 112.645924] niks3-server[1003]: 2026/09/20 09:34:17 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5903worker2 # [ 112.676373] nix-grpc-daemon[1570]: time=2026-09-20T09:34:16.881Z level=INFO msg="Upload complete. (407ms)"5904lb # [ 112.654585] niks3-server[1003]: 2026/09/20 09:34:17 INFO Completed upload id=375905worker2 # [ 112.678712] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=0 paths=1 nar_bytes_in=4245906worker2 # [ 112.681819] nix-grpc-daemon[1570]: time=2026-09-20T09:34:16.885Z level=WARN msg="Failed to register uploaded object" key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo error="server returned 404: object is not pending upload\n"5907worker2 # [ 112.685156] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585805908worker2 # [ 112.688425] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)5909lb # [ 112.669662] niks3-server[1003]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5910worker2 # [ 112.694723] nix-daemon[840]: reaped child process 1568, status = succeeded5911client # [ 112.692339] nix[1251]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5912worker2 # [ 112.699426] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)5913client: must succeed: systemctl kill -s INT intr5914client # [ 113.661314] systemd[1]: intr.service: Sent signal SIGINT to main process 1251 (nix) on client request.5915client # [ 113.664410] nix[1251]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5916client # [ 113.667490] nix[1251]: error: interrupted by the user5917worker2 # [ 113.672284] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:17Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5918client: (finished: must succeed: systemctl kill -s INT intr, in 0.03 seconds)5919client # [ 113.693814] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5920client # [ 113.695743] systemd[1]: intr.service: Failed with result 'exit-code'.5921client # [ 113.697421] systemd[1]: intr.service: Consumed 98ms CPU time over 2.386s wall clock time, 39.3M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.5922worker2 # [ 113.714337] nix-daemon[1599]: unexpected Nix daemon error: error: interrupted by the user5923worker2 # [ 113.717770] nix-daemon[840]: reaped child process 1597, status = succeeded5924worker2 # [ 113.720409] nix-daemon[840]: reaped child process 1599, status = failed with exit code 15925(finished: subtest: interrupting the client stops the build on the worker, in 2.49 seconds)5926subtest: a deploy mid-build drains: the build finishes, the new generation takes over5927client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw5928client # [ 113.796405] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.5929client # Running as unit: sw.service; invocation ID: 24528caee26c48aea35cf5db7faf16085930client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)5931worker1 # [ 114.000846] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334745932client # [ 113.994308] nix[1272]: don't know how to build these paths:5933client # [ 113.995251] nix[1272]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv5934worker2 # [ 114.007344] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:18Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=05935worker2 # [ 114.009345] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:18Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=15936worker1 # [ 114.060797] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334745937worker1 # [ 114.072708] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334745938client # [ 114.069813] nix[1272]: copying 1 paths to 'lb:50051'...5939client # [ 114.070761] nix[1272]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...5940worker1 # [ 114.083922] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334745941worker1 # [ 114.094737] nix-daemon[845]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5942worker1 # [ 114.279205] nix-daemon[845]: accepted connection from pid 1491, user nix-grpc-daemon (trusted)5943worker1 # [ 114.291773] nix-daemon[845]: reaped child process 1494, status = succeeded5944worker1 # [ 114.454344] nix-daemon[845]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)5945worker1 # [ 114.466388] nix-daemon[845]: reaped child process 1499, status = succeeded5946lb # [ 114.449179] niks3-server[1003]: 2026/09/20 09:34:19 INFO Received uploads request method=POST path=/api/pending_closures5947worker1 # [ 114.485504] nix-grpc-daemon[1451]: time=2026-09-20T09:34:19.656Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5948worker1 # [ 114.487054] nix-grpc-daemon[1451]: time=2026-09-20T09:34:19.657Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"5949lb # [ 114.472350] niks3-server[1003]: 2026/09/20 09:34:19 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst5950lb # [ 114.478131] niks3-server[1003]: 2026/09/20 09:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5951lb # [ 114.481300] niks3-server[1003]: 2026/09/20 09:34:19 INFO Signed narinfos id=38 count=15952worker1 # [ 114.512301] nix-grpc-daemon[1451]: time=2026-09-20T09:34:19.682Z level=INFO msg="Uploading 1 narinfos"5953lb # [ 114.485347] niks3-server[1003]: 2026/09/20 09:34:19 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls5954lb # [ 114.491170] niks3-server[1003]: 2026/09/20 09:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5955lb # [ 114.494637] niks3-server[1003]: 2026/09/20 09:34:19 INFO Completed upload id=385956worker1 # [ 114.523814] nix-grpc-daemon[1451]: time=2026-09-20T09:34:19.694Z level=INFO msg="Upload complete. (416ms)"5957worker1 # [ 114.525900] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33474 duration_s=0 paths=1 nar_bytes_in=4165958lb # [ 114.498242] niks3-server[1003]: 2026/09/20 09:34:19 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo5959worker1 # [ 114.530116] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334745960worker1 # [ 114.536064] nix-daemon[845]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5961worker1 # [ 114.538195] nix-daemon[845]: reaped child process 1489, status = succeeded5962client # [ 114.529730] nix[1272]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...5963worker1 # [ 114.544196] nix-daemon[845]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5964worker1: must succeed: systemctl reload nix-grpc-daemon.service5965worker1 # [ 116.276531] systemd[1]: Reloading Nix worker-protocol over gRPC...5966worker1 # [ 116.319389] systemd[1]: Reloaded Nix worker-protocol over gRPC.5967worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds)5968worker1: must succeed: pgrep -f 'read -t [9]0 x'5969worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)5970client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&25971worker1 # [ 116.600266] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334745972client # don't know how to build these paths:5973client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv5974worker2 # [ 116.601475] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585805975worker2 # [ 116.623678] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:20Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=05976worker2 # [ 116.628988] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:20Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=15977worker2 # [ 116.673627] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585805978worker2 # [ 116.687978] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585805979client # copying 1 paths to 'lb:50051'...5980worker2 # [ 116.701697] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585805981client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...5982worker2 # [ 116.723093] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)5983worker2 # [ 116.896754] nix-daemon[840]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)5984worker2 # [ 116.908864] nix-daemon[840]: reaped child process 1648, status = succeeded5985worker2 # [ 117.072940] nix-daemon[840]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)5986worker2 # [ 117.085762] nix-daemon[840]: reaped child process 1653, status = succeeded5987lb # [ 117.078657] niks3-server[1003]: 2026/09/20 09:34:22 INFO Received uploads request method=POST path=/api/pending_closures5988worker2 # [ 117.110273] nix-grpc-daemon[1570]: time=2026-09-20T09:34:21.314Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5989worker2 # [ 117.111710] nix-grpc-daemon[1570]: time=2026-09-20T09:34:21.314Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"5990lb # [ 117.112092] niks3-server[1003]: 2026/09/20 09:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5991lb # [ 117.119118] niks3-server[1003]: 2026/09/20 09:34:22 INFO Signed narinfos id=39 count=15992worker2 # [ 117.144244] nix-grpc-daemon[1570]: time=2026-09-20T09:34:21.349Z level=INFO msg="Uploading 1 narinfos"5993lb # [ 117.127554] niks3-server[1003]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst5994lb # [ 117.133169] niks3-server[1003]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls5995lb # [ 117.143521] niks3-server[1003]: 2026/09/20 09:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5996lb # [ 117.153434] niks3-server[1003]: 2026/09/20 09:34:22 INFO Completed upload id=395997worker2 # [ 117.175747] nix-grpc-daemon[1570]: time=2026-09-20T09:34:21.380Z level=INFO msg="Upload complete. (441ms)"5998lb # [ 117.155172] niks3-server[1003]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo5999worker2 # [ 117.179898] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=0 paths=1 nar_bytes_in=5286000worker2 # [ 117.186366] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806001worker2 # [ 117.195513] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6002worker2 # [ 117.204652] nix-daemon[840]: reaped child process 1643, status = succeeded6003worker2 # [ 117.212116] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6004client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6005client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6006worker1 # [ 117.284348] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:22Z level=info event=draining6007lb # [ 117.680668] envoy[910]: conn peer=[2001:db8:1::1]:47020 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3835 tx=1684 ms=94426008lb # [ 117.684807] envoy[910]: conn peer=[2001:db8:1::1]:42036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=732 ms=21636009client # farm-a> LOG-farm-a-during-drain6010worker2 # [ 117.825660] nix-daemon[840]: reaped child process 1657, status = succeeded6011worker2 # [ 117.982503] nix-daemon[840]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6012worker2 # [ 117.997812] nix-daemon[840]: reaped child process 1675, status = succeeded6013lb # [ 117.992255] niks3-server[1003]: 2026/09/20 09:34:22 INFO Received uploads request method=POST path=/api/pending_closures6014worker2 # [ 118.021693] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.226Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6015worker2 # [ 118.023229] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.228Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6016lb # [ 118.028779] niks3-server[1003]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6017lb # [ 118.038751] niks3-server[1003]: 2026/09/20 09:34:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6018lb # [ 118.041731] niks3-server[1003]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6019lb # [ 118.044216] niks3-server[1003]: 2026/09/20 09:34:23 INFO Signed narinfos id=40 count=16020worker2 # [ 118.066149] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.271Z level=INFO msg="Uploading 1 narinfos"6021lb # [ 118.048027] niks3-server[1003]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6022lb # [ 118.053434] niks3-server[1003]: 2026/09/20 09:34:23 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6023lb # [ 118.056383] niks3-server[1003]: 2026/09/20 09:34:23 INFO Completed upload id=406024worker2 # [ 118.079791] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.284Z level=INFO msg="Upload complete. (262ms)"6025lb # [ 118.059468] niks3-server[1003]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6026worker2 # [ 118.082295] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=0 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16027worker2 # [ 118.088993] nix-daemon[840]: reaped child process 1655, status = succeeded6028worker2 # [ 118.128931] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:22Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06029worker2 # [ 118.130664] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:22Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16030worker2 # [ 118.178270] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806031worker2 # [ 118.185988] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585806032worker2 # [ 118.195241] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.399Z level=INFO msg="All 1 paths already cached"6033client # copying 1 paths to 'lb:50051'...6034worker2 # [ 118.199722] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585806035client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6036worker2 # [ 118.216073] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6037worker2 # [ 118.358864] nix-daemon[840]: accepted connection from pid 1681, user nix-grpc-daemon (trusted)6038worker2 # [ 118.370733] nix-daemon[840]: reaped child process 1684, status = succeeded6039worker2 # [ 118.505958] nix-daemon[840]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6040worker2 # [ 118.517620] nix-daemon[840]: reaped child process 1689, status = succeeded6041lb # [ 118.506890] niks3-server[1003]: 2026/09/20 09:34:23 INFO Received uploads request method=POST path=/api/pending_closures6042worker2 # [ 118.538520] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.743Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6043worker2 # [ 118.539931] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.743Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6044lb # [ 118.543985] niks3-server[1003]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6045lb # [ 118.553329] niks3-server[1003]: 2026/09/20 09:34:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6046worker2 # [ 118.582088] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.786Z level=INFO msg="Uploading 1 narinfos"6047lb # [ 118.560069] niks3-server[1003]: 2026/09/20 09:34:23 INFO Signed narinfos id=41 count=16048lb # [ 118.564504] niks3-server[1003]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6049lb # [ 118.574963] niks3-server[1003]: 2026/09/20 09:34:23 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6050worker2 # [ 118.603667] nix-grpc-daemon[1570]: time=2026-09-20T09:34:22.808Z level=INFO msg="Upload complete. (382ms)"6051lb # [ 118.581927] niks3-server[1003]: 2026/09/20 09:34:23 INFO Completed upload id=416052lb # [ 118.583688] niks3-server[1003]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6053worker2 # [ 118.606304] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=0 paths=1 nar_bytes_in=6406054worker2 # [ 118.611761] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806055worker2 # [ 118.622105] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6056worker2 # [ 118.626519] nix-daemon[840]: reaped child process 1679, status = succeeded6057worker2 # [ 118.631514] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6058client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6059client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6060client # farm-b> LOG-farm-b-during-drain6061worker2 # [ 119.855723] nix-daemon[840]: reaped child process 1693, status = succeeded6062worker2 # [ 120.011304] nix-daemon[840]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6063worker2 # [ 120.023231] nix-daemon[840]: reaped child process 1711, status = succeeded6064lb # [ 120.016726] niks3-server[1003]: 2026/09/20 09:34:24 INFO Received uploads request method=POST path=/api/pending_closures6065worker2 # [ 120.046955] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.251Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6066worker2 # [ 120.048554] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.253Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6067lb # [ 120.046787] niks3-server[1003]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6068lb # [ 120.063221] niks3-server[1003]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6069lb # [ 120.072688] niks3-server[1003]: 2026/09/20 09:34:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6070lb # [ 120.077054] niks3-server[1003]: 2026/09/20 09:34:25 INFO Signed narinfos id=42 count=16071worker2 # [ 120.101929] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.306Z level=INFO msg="Uploading 1 narinfos"6072lb # [ 120.090217] niks3-server[1003]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6073lb # [ 120.095380] niks3-server[1003]: 2026/09/20 09:34:25 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6074lb # [ 120.099670] niks3-server[1003]: 2026/09/20 09:34:25 INFO Completed upload id=426075lb # [ 120.101131] niks3-server[1003]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6076worker2 # [ 120.121987] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.326Z level=INFO msg="Upload complete. (280ms)"6077worker2 # [ 120.134198] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16078worker2 # [ 120.140339] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:24Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06079worker2 # [ 120.143942] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:24Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16080worker2 # [ 120.152754] nix-daemon[840]: reaped child process 1691, status = succeeded6081worker2 # [ 120.176721] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806082worker2 # [ 120.190669] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585806083worker2 # [ 120.203490] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.408Z level=INFO msg="All 2 paths already cached"6084client # copying 1 paths to 'lb:50051'...6085client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6086worker2 # [ 120.210650] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585806087worker2 # [ 120.220284] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6088worker2 # [ 120.385756] nix-daemon[840]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6089worker2 # [ 120.397963] nix-daemon[840]: reaped child process 1720, status = succeeded6090worker2 # [ 120.553114] nix-daemon[840]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6091worker2 # [ 120.565103] nix-daemon[840]: reaped child process 1725, status = succeeded6092lb # [ 120.555611] niks3-server[1003]: 2026/09/20 09:34:25 INFO Received uploads request method=POST path=/api/pending_closures6093worker2 # [ 120.585225] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.790Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6094worker2 # [ 120.586724] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.791Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6095lb # [ 120.584074] niks3-server[1003]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6096lb # [ 120.595484] niks3-server[1003]: 2026/09/20 09:34:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6097worker2 # [ 120.622055] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.827Z level=INFO msg="Uploading 1 narinfos"6098lb # [ 120.600570] niks3-server[1003]: 2026/09/20 09:34:25 INFO Signed narinfos id=43 count=16099lb # [ 120.607601] niks3-server[1003]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6100lb # [ 120.620220] niks3-server[1003]: 2026/09/20 09:34:25 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6101lb # [ 120.626718] niks3-server[1003]: 2026/09/20 09:34:25 INFO Completed upload id=436102worker2 # [ 120.648755] nix-grpc-daemon[1570]: time=2026-09-20T09:34:24.853Z level=INFO msg="Upload complete. (416ms)"6103lb # [ 120.629512] niks3-server[1003]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6104worker2 # [ 120.652588] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=0 paths=1 nar_bytes_in=7686105worker2 # [ 120.657475] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806106worker2 # [ 120.666483] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6107worker2 # [ 120.673223] nix-daemon[840]: reaped child process 1715, status = succeeded6108client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6109client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6110worker2 # [ 120.678486] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6111client # farm-top> LOG-farm-top-during-drain6112worker2 # [ 122.094890] nix-daemon[840]: reaped child process 1729, status = succeeded6113worker2 # [ 122.223947] nix-daemon[840]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6114worker2 # [ 122.234703] nix-daemon[840]: reaped child process 1747, status = succeeded6115lb # [ 122.227660] niks3-server[1003]: 2026/09/20 09:34:27 INFO Received uploads request method=POST path=/api/pending_closures6116worker2 # [ 122.257858] nix-grpc-daemon[1570]: time=2026-09-20T09:34:26.462Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6117worker2 # [ 122.259654] nix-grpc-daemon[1570]: time=2026-09-20T09:34:26.464Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6118lb # [ 122.265578] niks3-server[1003]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6119lb # [ 122.275653] niks3-server[1003]: 2026/09/20 09:34:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6120worker2 # [ 122.308077] nix-grpc-daemon[1570]: time=2026-09-20T09:34:26.512Z level=INFO msg="Uploading 1 narinfos"6121lb # [ 122.286313] niks3-server[1003]: 2026/09/20 09:34:27 INFO Signed narinfos id=44 count=16122lb # [ 122.289871] niks3-server[1003]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6123lb # [ 122.293613] niks3-server[1003]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6124lb # [ 122.305526] niks3-server[1003]: 2026/09/20 09:34:27 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6125lb # [ 122.309150] niks3-server[1003]: 2026/09/20 09:34:27 INFO Completed upload id=446126lb # [ 122.310958] niks3-server[1003]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6127worker2 # [ 122.332648] nix-grpc-daemon[1570]: time=2026-09-20T09:34:26.537Z level=INFO msg="Upload complete. (246ms)"6128worker2 # [ 122.339670] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16129worker2 # [ 122.353688] nix-daemon[840]: reaped child process 1727, status = succeeded6130client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.03 seconds)6131client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6132client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.22 seconds)6133worker1: must succeed: pkill -f 'read -t [9]0 x'6134worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6135client: waiting for success: ! systemctl is-active sw6136worker1 # [ 122.771305] nix-daemon[845]: reaped child process 1503, status = succeeded6137worker1 # [ 122.927442] nix-daemon[845]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6138worker1 # [ 122.939330] nix-daemon[845]: reaped child process 1547, status = succeeded6139lb # [ 122.926923] niks3-server[1003]: 2026/09/20 09:34:27 INFO Received uploads request method=POST path=/api/pending_closures6140worker1 # [ 122.964880] nix-grpc-daemon[1451]: time=2026-09-20T09:34:28.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6141worker1 # [ 122.966375] nix-grpc-daemon[1451]: time=2026-09-20T09:34:28.134Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6142lb # [ 122.958970] niks3-server[1003]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6143lb # [ 122.964317] niks3-server[1003]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6144lb # [ 122.975893] niks3-server[1003]: 2026/09/20 09:34:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6145lb # [ 122.980083] niks3-server[1003]: 2026/09/20 09:34:27 INFO Signed narinfos id=45 count=16146worker1 # [ 123.012464] nix-grpc-daemon[1451]: time=2026-09-20T09:34:28.182Z level=INFO msg="Uploading 1 narinfos"6147lb # [ 122.989455] niks3-server[1003]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6148lb # [ 122.998219] niks3-server[1003]: 2026/09/20 09:34:27 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6149worker1 # [ 123.032207] nix-grpc-daemon[1451]: time=2026-09-20T09:34:28.202Z level=INFO msg="Upload complete. (275ms)"6150lb # [ 123.003112] niks3-server[1003]: 2026/09/20 09:34:27 INFO Completed upload id=456151lb # [ 123.004800] niks3-server[1003]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6152worker1 # [ 123.035300] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33474 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16153worker1 # [ 123.042069] nix-daemon[845]: reaped child process 1501, status = succeeded6154client # [ 123.063521] systemd[1]: sw.service: Deactivated successfully.6155client # [ 123.068311] systemd[1]: sw.service: Consumed 108ms CPU time over 9.269s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.6156worker1 # [ 123.287888] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:28Z level=info event=signal_exit6157client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6158client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6159client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6160worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6161worker1 # [ 124.505755] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:29Z level=info event=scheduler_restarting6162worker2 # [ 124.503428] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:585666163worker1 # [ 124.514575] nix-grpc-daemon[1350]: ts=2026-09-20T09:34:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6164worker2 # [ 124.507343] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:28Z level=info event=worker_gone addr=192.168.1.3:500516165worker1 # [ 124.525309] nix-daemon[845]: reaped child process 1403, status = succeeded6166worker1 # [ 125.124338] nix-grpc-daemon[1350]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6167worker1 # [ 125.125795] nix-grpc-daemon[1350]: E0000 00:00:1789896870.294941 1393 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6168worker1 # [ 125.127658] nix-grpc-daemon[1350]: E0000 00:00:1789896870.295047 1393 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6169worker1 # [ 126.323339] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6170worker1 # [ 126.327362] systemd[1]: nix-grpc-daemon.service: Consumed 628ms CPU time over 26.218s wall clock time, 32.4M memory peak, 20.5K incoming IP traffic, 26.8K outgoing IP traffic.6171worker1 # [ 126.343370] systemd[1]: Starting Nix worker-protocol over gRPC...6172worker1 # [ 126.504484] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6173worker1 # [ 126.508277] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16174worker1 # [ 126.514467] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6175worker1 # [ 126.516031] systemd[1]: Started Nix worker-protocol over gRPC.6176worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.73 seconds)6177lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6178lb: must succeed: curl -sf localhost:9901/clusters6179worker1 # [ 126.547564] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:31Z level=info event=scheduler_connected addr=lb:500516180worker2 # [ 126.544157] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06181lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6182lb: must succeed: curl -sf localhost:9901/clusters6183lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6184lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.15 seconds)6185lb: must succeed: curl -sf localhost:9901/clusters6186lb # [ 127.680786] envoy[910]: conn peer=[2001:db8:1::1]:51656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9602 tx=3866 ms=57676187lb # [ 127.682467] envoy[910]: conn peer=[2001:db8:1::1]:51640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2932 tx=1332 ms=90496188lb # [ 127.683932] envoy[910]: conn peer=[2001:db8:1::3]:33486 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1114 tx=448 ms=242116189lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6190worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6191worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)6192(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.00 seconds)6193subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6194client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16195worker2 # [ 127.971877] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585806196worker2 # [ 127.977795] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585806197worker2 # [ 128.024905] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:32Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06198worker2 # [ 128.029139] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:32Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16199worker2 # [ 128.033407] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:32Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06200worker2 # [ 128.037343] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:32Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06201worker1 # [ 128.080499] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406202worker1 # [ 128.091850] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6203worker1 # [ 128.108573] nix-grpc-daemon[1561]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 1/5)6204worker1 # [ 128.200527] nix-grpc-daemon[1561]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 2/5)6205worker1 # [ 128.223406] nix-grpc-daemon[1561]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 132 ms (attempt 3/5)6206worker1 # [ 128.361448] nix-grpc-daemon[1561]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 541 ms (attempt 4/5)6207worker1 # [ 128.908836] nix-grpc-daemon[1561]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6208worker1 # [ 128.927506] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406209worker1 # [ 128.939881] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406210worker1 # [ 128.948294] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6211worker1 # [ 128.969967] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:34Z level=info event=niks3_push_started pid=16176212worker1 # [ 129.317946] nix-daemon[845]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6213worker1 # [ 129.332093] nix-daemon[845]: reaped child process 1633, status = succeeded6214worker1 # [ 129.489257] nix-daemon[845]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6215worker1 # [ 129.500150] nix-daemon[845]: reaped child process 1638, status = succeeded6216lb # [ 129.486768] niks3-server[1003]: 2026/09/20 09:34:34 INFO Received uploads request method=POST path=/api/pending_closures6217worker1 # [ 129.526750] nix-grpc-daemon[1617]: time=2026-09-20T09:34:34.697Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6218worker1 # [ 129.529520] nix-grpc-daemon[1617]: time=2026-09-20T09:34:34.699Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6219lb # [ 129.523351] niks3-server[1003]: 2026/09/20 09:34:34 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6220lb # [ 129.535531] niks3-server[1003]: 2026/09/20 09:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6221lb # [ 129.541597] niks3-server[1003]: 2026/09/20 09:34:34 INFO Signed narinfos id=46 count=16222worker1 # [ 129.573889] nix-grpc-daemon[1617]: time=2026-09-20T09:34:34.744Z level=INFO msg="Uploading 1 narinfos"6223lb # [ 129.553833] niks3-server[1003]: 2026/09/20 09:34:34 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6224lb # [ 129.560450] niks3-server[1003]: 2026/09/20 09:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6225worker1 # [ 129.596043] nix-grpc-daemon[1617]: time=2026-09-20T09:34:34.766Z level=INFO msg="Upload complete. (434ms)"6226lb # [ 129.566844] niks3-server[1003]: 2026/09/20 09:34:34 INFO Completed upload id=466227worker1 # [ 129.598379] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=5046228lb # [ 129.571578] niks3-server[1003]: 2026/09/20 09:34:34 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6229worker1 # [ 129.603093] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406230worker1 # [ 129.609065] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6231worker1 # [ 129.611390] nix-daemon[845]: reaped child process 1615, status = succeeded6232worker1 # [ 129.616557] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6233worker1 # [ 130.428927] nix-daemon[845]: reaped child process 1642, status = succeeded6234worker1 # [ 130.572565] nix-daemon[845]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6235worker1 # [ 130.583298] nix-daemon[845]: reaped child process 1660, status = succeeded6236lb # [ 130.568336] niks3-server[1003]: 2026/09/20 09:34:35 INFO Received uploads request method=POST path=/api/pending_closures6237worker1 # [ 130.607495] nix-grpc-daemon[1617]: time=2026-09-20T09:34:35.778Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6238worker1 # [ 130.609143] nix-grpc-daemon[1617]: time=2026-09-20T09:34:35.779Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6239lb # [ 130.605730] niks3-server[1003]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6240lb # [ 130.612375] niks3-server[1003]: 2026/09/20 09:34:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6241lb # [ 130.614908] niks3-server[1003]: 2026/09/20 09:34:35 INFO Signed narinfos id=47 count=16242worker1 # [ 130.646103] nix-grpc-daemon[1617]: time=2026-09-20T09:34:35.816Z level=INFO msg="Uploading 1 narinfos"6243lb # [ 130.619838] niks3-server[1003]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6244lb # [ 130.625474] niks3-server[1003]: 2026/09/20 09:34:35 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6245lb # [ 130.628736] niks3-server[1003]: 2026/09/20 09:34:35 INFO Completed upload id=476246worker1 # [ 130.657746] nix-grpc-daemon[1617]: time=2026-09-20T09:34:35.828Z level=INFO msg="Upload complete. (235ms)"6247worker1 # [ 130.660365] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16248lb # [ 130.633847] niks3-server[1003]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6249worker2 # [ 130.654778] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:34Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16250lb # [ 130.637601] niks3-server[1003]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6251worker1 # [ 130.667954] nix-daemon[845]: reaped child process 1640, status = succeeded6252worker1 # [ 130.670731] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406253worker2 # [ 130.665188] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:34Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06254worker1 # [ 130.677480] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406255worker1 # [ 130.683291] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406256worker1 # [ 130.687991] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6257worker1 # [ 130.831520] nix-daemon[845]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6258worker1 # [ 130.843502] nix-daemon[845]: reaped child process 1667, status = succeeded6259worker1 # [ 131.012388] nix-daemon[845]: accepted connection from pid 1669, user nix-grpc-daemon (trusted)6260worker1 # [ 131.026800] nix-daemon[845]: reaped child process 1672, status = succeeded6261lb # [ 131.013844] niks3-server[1003]: 2026/09/20 09:34:35 INFO Received uploads request method=POST path=/api/pending_closures6262worker1 # [ 131.050888] nix-grpc-daemon[1617]: time=2026-09-20T09:34:36.221Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6263worker1 # [ 131.052643] nix-grpc-daemon[1617]: time=2026-09-20T09:34:36.221Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6264lb # [ 131.038533] niks3-server[1003]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6265lb # [ 131.051555] niks3-server[1003]: 2026/09/20 09:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6266worker1 # [ 131.085114] nix-grpc-daemon[1617]: time=2026-09-20T09:34:36.255Z level=INFO msg="Uploading 1 narinfos"6267lb # [ 131.055915] niks3-server[1003]: 2026/09/20 09:34:36 INFO Signed narinfos id=48 count=16268lb # [ 131.063155] niks3-server[1003]: 2026/09/20 09:34:36 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6269lb # [ 131.072433] niks3-server[1003]: 2026/09/20 09:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6270lb # [ 131.077536] niks3-server[1003]: 2026/09/20 09:34:36 INFO Completed upload id=486271worker1 # [ 131.108814] nix-grpc-daemon[1617]: time=2026-09-20T09:34:36.279Z level=INFO msg="Upload complete. (416ms)"6272lb # [ 131.081985] niks3-server[1003]: 2026/09/20 09:34:36 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6273worker1 # [ 131.112165] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=5046274worker1 # [ 131.118177] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406275worker1 # [ 131.126377] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6276worker1 # [ 131.131186] nix-daemon[845]: reaped child process 1662, status = succeeded6277worker1 # [ 131.137055] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6278worker1 # [ 132.763643] nix-daemon[845]: reaped child process 1676, status = succeeded6279worker1 # [ 132.916092] nix-daemon[845]: accepted connection from pid 1691, user nix-grpc-daemon (trusted)6280worker1 # [ 132.927510] nix-daemon[845]: reaped child process 1694, status = succeeded6281lb # [ 132.913650] niks3-server[1003]: 2026/09/20 09:34:37 INFO Received uploads request method=POST path=/api/pending_closures6282worker1 # [ 132.947345] nix-grpc-daemon[1617]: time=2026-09-20T09:34:38.117Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6283worker1 # [ 132.949583] nix-grpc-daemon[1617]: time=2026-09-20T09:34:38.119Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6284lb # [ 132.931905] niks3-server[1003]: 2026/09/20 09:34:37 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6285lb # [ 132.934362] niks3-server[1003]: 2026/09/20 09:34:37 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6286lb # [ 132.938433] niks3-server[1003]: 2026/09/20 09:34:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6287lb # [ 132.940837] niks3-server[1003]: 2026/09/20 09:34:37 INFO Signed narinfos id=49 count=16288worker1 # [ 132.970230] nix-grpc-daemon[1617]: time=2026-09-20T09:34:38.140Z level=INFO msg="Uploading 1 narinfos"6289lb # [ 132.944236] niks3-server[1003]: 2026/09/20 09:34:37 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6290lb # [ 132.949042] niks3-server[1003]: 2026/09/20 09:34:37 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6291lb # [ 132.951938] niks3-server[1003]: 2026/09/20 09:34:37 INFO Completed upload id=496292worker1 # [ 132.982446] nix-grpc-daemon[1617]: time=2026-09-20T09:34:38.153Z level=INFO msg="Upload complete. (237ms)"6293lb # [ 132.953564] niks3-server[1003]: 2026/09/20 09:34:37 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6294worker1 # [ 132.986161] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16295worker1 # [ 132.989441] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406296worker2 # [ 132.979956] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:37Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16297worker1 # [ 132.995970] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406298worker1 # [ 132.998495] nix-daemon[845]: reaped child process 1674, status = succeeded6299worker2 # [ 132.990613] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:37Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06300worker1 # [ 133.002731] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406301worker1 # [ 133.007596] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6302worker1 # [ 133.161498] nix-daemon[845]: accepted connection from pid 1698, user nix-grpc-daemon (trusted)6303worker1 # [ 133.173559] nix-daemon[845]: reaped child process 1701, status = succeeded6304worker1 # [ 133.328619] nix-daemon[845]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)6305worker1 # [ 133.340490] nix-daemon[845]: reaped child process 1706, status = succeeded6306lb # [ 133.322838] niks3-server[1003]: 2026/09/20 09:34:38 INFO Received uploads request method=POST path=/api/pending_closures6307worker1 # [ 133.360066] nix-grpc-daemon[1617]: time=2026-09-20T09:34:38.530Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6308worker1 # [ 133.361614] nix-grpc-daemon[1617]: time=2026-09-20T09:34:38.532Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6309lb # [ 133.352719] niks3-server[1003]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6310lb # [ 133.362967] niks3-server[1003]: 2026/09/20 09:34:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6311lb # [ 133.368111] niks3-server[1003]: 2026/09/20 09:34:38 INFO Signed narinfos id=50 count=16312worker1 # [ 133.400048] nix-grpc-daemon[1617]: time=2026-09-20T09:34:38.570Z level=INFO msg="Uploading 1 narinfos"6313lb # [ 133.374365] niks3-server[1003]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6314lb # [ 133.392155] niks3-server[1003]: 2026/09/20 09:34:38 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6315lb # [ 133.396751] niks3-server[1003]: 2026/09/20 09:34:38 INFO Completed upload id=506316worker1 # [ 133.426034] nix-grpc-daemon[1617]: time=2026-09-20T09:34:38.596Z level=INFO msg="Upload complete. (413ms)"6317lb # [ 133.398996] niks3-server[1003]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6318worker1 # [ 133.429115] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=5046319worker1 # [ 133.434267] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406320worker1 # [ 133.445111] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6321worker1 # [ 133.448558] nix-daemon[845]: reaped child process 1696, status = succeeded6322worker1 # [ 133.453523] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6323worker1 # [ 134.080751] nix-daemon[845]: reaped child process 1710, status = succeeded6324worker1 # [ 134.211787] nix-daemon[845]: accepted connection from pid 1725, user nix-grpc-daemon (trusted)6325worker1 # [ 134.223702] nix-daemon[845]: reaped child process 1728, status = succeeded6326lb # [ 134.214248] niks3-server[1003]: 2026/09/20 09:34:39 INFO Received uploads request method=POST path=/api/pending_closures6327worker1 # [ 134.251353] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.421Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6328worker1 # [ 134.253158] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.423Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6329lb # [ 134.248070] niks3-server[1003]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6330lb # [ 134.259717] niks3-server[1003]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6331lb # [ 134.269768] niks3-server[1003]: 2026/09/20 09:34:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6332lb # [ 134.276231] niks3-server[1003]: 2026/09/20 09:34:39 INFO Signed narinfos id=51 count=16333worker1 # [ 134.307888] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.478Z level=INFO msg="Uploading 1 narinfos"6334lb # [ 134.278690] niks3-server[1003]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6335lb # [ 134.289361] niks3-server[1003]: 2026/09/20 09:34:39 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6336lb # [ 134.294871] niks3-server[1003]: 2026/09/20 09:34:39 INFO Completed upload id=516337worker1 # [ 134.324157] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.494Z level=INFO msg="Upload complete. (264ms)"6338lb # [ 134.298791] niks3-server[1003]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6339worker2 # [ 134.324519] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:38Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16340worker2 # [ 134.330496] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:38Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06341worker1 # [ 134.335620] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16342worker1 # [ 134.341293] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406343worker1 # [ 134.351573] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406344worker1 # [ 134.359347] nix-daemon[845]: reaped child process 1708, status = succeeded6345worker1 # [ 134.361735] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.532Z level=INFO msg="All 1 paths already cached"6346worker1 # [ 134.365874] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406347worker1 # [ 134.369981] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6348worker1 # [ 134.531515] nix-daemon[845]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)6349worker1 # [ 134.543636] nix-daemon[845]: reaped child process 1735, status = succeeded6350worker1 # [ 134.726438] nix-daemon[845]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6351worker1 # [ 134.738692] nix-daemon[845]: reaped child process 1740, status = succeeded6352lb # [ 134.726222] niks3-server[1003]: 2026/09/20 09:34:39 INFO Received uploads request method=POST path=/api/pending_closures6353worker1 # [ 134.763856] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.934Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6354worker1 # [ 134.765272] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.934Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6355lb # [ 134.752893] niks3-server[1003]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6356lb # [ 134.765128] niks3-server[1003]: 2026/09/20 09:34:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6357lb # [ 134.770115] niks3-server[1003]: 2026/09/20 09:34:39 INFO Signed narinfos id=52 count=16358worker1 # [ 134.802545] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.973Z level=INFO msg="Uploading 1 narinfos"6359lb # [ 134.777569] niks3-server[1003]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6360lb # [ 134.788907] niks3-server[1003]: 2026/09/20 09:34:39 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6361lb # [ 134.794822] niks3-server[1003]: 2026/09/20 09:34:39 INFO Completed upload id=526362worker1 # [ 134.823787] nix-grpc-daemon[1617]: time=2026-09-20T09:34:39.994Z level=INFO msg="Upload complete. (447ms)"6363lb # [ 134.797650] niks3-server[1003]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6364worker1 # [ 134.827450] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=6246365worker1 # [ 134.833659] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406366worker1 # [ 134.843842] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6367worker1 # [ 134.846915] nix-daemon[845]: reaped child process 1730, status = succeeded6368worker1 # [ 134.852129] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6369worker1 # [ 135.677519] nix-daemon[845]: reaped child process 1744, status = succeeded6370worker1 # [ 135.822347] nix-daemon[845]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6371worker1 # [ 135.834673] nix-daemon[845]: reaped child process 1762, status = succeeded6372lb # [ 135.817805] niks3-server[1003]: 2026/09/20 09:34:40 INFO Received uploads request method=POST path=/api/pending_closures6373worker1 # [ 135.856429] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.026Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6374worker1 # [ 135.857956] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.028Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6375lb # [ 135.857926] niks3-server[1003]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6376lb # [ 135.865183] niks3-server[1003]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6377lb # [ 135.874921] niks3-server[1003]: 2026/09/20 09:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6378lb # [ 135.880829] niks3-server[1003]: 2026/09/20 09:34:40 INFO Signed narinfos id=53 count=16379worker1 # [ 135.909865] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.080Z level=INFO msg="Uploading 1 narinfos"6380lb # [ 135.885537] niks3-server[1003]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6381lb # [ 135.890895] niks3-server[1003]: 2026/09/20 09:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6382lb # [ 135.893886] niks3-server[1003]: 2026/09/20 09:34:40 INFO Completed upload id=536383worker1 # [ 135.923209] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.093Z level=INFO msg="Upload complete. (263ms)"6384lb # [ 135.896131] niks3-server[1003]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6385worker2 # [ 135.921305] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:40Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16386worker2 # [ 135.926712] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:40Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06387worker1 # [ 135.931331] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16388worker1 # [ 135.936195] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406389worker1 # [ 135.945668] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406390worker1 # [ 135.947898] nix-daemon[845]: reaped child process 1742, status = succeeded6391worker1 # [ 135.952025] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.122Z level=INFO msg="All 1 paths already cached"6392worker1 # [ 135.955749] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406393worker1 # [ 135.959644] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6394worker1 # [ 136.114981] nix-daemon[845]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6395worker1 # [ 136.128092] nix-daemon[845]: reaped child process 1769, status = succeeded6396worker1 # [ 136.286796] nix-daemon[845]: accepted connection from pid 1771, user nix-grpc-daemon (trusted)6397worker1 # [ 136.299183] nix-daemon[845]: reaped child process 1774, status = succeeded6398lb # [ 136.283244] niks3-server[1003]: 2026/09/20 09:34:41 INFO Received uploads request method=POST path=/api/pending_closures6399worker1 # [ 136.323842] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.494Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6400worker1 # [ 136.325426] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.496Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6401lb # [ 136.316262] niks3-server[1003]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6402lb # [ 136.320748] niks3-server[1003]: 2026/09/20 09:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6403lb # [ 136.322660] niks3-server[1003]: 2026/09/20 09:34:41 INFO Signed narinfos id=54 count=16404worker1 # [ 136.351910] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.522Z level=INFO msg="Uploading 1 narinfos"6405lb # [ 136.327807] niks3-server[1003]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6406lb # [ 136.331059] niks3-server[1003]: 2026/09/20 09:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6407lb # [ 136.334382] niks3-server[1003]: 2026/09/20 09:34:41 INFO Completed upload id=546408worker1 # [ 136.363678] nix-grpc-daemon[1617]: time=2026-09-20T09:34:41.534Z level=INFO msg="Upload complete. (398ms)"6409worker1 # [ 136.365572] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=6246410lb # [ 136.337564] niks3-server[1003]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6411worker1 # [ 136.369038] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406412worker1 # [ 136.374935] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6413worker1 # [ 136.378239] nix-daemon[845]: reaped child process 1764, status = succeeded6414worker1 # [ 136.383085] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6415worker1 # [ 137.209341] nix-daemon[845]: reaped child process 1778, status = succeeded6416worker1 # [ 137.329939] nix-daemon[845]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6417worker1 # [ 137.344137] nix-daemon[845]: reaped child process 1796, status = succeeded6418lb # [ 137.324868] niks3-server[1003]: 2026/09/20 09:34:42 INFO Received uploads request method=POST path=/api/pending_closures6419worker1 # [ 137.362919] nix-grpc-daemon[1617]: time=2026-09-20T09:34:42.533Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6420worker1 # [ 137.364449] nix-grpc-daemon[1617]: time=2026-09-20T09:34:42.535Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6421lb # [ 137.361091] niks3-server[1003]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6422lb # [ 137.369891] niks3-server[1003]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6423lb # [ 137.381363] niks3-server[1003]: 2026/09/20 09:34:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6424worker1 # [ 137.415782] nix-grpc-daemon[1617]: time=2026-09-20T09:34:42.586Z level=INFO msg="Uploading 1 narinfos"6425lb # [ 137.386648] niks3-server[1003]: 2026/09/20 09:34:42 INFO Signed narinfos id=55 count=16426lb # [ 137.391599] niks3-server[1003]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6427lb # [ 137.400264] niks3-server[1003]: 2026/09/20 09:34:42 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6428lb # [ 137.405679] niks3-server[1003]: 2026/09/20 09:34:42 INFO Completed upload id=556429lb # [ 137.408490] niks3-server[1003]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6430worker1 # [ 137.436799] nix-grpc-daemon[1617]: time=2026-09-20T09:34:42.607Z level=INFO msg="Upload complete. (247ms)"6431worker1 # [ 137.443969] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16432worker2 # [ 137.437973] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:41Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16433worker1 # [ 137.451442] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406434worker2 # [ 137.445881] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:41Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06435worker1 # [ 137.464396] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406436worker1 # [ 137.468800] nix-daemon[845]: reaped child process 1776, status = succeeded6437worker1 # [ 137.474868] nix-grpc-daemon[1617]: time=2026-09-20T09:34:42.645Z level=INFO msg="All 1 paths already cached"6438worker1 # [ 137.480542] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406439worker1 # [ 137.487524] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6440worker1 # [ 137.640132] nix-daemon[845]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6441worker1 # [ 137.652354] nix-daemon[845]: reaped child process 1803, status = succeeded6442worker1 # [ 137.804379] nix-daemon[845]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)6443worker1 # [ 137.815162] nix-daemon[845]: reaped child process 1808, status = succeeded6444lb # [ 137.798200] niks3-server[1003]: 2026/09/20 09:34:42 INFO Received uploads request method=POST path=/api/pending_closures6445worker1 # [ 137.835187] nix-grpc-daemon[1617]: time=2026-09-20T09:34:43.005Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6446worker1 # [ 137.836780] nix-grpc-daemon[1617]: time=2026-09-20T09:34:43.007Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6447lb # [ 137.825262] niks3-server[1003]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6448lb # [ 137.837614] niks3-server[1003]: 2026/09/20 09:34:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6449lb # [ 137.842225] niks3-server[1003]: 2026/09/20 09:34:42 INFO Signed narinfos id=56 count=16450worker1 # [ 137.874218] nix-grpc-daemon[1617]: time=2026-09-20T09:34:43.044Z level=INFO msg="Uploading 1 narinfos"6451lb # [ 137.848392] niks3-server[1003]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6452lb # [ 137.860794] niks3-server[1003]: 2026/09/20 09:34:42 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6453lb # [ 137.865505] niks3-server[1003]: 2026/09/20 09:34:42 INFO Completed upload id=566454worker1 # [ 137.894492] nix-grpc-daemon[1617]: time=2026-09-20T09:34:43.065Z level=INFO msg="Upload complete. (400ms)"6455lb # [ 137.868329] niks3-server[1003]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6456worker1 # [ 137.897992] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=6246457worker1 # [ 137.903904] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406458worker1 # [ 137.911077] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6459worker1 # [ 137.916180] nix-daemon[845]: reaped child process 1798, status = succeeded6460worker1 # [ 137.921923] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6461worker1 # [ 138.933506] nix-daemon[845]: reaped child process 1812, status = succeeded6462worker1 # [ 139.089109] nix-daemon[845]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)6463worker1 # [ 139.102383] nix-daemon[845]: reaped child process 1830, status = succeeded6464lb # [ 139.088850] niks3-server[1003]: 2026/09/20 09:34:44 INFO Received uploads request method=POST path=/api/pending_closures6465worker1 # [ 139.126885] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.296Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6466worker1 # [ 139.128340] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.296Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6467lb # [ 139.114244] niks3-server[1003]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6468lb # [ 139.131451] niks3-server[1003]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6469lb # [ 139.139903] niks3-server[1003]: 2026/09/20 09:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6470lb # [ 139.144812] niks3-server[1003]: 2026/09/20 09:34:44 INFO Signed narinfos id=57 count=16471worker1 # [ 139.176792] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.347Z level=INFO msg="Uploading 1 narinfos"6472lb # [ 139.149799] niks3-server[1003]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6473lb # [ 139.159156] niks3-server[1003]: 2026/09/20 09:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6474lb # [ 139.164571] niks3-server[1003]: 2026/09/20 09:34:44 INFO Completed upload id=576475worker1 # [ 139.193523] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.364Z level=INFO msg="Upload complete. (266ms)"6476lb # [ 139.170221] niks3-server[1003]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6477worker1 # [ 139.198608] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16478worker2 # [ 139.192399] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:43Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16479worker2 # [ 139.201946] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:43Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06480worker1 # [ 139.212533] nix-daemon[845]: reaped child process 1810, status = succeeded6481worker1 # [ 139.248931] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406482worker1 # [ 139.259325] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406483worker1 # [ 139.270916] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.441Z level=INFO msg="All 2 paths already cached"6484worker1 # [ 139.278383] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406485worker1 # [ 139.286441] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6486worker1 # [ 139.451072] nix-daemon[845]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6487worker1 # [ 139.463204] nix-daemon[845]: reaped child process 1837, status = succeeded6488worker1 # [ 139.628295] nix-daemon[845]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6489worker1 # [ 139.640291] nix-daemon[845]: reaped child process 1842, status = succeeded6490lb # [ 139.628106] niks3-server[1003]: 2026/09/20 09:34:44 INFO Received uploads request method=POST path=/api/pending_closures6491worker1 # [ 139.665108] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.835Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6492worker1 # [ 139.666711] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.837Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6493lb # [ 139.655648] niks3-server[1003]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6494lb # [ 139.666872] niks3-server[1003]: 2026/09/20 09:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6495lb # [ 139.672121] niks3-server[1003]: 2026/09/20 09:34:44 INFO Signed narinfos id=58 count=16496worker1 # [ 139.704491] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.874Z level=INFO msg="Uploading 1 narinfos"6497lb # [ 139.678367] niks3-server[1003]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6498lb # [ 139.690870] niks3-server[1003]: 2026/09/20 09:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6499worker1 # [ 139.724602] nix-grpc-daemon[1617]: time=2026-09-20T09:34:44.895Z level=INFO msg="Upload complete. (425ms)"6500lb # [ 139.695453] niks3-server[1003]: 2026/09/20 09:34:44 INFO Completed upload id=586501worker1 # [ 139.726728] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=7526502lb # [ 139.698704] niks3-server[1003]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6503worker1 # [ 139.732076] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406504worker1 # [ 139.741794] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6505worker1 # [ 139.744047] nix-daemon[845]: reaped child process 1832, status = succeeded6506worker1 # [ 139.749329] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6507worker1 # [ 140.762263] nix-daemon[845]: reaped child process 1846, status = succeeded6508worker1 # [ 140.914254] nix-daemon[845]: accepted connection from pid 1861, user nix-grpc-daemon (trusted)6509worker1 # [ 140.925107] nix-daemon[845]: reaped child process 1864, status = succeeded6510lb # [ 140.913829] niks3-server[1003]: 2026/09/20 09:34:45 INFO Received uploads request method=POST path=/api/pending_closures6511worker1 # [ 140.951479] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.121Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6512worker1 # [ 140.953164] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.123Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6513lb # [ 140.948742] niks3-server[1003]: 2026/09/20 09:34:45 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6514lb # [ 140.961516] niks3-server[1003]: 2026/09/20 09:34:45 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6515lb # [ 140.974245] niks3-server[1003]: 2026/09/20 09:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6516worker1 # [ 141.007719] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.178Z level=INFO msg="Uploading 1 narinfos"6517lb # [ 140.978904] niks3-server[1003]: 2026/09/20 09:34:45 INFO Signed narinfos id=59 count=16518lb # [ 140.984545] niks3-server[1003]: 2026/09/20 09:34:45 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6519lb # [ 140.992483] niks3-server[1003]: 2026/09/20 09:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6520lb # [ 140.998813] niks3-server[1003]: 2026/09/20 09:34:45 INFO Completed upload id=596521worker1 # [ 141.028272] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.198Z level=INFO msg="Upload complete. (272ms)"6522lb # [ 141.001735] niks3-server[1003]: 2026/09/20 09:34:45 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6523worker2 # [ 141.027670] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:45Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16524worker1 # [ 141.034628] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16525worker1 # [ 141.047523] nix-daemon[845]: reaped child process 1844, status = succeeded6526worker1 # [ 141.084485] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406527worker1 # [ 141.096760] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406528worker1 # [ 141.109498] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.280Z level=INFO msg="All 2 paths already cached"6529worker1 # [ 141.116695] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406530worker1 # [ 141.126158] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6531worker1 # [ 141.306598] nix-daemon[845]: accepted connection from pid 1868, user nix-grpc-daemon (trusted)6532worker1 # [ 141.318860] nix-daemon[845]: reaped child process 1871, status = succeeded6533worker1 # [ 141.492512] nix-daemon[845]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6534worker1 # [ 141.504658] nix-daemon[845]: reaped child process 1876, status = succeeded6535lb # [ 141.492692] niks3-server[1003]: 2026/09/20 09:34:46 INFO Received uploads request method=POST path=/api/pending_closures6536worker1 # [ 141.532237] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.702Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6537worker1 # [ 141.533642] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.702Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6538lb # [ 141.522291] niks3-server[1003]: 2026/09/20 09:34:46 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6539lb # [ 141.530715] niks3-server[1003]: 2026/09/20 09:34:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6540worker1 # [ 141.564411] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.734Z level=INFO msg="Uploading 1 narinfos"6541lb # [ 141.536260] niks3-server[1003]: 2026/09/20 09:34:46 INFO Signed narinfos id=60 count=16542lb # [ 141.542264] niks3-server[1003]: 2026/09/20 09:34:46 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6543lb # [ 141.554346] niks3-server[1003]: 2026/09/20 09:34:46 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6544worker1 # [ 141.589221] nix-grpc-daemon[1617]: time=2026-09-20T09:34:46.759Z level=INFO msg="Upload complete. (451ms)"6545lb # [ 141.560391] niks3-server[1003]: 2026/09/20 09:34:46 INFO Completed upload id=606546worker1 # [ 141.591756] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=7526547lb # [ 141.563570] niks3-server[1003]: 2026/09/20 09:34:46 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6548worker1 # [ 141.596788] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406549worker1 # [ 141.606431] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6550worker1 # [ 141.610432] nix-daemon[845]: reaped child process 1866, status = succeeded6551worker1 # [ 141.615537] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6552worker1 # [ 142.228250] nix-daemon[845]: reaped child process 1880, status = succeeded6553worker1 # [ 142.373566] nix-daemon[845]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6554worker1 # [ 142.384592] nix-daemon[845]: reaped child process 1898, status = succeeded6555lb # [ 142.372920] niks3-server[1003]: 2026/09/20 09:34:47 INFO Received uploads request method=POST path=/api/pending_closures6556worker1 # [ 142.411396] nix-grpc-daemon[1617]: time=2026-09-20T09:34:47.581Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6557worker1 # [ 142.413311] nix-grpc-daemon[1617]: time=2026-09-20T09:34:47.583Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6558lb # [ 142.410561] niks3-server[1003]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6559lb # [ 142.422578] niks3-server[1003]: 2026/09/20 09:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6560lb # [ 142.428481] niks3-server[1003]: 2026/09/20 09:34:47 INFO Signed narinfos id=61 count=16561worker1 # [ 142.460922] nix-grpc-daemon[1617]: time=2026-09-20T09:34:47.631Z level=INFO msg="Uploading 1 narinfos"6562lb # [ 142.434293] niks3-server[1003]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6563lb # [ 142.439339] niks3-server[1003]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6564lb # [ 142.448523] niks3-server[1003]: 2026/09/20 09:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6565lb # [ 142.453559] niks3-server[1003]: 2026/09/20 09:34:47 INFO Completed upload id=616566worker1 # [ 142.482966] nix-grpc-daemon[1617]: time=2026-09-20T09:34:47.653Z level=INFO msg="Upload complete. (261ms)"6567lb # [ 142.456283] niks3-server[1003]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6568worker2 # [ 142.481859] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:46Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16569worker1 # [ 142.490626] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16570worker1 # [ 142.496758] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406571worker1 # [ 142.509163] nix-daemon[845]: reaped child process 1878, status = succeeded6572worker1 # [ 142.533192] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406573worker1 # [ 142.545388] nix-grpc-daemon[1617]: time=2026-09-20T09:34:47.716Z level=INFO msg="All 2 paths already cached"6574worker1 # [ 142.552308] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384406575worker1 # [ 142.561156] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6576worker1 # [ 142.720064] nix-daemon[845]: accepted connection from pid 1902, user nix-grpc-daemon (trusted)6577worker1 # [ 142.733093] nix-daemon[845]: reaped child process 1905, status = succeeded6578worker1 # [ 142.894916] nix-daemon[845]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)6579worker1 # [ 142.906850] nix-daemon[845]: reaped child process 1910, status = succeeded6580lb # [ 142.895070] niks3-server[1003]: 2026/09/20 09:34:47 INFO Received uploads request method=POST path=/api/pending_closures6581worker1 # [ 142.932931] nix-grpc-daemon[1617]: time=2026-09-20T09:34:48.103Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6582worker1 # [ 142.934401] nix-grpc-daemon[1617]: time=2026-09-20T09:34:48.103Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6583lb # [ 142.919292] niks3-server[1003]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6584lb # [ 142.931409] niks3-server[1003]: 2026/09/20 09:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6585lb # [ 142.936732] niks3-server[1003]: 2026/09/20 09:34:47 INFO Signed narinfos id=62 count=16586worker1 # [ 142.968634] nix-grpc-daemon[1617]: time=2026-09-20T09:34:48.139Z level=INFO msg="Uploading 1 narinfos"6587lb # [ 142.942309] niks3-server[1003]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6588lb # [ 142.954539] niks3-server[1003]: 2026/09/20 09:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6589worker1 # [ 142.989866] nix-grpc-daemon[1617]: time=2026-09-20T09:34:48.160Z level=INFO msg="Upload complete. (418ms)"6590lb # [ 142.960930] niks3-server[1003]: 2026/09/20 09:34:47 INFO Completed upload id=626591worker1 # [ 142.992194] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=0 paths=1 nar_bytes_in=7526592lb # [ 142.964800] niks3-server[1003]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6593worker1 # [ 142.998232] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384406594worker1 # [ 143.008223] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6595worker1 # [ 143.011502] nix-daemon[845]: reaped child process 1900, status = succeeded6596worker1 # [ 143.016719] nix-daemon[845]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6597worker1 # [ 144.244468] nix-daemon[845]: reaped child process 1914, status = succeeded6598worker1 # [ 144.408958] nix-daemon[845]: accepted connection from pid 1929, user nix-grpc-daemon (trusted)6599worker1 # [ 144.420490] nix-daemon[845]: reaped child process 1932, status = succeeded6600lb # [ 144.407325] niks3-server[1003]: 2026/09/20 09:34:49 INFO Received uploads request method=POST path=/api/pending_closures6601worker1 # [ 144.444375] nix-grpc-daemon[1617]: time=2026-09-20T09:34:49.614Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6602worker1 # [ 144.445894] nix-grpc-daemon[1617]: time=2026-09-20T09:34:49.616Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6603lb # [ 144.441829] niks3-server[1003]: 2026/09/20 09:34:49 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6604lb # [ 144.456638] niks3-server[1003]: 2026/09/20 09:34:49 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6605lb # [ 144.465929] niks3-server[1003]: 2026/09/20 09:34:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6606worker1 # [ 144.498865] nix-grpc-daemon[1617]: time=2026-09-20T09:34:49.669Z level=INFO msg="Uploading 1 narinfos"6607lb # [ 144.469886] niks3-server[1003]: 2026/09/20 09:34:49 INFO Signed narinfos id=63 count=16608lb # [ 144.474851] niks3-server[1003]: 2026/09/20 09:34:49 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6609lb # [ 144.481879] niks3-server[1003]: 2026/09/20 09:34:49 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6610lb # [ 144.485102] niks3-server[1003]: 2026/09/20 09:34:49 INFO Completed upload id=636611worker1 # [ 144.514377] nix-grpc-daemon[1617]: time=2026-09-20T09:34:49.685Z level=INFO msg="Upload complete. (289ms)"6612lb # [ 144.486625] niks3-server[1003]: 2026/09/20 09:34:49 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6613worker1 # [ 144.519576] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38440 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16614worker1 # [ 144.531434] nix-daemon[845]: reaped child process 1912, status = succeeded6615client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.78 seconds)6616worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6617worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.05 seconds)6618client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16619worker2 # [ 144.801586] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585806620worker1 # [ 144.811266] nix-grpc-daemon[1561]: ts=2026-09-20T09:34:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406621worker2 # [ 144.826670] nix-grpc-daemon[1498]: ts=2026-09-20T09:34:49Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06622lb # [ 147.681976] envoy[910]: conn peer=[2001:db8:1::1]:51954 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27055 tx=10680 ms=165606623client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.25 seconds)6624(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.09 seconds)6625subtest: low disk drains a worker and builds go to the other6626worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6627worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.99 seconds)6628worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6629worker1 # [ 176.532683] nix-grpc-daemon[1561]: ts=2026-09-20T09:35:21Z level=info event=unhealthy reason=min_free6630worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)6631client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26632worker2 # [ 177.151754] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:585806633client # don't know how to build these paths:6634client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6635worker1 # [ 177.168200] nix-grpc-daemon[1561]: ts=2026-09-20T09:35:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384406636worker2 # [ 177.182907] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:21Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06637worker2 # [ 177.188186] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:21Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16638worker2 # [ 177.227777] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806639worker2 # [ 177.239807] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585806640client # copying 1 paths to 'lb:50051'...6641client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6642worker2 # [ 177.251607] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585806643worker2 # [ 177.262120] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6644worker2 # [ 177.417298] nix-daemon[840]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6645worker2 # [ 177.429250] nix-daemon[840]: reaped child process 1777, status = succeeded6646worker2 # [ 177.576805] nix-daemon[840]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6647worker2 # [ 177.588721] nix-daemon[840]: reaped child process 1782, status = succeeded6648lb # [ 177.580345] niks3-server[1003]: 2026/09/20 09:35:22 INFO Received uploads request method=POST path=/api/pending_closures6649worker2 # [ 177.608078] nix-grpc-daemon[1570]: time=2026-09-20T09:35:21.812Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6650worker2 # [ 177.609526] nix-grpc-daemon[1570]: time=2026-09-20T09:35:21.812Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6651lb # [ 177.594926] niks3-server[1003]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6652lb # [ 177.600583] niks3-server[1003]: 2026/09/20 09:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6653worker2 # [ 177.625608] nix-grpc-daemon[1570]: time=2026-09-20T09:35:21.830Z level=INFO msg="Uploading 1 narinfos"6654lb # [ 177.604026] niks3-server[1003]: 2026/09/20 09:35:22 INFO Signed narinfos id=64 count=16655lb # [ 177.606184] niks3-server[1003]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6656lb # [ 177.611222] niks3-server[1003]: 2026/09/20 09:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6657lb # [ 177.614848] niks3-server[1003]: 2026/09/20 09:35:22 INFO Completed upload id=646658worker2 # [ 177.636953] nix-grpc-daemon[1570]: time=2026-09-20T09:35:21.841Z level=INFO msg="Upload complete. (363ms)"6659lb # [ 177.616700] niks3-server[1003]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6660worker2 # [ 177.638891] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=0 paths=1 nar_bytes_in=5126661worker2 # [ 177.642590] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806662worker2 # [ 177.648270] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6663worker2 # [ 177.650567] nix-daemon[840]: reaped child process 1772, status = succeeded6664client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6665client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6666worker2 # [ 177.655792] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6667lb # [ 177.683550] envoy[910]: conn peer=[2001:db8:1::1]:59168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300416668client # farm-a> LOG-farm-a-drain6669worker2 # [ 178.880188] nix-daemon[840]: reaped child process 1786, status = succeeded6670worker2 # [ 179.009527] nix-daemon[840]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6671worker2 # [ 179.021159] nix-daemon[840]: reaped child process 1804, status = succeeded6672lb # [ 179.010454] niks3-server[1003]: 2026/09/20 09:35:23 INFO Received uploads request method=POST path=/api/pending_closures6673worker2 # [ 179.041487] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.246Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6674worker2 # [ 179.042968] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.247Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6675lb # [ 179.044190] niks3-server[1003]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6676lb # [ 179.061816] niks3-server[1003]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6677lb # [ 179.070433] niks3-server[1003]: 2026/09/20 09:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6678lb # [ 179.072654] niks3-server[1003]: 2026/09/20 09:35:24 INFO Signed narinfos id=65 count=16679worker2 # [ 179.094837] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.299Z level=INFO msg="Uploading 1 narinfos"6680lb # [ 179.075113] niks3-server[1003]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6681lb # [ 179.080446] niks3-server[1003]: 2026/09/20 09:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6682lb # [ 179.083713] niks3-server[1003]: 2026/09/20 09:35:24 INFO Completed upload id=656683lb # [ 179.085459] niks3-server[1003]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6684worker2 # [ 179.107092] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.311Z level=INFO msg="Upload complete. (244ms)"6685worker2 # [ 179.113667] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16686worker2 # [ 179.121863] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:23Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06687worker2 # [ 179.124899] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:23Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16688worker2 # [ 179.129830] nix-daemon[840]: reaped child process 1784, status = succeeded6689worker2 # [ 179.165599] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806690worker2 # [ 179.176204] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585806691worker2 # [ 179.186203] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.391Z level=INFO msg="All 1 paths already cached"6692client # copying 1 paths to 'lb:50051'...6693client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6694worker2 # [ 179.193648] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585806695worker2 # [ 179.201673] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6696worker2 # [ 179.357507] nix-daemon[840]: accepted connection from pid 1810, user nix-grpc-daemon (trusted)6697worker2 # [ 179.369419] nix-daemon[840]: reaped child process 1813, status = succeeded6698worker2 # [ 179.519179] nix-daemon[840]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6699worker2 # [ 179.530900] nix-daemon[840]: reaped child process 1818, status = succeeded6700lb # [ 179.518240] niks3-server[1003]: 2026/09/20 09:35:24 INFO Received uploads request method=POST path=/api/pending_closures6701worker2 # [ 179.544978] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.749Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6702worker2 # [ 179.546511] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.751Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6703lb # [ 179.533613] niks3-server[1003]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6704lb # [ 179.537564] niks3-server[1003]: 2026/09/20 09:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6705lb # [ 179.540040] niks3-server[1003]: 2026/09/20 09:35:24 INFO Signed narinfos id=66 count=16706worker2 # [ 179.561915] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.766Z level=INFO msg="Uploading 1 narinfos"6707lb # [ 179.542987] niks3-server[1003]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6708lb # [ 179.547982] niks3-server[1003]: 2026/09/20 09:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6709lb # [ 179.550775] niks3-server[1003]: 2026/09/20 09:35:24 INFO Completed upload id=666710worker2 # [ 179.574057] nix-grpc-daemon[1570]: time=2026-09-20T09:35:23.779Z level=INFO msg="Upload complete. (361ms)"6711lb # [ 179.552679] niks3-server[1003]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6712worker2 # [ 179.575966] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=0 paths=1 nar_bytes_in=6246713worker2 # [ 179.579148] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806714worker2 # [ 179.583253] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6715worker2 # [ 179.588489] nix-daemon[840]: reaped child process 1808, status = succeeded6716worker2 # [ 179.592533] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6717client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6718client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6719client # farm-b> LOG-farm-b-drain6720worker2 # [ 180.819229] nix-daemon[840]: reaped child process 1822, status = succeeded6721worker2 # [ 180.948809] nix-daemon[840]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6722worker2 # [ 180.960975] nix-daemon[840]: reaped child process 1840, status = succeeded6723lb # [ 180.949350] niks3-server[1003]: 2026/09/20 09:35:25 INFO Received uploads request method=POST path=/api/pending_closures6724worker2 # [ 180.979775] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.184Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6725worker2 # [ 180.983059] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.186Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6726lb # [ 180.992987] niks3-server[1003]: 2026/09/20 09:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6727lb # [ 181.002288] niks3-server[1003]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6728worker2 # [ 181.030145] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.234Z level=INFO msg="Uploading 1 narinfos"6729lb # [ 181.008305] niks3-server[1003]: 2026/09/20 09:35:25 INFO Signed narinfos id=67 count=16730lb # [ 181.012519] niks3-server[1003]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6731lb # [ 181.020115] niks3-server[1003]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6732lb # [ 181.029360] niks3-server[1003]: 2026/09/20 09:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6733lb # [ 181.035154] niks3-server[1003]: 2026/09/20 09:35:25 INFO Completed upload id=676734worker2 # [ 181.057426] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.262Z level=INFO msg="Upload complete. (257ms)"6735lb # [ 181.037338] niks3-server[1003]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6736worker2 # [ 181.066394] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16737worker2 # [ 181.074858] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:25Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06738worker2 # [ 181.078960] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:25Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16739worker2 # [ 181.083224] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806740worker2 # [ 181.091946] nix-daemon[840]: reaped child process 1820, status = succeeded6741worker2 # [ 181.094210] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:585806742worker2 # [ 181.101378] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.306Z level=INFO msg="All 2 paths already cached"6743client # copying 1 paths to 'lb:50051'...6744worker2 # [ 181.106507] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:585806745client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6746worker2 # [ 181.113408] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6747worker2 # [ 181.254062] nix-daemon[840]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6748worker2 # [ 181.266178] nix-daemon[840]: reaped child process 1849, status = succeeded6749worker2 # [ 181.405630] nix-daemon[840]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6750worker2 # [ 181.417835] nix-daemon[840]: reaped child process 1854, status = succeeded6751lb # [ 181.406829] niks3-server[1003]: 2026/09/20 09:35:26 INFO Received uploads request method=POST path=/api/pending_closures6752worker2 # [ 181.436879] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.641Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6753worker2 # [ 181.438354] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.643Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6754lb # [ 181.437147] niks3-server[1003]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6755lb # [ 181.446817] niks3-server[1003]: 2026/09/20 09:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6756lb # [ 181.451066] niks3-server[1003]: 2026/09/20 09:35:26 INFO Signed narinfos id=68 count=16757worker2 # [ 181.478516] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.683Z level=INFO msg="Uploading 1 narinfos"6758lb # [ 181.456843] niks3-server[1003]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6759lb # [ 181.468759] niks3-server[1003]: 2026/09/20 09:35:26 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6760lb # [ 181.473504] niks3-server[1003]: 2026/09/20 09:35:26 INFO Completed upload id=686761worker2 # [ 181.495517] nix-grpc-daemon[1570]: time=2026-09-20T09:35:25.700Z level=INFO msg="Upload complete. (377ms)"6762worker2 # [ 181.498868] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=0 paths=1 nar_bytes_in=7526763lb # [ 181.478230] niks3-server[1003]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6764worker2 # [ 181.504580] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:585806765worker2 # [ 181.512263] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6766worker2 # [ 181.516688] nix-daemon[840]: reaped child process 1844, status = succeeded6767client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6768worker2 # [ 181.522594] nix-daemon[840]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6769client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6770client # farm-top> LOG-farm-top-drain6771worker2 # [ 182.936324] nix-daemon[840]: reaped child process 1858, status = succeeded6772worker2 # [ 183.080270] nix-daemon[840]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6773worker2 # [ 183.093468] nix-daemon[840]: reaped child process 1876, status = succeeded6774lb # [ 183.089873] niks3-server[1003]: 2026/09/20 09:35:28 INFO Received uploads request method=POST path=/api/pending_closures6775worker2 # [ 183.118474] nix-grpc-daemon[1570]: time=2026-09-20T09:35:27.323Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6776worker2 # [ 183.119948] nix-grpc-daemon[1570]: time=2026-09-20T09:35:27.324Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6777lb # [ 183.124538] niks3-server[1003]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6778lb # [ 183.133359] niks3-server[1003]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6779lb # [ 183.143712] niks3-server[1003]: 2026/09/20 09:35:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6780worker2 # [ 183.170428] nix-grpc-daemon[1570]: time=2026-09-20T09:35:27.375Z level=INFO msg="Uploading 1 narinfos"6781lb # [ 183.149533] niks3-server[1003]: 2026/09/20 09:35:28 INFO Signed narinfos id=69 count=16782lb # [ 183.153476] niks3-server[1003]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6783lb # [ 183.161479] niks3-server[1003]: 2026/09/20 09:35:28 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6784lb # [ 183.167104] niks3-server[1003]: 2026/09/20 09:35:28 INFO Completed upload id=696785worker2 # [ 183.189268] nix-grpc-daemon[1570]: time=2026-09-20T09:35:27.393Z level=WARN msg="Failed to register uploaded object" key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo error="server returned 404: object is not pending upload\n"6786worker2 # [ 183.195387] nix-grpc-daemon[1570]: time=2026-09-20T09:35:27.394Z level=INFO msg="Upload complete. (260ms)"6787worker2 # [ 183.203405] nix-grpc-daemon[1498]: ts=2026-09-20T09:35:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58580 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16788worker2 # [ 183.213423] nix-daemon[840]: reaped child process 1856, status = succeeded6789client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.28 seconds)6790client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6791client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.20 seconds)6792worker1: must succeed: rm /nix/.rw-store/fill6793worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)6794worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6795worker1 # [ 184.540182] nix-grpc-daemon[1561]: ts=2026-09-20T09:35:29Z level=info event=healthy reason=min_free6796worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)6797(finished: subtest: low disk drains a worker and builds go to the other, in 9.90 seconds)6798(finished: run the VM test script, in 185.65 seconds)6799test script finished in 185.88s6800cleanup6801kill QemuMachine (pid 49)6802client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6803kill QemuMachine (pid 50)6804lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6805kill QemuMachine (pid 51)6806worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6807kill QemuMachine (pid 52)6808worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6809(finished: cleanup, in 1.00 seconds)6810additionally exposed symbols:6811 client, lb, worker1, worker2,6812 vlan1,6813 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh

succeeded vm-test-run-nix-grpc-farm checks.x86_64-linux.farm · build #279 · raw

succeeded `vm-test-run-nix-grpc-farm` `checks.x86_64-linux.farm` · build #279 · raw