vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #281
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13worker1 # Disk image does not exist, creating the virtualisation disk image...14lb: QEMU running (pid 50)15lb # Formatting '/build/vm-state-lb/tmp.HlxnRYIO9K', fmt=raw size=107374182416worker1: QEMU running (pid 51)17lb # mke2fs 1.47.4 (6-Mar-2025)18client: QEMU running (pid 49)19client # Disk image does not exist, creating the virtualisation disk image...20client # Formatting '/build/vm-state-client/tmp.NYRg3Pd2bs', fmt=raw size=107374182421lb # Discarding device blocks: 0/262144 done22client # mke2fs 1.47.4 (6-Mar-2025)23worker2: QEMU running (pid 52)24worker2 # Disk image does not exist, creating the virtualisation disk image...25client # Discarding device blocks: 0/262144 done26client # Creating filesystem with 262144 4k blocks and 65536 inodes27worker1 # Formatting '/build/vm-state-worker1/tmp.l62smR7vD3', fmt=raw size=107374182428lb # Creating filesystem with 262144 4k blocks and 65536 inodes29worker2 # Formatting '/build/vm-state-worker2/tmp.51oTxmgeMR', fmt=raw size=107374182430client # Filesystem UUID: ad66f794-6e61-40d0-9ce0-0d977233334631client # Superblock backups stored on blocks:32lb # Filesystem UUID: 14993162-0fd8-4a42-8c5e-061d2cec6d0033(finished: start all VMs, in 0.19 seconds)34worker1 # mke2fs 1.47.4 (6-Mar-2025)35lb # Superblock backups stored on blocks:36lb # 32768, 98304, 163840, 22937637lb # 38worker1 # Discarding device blocks: 0/262144 done39worker2 # mke2fs 1.47.4 (6-Mar-2025)40client: waiting for unit network-addresses-eth1.service41client: waiting for the VM to finish booting42worker2 # Discarding device blocks: 0/262144 done43lb # Allocating group tables: 0/8 done44client # 32768, 98304, 163840, 22937645worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes46worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes47lb # Writing inode tables: 0/8 done48client # 49worker1 # Filesystem UUID: d8d12c67-d14d-49a3-a34d-7f62eb44bd8c50worker2 # Filesystem UUID: ddac92e6-197b-4011-91d9-5dca67a89e4d51lb # Creating journal (8192 blocks): done52client # Allocating group tables: 0/8 done53worker1 # Superblock backups stored on blocks:54worker2 # Superblock backups stored on blocks:55lb # Writing superblocks and filesystem accounting information: 0/8 done56client # Writing inode tables: 0/8 done57worker1 # 32768, 98304, 163840, 22937658worker2 # 32768, 98304, 163840, 22937659lb # 60client # Creating journal (8192 blocks): done61worker1 # 62worker2 # 63lb # Virtualisation disk image created.64client # Writing superblocks and filesystem accounting information: 0/8 done65worker1 # Allocating group tables: 0/8 done66client # 67worker1 # Writing inode tables: 0/8 done68client # Virtualisation disk image created.69worker1 # Creating journal (8192 blocks): done70worker2 # Allocating group tables: 0/8 done71worker1 # Writing superblocks and filesystem accounting information: 0/8 done72worker2 # Writing inode tables: 0/8 done73worker1 # 74worker2 # Creating journal (8192 blocks): done75worker1 # Virtualisation disk image created.76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker1 # 84worker1 # 85worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87worker1 # 88worker1 # 89worker1 # 90worker1 # 91worker2 # 92worker2 # 93client # 94client # 95worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0096worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0097client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0098worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...99worker2 # 100worker2 # 101worker2 # 102worker2 # 103worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...104worker1 # 105worker1 # 106client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...107client # 108client # 109client # 110client # 111lb # 112lb # 113client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00114worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00115lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00116lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...117lb # 118lb # 119lb # 120lb # 121client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...122client # 123client # 124worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...125worker2 # 126worker2 # 127lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129lb # 130lb # 131worker1 # Booting from ROM...132client # Booting from ROM...133worker2 # Booting from ROM...134lb # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/98yf21c3jz4dira9wimzcqdwlb0l32kr-nixos-system-worker1-test/init regInfo=/nix/store/rgb3swjz9fl8q4w8lswl5z8wlc1zgzv4-closure-info/registration console=ttyS0,115200n8 console=tty0137worker1 # [ 0.000000] BIOS-provided physical RAM map:138worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026140worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved141worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable143worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved144client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test/init regInfo=/nix/store/z9v5mszvmn20hi4xvr0fypzq6a1x1rya-closure-info/registration console=ttyS0,115200n8 console=tty0145worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved146client # [ 0.000000] BIOS-provided physical RAM map:147worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved148client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable149worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026150worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved151client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved152client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved153worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved154client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable155worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved156worker1 # [ 0.000000] NX (Execute Disable) protection: active157client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved158worker1 # [ 0.000000] APIC: Static calls initialized159worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2y6kwi1mdv3l7s63cygabx3gwjz592yw-nixos-system-worker2-test/init regInfo=/nix/store/rxnsaix8qkg1nyh96vsafrk0l5rbj4zn-closure-info/registration console=ttyS0,115200n8 console=tty0160worker1 # [ 0.000000] SMBIOS 2.8 present.161client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved162worker2 # [ 0.000000] BIOS-provided physical RAM map:163client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved164worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014165worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable166worker1 # [ 0.000000] DMI: Memory slots populated: 1/1167client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved168worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved169worker1 # [ 0.000000] Hypervisor detected: KVM170client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved171worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000172worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved173client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved174worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00175lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026176worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable177client # [ 0.000000] NX (Execute Disable) protection: active178worker1 # [ 0.000000] kvm-clock: using sched offset of 452091989 cycles179client # [ 0.000000] APIC: Static calls initialized180worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved181client # [ 0.000000] SMBIOS 2.8 present.182worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns183worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved184worker1 # [ 0.000004] tsc: Detected 2400.012 MHz processor185client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014186worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved187worker1 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000188client # [ 0.000000] DMI: Memory slots populated: 1/1189client # [ 0.000000] Hypervisor detected: KVM190worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved191lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r7i2qny6zxqjgksiw8bnyjrpmwmq2adf-nixos-system-lb-test/init regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration console=ttyS0,115200n8 console=tty0192worker1 # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs193client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000194lb # [ 0.000000] BIOS-provided physical RAM map:195worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved196client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00197worker1 # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT198lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable199worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200client # [ 0.000000] kvm-clock: using sched offset of 452903795 cycles201worker1 # [ 0.002745] found SMP MP-table at [mem 0x000f5450-0x000f545f]202worker2 # [ 0.000000] NX (Execute Disable) protection: active203lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved204worker1 # [ 0.002757] Using GB pages for direct mapping205worker2 # [ 0.000000] APIC: Static calls initialized206worker1 # [ 0.002895] RAMDISK: [mem 0x5e367000-0x5ffcffff]207worker2 # [ 0.000000] SMBIOS 2.8 present.208lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved209client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns210client # [ 0.000005] tsc: Detected 2400.012 MHz processor211worker1 # [ 0.002904] ACPI: Early table checksum verification disabled212lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable213client # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000214worker1 # [ 0.002907] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )215worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014216lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved217worker2 # [ 0.000000] DMI: Memory slots populated: 1/1218client # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs219worker1 # [ 0.002911] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)220worker2 # [ 0.000000] Hypervisor detected: KVM221lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved222worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000223client # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT224worker1 # [ 0.002915] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)225lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved226worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00227client # [ 0.002734] found SMP MP-table at [mem 0x000f5450-0x000f545f]228client # [ 0.002745] Using GB pages for direct mapping229worker2 # [ 0.000001] kvm-clock: using sched offset of 467855780 cycles230lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved231worker1 # [ 0.002922] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)232client # [ 0.002901] RAMDISK: [mem 0x5e366000-0x5ffcffff]233worker1 # [ 0.002924] ACPI: FACS 0x000000005FFE0000 000040234lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved235client # [ 0.002909] ACPI: Early table checksum verification disabled236worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns237lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved238worker1 # [ 0.002925] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)239worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor240client # [ 0.002912] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )241lb # [ 0.000000] NX (Execute Disable) protection: active242worker2 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000243lb # [ 0.000000] APIC: Static calls initialized244worker1 # [ 0.002927] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)245client # [ 0.002915] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)246lb # [ 0.000000] SMBIOS 2.8 present.247worker2 # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs248worker1 # [ 0.002928] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)249client # [ 0.002919] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)250worker2 # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT251lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014252worker1 # [ 0.002930] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)253lb # [ 0.000000] DMI: Memory slots populated: 1/1254worker2 # [ 0.002736] found SMP MP-table at [mem 0x000f5450-0x000f545f]255client # [ 0.002925] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)256lb # [ 0.000000] Hypervisor detected: KVM257worker2 # [ 0.002747] Using GB pages for direct mapping258client # [ 0.002928] ACPI: FACS 0x000000005FFE0000 000040259worker1 # [ 0.002931] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]260lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000261worker2 # [ 0.002845] RAMDISK: [mem 0x5e367000-0x5ffcffff]262worker1 # [ 0.002932] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]263client # [ 0.002929] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)264lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00265worker2 # [ 0.002854] ACPI: Early table checksum verification disabled266lb # [ 0.000000] kvm-clock: using sched offset of 479933611 cycles267worker1 # [ 0.002933] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]268worker2 # [ 0.002856] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )269client # [ 0.002931] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)270worker1 # [ 0.002933] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]271worker2 # [ 0.002861] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)272client # [ 0.002932] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)273lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns274worker1 # [ 0.002934] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]275lb # [ 0.000004] tsc: Detected 2400.012 MHz processor276worker2 # [ 0.002865] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)277client # [ 0.002934] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)278worker1 # [ 0.002934] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]279lb # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000280client # [ 0.002935] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]281worker2 # [ 0.002872] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)282worker1 # [ 0.002934] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]283lb # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs284worker2 # [ 0.002874] ACPI: FACS 0x000000005FFE0000 000040285client # [ 0.002936] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]286worker1 # [ 0.003160] No NUMA configuration found287lb # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT288worker1 # [ 0.003161] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]289client # [ 0.002937] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]290worker2 # [ 0.002875] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)291lb # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]292worker1 # [ 0.003163] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]293lb # [ 0.002741] Using GB pages for direct mapping294client # [ 0.002937] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]295worker1 # [ 0.005291] Zone ranges:296worker2 # [ 0.002877] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)297lb # [ 0.002846] RAMDISK: [mem 0x5e367000-0x5ffcffff]298worker1 # [ 0.005291] DMA [mem 0x0000000000001000-0x0000000000ffffff]299client # [ 0.002938] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]300lb # [ 0.002854] ACPI: Early table checksum verification disabled301worker2 # [ 0.002879] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)302worker1 # [ 0.005293] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]303client # [ 0.002938] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]304lb # [ 0.002857] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )305worker1 # [ 0.005294] Normal empty306worker2 # [ 0.002880] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)307worker1 # [ 0.005295] Device empty308client # [ 0.002939] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]309lb # [ 0.002862] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)310worker1 # [ 0.005295] Movable zone start for each node311client # [ 0.003165] No NUMA configuration found312worker2 # [ 0.002882] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]313worker1 # [ 0.005296] Early memory node ranges314client # [ 0.003166] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]315lb # [ 0.002866] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)316worker1 # [ 0.005296] node 0: [mem 0x0000000000001000-0x000000000009efff]317worker2 # [ 0.002883] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]318client # [ 0.003168] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]319worker1 # [ 0.005297] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]320worker2 # [ 0.002883] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]321lb # [ 0.002873] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)322client # [ 0.005397] Zone ranges:323lb # [ 0.002875] ACPI: FACS 0x000000005FFE0000 000040324client # [ 0.005398] DMA [mem 0x0000000000001000-0x0000000000ffffff]325worker1 # [ 0.005298] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]326worker2 # [ 0.002884] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]327client # [ 0.005400] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]328worker1 # [ 0.005365] On node 0, zone DMA: 1 pages in unavailable ranges329lb # [ 0.002876] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)330worker2 # [ 0.002884] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]331client # [ 0.005401] Normal empty332client # [ 0.005402] Device empty333worker1 # [ 0.005614] On node 0, zone DMA: 97 pages in unavailable ranges334worker2 # [ 0.002885] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]335client # [ 0.005402] Movable zone start for each node336lb # [ 0.002878] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)337worker1 # [ 0.030598] On node 0, zone DMA32: 40 pages in unavailable ranges338client # [ 0.005403] Early memory node ranges339worker2 # [ 0.002885] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]340worker1 # [ 0.031059] ACPI: PM-Timer IO Port: 0x608341lb # [ 0.002880] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)342worker2 # [ 0.003110] No NUMA configuration found343client # [ 0.005403] node 0: [mem 0x0000000000001000-0x000000000009efff]344worker1 # [ 0.031070] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])345client # [ 0.005404] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]346worker2 # [ 0.003111] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]347lb # [ 0.002881] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)348worker1 # [ 0.031094] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23349worker2 # [ 0.003113] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]350client # [ 0.005405] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]351lb # [ 0.002882] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]352worker2 # [ 0.005503] Zone ranges:353worker1 # [ 0.031097] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)354client # [ 0.005424] On node 0, zone DMA: 1 pages in unavailable ranges355worker2 # [ 0.005504] DMA [mem 0x0000000000001000-0x0000000000ffffff]356worker1 # [ 0.031098] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)357lb # [ 0.002883] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]358client # [ 0.005668] On node 0, zone DMA: 97 pages in unavailable ranges359worker2 # [ 0.005505] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]360worker1 # [ 0.031099] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)361lb # [ 0.002884] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]362worker2 # [ 0.005506] Normal empty363client # [ 0.030729] On node 0, zone DMA32: 40 pages in unavailable ranges364worker2 # [ 0.005507] Device empty365client # [ 0.031186] ACPI: PM-Timer IO Port: 0x608366worker1 # [ 0.031100] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)367lb # [ 0.002884] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]368worker2 # [ 0.005508] Movable zone start for each node369client # [ 0.031197] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])370worker2 # [ 0.005508] Early memory node ranges371worker1 # [ 0.031101] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)372lb # [ 0.002885] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]373worker2 # [ 0.005509] node 0: [mem 0x0000000000001000-0x000000000009efff]374client # [ 0.031221] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23375lb # [ 0.002885] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]376worker1 # [ 0.031103] ACPI: Using ACPI (MADT) for SMP configuration information377client # [ 0.031224] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)378worker2 # [ 0.005510] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]379worker1 # [ 0.031104] ACPI: HPET id: 0x8086a201 base: 0xfed00000380lb # [ 0.002886] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]381worker1 # [ 0.031108] TSC deadline timer available382client # [ 0.031225] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)383lb # [ 0.003117] No NUMA configuration found384worker2 # [ 0.005511] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]385worker1 # [ 0.031112] CPU topo: Max. logical packages: 1386worker2 # [ 0.005530] On node 0, zone DMA: 1 pages in unavailable ranges387lb # [ 0.003117] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]388client # [ 0.031226] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)389worker1 # [ 0.031112] CPU topo: Max. logical dies: 1390worker1 # [ 0.031113] CPU topo: Max. dies per package: 1391worker2 # [ 0.005779] On node 0, zone DMA: 97 pages in unavailable ranges392lb # [ 0.003120] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]393client # [ 0.031227] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)394lb # [ 0.005469] Zone ranges:395worker1 # [ 0.031116] CPU topo: Max. threads per core: 1396worker2 # [ 0.031236] On node 0, zone DMA32: 40 pages in unavailable ranges397client # [ 0.031228] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)398lb # [ 0.005470] DMA [mem 0x0000000000001000-0x0000000000ffffff]399worker1 # [ 0.031117] CPU topo: Num. cores per package: 1400worker2 # [ 0.031687] ACPI: PM-Timer IO Port: 0x608401client # [ 0.031230] ACPI: Using ACPI (MADT) for SMP configuration information402worker1 # [ 0.031117] CPU topo: Num. threads per package: 1403lb # [ 0.005471] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]404worker2 # [ 0.031700] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])405lb # [ 0.005472] Normal empty406client # [ 0.031231] ACPI: HPET id: 0x8086a201 base: 0xfed00000407worker1 # [ 0.031117] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs408lb # [ 0.005473] Device empty409client # [ 0.031235] TSC deadline timer available410worker2 # [ 0.031729] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23411lb # [ 0.005474] Movable zone start for each node412worker1 # [ 0.031134] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()413client # [ 0.031239] CPU topo: Max. logical packages: 1414lb # [ 0.005475] Early memory node ranges415worker2 # [ 0.031732] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)416client # [ 0.031239] CPU topo: Max. logical dies: 1417lb # [ 0.005475] node 0: [mem 0x0000000000001000-0x000000000009efff]418worker2 # [ 0.031733] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)419worker1 # [ 0.031164] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]420client # [ 0.031240] CPU topo: Max. dies per package: 1421lb # [ 0.005476] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]422worker2 # [ 0.031735] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)423worker1 # [ 0.031166] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]424client # [ 0.031243] CPU topo: Max. threads per core: 1425client # [ 0.031244] CPU topo: Num. cores per package: 1426lb # [ 0.005477] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]427worker1 # [ 0.031167] [mem 0x60000000-0xafffffff] available for PCI devices428worker2 # [ 0.031736] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)429client # [ 0.031244] CPU topo: Num. threads per package: 1430worker1 # [ 0.031168] Booting paravirtualized kernel on KVM431lb # [ 0.005496] On node 0, zone DMA: 1 pages in unavailable ranges432worker2 # [ 0.031737] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)433client # [ 0.031244] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs434lb # [ 0.005748] On node 0, zone DMA: 97 pages in unavailable ranges435worker2 # [ 0.031739] ACPI: Using ACPI (MADT) for SMP configuration information436worker1 # [ 0.031171] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns437client # [ 0.031261] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()438lb # [ 0.031369] On node 0, zone DMA32: 40 pages in unavailable ranges439worker2 # [ 0.031740] ACPI: HPET id: 0x8086a201 base: 0xfed00000440lb # [ 0.031823] ACPI: PM-Timer IO Port: 0x608441client # [ 0.031291] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]442worker2 # [ 0.031743] TSC deadline timer available443worker1 # [ 0.035660] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1444lb # [ 0.031834] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])445worker2 # [ 0.031748] CPU topo: Max. logical packages: 1446worker1 # [ 0.037703] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152447client # [ 0.031292] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]448worker2 # [ 0.031749] CPU topo: Max. logical dies: 1449lb # [ 0.031862] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23450worker1 # [ 0.037744] kvm-guest: PV spinlocks disabled, single CPU451worker2 # [ 0.031749] CPU topo: Max. dies per package: 1452client # [ 0.031293] [mem 0x60000000-0xafffffff] available for PCI devices453lb # [ 0.031865] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)454worker2 # [ 0.031754] CPU topo: Max. threads per core: 1455client # [ 0.031295] Booting paravirtualized kernel on KVM456lb # [ 0.031867] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)457worker2 # [ 0.031755] CPU topo: Num. cores per package: 1458worker2 # [ 0.031755] CPU topo: Num. threads per package: 1459client # [ 0.031297] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns460lb # [ 0.031868] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)461worker2 # [ 0.031756] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs462client # [ 0.035724] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1463lb # [ 0.031869] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)464worker2 # [ 0.031774] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()465worker1 # [ 0.037745] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/98yf21c3jz4dira9wimzcqdwlb0l32kr-nixos-system-worker1-test/init regInfo=/nix/store/rgb3swjz9fl8q4w8lswl5z8wlc1zgzv4-closure-info/registration console=ttyS0,115200n8 console=tty0466client # [ 0.037762] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152467lb # [ 0.031870] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)468client # [ 0.037801] kvm-guest: PV spinlocks disabled, single CPU469worker2 # [ 0.031805] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]470lb # [ 0.031872] ACPI: Using ACPI (MADT) for SMP configuration information471lb # [ 0.031873] ACPI: HPET id: 0x8086a201 base: 0xfed00000472worker2 # [ 0.031807] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]473worker1 # [ 0.037842] Unknown kernel command line parameters "regInfo=/nix/store/rgb3swjz9fl8q4w8lswl5z8wlc1zgzv4-closure-info/registration", will be passed to user space.474lb # [ 0.031877] TSC deadline timer available475worker1 # [ 0.037855] random: crng init done476worker2 # [ 0.031808] [mem 0x60000000-0xafffffff] available for PCI devices477lb # [ 0.031881] CPU topo: Max. logical packages: 1478worker2 # [ 0.031809] Booting paravirtualized kernel on KVM479worker1 # [ 0.037855] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes480lb # [ 0.031882] CPU topo: Max. logical dies: 1481lb # [ 0.031882] CPU topo: Max. dies per package: 1482worker1 # [ 0.039877] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)483worker2 # [ 0.031812] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns484lb # [ 0.031885] CPU topo: Max. threads per core: 1485client # [ 0.037803] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test/init regInfo=/nix/store/z9v5mszvmn20hi4xvr0fypzq6a1x1rya-closure-info/registration console=ttyS0,115200n8 console=tty0486lb # [ 0.031886] CPU topo: Num. cores per package: 1487worker1 # [ 0.041030] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)488worker2 # [ 0.036265] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1489lb # [ 0.031886] CPU topo: Num. threads per package: 1490worker1 # [ 0.041061] Fallback order for Node 0: 0491worker2 # [ 0.038334] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152492client # [ 0.037893] Unknown kernel command line parameters "regInfo=/nix/store/z9v5mszvmn20hi4xvr0fypzq6a1x1rya-closure-info/registration", will be passed to user space.493lb # [ 0.031887] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs494worker1 # [ 0.041064] Built 1 zonelists, mobility grouping on. Total pages: 393078495client # [ 0.037905] random: crng init done496worker1 # [ 0.041065] Policy zone: DMA32497worker2 # [ 0.038383] kvm-guest: PV spinlocks disabled, single CPU498lb # [ 0.031915] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()499client # [ 0.037906] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes500worker1 # [ 0.043434] mem auto-init: stack:all(zero), heap alloc:on, heap free:off501lb # [ 0.031946] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]502worker1 # [ 0.046379] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1503client # [ 0.039910] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)504lb # [ 0.031947] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]505worker1 # [ 0.049517] allocated 3145728 bytes of page_ext506client # [ 0.040911] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)507worker1 # [ 0.059154] ftrace: allocating 48731 entries in 192 pages508lb # [ 0.031948] [mem 0x60000000-0xafffffff] available for PCI devices509client # [ 0.040941] Fallback order for Node 0: 0510lb # [ 0.031950] Booting paravirtualized kernel on KVM511worker1 # [ 0.059156] ftrace: allocated 192 pages with 2 groups512worker2 # [ 0.038385] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2y6kwi1mdv3l7s63cygabx3gwjz592yw-nixos-system-worker2-test/init regInfo=/nix/store/rxnsaix8qkg1nyh96vsafrk0l5rbj4zn-closure-info/registration console=ttyS0,115200n8 console=tty0513worker1 # [ 0.059975] Dynamic Preempt: lazy514client # [ 0.040944] Built 1 zonelists, mobility grouping on. Total pages: 393078515client # [ 0.040945] Policy zone: DMA32516worker1 # [ 0.060095] rcu: Preemptible hierarchical RCU implementation.517lb # [ 0.031952] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns518client # [ 0.043437] mem auto-init: stack:all(zero), heap alloc:on, heap free:off519worker1 # [ 0.060096] rcu: RCU event tracing is enabled.520worker2 # [ 0.038478] Unknown kernel command line parameters "regInfo=/nix/store/rxnsaix8qkg1nyh96vsafrk0l5rbj4zn-closure-info/registration", will be passed to user space.521lb # [ 0.036388] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1522client # [ 0.046364] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1523worker1 # [ 0.060097] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.524worker2 # [ 0.038490] random: crng init done525client # [ 0.049460] allocated 3145728 bytes of page_ext526lb # [ 0.038497] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152527worker1 # [ 0.060098] Trampoline variant of Tasks RCU enabled.528worker2 # [ 0.038491] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes529lb # [ 0.038541] kvm-guest: PV spinlocks disabled, single CPU530worker1 # [ 0.060099] Rude variant of Tasks RCU enabled.531client # [ 0.059043] ftrace: allocating 48731 entries in 192 pages532worker1 # [ 0.060099] Tracing variant of Tasks RCU enabled.533client # [ 0.059045] ftrace: allocated 192 pages with 2 groups534worker2 # [ 0.040544] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)535client # [ 0.059834] Dynamic Preempt: lazy536worker1 # [ 0.060100] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.537worker2 # [ 0.041558] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)538client # [ 0.059996] rcu: Preemptible hierarchical RCU implementation.539worker2 # [ 0.041589] Fallback order for Node 0: 0540client # [ 0.059996] rcu: RCU event tracing is enabled.541worker1 # [ 0.060100] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1542worker2 # [ 0.041592] Built 1 zonelists, mobility grouping on. Total pages: 393078543client # [ 0.059997] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.544worker2 # [ 0.041592] Policy zone: DMA32545worker1 # [ 0.060114] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.546client # [ 0.059998] Trampoline variant of Tasks RCU enabled.547lb # [ 0.038543] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r7i2qny6zxqjgksiw8bnyjrpmwmq2adf-nixos-system-lb-test/init regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration console=ttyS0,115200n8 console=tty0548worker2 # [ 0.044143] mem auto-init: stack:all(zero), heap alloc:on, heap free:off549client # [ 0.059999] Rude variant of Tasks RCU enabled.550worker1 # [ 0.060115] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.551client # [ 0.059999] Tracing variant of Tasks RCU enabled.552worker2 # [ 0.047172] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1553lb # [ 0.038637] Unknown kernel command line parameters "regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration", will be passed to user space.554worker2 # [ 0.050403] allocated 3145728 bytes of page_ext555worker1 # [ 0.060116] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.556client # [ 0.060000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.557lb # [ 0.038649] random: crng init done558worker2 # [ 0.060154] ftrace: allocating 48731 entries in 192 pages559worker1 # [ 0.064594] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16560client # [ 0.060000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1561lb # [ 0.038650] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes562worker2 # [ 0.060157] ftrace: allocated 192 pages with 2 groups563worker2 # [ 0.060980] Dynamic Preempt: lazy564worker1 # [ 0.064877] rcu: srcu_init: Setting srcu_struct sizes based on contention.565client # [ 0.060031] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.566lb # [ 0.040706] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)567worker2 # [ 0.061142] rcu: Preemptible hierarchical RCU implementation.568worker1 # [ 0.064883] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns569worker2 # [ 0.061143] rcu: RCU event tracing is enabled.570lb # [ 0.041743] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)571client # [ 0.060033] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.572lb # [ 0.041775] Fallback order for Node 0: 0573worker2 # [ 0.061143] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.574worker1 # [ 0.064983] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)575client # [ 0.060034] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.576worker2 # [ 0.061145] Trampoline variant of Tasks RCU enabled.577lb # [ 0.041778] Built 1 zonelists, mobility grouping on. Total pages: 393078578worker1 # [ 0.068531] Console: colour VGA+ 80x25579lb # [ 0.041779] Policy zone: DMA32580client # [ 0.064328] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16581worker2 # [ 0.061145] Rude variant of Tasks RCU enabled.582worker1 # [ 0.068534] printk: legacy console [tty0] enabled583worker2 # [ 0.061146] Tracing variant of Tasks RCU enabled.584worker1 # [ 0.098248] printk: legacy console [ttyS0] enabled585lb # [ 0.044356] mem auto-init: stack:all(zero), heap alloc:on, heap free:off586client # [ 0.064583] rcu: srcu_init: Setting srcu_struct sizes based on contention.587worker2 # [ 0.061146] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.588lb # [ 0.047347] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1589worker1 # [ 0.195050] ACPI: Core revision 20250807590client # [ 0.064588] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns591lb # [ 0.050553] allocated 3145728 bytes of page_ext592worker2 # [ 0.061147] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1593lb # [ 0.060186] ftrace: allocating 48731 entries in 192 pages594client # [ 0.064707] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)595worker1 # [ 0.195931] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns596lb # [ 0.060188] ftrace: allocated 192 pages with 2 groups597worker2 # [ 0.061169] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.598client # [ 0.068229] Console: colour VGA+ 80x25599lb # [ 0.061025] Dynamic Preempt: lazy600worker1 # [ 0.197451] APIC: Switch to symmetric I/O mode setup601client # [ 0.068231] printk: legacy console [tty0] enabled602lb # [ 0.061189] rcu: Preemptible hierarchical RCU implementation.603worker2 # [ 0.061171] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.604client # [ 0.097635] printk: legacy console [ttyS0] enabled605worker1 # [ 0.198404] x2apic enabled606lb # [ 0.061190] rcu: RCU event tracing is enabled.607worker2 # [ 0.061172] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.608client # [ 0.192682] ACPI: Core revision 20250807609lb # [ 0.061191] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.610worker1 # [ 0.199190] APIC: Switched APIC routing to: physical x2apic611worker2 # [ 0.065479] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16612lb # [ 0.061192] Trampoline variant of Tasks RCU enabled.613lb # [ 0.061193] Rude variant of Tasks RCU enabled.614worker2 # [ 0.065753] rcu: srcu_init: Setting srcu_struct sizes based on contention.615lb # [ 0.061193] Tracing variant of Tasks RCU enabled.616worker1 # [ 0.201017] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1617worker2 # [ 0.065760] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns618lb # [ 0.061194] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.619client # [ 0.193609] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns620lb # [ 0.061194] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1621worker2 # [ 0.065860] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)622client # [ 0.196427] APIC: Switch to symmetric I/O mode setup623worker1 # [ 0.202003] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns624worker2 # [ 0.069402] Console: colour VGA+ 80x25625lb # [ 0.061215] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.626client # [ 0.197571] x2apic enabled627worker2 # [ 0.069405] printk: legacy console [tty0] enabled628worker1 # [ 0.203687] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)629worker2 # [ 0.098926] printk: legacy console [ttyS0] enabled630lb # [ 0.061217] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.631client # [ 0.198405] APIC: Switched APIC routing to: physical x2apic632worker2 # [ 0.195093] ACPI: Core revision 20250807633worker1 # [ 0.204770] x86/cpu: User Mode Instruction Prevention (UMIP) activated634lb # [ 0.061218] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.635lb # [ 0.065546] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16636worker1 # [ 0.205824] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127637worker2 # [ 0.195998] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns638client # [ 0.200454] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1639lb # [ 0.065821] rcu: srcu_init: Setting srcu_struct sizes based on contention.640worker1 # [ 0.207380] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0641worker2 # [ 0.197540] APIC: Switch to symmetric I/O mode setup642worker2 # [ 0.198515] x2apic enabled643lb # [ 0.065828] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns644client # [ 0.201549] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns645worker1 # [ 0.208471] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto646worker2 # [ 0.199287] APIC: Switched APIC routing to: physical x2apic647lb # [ 0.065935] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)648lb # [ 0.069467] Console: colour VGA+ 80x25649client # [ 0.203319] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)650lb # [ 0.069470] printk: legacy console [tty0] enabled651worker1 # [ 0.209684] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl652lb # [ 0.098935] printk: legacy console [ttyS0] enabled653worker2 # [ 0.201103] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1654client # [ 0.205413] x86/cpu: User Mode Instruction Prevention (UMIP) activated655worker1 # [ 0.210685] Transient Scheduler Attacks: Vulnerable: No microcode656lb # [ 0.194623] ACPI: Core revision 20250807657worker1 # [ 0.212412] Spectre V2 : Mitigation: Enhanced / Automatic IBRS658client # [ 0.206480] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127659worker2 # [ 0.202078] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns660lb # [ 0.195570] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns661worker1 # [ 0.213401] Speculative Return Stack Overflow: Mitigation: Safe RET662client # [ 0.208036] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0663worker2 # [ 0.203767] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)664lb # [ 0.197085] APIC: Switch to symmetric I/O mode setup665lb # [ 0.198072] x2apic enabled666worker2 # [ 0.204850] x86/cpu: User Mode Instruction Prevention (UMIP) activated667client # [ 0.209121] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto668lb # [ 0.198824] APIC: Switched APIC routing to: physical x2apic669worker2 # [ 0.205908] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127670client # [ 0.210315] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl671worker1 # [ 0.214434] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization672worker2 # [ 0.207451] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0673client # [ 0.212314] Transient Scheduler Attacks: Vulnerable: No microcode674lb # [ 0.200623] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1675worker1 # [ 0.215692] Spectre V2 : Enabling IBPB for BPF676client # [ 0.213314] Spectre V2 : Mitigation: Enhanced / Automatic IBRS677worker2 # [ 0.207768] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto678worker1 # [ 0.217228] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier679lb # [ 0.201613] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns680client # [ 0.214313] Speculative Return Stack Overflow: Mitigation: Safe RET681worker1 # [ 0.218684] active return thunk: srso_alias_return_thunk682worker2 # [ 0.209764] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl683lb # [ 0.203271] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)684worker2 # [ 0.210765] Transient Scheduler Attacks: Vulnerable: No microcode685worker1 # [ 0.219574] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'686lb # [ 0.204357] x86/cpu: User Mode Instruction Prevention (UMIP) activated687worker2 # [ 0.212494] Spectre V2 : Mitigation: Enhanced / Automatic IBRS688worker1 # [ 0.220684] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'689lb # [ 0.205414] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127690worker2 # [ 0.213465] Speculative Return Stack Overflow: Mitigation: Safe RET691worker1 # [ 0.221684] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'692client # [ 0.215315] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization693lb # [ 0.206946] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0694worker1 # [ 0.222684] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'695client # [ 0.217322] Spectre V2 : Enabling IBPB for BPF696worker1 # [ 0.223684] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'697lb # [ 0.208048] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto698client # [ 0.218315] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier699worker1 # [ 0.224684] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'700client # [ 0.219315] active return thunk: srso_alias_return_thunk701worker2 # [ 0.214505] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization702lb # [ 0.209269] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl703worker2 # [ 0.215772] Spectre V2 : Enabling IBPB for BPF704worker1 # [ 0.225684] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'705lb # [ 0.210269] Transient Scheduler Attacks: Vulnerable: No microcode706client # [ 0.220337] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'707worker1 # [ 0.226685] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256708lb # [ 0.211960] Spectre V2 : Mitigation: Enhanced / Automatic IBRS709worker2 # [ 0.216764] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier710client # [ 0.221313] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'711worker1 # [ 0.227684] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64712lb # [ 0.212924] Speculative Return Stack Overflow: Mitigation: Safe RET713worker2 # [ 0.217765] active return thunk: srso_alias_return_thunk714client # [ 0.223038] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'715worker1 # [ 0.228684] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512716client # [ 0.223313] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'717worker2 # [ 0.218793] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'718worker1 # [ 0.229684] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024719client # [ 0.224313] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'720worker2 # [ 0.220674] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'721worker1 # [ 0.230684] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8722worker2 # [ 0.221509] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'723client # [ 0.225314] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'724lb # [ 0.213269] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization725worker2 # [ 0.222506] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'726worker1 # [ 0.231684] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.727client # [ 0.227130] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'728lb # [ 0.215277] Spectre V2 : Enabling IBPB for BPF729worker2 # [ 0.223506] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'730client # [ 0.228314] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256731lb # [ 0.216270] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier732worker2 # [ 0.224519] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'733client # [ 0.229313] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64734lb # [ 0.217269] active return thunk: srso_alias_return_thunk735client # [ 0.230313] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512736worker2 # [ 0.225557] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'737lb # [ 0.218290] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'738client # [ 0.231313] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024739worker2 # [ 0.226764] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256740lb # [ 0.220162] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'741client # [ 0.232313] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8742worker2 # [ 0.227764] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64743lb # [ 0.221014] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'744worker2 # [ 0.228764] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512745client # [ 0.233266] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.746lb # [ 0.222014] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'747worker2 # [ 0.229764] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024748lb # [ 0.223028] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'749worker2 # [ 0.230703] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8750lb # [ 0.224064] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'751worker2 # [ 0.231763] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.752lb # [ 0.225098] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753lb # [ 0.226269] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754lb # [ 0.227269] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755lb # [ 0.228269] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756lb # [ 0.229268] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757lb # [ 0.230269] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758lb # [ 0.231269] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker1 # [ 0.266931] Freeing SMP alternatives memory: 44K760worker1 # [ 0.267687] pid_max: default: 32768 minimum: 301761worker1 # [ 0.268542] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker1 # [ 0.269772] landlock: Up and running.763worker1 # [ 0.270409] Yama: becoming mindful.764worker1 # [ 0.270903] LSM support for eBPF active765worker1 # [ 0.271814] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker1 # [ 0.272721] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker2 # [ 0.266257] Freeing SMP alternatives memory: 44K768worker1 # [ 0.275761] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)769worker2 # [ 0.266767] pid_max: default: 32768 minimum: 301770worker1 # [ 0.277255] Performance Events: Fam17h+ core perfctr, AMD PMU driver.771worker2 # [ 0.267858] LSM: initializing lsm=capability,landlock,yama,bpf,ima772worker1 # [ 0.277694] ... version: 2773worker2 # [ 0.268868] landlock: Up and running.774worker2 # [ 0.269488] Yama: becoming mindful.775worker1 # [ 0.278438] ... bit width: 48776client # [ 0.274315] Freeing SMP alternatives memory: 44K777worker1 # [ 0.278687] ... generic counters: 6778worker2 # [ 0.270392] LSM support for eBPF active779client # [ 0.275107] pid_max: default: 32768 minimum: 301780worker1 # [ 0.279413] ... generic bitmap: 000000000000003f781client # [ 0.275427] LSM: initializing lsm=capability,landlock,yama,bpf,ima782worker2 # [ 0.270891] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)783worker1 # [ 0.279687] ... fixed-purpose counters: 0784client # [ 0.276439] landlock: Up and running.785worker1 # [ 0.280424] ... fixed-purpose bitmap: 0000000000000000786client # [ 0.277313] Yama: becoming mindful.787worker2 # [ 0.271803] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)788lb # [ 0.265168] Freeing SMP alternatives memory: 44K789worker1 # [ 0.280686] ... value mask: 0000ffffffffffff790client # [ 0.278547] LSM support for eBPF active791lb # [ 0.265815] pid_max: default: 32768 minimum: 301792worker1 # [ 0.281550] ... max period: 00007fffffffffff793client # [ 0.279303] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)794lb # [ 0.266375] LSM: initializing lsm=capability,landlock,yama,bpf,ima795worker1 # [ 0.282359] ... global_ctrl mask: 000000000000003f796worker2 # [ 0.274898] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)797lb # [ 0.267370] landlock: Up and running.798worker1 # [ 0.282785] signal: max sigframe size: 3376799client # [ 0.280350] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)800lb # [ 0.268270] Yama: becoming mindful.801worker1 # [ 0.283598] rcu: Hierarchical SRCU implementation.802worker2 # [ 0.276329] Performance Events: Fam17h+ core perfctr, AMD PMU driver.803lb # [ 0.269395] LSM support for eBPF active804worker2 # [ 0.276773] ... version: 2805worker1 # [ 0.284301] rcu: Max phase no-delay instances is 400.806worker2 # [ 0.277461] ... bit width: 48807lb # [ 0.270177] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)808client # [ 0.282909] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)809worker2 # [ 0.277766] ... generic counters: 6810worker2 # [ 0.278498] ... generic bitmap: 000000000000003f811lb # [ 0.271305] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)812client # [ 0.283831] Performance Events: Fam17h+ core perfctr, AMD PMU driver.813worker2 # [ 0.278767] ... fixed-purpose counters: 0814client # [ 0.284318] ... version: 2815worker2 # [ 0.279453] ... fixed-purpose bitmap: 0000000000000000816client # [ 0.285047] ... bit width: 48817worker2 # [ 0.279766] ... value mask: 0000ffffffffffff818client # [ 0.285315] ... generic counters: 6819lb # [ 0.273855] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820worker1 # [ 0.289153] smp: Bringing up secondary CPUs ...821worker2 # [ 0.280645] ... max period: 00007fffffffffff822client # [ 0.286005] ... generic bitmap: 000000000000003f823worker1 # [ 0.289702] smp: Brought up 1 node, 1 CPU824worker2 # [ 0.281442] ... global_ctrl mask: 000000000000003f825lb # [ 0.274793] Performance Events: Fam17h+ core perfctr, AMD PMU driver.826client # [ 0.286390] ... fixed-purpose counters: 0827worker1 # [ 0.290406] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)828lb # [ 0.275273] ... version: 2829worker2 # [ 0.281869] signal: max sigframe size: 3376830client # [ 0.287102] ... fixed-purpose bitmap: 0000000000000000831lb # [ 0.275986] ... bit width: 48832worker2 # [ 0.282637] rcu: Hierarchical SRCU implementation.833client # [ 0.287982] ... value mask: 0000ffffffffffff834lb # [ 0.276271] ... generic counters: 6835worker2 # [ 0.283352] rcu: Max phase no-delay instances is 400.836client # [ 0.288315] ... max period: 00007fffffffffff837worker1 # [ 0.290890] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)838lb # [ 0.276947] ... generic bitmap: 000000000000003f839client # [ 0.289216] ... global_ctrl mask: 000000000000003f840lb # [ 0.277282] ... fixed-purpose counters: 0841worker1 # [ 0.291925] devtmpfs: initialized842client # [ 0.290172] signal: max sigframe size: 3376843lb # [ 0.277995] ... fixed-purpose bitmap: 0000000000000000844worker1 # [ 0.292746] x86/mm: Memory block size: 128MB845client # [ 0.290902] rcu: Hierarchical SRCU implementation.846lb # [ 0.278271] ... value mask: 0000ffffffffffff847client # [ 0.291318] rcu: Max phase no-delay instances is 400.848lb # [ 0.279178] ... max period: 00007fffffffffff849worker1 # [ 0.294475] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)850lb # [ 0.279952] ... global_ctrl mask: 000000000000003f851worker2 # [ 0.288254] smp: Bringing up secondary CPUs ...852lb # [ 0.280384] signal: max sigframe size: 3376853worker2 # [ 0.288782] smp: Brought up 1 node, 1 CPU854worker1 # [ 0.295618] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).855lb # [ 0.281130] rcu: Hierarchical SRCU implementation.856worker1 # [ 0.296767] pinctrl core: initialized pinctrl subsystem857worker2 # [ 0.289434] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)858lb # [ 0.281839] rcu: Max phase no-delay instances is 400.859worker1 # [ 0.297912] PM: RTC time: 09:32:55, date: 2026-09-20860worker2 # [ 0.289944] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)861client # [ 0.296921] smp: Bringing up secondary CPUs ...862client # [ 0.297333] smp: Brought up 1 node, 1 CPU863worker2 # [ 0.291202] devtmpfs: initialized864client # [ 0.298013] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)865worker2 # [ 0.292019] x86/mm: Memory block size: 128MB866worker1 # [ 0.301310] NET: Registered PF_NETLINK/PF_ROUTE protocol family867lb # [ 0.286875] smp: Bringing up secondary CPUs ...868client # [ 0.298476] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)869worker1 # [ 0.302184] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations870lb # [ 0.287286] smp: Brought up 1 node, 1 CPU871worker2 # [ 0.293823] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)872client # [ 0.299621] devtmpfs: initialized873lb # [ 0.287914] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)874worker1 # [ 0.302726] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations875worker2 # [ 0.294791] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).876client # [ 0.300437] x86/mm: Memory block size: 128MB877worker2 # [ 0.295870] pinctrl core: initialized pinctrl subsystem878worker1 # [ 0.303954] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations879lb # [ 0.288457] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)880worker1 # [ 0.304696] audit: initializing netlink subsys (disabled)881worker2 # [ 0.296977] PM: RTC time: 09:32:55, date: 2026-09-20882client # [ 0.302236] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)883lb # [ 0.289539] devtmpfs: initialized884worker1 # [ 0.305812] thermal_sys: Registered thermal governor 'fair_share'885lb # [ 0.290237] x86/mm: Memory block size: 128MB886client # [ 0.303207] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).887worker1 # [ 0.305814] thermal_sys: Registered thermal governor 'bang_bang'888client # [ 0.304413] pinctrl core: initialized pinctrl subsystem889worker1 # [ 0.306688] thermal_sys: Registered thermal governor 'step_wise'890lb # [ 0.291812] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)891worker2 # [ 0.300362] NET: Registered PF_NETLINK/PF_ROUTE protocol family892client # [ 0.305599] PM: RTC time: 09:32:55, date: 2026-09-20893worker1 # [ 0.307660] thermal_sys: Registered thermal governor 'user_space'894lb # [ 0.292296] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).895worker2 # [ 0.301237] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations896worker1 # [ 0.308438] audit: type=2000 audit(1789896776.414:1): state=initialized audit_enabled=0 res=1897lb # [ 0.293365] pinctrl core: initialized pinctrl subsystem898worker2 # [ 0.301788] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations899worker1 # [ 0.309690] thermal_sys: Registered thermal governor 'power_allocator'900lb # [ 0.294501] PM: RTC time: 09:32:55, date: 2026-09-20901worker1 # [ 0.309720] cpuidle: using governor menu902client # [ 0.309084] NET: Registered PF_NETLINK/PF_ROUTE protocol family903worker2 # [ 0.303031] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations904worker2 # [ 0.303775] audit: initializing netlink subsys (disabled)905worker1 # [ 0.311930] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5906client # [ 0.310539] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations907worker2 # [ 0.304871] thermal_sys: Registered thermal governor 'fair_share'908worker1 # [ 0.312921] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]909client # [ 0.311340] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations910lb # [ 0.297947] NET: Registered PF_NETLINK/PF_ROUTE protocol family911worker2 # [ 0.304874] thermal_sys: Registered thermal governor 'bang_bang'912worker1 # [ 0.313691] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry913worker2 # [ 0.305767] thermal_sys: Registered thermal governor 'step_wise'914client # [ 0.312595] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations915lb # [ 0.298771] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations916worker1 # [ 0.314700] PCI: Using configuration type 1 for base access917worker2 # [ 0.306661] thermal_sys: Registered thermal governor 'user_space'918client # [ 0.313328] audit: initializing netlink subsys (disabled)919lb # [ 0.299292] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations920worker1 # [ 0.315782] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.921worker2 # [ 0.307463] audit: type=2000 audit(1789896776.424:1): state=initialized audit_enabled=0 res=1922client # [ 0.314318] audit: type=2000 audit(1789896776.430:1): state=initialized audit_enabled=0 res=1923worker2 # [ 0.308771] thermal_sys: Registered thermal governor 'power_allocator'924lb # [ 0.300542] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations925client # [ 0.315485] thermal_sys: Registered thermal governor 'fair_share'926worker2 # [ 0.308786] cpuidle: using governor menu927lb # [ 0.301282] audit: initializing netlink subsys (disabled)928client # [ 0.315488] thermal_sys: Registered thermal governor 'bang_bang'929lb # [ 0.302442] thermal_sys: Registered thermal governor 'fair_share'930worker2 # [ 0.311001] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5931client # [ 0.316316] thermal_sys: Registered thermal governor 'step_wise'932lb # [ 0.302444] thermal_sys: Registered thermal governor 'bang_bang'933client # [ 0.317266] thermal_sys: Registered thermal governor 'user_space'934worker2 # [ 0.312004] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]935lb # [ 0.303272] thermal_sys: Registered thermal governor 'step_wise'936worker1 # [ 0.320971] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages937client # [ 0.318025] thermal_sys: Registered thermal governor 'power_allocator'938worker2 # [ 0.312769] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry939client # [ 0.318334] cpuidle: using governor menu940lb # [ 0.304243] thermal_sys: Registered thermal governor 'user_space'941worker1 # [ 0.321688] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page942worker2 # [ 0.313772] PCI: Using configuration type 1 for base access943lb # [ 0.304996] audit: type=2000 audit(1789896776.429:1): state=initialized audit_enabled=0 res=1944client # [ 0.320525] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5945worker2 # [ 0.314888] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.946lb # [ 0.306274] thermal_sys: Registered thermal governor 'power_allocator'947client # [ 0.321563] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]948lb # [ 0.306287] cpuidle: using governor menu949client # [ 0.322318] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry950lb # [ 0.308508] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5951client # [ 0.323331] PCI: Using configuration type 1 for base access952worker1 # [ 0.326688] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages953lb # [ 0.309514] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]954worker1 # [ 0.327686] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page955client # [ 0.324452] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.956lb # [ 0.310274] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry957worker2 # [ 0.320011] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages958lb # [ 0.311283] PCI: Using configuration type 1 for base access959worker2 # [ 0.320768] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page960lb # [ 0.312411] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.961worker2 # [ 0.325767] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages962client # [ 0.331343] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages963worker2 # [ 0.326766] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page964client # [ 0.332316] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page965lb # [ 0.317546] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages966lb # [ 0.318272] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page967worker1 # [ 0.338036] ACPI: Added _OSI(Module Device)968worker1 # [ 0.338689] ACPI: Added _OSI(Processor Device)969worker1 # [ 0.339437] ACPI: Added _OSI(Processor Aggregator Device)970client # [ 0.337317] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages971client # [ 0.338316] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page972lb # [ 0.323272] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages973lb # [ 0.324271] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page974worker2 # [ 0.337172] ACPI: Added _OSI(Module Device)975worker2 # [ 0.337768] ACPI: Added _OSI(Processor Device)976worker2 # [ 0.338527] ACPI: Added _OSI(Processor Aggregator Device)977worker1 # [ 0.348241] ACPI: 1 ACPI AML tables successfully acquired and loaded978worker1 # [ 0.352110] ACPI: Interpreter enabled979client # [ 0.349617] ACPI: Added _OSI(Module Device)980worker1 # [ 0.352702] ACPI: PM: (supports S0 S3 S4 S5)981client # [ 0.350317] ACPI: Added _OSI(Processor Device)982lb # [ 0.334672] ACPI: Added _OSI(Module Device)983worker1 # [ 0.353387] ACPI: Using IOAPIC for interrupt routing984lb # [ 0.335273] ACPI: Added _OSI(Processor Device)985client # [ 0.351031] ACPI: Added _OSI(Processor Aggregator Device)986lb # [ 0.336000] ACPI: Added _OSI(Processor Aggregator Device)987worker1 # [ 0.353733] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug988worker1 # [ 0.354688] PCI: Using E820 reservations for host bridge windows989worker2 # [ 0.347317] ACPI: 1 ACPI AML tables successfully acquired and loaded990client # [ 0.355264] ACPI: 1 ACPI AML tables successfully acquired and loaded991worker1 # [ 0.357806] ACPI: Enabled 2 GPEs in block 00 to 3F992worker2 # [ 0.351151] ACPI: Interpreter enabled993worker2 # [ 0.351783] ACPI: PM: (supports S0 S3 S4 S5)994worker2 # [ 0.352513] ACPI: Using IOAPIC for interrupt routing995lb # [ 0.343891] ACPI: 1 ACPI AML tables successfully acquired and loaded996worker2 # [ 0.352813] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug997worker2 # [ 0.353767] PCI: Using E820 reservations for host bridge windows998client # [ 0.361294] ACPI: Interpreter enabled999client # [ 0.361801] ACPI: PM: (supports S0 S3 S4 S5)1000client # [ 0.362317] ACPI: Using IOAPIC for interrupt routing1001lb # [ 0.347679] ACPI: Interpreter enabled1002worker2 # [ 0.356876] ACPI: Enabled 2 GPEs in block 00 to 3F1003client # [ 0.363173] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1004worker1 # [ 0.366734] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1005lb # [ 0.348287] ACPI: PM: (supports S0 S3 S4 S5)1006lb # [ 0.349005] ACPI: Using IOAPIC for interrupt routing1007worker1 # [ 0.368476] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1008worker1 # [ 0.369777] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1009lb # [ 0.351362] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1010client # [ 0.368316] PCI: Using E820 reservations for host bridge windows1011worker1 # [ 0.370813] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1012lb # [ 0.352272] PCI: Using E820 reservations for host bridge windows1013client # [ 0.369416] ACPI: Enabled 2 GPEs in block 00 to 3F1014worker1 # [ 0.372054] PCI host bridge to bus 0000:001015worker1 # [ 0.372692] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1016worker1 # [ 0.373687] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1017worker2 # [ 0.365536] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1018lb # [ 0.355389] ACPI: Enabled 2 GPEs in block 00 to 3F1019worker1 # [ 0.374687] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1020worker2 # [ 0.366528] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1021worker1 # [ 0.375687] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1022worker2 # [ 0.368367] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1023worker1 # [ 0.376687] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1024worker2 # [ 0.368893] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1025worker1 # [ 0.377688] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1026client # [ 0.377174] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1027worker2 # [ 0.370171] PCI host bridge to bus 0000:001028worker1 # [ 0.378688] pci_bus 0000:00: root bus resource [bus 00-ff]1029worker2 # [ 0.370771] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1030client # [ 0.378104] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1031worker1 # [ 0.379741] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1032worker2 # [ 0.371767] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1033client # [ 0.379396] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1034lb # [ 0.363297] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1035worker2 # [ 0.372767] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1036worker1 # [ 0.381153] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1037client # [ 0.380440] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1038lb # [ 0.364278] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1039worker2 # [ 0.373766] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1040client # [ 0.381685] PCI host bridge to bus 0000:001041lb # [ 0.365349] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1042worker2 # [ 0.374766] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1043client # [ 0.382320] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1044lb # [ 0.366395] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1045client # [ 0.383316] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1046worker2 # [ 0.375766] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1047lb # [ 0.367632] PCI host bridge to bus 0000:001048worker2 # [ 0.376767] pci_bus 0000:00: root bus resource [bus 00-ff]1049client # [ 0.384316] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1050lb # [ 0.368276] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1051client # [ 0.385316] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1052worker2 # [ 0.377828] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1053lb # [ 0.369272] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1054client # [ 0.386316] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1055worker1 # [ 0.383869] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1056lb # [ 0.370272] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1057worker2 # [ 0.379213] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1058worker1 # [ 0.384743] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1059client # [ 0.387316] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1060lb # [ 0.371271] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1061client # [ 0.388316] pci_bus 0000:00: root bus resource [bus 00-ff]1062worker1 # [ 0.385698] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1063lb # [ 0.372272] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1064client # [ 0.389358] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1065lb # [ 0.373272] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1066worker1 # [ 0.387017] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1067lb # [ 0.374272] pci_bus 0000:00: root bus resource [bus 00-ff]1068client # [ 0.390798] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1069worker1 # [ 0.388395] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1070lb # [ 0.375274] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1071worker2 # [ 0.381846] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1072lb # [ 0.376719] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1073worker2 # [ 0.382780] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1074worker1 # [ 0.391706] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1075worker2 # [ 0.383910] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1076worker1 # [ 0.392534] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1077worker1 # [ 0.393431] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1078worker2 # [ 0.385140] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1079worker1 # [ 0.393693] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1080client # [ 0.393426] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1081worker2 # [ 0.386519] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1082worker1 # [ 0.395225] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1083client # [ 0.394328] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1084lb # [ 0.379335] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1085client # [ 0.395326] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1086lb # [ 0.380284] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1087worker2 # [ 0.388776] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1088worker1 # [ 0.397623] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1089lb # [ 0.381282] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1090client # [ 0.396774] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1091worker2 # [ 0.389657] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1092worker1 # [ 0.398283] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1093worker2 # [ 0.390472] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1094worker1 # [ 0.398708] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1095lb # [ 0.382404] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1096worker2 # [ 0.390775] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1097client # [ 0.398172] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1098worker1 # [ 0.400248] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1099lb # [ 0.384039] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1100worker2 # [ 0.392300] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1101client # [ 0.401346] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1102client # [ 0.402282] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1103lb # [ 0.386302] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1104worker1 # [ 0.402696] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1105worker2 # [ 0.393774] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1106lb # [ 0.387124] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1107worker2 # [ 0.394589] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1108client # [ 0.403082] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1109worker2 # [ 0.395462] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1110worker1 # [ 0.403497] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1111lb # [ 0.388000] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1112client # [ 0.404206] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1113lb # [ 0.388277] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1114worker1 # [ 0.404391] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1115worker2 # [ 0.396322] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1116client # [ 0.405945] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1117worker1 # [ 0.405240] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118lb # [ 0.389967] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1119worker2 # [ 0.398776] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1120client # [ 0.408208] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1121worker1 # [ 0.407495] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1122lb # [ 0.392303] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1123client # [ 0.409004] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1124worker1 # [ 0.408301] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1125lb # [ 0.393150] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1126worker2 # [ 0.399687] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1127client # [ 0.409339] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1128worker1 # [ 0.408708] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1129lb # [ 0.394010] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1130worker2 # [ 0.400477] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1131client # [ 0.410875] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1132lb # [ 0.394828] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1133worker2 # [ 0.401344] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1134worker1 # [ 0.410266] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1135worker2 # [ 0.402774] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1136lb # [ 0.397278] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1137client # [ 0.413322] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1138worker2 # [ 0.403664] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1139worker1 # [ 0.412445] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1140client # [ 0.414143] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1141worker2 # [ 0.404475] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1142worker1 # [ 0.412694] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1143lb # [ 0.398081] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144client # [ 0.415023] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1145worker1 # [ 0.413622] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1146lb # [ 0.399315] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1147worker2 # [ 0.405330] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1148client # [ 0.415891] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1149worker1 # [ 0.415081] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1150lb # [ 0.400831] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1151worker2 # [ 0.407714] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1152client # [ 0.417345] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1153lb # [ 0.402304] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1154worker1 # [ 0.417694] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1155worker2 # [ 0.408460] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1156client # [ 0.418169] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1157lb # [ 0.403150] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158worker2 # [ 0.408788] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1159lb # [ 0.403997] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1160worker1 # [ 0.418508] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1161client # [ 0.418992] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1162worker2 # [ 0.410380] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1163worker1 # [ 0.419709] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1164lb # [ 0.404821] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1165client # [ 0.419880] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1166worker1 # [ 0.421286] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1167worker2 # [ 0.411778] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1168worker2 # [ 0.412622] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1169client # [ 0.421346] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1170lb # [ 0.407148] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1171worker2 # [ 0.413497] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1172client # [ 0.422152] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1173lb # [ 0.407939] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1174worker1 # [ 0.423426] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1175client # [ 0.423009] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1176lb # [ 0.408293] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1177worker1 # [ 0.423694] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1178worker1 # [ 0.424665] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1179worker2 # [ 0.414935] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1180client # [ 0.423913] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1181lb # [ 0.409876] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1182worker1 # [ 0.425519] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1183worker2 # [ 0.416774] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1184worker2 # [ 0.417684] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1185lb # [ 0.411305] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1186worker1 # [ 0.426976] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1187worker2 # [ 0.418516] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1188lb # [ 0.412148] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1189client # [ 0.426565] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1190worker2 # [ 0.418772] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1191lb # [ 0.413003] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1192client # [ 0.427322] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1193worker1 # [ 0.429701] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1194client # [ 0.428297] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1195worker2 # [ 0.420281] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1196worker1 # [ 0.430666] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1197lb # [ 0.414389] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1198client # [ 0.429765] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1199worker1 # [ 0.432044] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1200worker2 # [ 0.421780] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1201lb # [ 0.416301] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1202worker2 # [ 0.422791] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1203lb # [ 0.417226] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1204client # [ 0.431350] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1205worker1 # [ 0.434466] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1206lb # [ 0.418045] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1207client # [ 0.432202] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1208worker1 # [ 0.434694] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1209worker2 # [ 0.424314] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1210lb # [ 0.419090] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1211client # [ 0.433035] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1212worker2 # [ 0.425775] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1213worker1 # [ 0.435589] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1214client # [ 0.433322] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1215lb # [ 0.420547] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1216worker2 # [ 0.426624] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1217worker1 # [ 0.437246] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1218client # [ 0.434880] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1219worker2 # [ 0.427559] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1220lb # [ 0.422308] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1221lb # [ 0.423293] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1222worker2 # [ 0.429181] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1223client # [ 0.436327] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1224worker1 # [ 0.439411] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1225client # [ 0.437321] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1226worker1 # [ 0.439694] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1227lb # [ 0.424819] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1228worker1 # [ 0.440614] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1229worker2 # [ 0.431277] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1230client # [ 0.438885] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1231worker2 # [ 0.431774] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1232lb # [ 0.426320] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1233worker1 # [ 0.442035] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1234lb # [ 0.427177] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1235worker2 # [ 0.432723] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1236client # [ 0.440353] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1237lb # [ 0.428037] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1238worker2 # [ 0.434175] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1239worker1 # [ 0.443625] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1240client # [ 0.441175] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1241lb # [ 0.428823] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1242client # [ 0.442338] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1243worker2 # [ 0.435341] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1244worker1 # [ 0.444903] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1245worker2 # [ 0.436015] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1246client # [ 0.443863] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1247worker1 # [ 0.446479] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1248lb # [ 0.430786] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1249lb # [ 0.431278] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1250worker2 # [ 0.437441] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1251client # [ 0.445323] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1252lb # [ 0.432252] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1253client # [ 0.446160] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1254client # [ 0.447045] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1255lb # [ 0.433707] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1256lb # [ 0.434836] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1257client # [ 0.447894] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1258lb # [ 0.435473] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1259client # [ 0.449327] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1260lb # [ 0.436818] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1261client # [ 0.450418] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1262client # [ 0.452049] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1263worker1 # [ 0.447481] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1264worker1 # [ 0.449498] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1265worker2 # [ 0.438334] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1266worker1 # [ 0.450568] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1267worker2 # [ 0.441336] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1268worker1 # [ 0.452420] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1269worker2 # [ 0.442032] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1270worker1 # [ 0.453679] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1271worker2 # [ 0.443718] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1272worker1 # [ 0.454971] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1273lb # [ 0.437977] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1274worker2 # [ 0.444807] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1275worker1 # [ 0.455936] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1276client # [ 0.452775] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1277worker2 # [ 0.446050] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1278worker1 # [ 0.457714] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1279lb # [ 0.440844] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1280worker1 # [ 0.458566] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1281worker2 # [ 0.447066] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1282client # [ 0.455113] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1283lb # [ 0.441558] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1284client # [ 0.456215] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1285worker1 # [ 0.459723] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1286worker2 # [ 0.449334] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1287lb # [ 0.443166] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1288worker2 # [ 0.449773] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1289client # [ 0.458015] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1290worker1 # [ 0.461723] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1291lb # [ 0.444255] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1292client # [ 0.458618] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1293worker2 # [ 0.451081] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1294lb # [ 0.445583] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1295client # [ 0.459618] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1296worker2 # [ 0.452504] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1297lb # [ 0.446518] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1298client # [ 0.460592] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1299worker1 # [ 0.466087] ACPI: PCI: Interrupt link LNKA configured for IRQ 101300worker1 # [ 0.466794] ACPI: PCI: Interrupt link LNKB configured for IRQ 101301client # [ 0.462343] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1302lb # [ 0.449133] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1303worker1 # [ 0.467788] ACPI: PCI: Interrupt link LNKC configured for IRQ 111304worker2 # [ 0.455788] ACPI: PCI: Interrupt link LNKA configured for IRQ 101305client # [ 0.463236] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1306lb # [ 0.450011] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1307worker1 # [ 0.468784] ACPI: PCI: Interrupt link LNKD configured for IRQ 111308worker2 # [ 0.456877] ACPI: PCI: Interrupt link LNKB configured for IRQ 101309worker1 # [ 0.469929] ACPI: PCI: Interrupt link LNKE configured for IRQ 101310worker2 # [ 0.457877] ACPI: PCI: Interrupt link LNKC configured for IRQ 111311lb # [ 0.451454] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1312client # [ 0.464380] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1313worker1 # [ 0.470784] ACPI: PCI: Interrupt link LNKF configured for IRQ 101314worker2 # [ 0.458872] ACPI: PCI: Interrupt link LNKD configured for IRQ 111315client # [ 0.466054] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1316lb # [ 0.453065] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1317worker1 # [ 0.471797] ACPI: PCI: Interrupt link LNKG configured for IRQ 111318worker2 # [ 0.459866] ACPI: PCI: Interrupt link LNKE configured for IRQ 101319worker1 # [ 0.472794] ACPI: PCI: Interrupt link LNKH configured for IRQ 111320worker2 # [ 0.460861] ACPI: PCI: Interrupt link LNKF configured for IRQ 101321worker1 # [ 0.473731] ACPI: PCI: Interrupt link GSIA configured for IRQ 161322worker2 # [ 0.461864] ACPI: PCI: Interrupt link LNKG configured for IRQ 111323worker1 # [ 0.474692] ACPI: PCI: Interrupt link GSIB configured for IRQ 171324worker2 # [ 0.462861] ACPI: PCI: Interrupt link LNKH configured for IRQ 111325client # [ 0.469383] ACPI: PCI: Interrupt link LNKA configured for IRQ 101326worker1 # [ 0.475634] ACPI: PCI: Interrupt link GSIC configured for IRQ 181327worker2 # [ 0.463833] ACPI: PCI: Interrupt link GSIA configured for IRQ 161328lb # [ 0.457012] ACPI: PCI: Interrupt link LNKA configured for IRQ 101329worker1 # [ 0.476409] ACPI: PCI: Interrupt link GSID configured for IRQ 191330client # [ 0.470425] ACPI: PCI: Interrupt link LNKB configured for IRQ 101331worker2 # [ 0.464750] ACPI: PCI: Interrupt link GSIB configured for IRQ 171332lb # [ 0.457402] ACPI: PCI: Interrupt link LNKB configured for IRQ 101333worker1 # [ 0.476699] ACPI: PCI: Interrupt link GSIE configured for IRQ 201334client # [ 0.471418] ACPI: PCI: Interrupt link LNKC configured for IRQ 111335worker2 # [ 0.465523] ACPI: PCI: Interrupt link GSIC configured for IRQ 181336worker1 # [ 0.477625] ACPI: PCI: Interrupt link GSIF configured for IRQ 211337lb # [ 0.458401] ACPI: PCI: Interrupt link LNKC configured for IRQ 111338client # [ 0.472425] ACPI: PCI: Interrupt link LNKD configured for IRQ 111339worker2 # [ 0.466489] ACPI: PCI: Interrupt link GSID configured for IRQ 191340worker1 # [ 0.478414] ACPI: PCI: Interrupt link GSIG configured for IRQ 221341lb # [ 0.459397] ACPI: PCI: Interrupt link LNKD configured for IRQ 111342worker2 # [ 0.467524] ACPI: PCI: Interrupt link GSIE configured for IRQ 201343client # [ 0.473423] ACPI: PCI: Interrupt link LNKE configured for IRQ 101344worker1 # [ 0.478702] ACPI: PCI: Interrupt link GSIH configured for IRQ 231345lb # [ 0.460402] ACPI: PCI: Interrupt link LNKE configured for IRQ 101346worker2 # [ 0.467782] ACPI: PCI: Interrupt link GSIF configured for IRQ 211347client # [ 0.474439] ACPI: PCI: Interrupt link LNKF configured for IRQ 101348worker2 # [ 0.468697] ACPI: PCI: Interrupt link GSIG configured for IRQ 221349lb # [ 0.461399] ACPI: PCI: Interrupt link LNKF configured for IRQ 101350client # [ 0.475430] ACPI: PCI: Interrupt link LNKG configured for IRQ 111351worker1 # [ 0.480719] iommu: Default domain type: Translated1352worker2 # [ 0.469491] ACPI: PCI: Interrupt link GSIH configured for IRQ 231353lb # [ 0.462398] ACPI: PCI: Interrupt link LNKG configured for IRQ 111354worker1 # [ 0.481492] iommu: DMA domain TLB invalidation policy: lazy mode1355client # [ 0.476418] ACPI: PCI: Interrupt link LNKH configured for IRQ 111356worker1 # [ 0.482656] ACPI: bus type USB registered1357client # [ 0.477353] ACPI: PCI: Interrupt link GSIA configured for IRQ 161358lb # [ 0.463397] ACPI: PCI: Interrupt link LNKH configured for IRQ 111359worker2 # [ 0.470808] iommu: Default domain type: Translated1360worker1 # [ 0.483230] usbcore: registered new interface driver usbfs1361client # [ 0.478305] ACPI: PCI: Interrupt link GSIB configured for IRQ 171362worker2 # [ 0.471557] iommu: DMA domain TLB invalidation policy: lazy mode1363lb # [ 0.464320] ACPI: PCI: Interrupt link GSIA configured for IRQ 161364worker1 # [ 0.483699] usbcore: registered new interface driver hub1365client # [ 0.479023] ACPI: PCI: Interrupt link GSIC configured for IRQ 181366worker2 # [ 0.472750] ACPI: bus type USB registered1367worker1 # [ 0.484552] usbcore: registered new device driver usb1368lb # [ 0.465293] ACPI: PCI: Interrupt link GSIB configured for IRQ 171369client # [ 0.479328] ACPI: PCI: Interrupt link GSID configured for IRQ 191370worker2 # [ 0.473293] usbcore: registered new interface driver usbfs1371lb # [ 0.466296] ACPI: PCI: Interrupt link GSIC configured for IRQ 181372client # [ 0.480255] ACPI: PCI: Interrupt link GSIE configured for IRQ 201373worker1 # [ 0.486159] NetLabel: Initializing1374worker2 # [ 0.473780] usbcore: registered new interface driver hub1375client # [ 0.481054] ACPI: PCI: Interrupt link GSIF configured for IRQ 211376lb # [ 0.467293] ACPI: PCI: Interrupt link GSID configured for IRQ 191377worker1 # [ 0.486687] NetLabel: domain hash size = 1281378worker2 # [ 0.474628] usbcore: registered new device driver usb1379client # [ 0.481328] ACPI: PCI: Interrupt link GSIG configured for IRQ 221380worker1 # [ 0.487509] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1381lb # [ 0.468296] ACPI: PCI: Interrupt link GSIE configured for IRQ 201382worker2 # [ 0.476228] NetLabel: Initializing1383client # [ 0.482331] ACPI: PCI: Interrupt link GSIH configured for IRQ 231384worker1 # [ 0.488494] NetLabel: unlabeled traffic allowed by default1385lb # [ 0.469292] ACPI: PCI: Interrupt link GSIF configured for IRQ 211386worker2 # [ 0.476767] NetLabel: domain hash size = 1281387worker1 # [ 0.489421] PCI: Using ACPI for IRQ routing1388worker2 # [ 0.477498] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1389lb # [ 0.470292] ACPI: PCI: Interrupt link GSIG configured for IRQ 221390client # [ 0.484455] iommu: Default domain type: Translated1391worker2 # [ 0.477836] NetLabel: unlabeled traffic allowed by default1392lb # [ 0.471296] ACPI: PCI: Interrupt link GSIH configured for IRQ 231393client # [ 0.485268] iommu: DMA domain TLB invalidation policy: lazy mode1394worker2 # [ 0.478750] PCI: Using ACPI for IRQ routing1395client # [ 0.486354] ACPI: bus type USB registered1396lb # [ 0.473372] iommu: Default domain type: Translated1397client # [ 0.487115] usbcore: registered new interface driver usbfs1398lb # [ 0.474262] iommu: DMA domain TLB invalidation policy: lazy mode1399client # [ 0.488012] usbcore: registered new interface driver hub1400lb # [ 0.475556] ACPI: bus type USB registered1401client # [ 0.488343] usbcore: registered new device driver usb1402lb # [ 0.476342] usbcore: registered new interface driver usbfs1403client # [ 0.490124] NetLabel: Initializing1404lb # [ 0.477290] usbcore: registered new interface driver hub1405client # [ 0.490322] NetLabel: domain hash size = 1281406lb # [ 0.478279] usbcore: registered new device driver usb1407client # [ 0.491161] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1408client # [ 0.492089] NetLabel: unlabeled traffic allowed by default1409lb # [ 0.480116] NetLabel: Initializing1410client # [ 0.493051] PCI: Using ACPI for IRQ routing1411lb # [ 0.480783] NetLabel: domain hash size = 1281412lb # [ 0.481273] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413lb # [ 0.482336] NetLabel: unlabeled traffic allowed by default1414lb # [ 0.483290] PCI: Using ACPI for IRQ routing1415worker1 # [ 0.532507] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker1 # [ 0.532683] pci 0000:00:01.0: vgaarb: bridge control possible1417worker2 # [ 0.523446] pci 0000:00:01.0: vgaarb: setting as boot VGA device1418worker1 # [ 0.532683] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1419worker1 # [ 0.532691] vgaarb: loaded1420worker2 # [ 0.523763] pci 0000:00:01.0: vgaarb: bridge control possible1421worker1 # [ 0.533433] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01422worker2 # [ 0.523763] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1423worker1 # [ 0.533687] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1424worker2 # [ 0.523771] vgaarb: loaded1425worker2 # [ 0.524523] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01426worker2 # [ 0.524768] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1427worker1 # [ 0.537775] clocksource: Switched to clocksource kvm-clock1428worker2 # [ 0.528857] clocksource: Switched to clocksource kvm-clock1429client # [ 0.536914] pci 0000:00:01.0: vgaarb: setting as boot VGA device1430client # [ 0.537312] pci 0000:00:01.0: vgaarb: bridge control possible1431worker1 # [ 0.541255] VFS: Disk quotas dquot_6.6.01432client # [ 0.537312] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1433lb # [ 0.527675] pci 0000:00:01.0: vgaarb: setting as boot VGA device1434client # [ 0.537321] vgaarb: loaded1435worker1 # [ 0.541952] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1436worker2 # [ 0.532373] VFS: Disk quotas dquot_6.6.01437worker1 # [ 0.543287] pnp: PnP ACPI init1438lb # [ 0.528267] pci 0000:00:01.0: vgaarb: bridge control possible1439client # [ 0.538082] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01440worker2 # [ 0.533088] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1441worker1 # [ 0.544039] ACPI: IRQ 4 override to edge(!), high(!)1442client # [ 0.538315] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1443worker2 # [ 0.534417] pnp: PnP ACPI init1444lb # [ 0.528267] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1445lb # [ 0.528279] vgaarb: loaded1446worker1 # [ 0.545025] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1447worker2 # [ 0.535266] ACPI: IRQ 4 override to edge(!), high(!)1448lb # [ 0.529089] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01449worker1 # [ 0.546522] pnp: PnP ACPI: found 6 devices1450worker2 # [ 0.536210] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1451lb # [ 0.529950] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1452worker2 # [ 0.537755] pnp: PnP ACPI: found 6 devices1453client # [ 0.541439] clocksource: Switched to clocksource kvm-clock1454lb # [ 0.533365] clocksource: Switched to clocksource kvm-clock1455client # [ 0.545231] VFS: Disk quotas dquot_6.6.01456client # [ 0.545995] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1457lb # [ 0.535172] VFS: Disk quotas dquot_6.6.01458client # [ 0.547420] pnp: PnP ACPI init1459worker1 # [ 0.554092] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1460client # [ 0.548285] ACPI: IRQ 4 override to edge(!), high(!)1461worker1 # [ 0.555571] clocksource: Switched to clocksource acpi_pm1462lb # [ 0.537765] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1463worker2 # [ 0.545378] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1464client # [ 0.549306] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1465worker1 # [ 0.556512] NET: Registered PF_INET protocol family1466worker2 # [ 0.546784] clocksource: Switched to clocksource acpi_pm1467lb # [ 0.539307] pnp: PnP ACPI init1468client # [ 0.550860] pnp: PnP ACPI: found 6 devices1469worker1 # [ 0.557633] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1470worker2 # [ 0.547774] NET: Registered PF_INET protocol family1471lb # [ 0.540207] ACPI: IRQ 4 override to edge(!), high(!)1472worker2 # [ 0.548995] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1473lb # [ 0.541305] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1474lb # [ 0.542980] pnp: PnP ACPI: found 6 devices1475client # [ 0.558483] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1476client # [ 0.560021] clocksource: Switched to clocksource acpi_pm1477client # [ 0.561034] NET: Registered PF_INET protocol family1478lb # [ 0.550671] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1479client # [ 0.562225] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1480lb # [ 0.552310] clocksource: Switched to clocksource acpi_pm1481lb # [ 0.553404] NET: Registered PF_INET protocol family1482lb # [ 0.554720] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1483worker1 # [ 0.573939] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1484worker1 # [ 0.575381] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1485worker2 # [ 0.565501] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1486worker1 # [ 0.576721] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1487worker2 # [ 0.567146] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1488worker1 # [ 0.578098] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1489worker2 # [ 0.568467] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1490worker1 # [ 0.579362] TCP: Hash tables configured (established 16384 bind 16384)1491worker2 # [ 0.569842] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1492worker1 # [ 0.580472] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1493worker2 # [ 0.571126] TCP: Hash tables configured (established 16384 bind 16384)1494worker1 # [ 0.581731] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1495worker2 # [ 0.572252] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1496worker1 # [ 0.582859] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1497worker2 # [ 0.573535] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1498worker1 # [ 0.584053] NET: Registered PF_UNIX/PF_LOCAL protocol family1499worker1 # [ 0.585031] NET: Registered PF_XDP protocol family1500worker2 # [ 0.574689] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1501worker1 # [ 0.585856] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1502worker2 # [ 0.575951] NET: Registered PF_UNIX/PF_LOCAL protocol family1503client # [ 0.579229] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1504worker1 # [ 0.586878] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1505worker2 # [ 0.576899] NET: Registered PF_XDP protocol family1506worker1 # [ 0.587880] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1507client # [ 0.580816] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1508worker2 # [ 0.577757] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1509worker1 # [ 0.588949] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1510worker2 # [ 0.578746] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1511client # [ 0.582134] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1512lb # [ 0.571183] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1513worker1 # [ 0.590000] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1514worker2 # [ 0.579719] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1515client # [ 0.583442] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1516worker1 # [ 0.591056] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1517worker2 # [ 0.580792] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1518lb # [ 0.572869] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1519client # [ 0.584637] TCP: Hash tables configured (established 16384 bind 16384)1520worker2 # [ 0.581870] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1521worker1 # [ 0.592679] ACPI: \_SB_.GSIA: Enabled at IRQ 161522client # [ 0.585765] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1523lb # [ 0.574314] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1524worker2 # [ 0.582970] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1525client # [ 0.586993] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1526lb # [ 0.575797] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1527worker2 # [ 0.584653] ACPI: \_SB_.GSIA: Enabled at IRQ 161528worker1 # [ 0.595227] ACPI: \_SB_.GSIB: Enabled at IRQ 171529client # [ 0.588077] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1530lb # [ 0.577202] TCP: Hash tables configured (established 16384 bind 16384)1531client # [ 0.589278] NET: Registered PF_UNIX/PF_LOCAL protocol family1532worker1 # [ 0.597066] ACPI: \_SB_.GSIC: Enabled at IRQ 181533lb # [ 0.578509] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1534client # [ 0.590308] NET: Registered PF_XDP protocol family1535worker2 # [ 0.586987] ACPI: \_SB_.GSIB: Enabled at IRQ 171536client # [ 0.591132] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1537lb # [ 0.579882] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1538worker1 # [ 0.598919] ACPI: \_SB_.GSID: Enabled at IRQ 191539client # [ 0.592165] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1540worker2 # [ 0.588901] ACPI: \_SB_.GSIC: Enabled at IRQ 181541lb # [ 0.581104] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1542client # [ 0.593159] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1543worker1 # [ 0.600554] PCI: CLS 0 bytes, default 641544lb # [ 0.582433] NET: Registered PF_UNIX/PF_LOCAL protocol family1545worker2 # [ 0.590908] ACPI: \_SB_.GSID: Enabled at IRQ 191546client # [ 0.594276] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1547lb # [ 0.583543] NET: Registered PF_XDP protocol family1548worker1 # [ 0.601391] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1549client # [ 0.595359] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1550worker2 # [ 0.592612] PCI: CLS 0 bytes, default 641551lb # [ 0.584482] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1552worker1 # [ 0.603060] Trying to unpack rootfs image as initramfs...1553client # [ 0.596463] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1554lb # [ 0.585596] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1555worker2 # [ 0.593503] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1556client # [ 0.598202] ACPI: \_SB_.GSIA: Enabled at IRQ 161557lb # [ 0.586724] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1558worker2 # [ 0.595234] Trying to unpack rootfs image as initramfs...1559lb # [ 0.587959] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1560client # [ 0.600647] ACPI: \_SB_.GSIB: Enabled at IRQ 171561lb # [ 0.589187] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1562lb # [ 0.590419] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1563client # [ 0.603117] ACPI: \_SB_.GSIC: Enabled at IRQ 181564lb # [ 0.592297] ACPI: \_SB_.GSIA: Enabled at IRQ 161565client # [ 0.605047] ACPI: \_SB_.GSID: Enabled at IRQ 191566lb # [ 0.594630] ACPI: \_SB_.GSIB: Enabled at IRQ 171567client # [ 0.606772] PCI: CLS 0 bytes, default 641568lb # [ 0.596719] ACPI: \_SB_.GSIC: Enabled at IRQ 181569client # [ 0.607639] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1570client # [ 0.609481] Trying to unpack rootfs image as initramfs...1571lb # [ 0.598843] ACPI: \_SB_.GSID: Enabled at IRQ 191572lb # [ 0.600755] PCI: CLS 0 bytes, default 641573lb # [ 0.601753] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574lb # [ 0.603664] Trying to unpack rootfs image as initramfs...1575worker1 # [ 0.646913] Initialise system trusted keyrings1576worker1 # [ 0.649866] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.640310] Initialise system trusted keyrings1578worker2 # [ 0.643071] workingset: timestamp_bits=40 max_order=19 bucket_order=01579client # [ 0.657294] Initialise system trusted keyrings1580client # [ 0.658322] workingset: timestamp_bits=40 max_order=19 bucket_order=01581lb # [ 0.649951] Initialise system trusted keyrings1582lb # [ 0.651022] workingset: timestamp_bits=40 max_order=19 bucket_order=01583worker1 # [ 0.672966] Key type asymmetric registered1584worker1 # [ 0.673673] Asymmetric key parser 'x509' registered1585worker1 # [ 0.674524] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1586worker2 # [ 0.665254] Key type asymmetric registered1587worker2 # [ 0.665998] Asymmetric key parser 'x509' registered1588worker2 # [ 0.666863] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1589worker1 # [ 0.678934] io scheduler mq-deadline registered1590worker1 # [ 0.679683] io scheduler kyber registered1591worker2 # [ 0.672194] io scheduler mq-deadline registered1592worker2 # [ 0.672969] io scheduler kyber registered1593worker1 # [ 0.683346] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1594worker1 # [ 0.684587] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1595worker2 # [ 0.676590] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1596worker1 # [ 0.688659] Linux agpgart interface v0.1031597client # [ 0.681872] Key type asymmetric registered1598worker2 # [ 0.677912] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1599worker1 # [ 0.689423] ACPI: bus type drm_connector registered1600client # [ 0.682585] Asymmetric key parser 'x509' registered1601client # [ 0.683447] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1602worker2 # [ 0.681802] Linux agpgart interface v0.1031603worker1 # [ 0.692288] usbcore: registered new interface driver usbserial_generic1604lb # [ 0.674583] Key type asymmetric registered1605worker2 # [ 0.682626] ACPI: bus type drm_connector registered1606worker1 # [ 0.693319] usbserial: USB Serial support registered for generic1607lb # [ 0.675377] Asymmetric key parser 'x509' registered1608client # [ 0.686913] io scheduler mq-deadline registered1609client # [ 0.687675] io scheduler kyber registered1610lb # [ 0.677470] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1611worker1 # [ 0.696836] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1612worker2 # [ 0.687481] usbcore: registered new interface driver usbserial_generic1613worker1 # [ 0.696836] Please enable it if your BIOS has the CPPC option.1614lb # [ 0.680511] io scheduler mq-deadline registered1615client # [ 0.691933] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1616worker2 # [ 0.688580] usbserial: USB Serial support registered for generic1617lb # [ 0.681353] io scheduler kyber registered1618worker1 # [ 0.699049] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1619client # [ 0.693211] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1620worker2 # [ 0.692053] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1621worker1 # [ 0.702946] drop_monitor: Initializing network drop monitor service1622lb # [ 0.685090] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1623worker2 # [ 0.692053] Please enable it if your BIOS has the CPPC option.1624worker1 # [ 0.704063] NET: Registered PF_INET6 protocol family1625worker2 # [ 0.694294] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1626lb # [ 0.686497] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1627client # [ 0.698655] Linux agpgart interface v0.1031628client # [ 0.699454] ACPI: bus type drm_connector registered1629worker1 # [ 0.707287] Segment Routing with IPv61630worker2 # [ 0.698172] drop_monitor: Initializing network drop monitor service1631worker2 # [ 0.699336] NET: Registered PF_INET6 protocol family1632lb # [ 0.691623] Linux agpgart interface v0.1031633worker1 # [ 0.709846] In-situ OAM (IOAM) with IPv61634lb # [ 0.692516] ACPI: bus type drm_connector registered1635worker1 # [ 0.710868] IPI shorthand broadcast: enabled1636client # [ 0.704215] usbcore: registered new interface driver usbserial_generic1637client # [ 0.705325] usbserial: USB Serial support registered for generic1638worker2 # [ 0.702347] Segment Routing with IPv61639lb # [ 0.695991] usbcore: registered new interface driver usbserial_generic1640worker2 # [ 0.705058] In-situ OAM (IOAM) with IPv61641lb # [ 0.697185] usbserial: USB Serial support registered for generic1642client # [ 0.708757] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1643worker2 # [ 0.706151] IPI shorthand broadcast: enabled1644client # [ 0.708757] Please enable it if your BIOS has the CPPC option.1645client # [ 0.711062] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1646lb # [ 0.700473] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1647worker1 # [ 0.719025] sched_clock: Marking stable (584013568, 134799956)->(789996430, -71182906)1648lb # [ 0.700473] Please enable it if your BIOS has the CPPC option.1649client # [ 0.713901] drop_monitor: Initializing network drop monitor service1650lb # [ 0.702935] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1651client # [ 0.715132] NET: Registered PF_INET6 protocol family1652worker1 # [ 0.723057] registered taskstats version 11653worker1 # [ 0.724030] Loading compiled-in X.509 certificates1654lb # [ 0.706604] drop_monitor: Initializing network drop monitor service1655worker2 # [ 0.714179] sched_clock: Marking stable (580014741, 134002810)->(785412723, -71395172)1656client # [ 0.719306] Segment Routing with IPv61657lb # [ 0.707904] NET: Registered PF_INET6 protocol family1658client # [ 0.721772] In-situ OAM (IOAM) with IPv61659worker2 # [ 0.719205] registered taskstats version 11660client # [ 0.722879] IPI shorthand broadcast: enabled1661worker2 # [ 0.720207] Loading compiled-in X.509 certificates1662lb # [ 0.712774] Segment Routing with IPv61663lb # [ 0.713562] In-situ OAM (IOAM) with IPv61664lb # [ 0.716734] IPI shorthand broadcast: enabled1665client # [ 0.731170] sched_clock: Marking stable (597014221, 133724427)->(802466328, -71727680)1666worker1 # [ 0.741836] Demotion targets for Node 0: null1667lb # [ 0.724827] sched_clock: Marking stable (591015951, 133440262)->(796278668, -71822455)1668client # [ 0.736971] registered taskstats version 11669worker1 # [ 0.743872] Key type .fscrypt registered1670worker1 # [ 0.744540] Key type fscrypt-provisioning registered1671client # [ 0.737963] Loading compiled-in X.509 certificates1672worker1 # [ 0.745501] ima: No TPM chip found, activating TPM-bypass!1673lb # [ 0.728580] registered taskstats version 11674worker2 # [ 0.736667] Demotion targets for Node 0: null1675lb # [ 0.729672] Loading compiled-in X.509 certificates1676worker2 # [ 0.739076] Key type .fscrypt registered1677worker1 # [ 0.749829] ima: Allocated hash algorithm: sha11678worker2 # [ 0.739798] Key type fscrypt-provisioning registered1679worker1 # [ 0.750681] ima: No architecture policies found1680worker2 # [ 0.740735] ima: No TPM chip found, activating TPM-bypass!1681worker1 # [ 0.752834] PM: Magic number: 10:951:5251682worker1 # [ 0.753567] PM: hash matches drivers/base/power/main.c:8931683worker2 # [ 0.745132] ima: Allocated hash algorithm: sha11684worker2 # [ 0.745958] ima: No architecture policies found1685worker1 # [ 0.757412] RAS: Correctable Errors collector initialized.1686worker2 # [ 0.748038] PM: Magic number: 10:951:5251687worker2 # [ 0.748777] PM: hash matches drivers/base/power/main.c:8931688worker2 # [ 0.752667] RAS: Correctable Errors collector initialized.1689client # [ 0.756767] Demotion targets for Node 0: null1690client # [ 0.757693] Key type .fscrypt registered1691client # [ 0.758429] Key type fscrypt-provisioning registered1692lb # [ 0.747475] Demotion targets for Node 0: null1693worker1 # [ 0.766242] clk: Disabling unused clocks1694client # [ 0.759464] ima: No TPM chip found, activating TPM-bypass!1695lb # [ 0.749517] Key type .fscrypt registered1696lb # [ 0.750254] Key type fscrypt-provisioning registered1697worker1 # [ 0.768833] PM: genpd: Disabling unused power domains1698client # [ 0.762769] ima: Allocated hash algorithm: sha11699client # [ 0.763578] ima: No architecture policies found1700worker2 # [ 0.761405] clk: Disabling unused clocks1701lb # [ 0.753591] ima: No TPM chip found, activating TPM-bypass!1702lb # [ 0.754630] ima: Allocated hash algorithm: sha11703worker2 # [ 0.764031] PM: genpd: Disabling unused power domains1704client # [ 0.767768] PM: Magic number: 10:951:5251705client # [ 0.768517] PM: hash matches drivers/base/power/main.c:8931706lb # [ 0.757496] ima: No architecture policies found1707lb # [ 0.758594] PM: Magic number: 10:951:5251708lb # [ 0.759384] PM: hash matches drivers/base/power/main.c:8931709client # [ 0.772660] RAS: Correctable Errors collector initialized.1710lb # [ 0.763366] RAS: Correctable Errors collector initialized.1711client # [ 0.781638] clk: Disabling unused clocks1712lb # [ 0.772200] clk: Disabling unused clocks1713client # [ 0.783771] PM: genpd: Disabling unused power domains1714lb # [ 0.776469] PM: genpd: Disabling unused power domains1715worker1 # [ 0.925545] Freeing initrd memory: 29092K1716worker1 # [ 0.928645] Freeing unused decrypted memory: 2028K1717worker1 # [ 0.931232] Freeing unused kernel image (initmem) memory: 3644K1718worker1 # [ 0.932343] Write protecting the kernel read-only data: 32768k1719worker1 # [ 0.934264] Freeing unused kernel image (text/rodata gap) memory: 1208K1720worker2 # [ 0.924754] Freeing initrd memory: 29092K1721worker1 # [ 0.935773] Freeing unused kernel image (rodata/data gap) memory: 752K1722worker2 # [ 0.927994] Freeing unused decrypted memory: 2028K1723worker2 # [ 0.930604] Freeing unused kernel image (initmem) memory: 3644K1724worker2 # [ 0.931738] Write protecting the kernel read-only data: 32768k1725worker2 # [ 0.933679] Freeing unused kernel image (text/rodata gap) memory: 1208K1726worker2 # [ 0.935281] Freeing unused kernel image (rodata/data gap) memory: 752K1727lb # [ 0.934285] Freeing initrd memory: 29092K1728lb # [ 0.937641] Freeing unused decrypted memory: 2028K1729client # [ 0.950893] Freeing initrd memory: 29096K1730lb # [ 0.940325] Freeing unused kernel image (initmem) memory: 3644K1731lb # [ 0.941554] Write protecting the kernel read-only data: 32768k1732client # [ 0.954235] Freeing unused decrypted memory: 2028K1733lb # [ 0.943513] Freeing unused kernel image (text/rodata gap) memory: 1208K1734lb # [ 0.945136] Freeing unused kernel image (rodata/data gap) memory: 752K1735client # [ 0.957097] Freeing unused kernel image (initmem) memory: 3644K1736client # [ 0.958253] Write protecting the kernel read-only data: 32768k1737client # [ 0.960243] Freeing unused kernel image (text/rodata gap) memory: 1208K1738client # [ 0.961858] Freeing unused kernel image (rodata/data gap) memory: 752K1739worker1 # [ 0.987031] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740worker1 # [ 0.988126] Run /init as init process1741worker2 # [ 0.986422] x86/mm: Checked W+X mappings: passed, no W+X pages found.1742worker2 # [ 0.987608] Run /init as init process1743worker1 # [ 0.998067] systemd[1]: Inserted module 'autofs4'1744worker2 # [ 0.997813] systemd[1]: Inserted module 'autofs4'1745worker1 # [ 1.014381] fuse: init (API version 7.45)1746lb # [ 0.996309] x86/mm: Checked W+X mappings: passed, no W+X pages found.1747lb # [ 0.997533] Run /init as init process1748client # [ 1.013076] x86/mm: Checked W+X mappings: passed, no W+X pages found.1749worker1 # [ 1.020216] ACPI: \_SB_.GSIG: Enabled at IRQ 221750client # [ 1.014151] Run /init as init process1751worker1 # [ 1.023105] ACPI: \_SB_.GSIH: Enabled at IRQ 231752worker2 # [ 1.014289] fuse: init (API version 7.45)1753lb # [ 1.007799] systemd[1]: Inserted module 'autofs4'1754worker1 # [ 1.026411] ACPI: \_SB_.GSIE: Enabled at IRQ 201755worker1 # [ 1.028928] ACPI: \_SB_.GSIF: Enabled at IRQ 211756worker2 # [ 1.020157] ACPI: \_SB_.GSIG: Enabled at IRQ 221757client # [ 1.025171] systemd[1]: Inserted module 'autofs4'1758worker2 # [ 1.022586] ACPI: \_SB_.GSIH: Enabled at IRQ 231759worker2 # [ 1.025537] ACPI: \_SB_.GSIE: Enabled at IRQ 201760worker2 # [ 1.027920] ACPI: \_SB_.GSIF: Enabled at IRQ 211761lb # [ 1.024159] fuse: init (API version 7.45)1762lb # [ 1.029998] ACPI: \_SB_.GSIG: Enabled at IRQ 221763client # [ 1.042072] fuse: init (API version 7.45)1764lb # [ 1.032533] ACPI: \_SB_.GSIH: Enabled at IRQ 231765lb # [ 1.036113] ACPI: \_SB_.GSIE: Enabled at IRQ 201766client # [ 1.048166] ACPI: \_SB_.GSIG: Enabled at IRQ 221767lb # [ 1.038658] ACPI: \_SB_.GSIF: Enabled at IRQ 211768client # [ 1.050571] ACPI: \_SB_.GSIH: Enabled at IRQ 231769client # [ 1.053595] ACPI: \_SB_.GSIE: Enabled at IRQ 201770client # [ 1.055812] ACPI: \_SB_.GSIF: Enabled at IRQ 211771worker1 # [ 1.063946] systemd[1]: Successfully made /usr/ read-only.1772worker2 # [ 1.058775] systemd[1]: Successfully made /usr/ read-only.1773lb # [ 1.065314] systemd[1]: Successfully made /usr/ read-only.1774client # [ 1.084767] systemd[1]: Successfully made /usr/ read-only.1775worker1 # [ 1.399655] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776worker1 # [ 1.404706] systemd[1]: Detected virtualization kvm.1777worker1 # [ 1.405580] systemd[1]: Detected architecture x86-64.1778worker1 # [ 1.406521] systemd[1]: Running in initrd.1779worker1 # [ 1.407750] systemd[1]: Initializing machine ID from random generator.1780worker1 # [ 1.409043] systemd[1]: Hostname set to <worker1>.1781worker2 # [ 1.395300] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1782worker2 # [ 1.407278] systemd[1]: Detected virtualization kvm.1783worker2 # [ 1.409387] systemd[1]: Detected architecture x86-64.1784worker2 # [ 1.411545] systemd[1]: Running in initrd.1785worker2 # [ 1.414080] systemd[1]: Initializing machine ID from random generator.1786worker2 # [ 1.416974] systemd[1]: Hostname set to <worker2>.1787lb # [ 1.401967] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1788lb # [ 1.414268] systemd[1]: Detected virtualization kvm.1789lb # [ 1.416411] systemd[1]: Detected architecture x86-64.1790lb # [ 1.418915] systemd[1]: Running in initrd.1791client # [ 1.421406] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1792lb # [ 1.421518] systemd[1]: Initializing machine ID from random generator.1793client # [ 1.433564] systemd[1]: Detected virtualization kvm.1794lb # [ 1.424607] systemd[1]: Hostname set to <lb>.1795client # [ 1.435817] systemd[1]: Detected architecture x86-64.1796client # [ 1.438001] systemd[1]: Running in initrd.1797client # [ 1.440482] systemd[1]: Initializing machine ID from random generator.1798client # [ 1.443404] systemd[1]: Hostname set to <client>.1799worker1 # [ 1.632227] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker2 # [ 1.640845] systemd[1]: bpf-restrict-fs: LSM BPF program attached1801lb # [ 1.647802] systemd[1]: bpf-restrict-fs: LSM BPF program attached1802client # [ 1.662411] systemd[1]: bpf-restrict-fs: LSM BPF program attached1803worker1 # [ 1.687679] systemd[1]: Queued start job for default target Initrd Default Target.1804worker1 # [ 1.692264] systemd[1]: Created slice Slice /system/modprobe.1805worker1 # [ 1.693408] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1806worker1 # [ 1.694764] systemd[1]: Expecting device /dev/disk/by-label/nixos...1807worker1 # [ 1.695891] systemd[1]: Reached target Path Units.1808worker1 # [ 1.696732] systemd[1]: Reached target Slice Units.1809worker1 # [ 1.697608] systemd[1]: Reached target Swaps.1810worker1 # [ 1.698438] systemd[1]: Reached target Timer Units.1811worker1 # [ 1.699376] systemd[1]: Listening on D-Bus System Message Bus Socket.1812worker1 # [ 1.700564] systemd[1]: Listening on Journal Socket (/dev/log).1813worker1 # [ 1.701702] systemd[1]: Listening on Journal Sockets.1814worker1 # [ 1.702690] systemd[1]: Listening on udev Control Socket.1815worker1 # [ 1.703774] systemd[1]: Listening on udev Kernel Socket.1816worker1 # [ 1.704747] systemd[1]: Reached target Socket Units.1817worker2 # [ 1.695179] systemd[1]: Queued start job for default target Initrd Default Target.1818worker1 # [ 1.706654] systemd[1]: Starting Create List of Static Device Nodes...1819worker2 # [ 1.698360] systemd[1]: Created slice Slice /system/modprobe.1820worker2 # [ 1.699499] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1821worker1 # [ 1.710888] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1822worker2 # [ 1.700859] systemd[1]: Expecting device /dev/disk/by-label/nixos...1823worker2 # [ 1.701917] systemd[1]: Reached target Path Units.1824worker2 # [ 1.702823] systemd[1]: Reached target Slice Units.1825worker2 # [ 1.703717] systemd[1]: Reached target Swaps.1826lb # [ 1.695471] systemd[1]: Queued start job for default target Initrd Default Target.1827worker2 # [ 1.704551] systemd[1]: Reached target Timer Units.1828worker2 # [ 1.705549] systemd[1]: Listening on D-Bus System Message Bus Socket.1829lb # [ 1.698734] systemd[1]: Created slice Slice /system/modprobe.1830worker2 # [ 1.706747] systemd[1]: Listening on Journal Socket (/dev/log).1831worker1 # [ 1.717521] systemd[1]: Starting Load Kernel Module configfs...1832worker2 # [ 1.707914] systemd[1]: Listening on Journal Sockets.1833lb # [ 1.699907] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1834worker2 # [ 1.708933] systemd[1]: Listening on udev Control Socket.1835lb # [ 1.701315] systemd[1]: Expecting device /dev/disk/by-label/nixos...1836worker2 # [ 1.709942] systemd[1]: Listening on udev Kernel Socket.1837lb # [ 1.702455] systemd[1]: Reached target Path Units.1838worker2 # [ 1.710897] systemd[1]: Reached target Socket Units.1839lb # [ 1.703306] systemd[1]: Reached target Slice Units.1840lb # [ 1.704177] systemd[1]: Reached target Swaps.1841lb # [ 1.705029] systemd[1]: Reached target Timer Units.1842worker2 # [ 1.712705] systemd[1]: Starting Create List of Static Device Nodes...1843lb # [ 1.705991] systemd[1]: Listening on D-Bus System Message Bus Socket.1844client # [ 1.717092] systemd[1]: Queued start job for default target Initrd Default Target.1845lb # [ 1.707175] systemd[1]: Listening on Journal Socket (/dev/log).1846lb # [ 1.708333] systemd[1]: Listening on Journal Sockets.1847lb # [ 1.709350] systemd[1]: Listening on udev Control Socket.1848worker2 # [ 1.717089] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1849client # [ 1.721063] systemd[1]: Created slice Slice /system/modprobe.1850lb # [ 1.710355] systemd[1]: Listening on udev Kernel Socket.1851lb # [ 1.711324] systemd[1]: Reached target Socket Units.1852client # [ 1.722236] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1853client # [ 1.723642] systemd[1]: Expecting device /dev/disk/by-label/nixos...1854lb # [ 1.713126] systemd[1]: Starting Create List of Static Device Nodes...1855client # [ 1.724901] systemd[1]: Reached target Path Units.1856client # [ 1.725771] systemd[1]: Reached target Slice Units.1857worker1 # [ 1.732891] systemd[1]: Starting Journal Service...1858client # [ 1.726633] systemd[1]: Reached target Swaps.1859client # [ 1.727527] systemd[1]: Reached target Timer Units.1860client # [ 1.728577] systemd[1]: Listening on D-Bus System Message Bus Socket.1861worker2 # [ 1.725201] systemd[1]: Starting Load Kernel Module configfs...1862lb # [ 1.717529] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1863client # [ 1.729749] systemd[1]: Listening on Journal Socket (/dev/log).1864client # [ 1.730986] systemd[1]: Listening on Journal Sockets.1865client # [ 1.732059] systemd[1]: Listening on udev Control Socket.1866client # [ 1.733177] systemd[1]: Listening on udev Kernel Socket.1867client # [ 1.734216] systemd[1]: Reached target Socket Units.1868client # [ 1.736117] systemd[1]: Starting Create List of Static Device Nodes...1869lb # [ 1.725644] systemd[1]: Starting Load Kernel Module configfs...1870client # [ 1.739810] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1871worker2 # [ 1.741089] systemd[1]: Starting Journal Service...1872client # [ 1.746497] systemd[1]: Starting Load Kernel Module configfs...1873lb # [ 1.739766] systemd[1]: Starting Journal Service...1874worker1 # [ 1.746288] systemd[1]: Starting Load Kernel Modules...1875worker1 # [ 1.767437] netfs: FS-Cache loaded1876client # [ 1.760933] systemd[1]: Starting Journal Service...1877worker1 # [ 1.772271] 9pnet: Installing 9P2000 support1878worker2 # [ 1.753087] systemd[1]: Starting Load Kernel Modules...1879lb # [ 1.748640] systemd[1]: Starting Load Kernel Modules...1880worker2 # [ 1.770861] netfs: FS-Cache loaded1881worker1 # [ 1.782917] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1882worker2 # [ 1.775621] 9pnet: Installing 9P2000 support1883lb # [ 1.769987] netfs: FS-Cache loaded1884worker1 # [ 1.792247] systemd[1]: Starting Coldplug All udev Devices...1885client # [ 1.774834] systemd[1]: Starting Load Kernel Modules...1886lb # [ 1.775594] 9pnet: Installing 9P2000 support1887worker2 # [ 1.786554] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1888client # [ 1.794604] netfs: FS-Cache loaded1889worker1 # [ 1.805313] systemd[1]: Finished Create List of Static Device Nodes.1890client # [ 1.799279] 9pnet: Installing 9P2000 support1891lb # [ 1.787639] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1892worker2 # [ 1.796095] systemd[1]: Starting Coldplug All udev Devices...1893worker1 # [ 1.811446] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1894lb # [ 1.796535] systemd[1]: Starting Coldplug All udev Devices...1895client # [ 1.807480] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1896worker1 # [ 1.816829] systemd-journald[67]: Collecting audit messages is disabled.1897worker2 # [ 1.808517] systemd[1]: Finished Create List of Static Device Nodes.1898worker1 # [ 1.821191] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1899client # [ 1.816937] systemd[1]: Starting Coldplug All udev Devices...1900worker2 # [ 1.814598] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1901worker1 # [ 1.826252] systemd[1]: modprobe@configfs.service: Deactivated successfully.1902lb # [ 1.809022] systemd[1]: Finished Create List of Static Device Nodes.1903worker1 # [ 1.831861] systemd[1]: Finished Load Kernel Module configfs.1904lb # [ 1.814083] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1905worker2 # [ 1.822895] systemd-journald[67]: Collecting audit messages is disabled.1906worker2 # [ 1.824440] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1907worker1 # [ 1.834067] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1908client # [ 1.829858] systemd[1]: Finished Create List of Static Device Nodes.1909worker2 # [ 1.828490] systemd[1]: modprobe@configfs.service: Deactivated successfully.1910lb # [ 1.821774] systemd-journald[67]: Collecting audit messages is disabled.1911worker1 # [ 1.839273] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1912lb # [ 1.823334] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1913client # [ 1.836449] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1914worker1 # [ 1.845963] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1915worker2 # [ 1.835772] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1916lb # [ 1.829874] systemd[1]: modprobe@configfs.service: Deactivated successfully.1917worker2 # [ 1.837798] systemd[1]: Finished Load Kernel Module configfs.1918client # [ 1.843684] systemd-journald[67]: Collecting audit messages is disabled.1919worker1 # [ 1.851024] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1920lb # [ 1.832832] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1921client # [ 1.846152] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1922worker2 # [ 1.842495] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1923lb # [ 1.836824] systemd[1]: Finished Load Kernel Module configfs.1924client # [ 1.850210] systemd[1]: modprobe@configfs.service: Deactivated successfully.1925lb # [ 1.840916] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1926worker2 # [ 1.849180] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1927client # [ 1.857010] systemd[1]: Finished Load Kernel Module configfs.1928lb # [ 1.845620] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1929worker2 # [ 1.854138] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1930client # [ 1.860371] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1931client # [ 1.864222] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1932lb # [ 1.853743] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1933worker1 # [ 1.874342] systemd[1]: Finished Load Kernel Modules.1934client # [ 1.869896] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1935client # [ 1.873696] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1936worker1 # [ 1.882020] systemd[1]: Starting Apply Kernel Variables...1937worker2 # [ 1.879548] systemd[1]: Finished Load Kernel Modules.1938lb # [ 1.875967] systemd[1]: Finished Load Kernel Modules.1939worker2 # [ 1.886213] systemd[1]: Starting Apply Kernel Variables...1940worker1 # [ 1.898142] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1941lb # [ 1.882623] systemd[1]: Starting Apply Kernel Variables...1942client # [ 1.900274] systemd[1]: Finished Load Kernel Modules.1943worker1 # [ 1.910008] systemd[1]: Starting Create Static Device Nodes in /dev...1944worker2 # [ 1.899725] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1945client # [ 1.906957] systemd[1]: Starting Apply Kernel Variables...1946lb # [ 1.899548] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1947worker1 # [ 1.920393] systemd[1]: Finished Apply Kernel Variables.1948worker2 # [ 1.911249] systemd[1]: Starting Create Static Device Nodes in /dev...1949lb # [ 1.908636] systemd[1]: Starting Create Static Device Nodes in /dev...1950client # [ 1.922835] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1951worker1 # [ 1.935490] systemd[1]: Started Journal Service.1952worker2 # [ 1.925131] systemd[1]: Finished Apply Kernel Variables.1953worker1 # [ 1.802293] systemd-modules-load[68]: Inserted module 'dm_mod'1954lb # [ 1.920996] systemd[1]: Finished Apply Kernel Variables.1955client # [ 1.931937] systemd[1]: Starting Create Static Device Nodes in /dev...1956worker1 # [ 1.806102] systemd-modules-load[68]: Inserted module 'virtio_balloon'1957worker1 # [ 1.808238] systemd-modules-load[68]: Inserted module 'virtio_gpu'1958worker1 # [ 1.816887] systemd[1]: Finished Create Static Device Nodes in /dev.1959worker2 # [ 1.942688] systemd[1]: Started Journal Service.1960worker2 # [ 1.807857] systemd-modules-load[68]: Inserted module 'dm_mod'1961worker1 # [ 1.819727] systemd[1]: Reached target Preparation for Local File Systems.1962worker2 # [ 1.812222] systemd-modules-load[68]: Inserted module 'virtio_balloon'1963lb # [ 1.938609] systemd[1]: Started Journal Service.1964worker2 # [ 1.813227] systemd-modules-load[68]: Inserted module 'virtio_gpu'1965worker1 # [ 1.822753] systemd[1]: Reached target Local File Systems.1966worker1 # [ 1.823807] systemd[1]: Starting Create System Files and Directories...1967lb # [ 1.806129] systemd-modules-load[68]: Inserted module 'dm_mod'1968client # [ 1.952096] systemd[1]: Finished Apply Kernel Variables.1969worker2 # [ 1.815197] systemd[1]: Finished Create Static Device Nodes in /dev.1970lb # [ 1.808302] systemd-modules-load[68]: Inserted module 'virtio_balloon'1971worker2 # [ 1.816287] systemd[1]: Reached target Preparation for Local File Systems.1972lb # [ 1.809942] systemd-modules-load[68]: Inserted module 'virtio_gpu'1973worker2 # [ 1.819213] systemd[1]: Reached target Local File Systems.1974worker1 # [ 1.829396] systemd[1]: Starting Rule-based Manager for Device Events and Files...1975lb # [ 1.813932] systemd[1]: Finished Create Static Device Nodes in /dev.1976worker2 # [ 1.823069] systemd[1]: Starting Create System Files and Directories...1977lb # [ 1.820881] systemd[1]: Reached target Preparation for Local File Systems.1978lb # [ 1.821974] systemd[1]: Reached target Local File Systems.1979client # [ 1.966520] systemd[1]: Started Journal Service.1980worker2 # [ 1.829044] systemd[1]: Starting Rule-based Manager for Device Events and Files...1981client # [ 1.834792] systemd-modules-load[68]: Inserted module 'dm_mod'1982lb # [ 1.825098] systemd[1]: Starting Create System Files and Directories...1983client # [ 1.837360] systemd-modules-load[68]: Inserted module 'virtio_balloon'1984client # [ 1.839284] systemd-modules-load[68]: Inserted module 'virtio_gpu'1985lb # [ 1.829771] systemd[1]: Starting Rule-based Manager for Device Events and Files...1986client # [ 1.841299] systemd[1]: Finished Create Static Device Nodes in /dev.1987worker1 # [ 1.856560] systemd[1]: Finished Create System Files and Directories.1988client # [ 1.851107] systemd[1]: Reached target Preparation for Local File Systems.1989client # [ 1.852146] systemd[1]: Reached target Local File Systems.1990client # [ 1.855129] systemd[1]: Starting Create System Files and Directories...1991client # [ 1.859803] systemd[1]: Starting Rule-based Manager for Device Events and Files...1992worker2 # [ 1.860055] systemd[1]: Finished Create System Files and Directories.1993lb # [ 1.854037] systemd[1]: Finished Create System Files and Directories.1994worker1 # [ 1.878365] systemd-udevd[83]: Using default interface naming scheme 'v261'.1995worker2 # [ 1.881404] systemd-udevd[77]: Using default interface naming scheme 'v261'.1996lb # [ 1.878131] systemd-udevd[83]: Using default interface naming scheme 'v261'.1997client # [ 1.890135] systemd[1]: Finished Create System Files and Directories.1998worker1 # [ 1.902621] systemd[1]: Started Rule-based Manager for Device Events and Files.1999worker2 # [ 1.906066] systemd[1]: Started Rule-based Manager for Device Events and Files.2000client # [ 1.913369] systemd-udevd[83]: Using default interface naming scheme 'v261'.
loading 1,917 hidden lines…
3918worker2 # [ 12.146376] dhcpcd[695]: eth0: soliciting a DHCP lease3919worker2 # [ 12.304603] NET: Registered PF_PACKET protocol family3920worker2 # [ 12.176938] dhcpcd[695]: eth0: offered 10.0.2.15 from 10.0.2.23921worker2 # [ 12.182140] dhcpcd[695]: eth0: probing address 10.0.2.15/243922lb # [ 12.342403] dhcpcd[706]: eth0: soliciting a DHCP lease3923lb # [ 12.490179] NET: Registered PF_PACKET protocol family3924lb # [ 12.360467] dhcpcd[706]: eth0: offered 10.0.2.15 from 10.0.2.23925lb # [ 12.363206] dhcpcd[706]: eth0: probing address 10.0.2.15/243926worker2 # [ 12.526705] dhcpcd[695]: eth0: soliciting an IPv6 router3927worker2 # [ 12.527985] dhcpcd[695]: eth0: Router Advertisement from fe80::23928worker2 # [ 12.528932] dhcpcd[695]: eth0: adding address fec0::5054:ff:fe12:3456/643929worker2 # [ 12.530144] dhcpcd[695]: eth0: adding route to fec0::/643930worker2 # [ 12.530931] dhcpcd[695]: eth0: adding default route via fe80::23931lb # [ 12.542649] dhcpcd[706]: eth0: soliciting an IPv6 router3932lb # [ 12.543740] dhcpcd[706]: eth0: Router Advertisement from fe80::23933lb # [ 12.544851] dhcpcd[706]: eth0: adding address fec0::5054:ff:fe12:3456/643934lb # [ 12.545837] dhcpcd[706]: eth0: adding route to fec0::/643935lb # [ 12.546852] dhcpcd[706]: eth0: adding default route via fe80::23936lb # [ 12.866266] postgresql-pre-start[729]: selecting default time zone ... UTC3937lb # [ 12.871152] postgresql-pre-start[729]: creating configuration files ... ok3938lb # [ 13.093933] postgresql-pre-start[729]: running bootstrap script ... ok3939lb # [ 13.616962] postgresql-pre-start[729]: performing post-bootstrap initialization ... ok3940lb # [ 13.732347] postgresql-pre-start[729]: syncing data to disk ... ok3941lb # [ 13.734064] postgresql-pre-start[729]: initdb: warning: enabling "trust" authentication for local connections3942lb # [ 13.735338] postgresql-pre-start[729]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3943lb # [ 13.737124] postgresql-pre-start[729]: Success. You can now start the database server using:3944lb # [ 13.738168] postgresql-pre-start[729]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3945lb # [ 13.842188] postgres[799]: [799] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3946lb # [ 13.846384] postgres[799]: [799] LOG: listening on IPv6 address "::1", port 54323947lb # [ 13.847585] postgres[799]: [799] LOG: listening on IPv4 address "127.0.0.1", port 54323948lb # [ 13.849958] postgres[799]: [799] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3949lb # [ 13.863405] postgres[808]: [808] LOG: database system was shut down at 2026-09-20 09:33:08 GMT3950lb # [ 13.867961] postgres[799]: [799] LOG: database system is ready to accept connections3951lb # [ 13.875289] systemd[1]: Started PostgreSQL Server.3952lb # [ 13.880606] systemd[1]: Starting PostgreSQL Setup Scripts...3953lb # [ 14.051290] postgresql-setup-start[824]: CREATE DATABASE3954lb # [ 14.096156] postgresql-setup-start[829]: CREATE ROLE3955lb # [ 14.116968] postgresql-setup-start[831]: ALTER DATABASE3956lb # [ 14.121771] systemd[1]: Finished PostgreSQL Setup Scripts.3957lb # [ 14.123475] systemd[1]: Reached target PostgreSQL.3958client # [ 16.690188] dhcpcd[563]: eth0: leased 10.0.2.15 for 86400 seconds3959client # [ 16.691133] dhcpcd[563]: eth0: adding route to 10.0.2.0/243960client # [ 16.691829] dhcpcd[563]: eth0: adding default route via 10.0.2.23961client # [ 16.768447] systemd[1]: Started DHCP Client.3962client # [ 16.770799] systemd[1]: Reached target Multi-User System.3963client # [ 16.771864] systemd[1]: Startup finished in 884ms (kernel) + 4.030s (initrd) + 11.856s (userspace) = 16.771s.3964worker2 # [ 17.268536] dhcpcd[695]: eth0: leased 10.0.2.15 for 86400 seconds3965worker2 # [ 17.271228] dhcpcd[695]: eth0: adding route to 10.0.2.0/243966worker2 # [ 17.273426] dhcpcd[695]: eth0: adding default route via 10.0.2.23967worker2 # [ 17.386559] systemd[1]: Started DHCP Client.3968worker2 # [ 17.388613] systemd[1]: Reached target Multi-User System.3969worker2 # [ 17.389489] systemd[1]: Startup finished in 857ms (kernel) + 5.068s (initrd) + 11.462s (userspace) = 17.388s.3970worker1 # [ 17.586413] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds3971worker1 # [ 17.589323] dhcpcd[692]: eth0: adding route to 10.0.2.0/243972worker1 # [ 17.591617] dhcpcd[692]: eth0: adding default route via 10.0.2.23973lb # [ 17.698515] dhcpcd[706]: eth0: leased 10.0.2.15 for 86400 seconds3974lb # [ 17.699844] dhcpcd[706]: eth0: adding route to 10.0.2.0/243975lb # [ 17.700653] dhcpcd[706]: eth0: adding default route via 10.0.2.23976worker1 # [ 17.719455] systemd[1]: Started DHCP Client.3977worker1 # [ 17.721478] systemd[1]: Reached target Multi-User System.3978worker1 # [ 17.722426] systemd[1]: Startup finished in 856ms (kernel) + 4.072s (initrd) + 12.792s (userspace) = 17.721s.3979lb # [ 17.792597] systemd[1]: Started DHCP Client.3980lb # [ 17.795081] systemd[1]: Reached target Network is Online.3981lb # [ 17.799294] systemd[1]: Started Envoy reverse proxy.3982lb # [ 17.802684] systemd[1]: Started mock-oidc.service.3983lb # [ 17.893933] mock-oidc-server[927]: Mock OIDC Server running3984lb # [ 17.894786] mock-oidc-server[927]: OIDC Address: 192.168.1.2:80803985lb # [ 17.895608] mock-oidc-server[927]: Issue Address: 0.0.0.0:80813986lb # [ 17.896543] mock-oidc-server[927]: Issuer: http://192.168.1.2:8080/oidc3987lb # [ 17.897854] mock-oidc-server[927]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3988lb # [ 17.899619] mock-oidc-server[927]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3989lb # [ 17.901135] mock-oidc-server[927]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3990lb # [ 18.327260] envoy[926]: [2026-09-20 09:33:13.584][926][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3991lb # [ 18.329388] envoy[926]: [2026-09-20 09:33:13.585][926][info][main] [source/server/server.cc:469] statically linked extensions:3992lb # [ 18.331041] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3993lb # [ 18.334825] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3994lb # [ 18.336364] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3995lb # [ 18.338521] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3996lb # [ 18.340359] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3997lb # [ 18.342071] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23998lb # [ 18.343809] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3999lb # [ 18.348255] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4000lb # [ 18.362888] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4001lb # [ 18.364545] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4002lb # [ 18.366058] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4003lb # [ 18.367714] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4004lb # [ 18.370107] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4005lb # [ 18.371838] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4006lb # [ 18.378662] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84007lb # [ 18.380500] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4008lb # [ 18.382299] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4009lb # [ 18.385430] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4010lb # [ 18.387311] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4011lb # [ 18.389453] envoy[926]: [2026-09-20 09:33:13.587][926][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4012lb # [ 18.391155] envoy[926]: [2026-09-20 09:33:13.588][926][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4013lb # [ 18.393135] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4014lb # [ 18.395405] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4015lb # [ 18.398108] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4016lb # [ 18.400453] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4017lb # [ 18.403541] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4018lb # [ 18.405802] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4019lb # [ 18.407806] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4020lb # [ 18.412744] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4021lb # [ 18.414935] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24022lb # [ 18.417344] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4023lb # [ 18.419495] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4024lb # [ 18.422083] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4025lb # [ 18.426063] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4026lb # [ 18.427750] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4027lb # [ 18.430122] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4028lb # [ 18.431703] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4029lb # [ 18.434094] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4030lb # [ 18.436885] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4031lb # [ 18.439091] envoy[926]: [2026-09-20 09:33:13.589][926][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4032lb # [ 18.440661] envoy[926]: [2026-09-20 09:33:13.596][926][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4033lb # [ 18.443243] envoy[926]: [2026-09-20 09:33:13.596][926][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4034lb # [ 18.452378] envoy[926]: [2026-09-20 09:33:13.596][926][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4035lb # [ 18.457654] envoy[926]: [2026-09-20 09:33:13.596][926][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4036lb # [ 18.459823] envoy[926]: [2026-09-20 09:33:13.596][926][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4037lb # [ 18.465373] envoy[926]: [2026-09-20 09:33:13.596][926][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4038lb # [ 18.467348] envoy[926]: [2026-09-20 09:33:13.598][926][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4039lb # [ 18.470349] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4040lb # [ 18.472719] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4041lb # [ 18.474484] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4042lb # [ 18.476085] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4043lb # [ 18.478431] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4044lb # [ 18.484456] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4045lb # [ 18.486284] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4046lb # [ 18.488063] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4047lb # [ 18.489702] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4048lb # [ 18.491653] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4049lb # [ 18.493691] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4050lb # [ 18.495511] envoy[926]: [2026-09-20 09:33:13.600][926][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4051lb # [ 18.497559] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4052lb # [ 18.503785] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4053lb # [ 18.507696] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4054lb # [ 18.510154] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4055lb # [ 18.512395] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4056lb # [ 18.514686] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4057lb # [ 18.518837] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14058lb # [ 18.521077] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4059lb # [ 18.522795] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4060lb # [ 18.524619] envoy[926]: [2026-09-20 09:33:13.606][926][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4061lb # [ 18.526403] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4062lb # [ 18.528385] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4063lb # [ 18.530674] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4064lb # [ 18.541579] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4065lb # [ 18.544074] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4066lb # [ 18.545933] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4067lb # [ 18.547423] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4068lb # [ 18.549783] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4069lb # [ 18.552396] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4070lb # [ 18.556974] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts4071lb # [ 18.559746] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4072lb # [ 18.561743] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4073lb # [ 18.563487] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4074lb # [ 18.568306] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4075lb # [ 18.593843] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4076lb # [ 18.596561] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4077lb # [ 18.599526] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4078lb # [ 18.602814] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4079lb # [ 18.605298] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4080lb # [ 18.608092] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4081lb # [ 18.610240] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4082lb # [ 18.612271] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4083lb # [ 18.614189] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4084lb # [ 18.617079] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4085lb # [ 18.623572] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4086lb # [ 18.626157] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4087lb # [ 18.628230] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4088lb # [ 18.630808] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4089lb # [ 18.634567] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4090lb # [ 18.636266] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4091lb # [ 18.638263] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4092lb # [ 18.640257] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4093lb # [ 18.642566] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4094lb # [ 18.644576] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24095lb # [ 18.646135] envoy[926]: [2026-09-20 09:33:13.612][926][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4096lb # [ 18.649074] envoy[926]: [2026-09-20 09:33:13.723][926][info][main] [source/server/server.cc:531] HTTP header map info:4097lb # [ 18.650292] envoy[926]: [2026-09-20 09:33:13.770][926][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4098lb # [ 18.663346] envoy[926]: [2026-09-20 09:33:13.770][926][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4099lb # [ 18.664806] envoy[926]: [2026-09-20 09:33:13.770][926][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4100lb # [ 18.672275] envoy[926]: [2026-09-20 09:33:13.770][926][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4101lb # [ 18.705910] envoy[926]: [2026-09-20 09:33:13.964][926][info][main] [source/server/server.cc:968] runtime: {}4102lb # [ 18.713132] envoy[926]: [2026-09-20 09:33:13.971][926][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4103lb # [ 18.715206] envoy[926]: [2026-09-20 09:33:13.973][926][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014104lb # [ 18.717832] envoy[926]: [2026-09-20 09:33:13.975][926][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014105lb # [ 18.721537] envoy[926]: [2026-09-20 09:33:13.979][926][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4106lb # [ 18.723248] envoy[926]: [2026-09-20 09:33:13.981][926][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4107lb # [ 18.724759] envoy[926]: [2026-09-20 09:33:13.981][926][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4108lb # [ 18.776865] envoy[926]: [2026-09-20 09:33:14.035][926][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4109worker1 # [ 18.796503] systemd[1]: Starting Nix worker-protocol over gRPC...4110worker2 # [ 18.792400] systemd[1]: Starting Nix worker-protocol over gRPC...4111lb # [ 18.798077] envoy[926]: [2026-09-20 09:33:14.056][926][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4112lb # [ 18.800534] envoy[926]: [2026-09-20 09:33:14.058][926][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4113lb # [ 18.803848] envoy[926]: [2026-09-20 09:33:14.058][926][info][main] [source/server/server.cc:1077] starting main dispatch loop4114worker2 # [ 19.006367] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4115worker1 # [ 19.015960] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4116worker1 # [ 19.029372] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14117worker2 # [ 19.019890] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14118worker1 # [ 19.064440] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4119worker2 # [ 19.056451] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4120worker1 # [ 19.067626] systemd[1]: Started Nix worker-protocol over gRPC.4121worker2 # [ 19.059571] systemd[1]: Started Nix worker-protocol over gRPC.4122lb # [ 19.103317] envoy[926]: [2026-09-20 09:33:14.361][926][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4123lb # [ 19.108478] envoy[926]: [2026-09-20 09:33:14.366][926][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4124lb # [ 19.114807] envoy[926]: [2026-09-20 09:33:14.366][926][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4125lb # [ 19.119778] envoy[926]: [2026-09-20 09:33:14.371][926][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4126worker1 # [ 19.160150] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500514127worker2 # [ 19.151042] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500514128worker1 # [ 19.170405] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4129worker2 # [ 19.161217] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4130worker1 # [ 19.371543] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500514131worker2 # [ 19.363239] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500514132worker1 # [ 19.375627] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker2 # [ 19.367077] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4134worker2 # [ 19.567322] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500514135worker1 # [ 19.576303] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500514136worker2 # [ 19.569149] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker1 # [ 19.580267] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker2 # [ 19.770452] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500514139worker1 # [ 19.780997] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_connected addr=lb:500514140worker2 # [ 19.774460] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4141worker1 # [ 19.785726] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 19.974926] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514143worker1 # [ 19.982844] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514144worker2 # [ 19.976747] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker1 # [ 19.986933] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4146worker2 # [ 20.177589] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514147worker2 # [ 20.179517] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4148worker1 # [ 20.189603] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514149worker1 # [ 20.193075] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4150worker2 # [ 20.380207] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514151worker2 # [ 20.382060] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker1 # [ 20.390327] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514153worker1 # [ 20.393979] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4154worker2 # [ 20.582634] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514155worker1 # [ 20.592107] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514156worker2 # [ 20.584597] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker1 # [ 20.595926] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4158worker2 # [ 20.785169] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514159worker2 # [ 20.786813] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4160worker1 # [ 20.796921] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_connected addr=lb:500514161worker1 # [ 20.800805] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker2 # [ 20.987971] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514163worker2 # [ 20.991844] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164worker1 # [ 21.002378] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514165worker1 # [ 21.005450] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4166worker2 # [ 21.193464] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514167worker1 # [ 21.204612] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514168worker2 # [ 21.197147] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 21.208258] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4170worker2 # [ 21.395268] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514171worker1 # [ 21.405276] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514172worker2 # [ 21.399316] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 21.410464] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4174worker2 # [ 21.600132] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514175worker1 # [ 21.611240] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 21.604122] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 21.615096] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4178worker2 # [ 21.804491] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 21.806293] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 21.816000] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_connected addr=lb:500514181worker1 # [ 21.818510] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker2 # [ 22.006953] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514183worker2 # [ 22.008681] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker1 # [ 22.019772] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514185worker1 # [ 22.024164] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4186worker2 # [ 22.211112] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514187worker2 # [ 22.214509] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker1 # [ 22.226537] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514189worker1 # [ 22.229625] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4190worker2 # [ 22.411942] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514191worker2 # [ 22.415913] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker1 # [ 22.427422] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514193worker1 # [ 22.431261] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4194worker2 # [ 22.616733] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514195worker2 # [ 22.620624] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 22.633178] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514197worker1 # [ 22.636657] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198worker2 # [ 22.821481] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514199worker1 # [ 22.834450] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_connected addr=lb:500514200worker2 # [ 22.825585] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker1 # [ 22.838661] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker2 # [ 23.026177] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514203worker1 # [ 23.039703] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514204worker2 # [ 23.030104] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 23.042222] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4206worker2 # [ 23.232312] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514207worker1 # [ 23.244418] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 23.235469] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker1 # [ 23.247508] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker2 # [ 23.433531] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514211worker1 # [ 23.445702] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 23.437583] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker1 # [ 23.449904] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker2 # [ 23.638536] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514215worker1 # [ 23.647635] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514216worker2 # [ 23.642399] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker1 # [ 23.652383] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker1 # [ 23.853188] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514219worker2 # [ 23.843273] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 23.855305] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 23.847331] nix-grpc-daemon[784]: ts=2026-09-20T09:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4222worker1 # [ 24.056206] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514223worker1 # [ 24.058213] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker2 # [ 24.048153] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514225worker2 # [ 24.052066] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker1 # [ 24.260823] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 24.254292] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 24.264000] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 24.257504] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker1 # [ 24.461520] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514231worker2 # [ 24.454623] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 24.465961] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker2 # [ 24.458363] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker2 # [ 24.656074] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514235worker2 # [ 24.658169] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker1 # [ 24.667904] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514237worker1 # [ 24.671312] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker2 # [ 24.858771] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 24.869415] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_connected addr=lb:500514240worker2 # [ 24.861302] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 24.873431] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker2 # [ 25.061813] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514243worker2 # [ 25.064147] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker1 # [ 25.074517] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514245worker1 # [ 25.078416] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker2 # [ 25.265888] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514247worker1 # [ 25.278225] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514248worker2 # [ 25.269752] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker1 # [ 25.281915] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker1 # [ 25.479405] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 25.470959] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514252worker2 # [ 25.475240] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 25.484491] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4254worker2 # [ 25.672462] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514255worker2 # [ 25.674316] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker1 # [ 25.685685] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514257worker1 # [ 25.689854] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker2 # [ 25.874923] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514259worker2 # [ 25.876704] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker1 # [ 25.891497] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_connected addr=lb:500514261worker1 # [ 25.895498] nix-grpc-daemon[784]: ts=2026-09-20T09:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker2 # [ 26.077549] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514263worker2 # [ 26.081653] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker1 # [ 26.095846] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514265worker1 # [ 26.097938] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4266worker2 # [ 26.284293] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514267worker2 # [ 26.288071] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4268worker1 # [ 26.300643] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514269worker1 # [ 26.303691] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4270worker2 # [ 26.484895] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514271worker2 # [ 26.488910] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker1 # [ 26.501307] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514273worker1 # [ 26.505205] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4274worker2 # [ 26.689438] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514275worker2 # [ 26.691242] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4276worker1 # [ 26.705934] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514277worker1 # [ 26.707768] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4278worker2 # [ 26.891747] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514279worker1 # [ 26.907351] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 26.895751] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 26.909196] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4282worker2 # [ 27.096773] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514283worker1 # [ 27.110542] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514284worker2 # [ 27.101649] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker1 # [ 27.114381] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4286worker2 # [ 27.303234] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514287worker1 # [ 27.313980] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 27.306952] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 27.317186] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290worker1 # [ 27.515198] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514291worker2 # [ 27.508635] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514292worker1 # [ 27.519350] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker2 # [ 27.511699] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4294worker2 # [ 27.709563] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514295worker1 # [ 27.720418] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514296worker2 # [ 27.713407] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker1 # [ 27.724264] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4298worker1 # [ 27.922501] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_connected addr=lb:500514299worker2 # [ 27.914291] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514300worker1 # [ 27.926483] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker2 # [ 27.918103] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4302worker1 # [ 28.127501] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514303worker2 # [ 28.118784] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514304worker1 # [ 28.131450] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker2 # [ 28.122704] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4306worker2 # [ 28.323640] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514307worker1 # [ 28.333615] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 28.325576] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 28.337468] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4310worker2 # [ 28.526391] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514311worker1 # [ 28.538498] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 28.530341] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker1 # [ 28.542794] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4314worker2 # [ 28.731267] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514315worker1 # [ 28.743998] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514316worker2 # [ 28.735140] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker1 # [ 28.748514] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4318worker2 # [ 28.935845] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514319worker2 # [ 28.939706] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4320worker1 # [ 28.949567] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_connected addr=lb:500514321worker1 # [ 28.954082] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4322worker2 # [ 29.139926] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514323worker2 # [ 29.141933] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4324worker1 # [ 29.155322] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514325worker1 # [ 29.159651] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4326worker2 # [ 29.344227] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514327worker2 # [ 29.347500] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4328worker1 # [ 29.362786] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514329worker1 # [ 29.366394] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4330worker2 # [ 29.544716] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514331worker2 # [ 29.546486] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4332worker1 # [ 29.562712] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514333worker1 # [ 29.566564] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4334worker2 # [ 29.747544] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514335worker2 # [ 29.751442] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4336worker1 # [ 29.767863] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_connected addr=lb:500514337worker1 # [ 29.771948] nix-grpc-daemon[784]: ts=2026-09-20T09:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4338worker2 # [ 29.952471] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514339worker2 # [ 29.956548] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4340worker1 # [ 29.973891] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514341worker1 # [ 29.977178] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4342lb # [ 30.068974] rustfs-bucket-start[1045]: mb s3://farm4343lb # [ 30.074258] systemd[1]: Finished rustfs-bucket.service.4344lb # [ 30.080434] systemd[1]: Starting niks3 server...4345worker2 # [ 30.157353] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514346worker2 # [ 30.161210] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4347worker1 # [ 30.174671] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514348worker1 # [ 30.178805] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349lb # [ 30.287773] postgres[1061]: [1061] ERROR: relation "goose_db_version" does not exist at character 364350lb # [ 30.289102] postgres[1061]: [1061] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4351lb # [ 30.328777] niks3-server[1054]: 2026/09/20 09:33:25 OK 20241026095416_initial_model.sql (20.22ms)4352lb # [ 30.332229] niks3-server[1054]: 2026/09/20 09:33:25 OK 20251210153512_drop_unused_gin_index.sql (3.58ms)4353lb # [ 30.337882] niks3-server[1054]: 2026/09/20 09:33:25 OK 20251218171726_add_pins.sql (4.46ms)4354lb # [ 30.342945] niks3-server[1054]: 2026/09/20 09:33:25 OK 20260628120000_add_object_size_and_stats.sql (3.84ms)4355lb # [ 30.347243] niks3-server[1054]: 2026/09/20 09:33:25 OK 20260905000000_add_claims.sql (2.93ms)4356lb # [ 30.351194] niks3-server[1054]: 2026/09/20 09:33:25 OK 20260920000000_drop_claims.sql (4.02ms)4357lb # [ 30.352470] niks3-server[1054]: 2026/09/20 09:33:25 goose: successfully migrated database to version: 202609200000004358worker2 # [ 30.363323] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514359lb # [ 30.358564] niks3-server[1054]: 2026/09/20 09:33:25 OK 1_commit_pending_closure.sql (4.8ms)4360worker2 # [ 30.365871] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4361lb # [ 30.361495] niks3-server[1054]: 2026/09/20 09:33:25 OK 2_object_stats_trigger.sql (1.29ms)4362lb # [ 30.362691] niks3-server[1054]: 2026/09/20 09:33:25 goose: up to current file version: 24363worker1 # [ 30.378105] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514364worker1 # [ 30.381248] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4365lb # [ 30.368370] niks3-server[1054]: 2026/09/20 09:33:25 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4366lb # [ 30.430832] niks3-server[1054]: 2026/09/20 09:33:25 INFO Read proxy enabled — serving cache objects from S34367lb # [ 30.432529] niks3-server[1054]: 2026/09/20 09:33:25 INFO Using socket-activated listener address=0.0.0.0:57514368lb # [ 30.437123] niks3-server[1054]: 2026/09/20 09:33:25 INFO systemd watchdog enabled interval=15s4369lb # [ 30.438280] systemd[1]: Started niks3 server.4370lb # [ 30.438904] systemd[1]: Reached target Multi-User System.4371lb # [ 30.440823] systemd[1]: Startup finished in 867ms (kernel) + 5.038s (initrd) + 24.534s (userspace) = 30.440s.4372lb # [ 30.442733] niks3-server[1054]: 2026/09/20 09:33:25 INFO Starting HTTP server address=0.0.0.0:57514373lb # [ 30.448241] niks3-server[1054]: 2026/09/20 09:33:25 INFO lead: acquired remote=192.168.1.4:496864374worker2 # [ 30.461677] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_take_over4375worker2 # [ 30.563863] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514376worker2 # [ 30.567744] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4377worker1 # [ 30.578715] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514378worker1 # [ 30.582635] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4379lb: (finished: waiting for unit niks3.service, in 19.64 seconds)4380lb: waiting for TCP port 5751 on localhost4381worker2 # [ 30.768785] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514382worker2 # [ 30.772667] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4383worker1 # [ 30.783758] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_connected addr=lb:500514384worker1 # [ 30.787843] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4385lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4386lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds)4387lb: waiting for unit envoy.service4388lb: (finished: waiting for unit envoy.service, in 0.06 seconds)4389lb: waiting for TCP port 50051 on localhost4390lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4391lb # [ 30.902346] envoy[926]: conn peer=[::1]:53650 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34392lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4393worker2 # [ 30.973658] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=scheduler_connected addr=lb:500514394worker2 # [ 30.977697] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4395worker1 # [ 30.990467] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=scheduler_connected addr=lb:500514396worker1 # [ 30.993658] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4397lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4398??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4399 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284400lb: must succeed: curl -sf localhost:9901/clusters4401lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4402lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)4403subtest: one node schedules and both builders hold a WorkerSession on it4404lb: must succeed: curl -sf localhost:9901/clusters4405lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4406worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4407worker2 # [ 31.178496] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=scheduler_connected addr=lb:500514408worker1 # [ 31.193579] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=scheduler_connected addr=lb:500514409worker2 # [ 31.193550] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04410worker2 # [ 31.199892] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04411worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.10 seconds)4412(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.14 seconds)4413subtest: balancer auth: client cert, bearer token, nothing, unknown CN4414worker1 # [ 31.604217] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380384415worker1 # [ 31.616456] systemd[1]: Started Nix Daemon.4416worker1 # [ 31.842696] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4417worker1 # [ 31.886775] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)4418worker1 # [ 31.962794] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 142 ms (attempt 2/5)4419worker1 # [ 32.110893] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 3/5)4420worker1 # [ 32.132954] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 4/5)4421worker1 # [ 32.151356] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4422worker1 # [ 32.179830] nix-grpc-daemon[784]: ts=2026-09-20T09:33:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380384423worker1 # [ 32.199133] nix-grpc-daemon[784]: ts=2026-09-20T09:33:28Z level=debug event=path_info_miss path=3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test error="error: path '/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test' is required, but there is no substituter that can build it"4424worker2 # [ 32.201786] nix-grpc-daemon[784]: ts=2026-09-20T09:33:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329744425worker2 # [ 32.214981] systemd[1]: Started Nix Daemon.4426worker2 # [ 32.410948] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4427worker2 # [ 32.438621] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)4428worker2 # [ 32.469053] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 192 ms (attempt 2/5)4429worker2 # [ 32.663769] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 3/5)4430worker2 # [ 32.809754] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 561 ms (attempt 4/5)4431worker2 # [ 33.380511] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4432worker2 # [ 33.393209] nix-grpc-daemon[784]: ts=2026-09-20T09:33:29Z level=debug event=path_info_miss path=3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test error="error: path '/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test' is required, but there is no substituter that can build it"4433client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4434client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)4435client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4436client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4437worker1 # [ 33.794307] nix-grpc-daemon[784]: ts=2026-09-20T09:33:29Z level=info event=oidc_jwks provider=mock keys=14438worker1 # [ 33.798503] nix-grpc-daemon[784]: ts=2026-09-20T09:33:29Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:380384439worker1 # [ 33.807882] nix-grpc-daemon[784]: ts=2026-09-20T09:33:29Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:380384440worker1 # [ 33.816956] nix-grpc-daemon[784]: ts=2026-09-20T09:33:29Z level=debug event=path_info_miss path=3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test error="error: path '/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test' is required, but there is no substituter that can build it"4441worker1 # [ 33.826837] nix-grpc-daemon[784]: ts=2026-09-20T09:33:29Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:380384442worker1 # [ 33.835984] nix-grpc-daemon[784]: ts=2026-09-20T09:33:29Z level=debug event=path_info_miss path=3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test error="error: path '/nix/store/3yn33ipjwn92p38w4qwdnvyyq096zfzd-nixos-system-client-test' is required, but there is no substituter that can build it"4443client: must succeed: rm -r /var/lib/nix-grpc-store4444client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4445worker2 # [ 36.079763] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=info event=denied method=QueryMissing cn=- role=none4446worker2 # [ 36.297886] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=info event=denied method=QueryMissing cn=stranger role=none4447(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.06 seconds)4448subtest: DAG build is scheduled across workers and lands in the cache4449client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24450client # don't know how to build these paths:4451client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4452worker1 # [ 36.541942] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380384453worker1 # [ 36.550135] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380384454worker2 # [ 36.568274] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04455worker2 # [ 36.572874] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14456worker1 # [ 36.633769] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384457worker1 # [ 36.655448] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380384458client # copying 1 paths to 'lb:50051'...4459worker1 # [ 36.667133] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380384460client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4461worker1 # [ 36.695484] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4462worker1 # [ 36.721866] nix-grpc-daemon[784]: ts=2026-09-20T09:33:32Z level=info event=niks3_push_started pid=8534463worker1 # [ 37.219801] nix-daemon[842]: accepted connection from pid 865, user nix-grpc-daemon (trusted)4464worker1 # [ 37.236521] nix-daemon[842]: reaped child process 868, status = succeeded4465worker1 # [ 37.411576] nix-daemon[842]: accepted connection from pid 871, user nix-grpc-daemon (trusted)4466worker1 # [ 37.427656] nix-daemon[842]: reaped child process 874, status = succeeded4467lb # [ 37.427709] niks3-server[1054]: 2026/09/20 09:33:32 INFO Received uploads request method=POST path=/api/pending_closures4468worker1 # [ 37.474487] nix-grpc-daemon[853]: time=2026-09-20T09:33:33.513Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4469worker1 # [ 37.477646] nix-grpc-daemon[853]: time=2026-09-20T09:33:33.517Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4470lb # [ 37.509855] niks3-server[1054]: 2026/09/20 09:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4471lb # [ 37.521351] niks3-server[1054]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4472lb # [ 37.535540] niks3-server[1054]: 2026/09/20 09:33:32 INFO Signed narinfos id=1 count=14473worker1 # [ 37.554253] nix-grpc-daemon[853]: time=2026-09-20T09:33:33.594Z level=INFO msg="Uploading 1 narinfos"4474lb # [ 37.542751] niks3-server[1054]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4475lb # [ 37.547966] niks3-server[1054]: 2026/09/20 09:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4476lb # [ 37.562397] niks3-server[1054]: 2026/09/20 09:33:32 INFO Completed upload id=14477worker1 # [ 37.579478] nix-grpc-daemon[853]: time=2026-09-20T09:33:33.620Z level=INFO msg="Upload complete. (545ms)"4478worker1 # [ 37.582095] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_in=5044479worker1 # [ 37.586154] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384480lb # [ 37.571391] niks3-server[1054]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4481worker1 # [ 37.592289] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4482worker1 # [ 37.594632] nix-daemon[842]: reaped child process 851, status = succeeded4483client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4484worker1 # [ 37.601298] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4485client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4486lb # [ 38.787068] envoy[926]: conn peer=[2001:db8:1::1]:34300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=18614487lb # [ 38.790755] envoy[926]: conn peer=[2001:db8:1::1]:41354 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=388 ms=734488lb # [ 38.794064] envoy[926]: conn peer=[2001:db8:1::1]:41364 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=144489lb # [ 38.797343] envoy[926]: conn peer=[2001:db8:1::1]:41366 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=104490client # farm-a> LOG-farm-a-t14491worker1 # [ 39.221691] nix-daemon[842]: reaped child process 878, status = succeeded4492worker1 # [ 39.371273] nix-daemon[842]: accepted connection from pid 894, user nix-grpc-daemon (trusted)4493worker1 # [ 39.383652] nix-daemon[842]: reaped child process 897, status = succeeded4494lb # [ 39.382943] niks3-server[1054]: 2026/09/20 09:33:34 INFO Received uploads request method=POST path=/api/pending_closures4495worker1 # [ 39.411797] nix-grpc-daemon[853]: time=2026-09-20T09:33:35.452Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4496worker1 # [ 39.413418] nix-grpc-daemon[853]: time=2026-09-20T09:33:35.453Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4497lb # [ 39.428477] niks3-server[1054]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4498lb # [ 39.439920] niks3-server[1054]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4499lb # [ 39.447237] niks3-server[1054]: 2026/09/20 09:33:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4500lb # [ 39.450751] niks3-server[1054]: 2026/09/20 09:33:34 INFO Signed narinfos id=2 count=14501worker1 # [ 39.470445] nix-grpc-daemon[853]: time=2026-09-20T09:33:35.510Z level=INFO msg="Uploading 1 narinfos"4502lb # [ 39.457876] niks3-server[1054]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4503lb # [ 39.468448] niks3-server[1054]: 2026/09/20 09:33:34 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4504lb # [ 39.472699] niks3-server[1054]: 2026/09/20 09:33:34 INFO Completed upload id=24505worker1 # [ 39.489787] nix-grpc-daemon[853]: time=2026-09-20T09:33:35.530Z level=INFO msg="Upload complete. (279ms)"4506lb # [ 39.474142] niks3-server[1054]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4507worker1 # [ 39.495817] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14508worker1 # [ 39.508405] nix-daemon[842]: reaped child process 876, status = succeeded4509worker2 # [ 39.503812] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04510worker2 # [ 39.508695] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14511worker1 # [ 39.557704] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384512worker1 # [ 39.571363] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380384513worker1 # [ 39.585778] nix-grpc-daemon[853]: time=2026-09-20T09:33:35.626Z level=INFO msg="All 1 paths already cached"4514client # copying 1 paths to 'lb:50051'...4515worker1 # [ 39.593902] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380384516client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4517worker1 # [ 39.613165] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4518worker1 # [ 39.790654] nix-daemon[842]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4519worker1 # [ 39.804138] nix-daemon[842]: reaped child process 904, status = succeeded4520worker1 # [ 39.961366] nix-daemon[842]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4521worker1 # [ 39.973570] nix-daemon[842]: reaped child process 909, status = succeeded4522lb # [ 39.971317] niks3-server[1054]: 2026/09/20 09:33:35 INFO Received uploads request method=POST path=/api/pending_closures4523worker1 # [ 39.998696] nix-grpc-daemon[853]: time=2026-09-20T09:33:36.038Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4524worker1 # [ 40.000097] nix-grpc-daemon[853]: time=2026-09-20T09:33:36.038Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4525lb # [ 40.001646] niks3-server[1054]: 2026/09/20 09:33:35 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4526lb # [ 40.010325] niks3-server[1054]: 2026/09/20 09:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4527worker1 # [ 40.032406] nix-grpc-daemon[853]: time=2026-09-20T09:33:36.072Z level=INFO msg="Uploading 1 narinfos"4528lb # [ 40.016121] niks3-server[1054]: 2026/09/20 09:33:35 INFO Signed narinfos id=3 count=14529lb # [ 40.022133] niks3-server[1054]: 2026/09/20 09:33:35 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4530lb # [ 40.033780] niks3-server[1054]: 2026/09/20 09:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4531lb # [ 40.045683] niks3-server[1054]: 2026/09/20 09:33:35 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4532lb # [ 40.051107] niks3-server[1054]: 2026/09/20 09:33:35 INFO Completed upload id=34533worker1 # [ 40.068110] nix-grpc-daemon[853]: time=2026-09-20T09:33:36.108Z level=INFO msg="Upload complete. (441ms)"4534worker1 # [ 40.073163] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_in=6244535worker1 # [ 40.079299] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384536worker1 # [ 40.090132] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4537worker1 # [ 40.101613] nix-daemon[842]: reaped child process 899, status = succeeded4538worker1 # [ 40.104762] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4539client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4540client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4541client # farm-b> LOG-farm-b-t14542worker1 # [ 41.531236] nix-daemon[842]: reaped child process 913, status = succeeded4543worker1 # [ 41.686972] nix-daemon[842]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4544worker1 # [ 41.699171] nix-daemon[842]: reaped child process 931, status = succeeded4545lb # [ 41.699449] niks3-server[1054]: 2026/09/20 09:33:36 INFO Received uploads request method=POST path=/api/pending_closures4546worker1 # [ 41.724992] nix-grpc-daemon[853]: time=2026-09-20T09:33:37.765Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4547worker1 # [ 41.726517] nix-grpc-daemon[853]: time=2026-09-20T09:33:37.767Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4548lb # [ 41.739119] niks3-server[1054]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4549lb # [ 41.744385] niks3-server[1054]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4550lb # [ 41.756566] niks3-server[1054]: 2026/09/20 09:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4551lb # [ 41.762061] niks3-server[1054]: 2026/09/20 09:33:37 INFO Signed narinfos id=4 count=14552worker1 # [ 41.781761] nix-grpc-daemon[853]: time=2026-09-20T09:33:37.822Z level=INFO msg="Uploading 1 narinfos"4553lb # [ 41.767227] niks3-server[1054]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4554lb # [ 41.777094] niks3-server[1054]: 2026/09/20 09:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4555lb # [ 41.782581] niks3-server[1054]: 2026/09/20 09:33:37 INFO Completed upload id=44556worker1 # [ 41.799523] nix-grpc-daemon[853]: time=2026-09-20T09:33:37.840Z level=INFO msg="Upload complete. (285ms)"4557lb # [ 41.785465] niks3-server[1054]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4558worker2 # [ 41.799916] nix-grpc-daemon[784]: ts=2026-09-20T09:33:37Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04559worker1 # [ 41.806115] nix-grpc-daemon[784]: ts=2026-09-20T09:33:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14560worker2 # [ 41.801663] nix-grpc-daemon[784]: ts=2026-09-20T09:33:37Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14561worker1 # [ 41.819582] nix-daemon[842]: reaped child process 911, status = succeeded4562worker1 # [ 41.854177] nix-grpc-daemon[784]: ts=2026-09-20T09:33:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384563worker1 # [ 41.866872] nix-grpc-daemon[784]: ts=2026-09-20T09:33:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380384564worker1 # [ 41.880140] nix-grpc-daemon[853]: time=2026-09-20T09:33:37.920Z level=INFO msg="All 2 paths already cached"4565client # copying 1 paths to 'lb:50051'...4566worker1 # [ 41.888186] nix-grpc-daemon[784]: ts=2026-09-20T09:33:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380384567client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4568worker1 # [ 41.908143] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4569worker1 # [ 42.052245] nix-daemon[842]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4570worker1 # [ 42.064316] nix-daemon[842]: reaped child process 938, status = succeeded4571worker1 # [ 42.218797] nix-daemon[842]: accepted connection from pid 940, user nix-grpc-daemon (trusted)4572worker1 # [ 42.234271] nix-daemon[842]: reaped child process 943, status = succeeded4573lb # [ 42.227435] niks3-server[1054]: 2026/09/20 09:33:37 INFO Received uploads request method=POST path=/api/pending_closures4574worker1 # [ 42.249399] nix-grpc-daemon[853]: time=2026-09-20T09:33:38.290Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4575worker1 # [ 42.250940] nix-grpc-daemon[853]: time=2026-09-20T09:33:38.291Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4576lb # [ 42.244165] niks3-server[1054]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4577lb # [ 42.247139] niks3-server[1054]: 2026/09/20 09:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4578lb # [ 42.248850] niks3-server[1054]: 2026/09/20 09:33:37 INFO Signed narinfos id=5 count=14579worker1 # [ 42.265812] nix-grpc-daemon[853]: time=2026-09-20T09:33:38.306Z level=INFO msg="Uploading 1 narinfos"4580lb # [ 42.253217] niks3-server[1054]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4581lb # [ 42.256298] niks3-server[1054]: 2026/09/20 09:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4582lb # [ 42.260313] niks3-server[1054]: 2026/09/20 09:33:37 INFO Completed upload id=54583worker1 # [ 42.277130] nix-grpc-daemon[853]: time=2026-09-20T09:33:38.317Z level=INFO msg="Upload complete. (357ms)"4584lb # [ 42.261752] niks3-server[1054]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4585worker1 # [ 42.279028] nix-grpc-daemon[784]: ts=2026-09-20T09:33:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_in=7524586worker1 # [ 42.283290] nix-grpc-daemon[784]: ts=2026-09-20T09:33:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384587worker1 # [ 42.288638] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4588worker1 # [ 42.290808] nix-daemon[842]: reaped child process 933, status = succeeded4589worker1 # [ 42.296045] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4590client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4591client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4592client # farm-top> LOG-farm-top-t14593worker1 # [ 43.318187] nix-daemon[842]: reaped child process 948, status = succeeded4594worker1 # [ 43.450744] nix-daemon[842]: accepted connection from pid 963, user nix-grpc-daemon (trusted)4595worker1 # [ 43.462317] nix-daemon[842]: reaped child process 966, status = succeeded4596lb # [ 43.457510] niks3-server[1054]: 2026/09/20 09:33:38 INFO Received uploads request method=POST path=/api/pending_closures4597worker1 # [ 43.482687] nix-grpc-daemon[853]: time=2026-09-20T09:33:39.523Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4598worker1 # [ 43.484326] nix-grpc-daemon[853]: time=2026-09-20T09:33:39.524Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4599lb # [ 43.501287] niks3-server[1054]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4600lb # [ 43.506571] niks3-server[1054]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4601lb # [ 43.516124] niks3-server[1054]: 2026/09/20 09:33:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4602lb # [ 43.523389] niks3-server[1054]: 2026/09/20 09:33:38 INFO Signed narinfos id=6 count=14603worker1 # [ 43.543236] nix-grpc-daemon[853]: time=2026-09-20T09:33:39.583Z level=INFO msg="Uploading 1 narinfos"4604lb # [ 43.527565] niks3-server[1054]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4605lb # [ 43.537868] niks3-server[1054]: 2026/09/20 09:33:38 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4606lb # [ 43.540896] niks3-server[1054]: 2026/09/20 09:33:38 INFO Completed upload id=64607lb # [ 43.542388] niks3-server[1054]: 2026/09/20 09:33:38 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4608worker1 # [ 43.559136] nix-grpc-daemon[853]: time=2026-09-20T09:33:39.599Z level=INFO msg="Upload complete. (255ms)"4609worker1 # [ 43.564822] nix-grpc-daemon[784]: ts=2026-09-20T09:33:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14610worker1 # [ 43.577840] nix-daemon[842]: reaped child process 946, status = succeeded4611client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.28 seconds)4612client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4613client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4614client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4615client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4616client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4617client # copying 3 paths...4618client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4619client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4620client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4621client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4622(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.85 seconds)4623subtest: repeat is answered Cached by the scheduler without building4624worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4625worker1 # finding garbage collector roots...4626worker1 # deleting specified paths...4627worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4628worker1 # deleting unused links...4629worker1 # note: hard linking is currently saving 0.0 KiB4630worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4631worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4632worker2 # finding garbage collector roots...4633worker2 # deleting specified paths...4634worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4635worker2 # deleting unused links...4636worker2 # note: hard linking is currently saving 0.0 KiB4637worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4638client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24639worker2 # [ 44.899476] nix-grpc-daemon[784]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329744640client # don't know how to build these paths:4641client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4642worker2 # [ 44.907263] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329744643worker2 # [ 44.928626] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14644client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.27 seconds)4645client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4646client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4647lb: must succeed: curl -sf localhost:9901/clusters4648lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4649lb: must succeed: curl -sf localhost:9901/clusters4650lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4651worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4652worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.07 seconds)4653(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.22 seconds)4654subtest: inputs already in the cache are not sent to the scheduler4655lb: must succeed: curl -sf localhost:9901/clusters4656lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4657lb: must succeed: curl -sf localhost:9901/clusters4658lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4659worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4660worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4661client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24662client # don't know how to build these paths:4663client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4664worker1 # [ 45.743279] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380384665worker2 # [ 45.737457] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329744666worker2 # [ 45.750071] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04667worker2 # [ 45.754223] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14668worker1 # [ 45.803466] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384669worker1 # [ 45.814812] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380384670worker1 # [ 45.826662] nix-grpc-daemon[853]: time=2026-09-20T09:33:41.866Z level=INFO msg="All 2 paths already cached"4671client # copying 1 paths to 'lb:50051'...4672client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4673worker1 # [ 45.834659] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380384674worker1 # [ 45.848373] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4675worker1 # [ 46.031394] nix-daemon[842]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4676worker1 # [ 46.046547] nix-daemon[842]: reaped child process 986, status = succeeded4677worker1 # [ 46.219979] nix-daemon[842]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4678worker1 # [ 46.231702] nix-daemon[842]: reaped child process 991, status = succeeded4679lb # [ 46.230691] niks3-server[1054]: 2026/09/20 09:33:41 INFO Received uploads request method=POST path=/api/pending_closures4680worker1 # [ 46.255260] nix-grpc-daemon[853]: time=2026-09-20T09:33:42.295Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4681worker1 # [ 46.256830] nix-grpc-daemon[853]: time=2026-09-20T09:33:42.297Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4682lb # [ 46.264167] niks3-server[1054]: 2026/09/20 09:33:41 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4683lb # [ 46.272987] niks3-server[1054]: 2026/09/20 09:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4684worker1 # [ 46.294428] nix-grpc-daemon[853]: time=2026-09-20T09:33:42.334Z level=INFO msg="Uploading 1 narinfos"4685lb # [ 46.277893] niks3-server[1054]: 2026/09/20 09:33:41 INFO Signed narinfos id=7 count=14686lb # [ 46.282985] niks3-server[1054]: 2026/09/20 09:33:41 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4687lb # [ 46.294076] niks3-server[1054]: 2026/09/20 09:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4688lb # [ 46.299867] niks3-server[1054]: 2026/09/20 09:33:41 INFO Completed upload id=74689worker1 # [ 46.316893] nix-grpc-daemon[853]: time=2026-09-20T09:33:42.357Z level=INFO msg="Upload complete. (453ms)"4690worker1 # [ 46.319239] nix-grpc-daemon[784]: ts=2026-09-20T09:33:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_in=7524691lb # [ 46.304354] niks3-server[1054]: 2026/09/20 09:33:41 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4692worker1 # [ 46.325375] nix-grpc-daemon[784]: ts=2026-09-20T09:33:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384693worker1 # [ 46.334673] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4694worker1 # [ 46.336042] nix-daemon[842]: reaped child process 981, status = succeeded4695worker1 # [ 46.341615] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4696client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4697client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4698worker1 # [ 46.955197] nix-daemon[842]: reaped child process 995, status = succeeded4699client # farm-top> LOG-farm-top-t1b4700worker1 # [ 47.079824] nix-daemon[842]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)4701worker1 # [ 47.092875] nix-daemon[842]: reaped child process 1013, status = succeeded4702lb # [ 47.086330] niks3-server[1054]: 2026/09/20 09:33:42 INFO Received uploads request method=POST path=/api/pending_closures4703worker1 # [ 47.112721] nix-grpc-daemon[853]: time=2026-09-20T09:33:43.153Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4704worker1 # [ 47.114263] nix-grpc-daemon[853]: time=2026-09-20T09:33:43.154Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4705lb # [ 47.126063] niks3-server[1054]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4706lb # [ 47.130979] niks3-server[1054]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4707lb # [ 47.144941] niks3-server[1054]: 2026/09/20 09:33:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4708worker1 # [ 47.167074] nix-grpc-daemon[853]: time=2026-09-20T09:33:43.207Z level=INFO msg="Uploading 1 narinfos"4709lb # [ 47.150117] niks3-server[1054]: 2026/09/20 09:33:42 INFO Signed narinfos id=8 count=14710lb # [ 47.155727] niks3-server[1054]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4711lb # [ 47.165414] niks3-server[1054]: 2026/09/20 09:33:42 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4712lb # [ 47.171780] niks3-server[1054]: 2026/09/20 09:33:42 INFO Completed upload id=84713worker1 # [ 47.188641] nix-grpc-daemon[853]: time=2026-09-20T09:33:43.229Z level=INFO msg="Upload complete. (240ms)"4714lb # [ 47.173428] niks3-server[1054]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4715worker1 # [ 47.192259] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14716worker1 # [ 47.198198] nix-daemon[842]: reaped child process 993, status = succeeded4717client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.70 seconds)4718client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4719client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds)4720lb: must succeed: curl -sf localhost:9901/clusters4721lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4722lb: must succeed: curl -sf localhost:9901/clusters4723lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4724worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4725worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4726(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.16 seconds)4727subtest: two clients wanting the same drv build it once4728client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4729client # [ 47.601760] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4730client # Running as unit: dedup1.service; invocation ID: 13cd3fe63e8b4a1bb906107e881497104731client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.06 seconds)4732client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4733worker1 # [ 47.806545] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380384734client # [ 47.806858] nix[888]: don't know how to build these paths:4735client # [ 47.807908] nix[888]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4736worker2 # [ 47.808385] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04737worker2 # [ 47.810300] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14738worker2 # [ 47.816291] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329744739worker2 # [ 47.829608] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329744740client # [ 47.841652] nix[888]: copying 1 paths to 'lb:50051'...4741client # [ 47.842491] nix[888]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4742worker2 # [ 47.838421] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329744743worker2 # [ 47.849443] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4744worker2 # [ 47.868941] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=info event=niks3_push_started pid=9004745worker2 # [ 48.297650] nix-daemon[847]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4746worker2 # [ 48.313565] nix-daemon[847]: reaped child process 915, status = succeeded4747worker2 # [ 48.464144] nix-daemon[847]: accepted connection from pid 917, user nix-grpc-daemon (trusted)4748worker2 # [ 48.476937] nix-daemon[847]: reaped child process 920, status = succeeded4749lb # [ 48.482077] niks3-server[1054]: 2026/09/20 09:33:43 INFO Received uploads request method=POST path=/api/pending_closures4750worker2 # [ 48.495670] nix-grpc-daemon[900]: time=2026-09-20T09:33:44.594Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4751worker2 # [ 48.497933] nix-grpc-daemon[900]: time=2026-09-20T09:33:44.596Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4752lb # [ 48.506668] niks3-server[1054]: 2026/09/20 09:33:43 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4753lb # [ 48.509644] niks3-server[1054]: 2026/09/20 09:33:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4754lb # [ 48.511432] niks3-server[1054]: 2026/09/20 09:33:43 INFO Signed narinfos id=9 count=14755worker2 # [ 48.520327] nix-grpc-daemon[900]: time=2026-09-20T09:33:44.620Z level=INFO msg="Uploading 1 narinfos"4756lb # [ 48.518661] niks3-server[1054]: 2026/09/20 09:33:43 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4757lb # [ 48.522417] niks3-server[1054]: 2026/09/20 09:33:43 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4758lb # [ 48.525400] niks3-server[1054]: 2026/09/20 09:33:43 INFO Completed upload id=94759worker2 # [ 48.534212] nix-grpc-daemon[900]: time=2026-09-20T09:33:44.634Z level=INFO msg="Upload complete. (411ms)"4760lb # [ 48.527574] niks3-server[1054]: 2026/09/20 09:33:43 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4761worker2 # [ 48.536499] nix-grpc-daemon[784]: ts=2026-09-20T09:33:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=0 paths=1 nar_bytes_in=4324762worker2 # [ 48.539557] nix-grpc-daemon[784]: ts=2026-09-20T09:33:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329744763worker2 # [ 48.546083] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4764worker2 # [ 48.548520] nix-daemon[847]: reaped child process 898, status = succeeded4765worker2 # [ 48.553838] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4766client # [ 48.557099] nix[888]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4767client # [ 48.677856] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4768client # Running as unit: dedup2.service; invocation ID: 381862001b0740089b7e7790aca1e7a84769client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4770lb # [ 48.786605] envoy[926]: conn peer=[2001:db8:1::1]:41382 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=3817 ms=70704771lb # [ 48.788257] envoy[926]: conn peer=[2001:db8:1::1]:38498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=424772lb # [ 48.789638] envoy[926]: conn peer=[2001:db8:1::1]:38510 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4139 tx=1710 ms=14634773client # [ 48.927077] nix[903]: don't know how to build these paths:4774client # [ 48.928485] nix[903]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4775worker1 # [ 48.932836] nix-grpc-daemon[784]: ts=2026-09-20T09:33:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380384776worker2 # [ 48.932641] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04777worker2 # [ 48.941188] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329744778client # [ 48.949833] nix[903]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4779client: must succeed: sleep 24780client: (finished: must succeed: sleep 2, in 2.05 seconds)4781??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4782 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394783client: waiting for success: ! systemctl is-active dedup1 dedup24784??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4785 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394786worker2 # [ 52.169724] nix-daemon[847]: reaped child process 924, status = succeeded4787worker2 # [ 52.313386] nix-daemon[847]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4788worker2 # [ 52.324597] nix-daemon[847]: reaped child process 959, status = succeeded4789lb # [ 52.332901] niks3-server[1054]: 2026/09/20 09:33:47 INFO Received uploads request method=POST path=/api/pending_closures4790worker2 # [ 52.348520] nix-grpc-daemon[900]: time=2026-09-20T09:33:48.448Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4791worker2 # [ 52.349972] nix-grpc-daemon[900]: time=2026-09-20T09:33:48.450Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4792lb # [ 52.373658] niks3-server[1054]: 2026/09/20 09:33:47 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4793lb # [ 52.390229] niks3-server[1054]: 2026/09/20 09:33:47 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4794lb # [ 52.398862] niks3-server[1054]: 2026/09/20 09:33:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4795lb # [ 52.403905] niks3-server[1054]: 2026/09/20 09:33:47 INFO Signed narinfos id=10 count=14796worker2 # [ 52.414330] nix-grpc-daemon[900]: time=2026-09-20T09:33:48.513Z level=INFO msg="Uploading 1 narinfos"4797lb # [ 52.406413] niks3-server[1054]: 2026/09/20 09:33:47 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4798lb # [ 52.417393] niks3-server[1054]: 2026/09/20 09:33:47 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4799lb # [ 52.421722] niks3-server[1054]: 2026/09/20 09:33:47 INFO Completed upload id=104800lb # [ 52.424356] niks3-server[1054]: 2026/09/20 09:33:47 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4801worker2 # [ 52.430796] nix-grpc-daemon[900]: time=2026-09-20T09:33:48.530Z level=INFO msg="Upload complete. (266ms)"4802worker2 # [ 52.439956] nix-grpc-daemon[784]: ts=2026-09-20T09:33:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14803worker2 # [ 52.446406] nix-grpc-daemon[784]: ts=2026-09-20T09:33:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14804worker2 # [ 52.457210] nix-daemon[847]: reaped child process 922, status = succeeded4805client # [ 52.490402] systemd[1]: dedup1.service: Deactivated successfully.4806client # [ 52.494632] systemd[1]: dedup1.service: Consumed 115ms CPU time over 4.891s wall clock time, 39.3M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.4807client # [ 52.502348] systemd[1]: dedup2.service: Deactivated successfully.4808client # [ 52.504433] systemd[1]: dedup2.service: Consumed 128ms CPU time over 3.826s wall clock time, 31.2M memory peak, 7.5K incoming IP traffic, 6.4K outgoing IP traffic.4809client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4810client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4811client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4812worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4813worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4814worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4815worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4816worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4817worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.05 seconds)4818worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4819worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.06 seconds)4820(finished: subtest: two clients wanting the same drv build it once, in 5.95 seconds)4821subtest: build hook: nix-daemon with builders = grpc://lb4822client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4823client # [ 53.674320] systemd[1]: Started Nix Daemon.4824client # [ 53.846932] nix-daemon[944]: accepted connection from pid 941, user root (trusted)4825worker1 # [ 54.034668] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:380384826worker2 # [ 54.031802] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329744827worker2 # [ 54.040529] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04828worker2 # [ 54.042686] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14829worker1 # [ 54.093727] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384830worker1 # [ 54.102208] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380384831worker1 # [ 54.110412] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380384832worker1 # [ 54.121539] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4833worker1 # [ 54.277821] nix-daemon[842]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)4834worker1 # [ 54.290899] nix-daemon[842]: reaped child process 1043, status = succeeded4835worker1 # [ 54.462301] nix-daemon[842]: accepted connection from pid 1045, user nix-grpc-daemon (trusted)4836worker1 # [ 54.474509] nix-daemon[842]: reaped child process 1048, status = succeeded4837lb # [ 54.474897] niks3-server[1054]: 2026/09/20 09:33:49 INFO Received uploads request method=POST path=/api/pending_closures4838worker1 # [ 54.501902] nix-grpc-daemon[853]: time=2026-09-20T09:33:50.541Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4839worker1 # [ 54.503320] nix-grpc-daemon[853]: time=2026-09-20T09:33:50.542Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4840lb # [ 54.503098] niks3-server[1054]: 2026/09/20 09:33:49 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4841lb # [ 54.511778] niks3-server[1054]: 2026/09/20 09:33:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4842worker1 # [ 54.533659] nix-grpc-daemon[853]: time=2026-09-20T09:33:50.574Z level=INFO msg="Uploading 1 narinfos"4843lb # [ 54.518241] niks3-server[1054]: 2026/09/20 09:33:49 INFO Signed narinfos id=11 count=14844lb # [ 54.523310] niks3-server[1054]: 2026/09/20 09:33:49 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4845lb # [ 54.535754] niks3-server[1054]: 2026/09/20 09:33:49 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4846lb # [ 54.543595] niks3-server[1054]: 2026/09/20 09:33:49 INFO Completed upload id=114847worker1 # [ 54.560428] nix-grpc-daemon[853]: time=2026-09-20T09:33:50.600Z level=INFO msg="Upload complete. (433ms)"4848lb # [ 54.546315] niks3-server[1054]: 2026/09/20 09:33:49 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4849worker1 # [ 54.564118] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_in=5124850worker1 # [ 54.569236] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384851worker1 # [ 54.579233] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4852worker1 # [ 54.582249] nix-daemon[842]: reaped child process 1038, status = succeeded4853worker1 # [ 54.587356] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4854worker1 # [ 55.600809] nix-daemon[842]: reaped child process 1052, status = succeeded4855worker1 # [ 55.745087] nix-daemon[842]: accepted connection from pid 1067, user nix-grpc-daemon (trusted)4856worker1 # [ 55.755995] nix-daemon[842]: reaped child process 1070, status = succeeded4857lb # [ 55.754872] niks3-server[1054]: 2026/09/20 09:33:51 INFO Received uploads request method=POST path=/api/pending_closures4858worker1 # [ 55.779753] nix-grpc-daemon[853]: time=2026-09-20T09:33:51.820Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4859worker1 # [ 55.783058] nix-grpc-daemon[853]: time=2026-09-20T09:33:51.821Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4860lb # [ 55.797979] niks3-server[1054]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4861lb # [ 55.802520] niks3-server[1054]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4862lb # [ 55.813718] niks3-server[1054]: 2026/09/20 09:33:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4863lb # [ 55.819680] niks3-server[1054]: 2026/09/20 09:33:51 INFO Signed narinfos id=12 count=14864worker1 # [ 55.839887] nix-grpc-daemon[853]: time=2026-09-20T09:33:51.880Z level=INFO msg="Uploading 1 narinfos"4865lb # [ 55.824199] niks3-server[1054]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4866lb # [ 55.834199] niks3-server[1054]: 2026/09/20 09:33:51 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4867lb # [ 55.838821] niks3-server[1054]: 2026/09/20 09:33:51 INFO Completed upload id=124868worker1 # [ 55.855864] nix-grpc-daemon[853]: time=2026-09-20T09:33:51.896Z level=INFO msg="Upload complete. (262ms)"4869lb # [ 55.842545] niks3-server[1054]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4870worker1 # [ 55.862132] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14871worker2 # [ 55.857859] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329744872worker1 # [ 55.874209] nix-daemon[842]: reaped child process 1050, status = succeeded4873worker2 # [ 55.870329] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4874worker1 # [ 55.919640] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:380384875worker1 # [ 55.925292] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_out=1284876worker1 # [ 56.127967] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:380384877worker2 # [ 56.121327] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329744878worker2 # [ 56.130792] nix-grpc-daemon[900]: time=2026-09-20T09:33:52.230Z level=INFO msg="All 1 paths already cached"4879worker2 # [ 56.144256] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04880worker2 # [ 56.148286] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14881worker1 # [ 56.164913] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384882worker1 # [ 56.176189] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380384883worker1 # [ 56.188370] nix-grpc-daemon[853]: time=2026-09-20T09:33:52.228Z level=INFO msg="All 1 paths already cached"4884worker1 # [ 56.194597] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380384885worker1 # [ 56.204157] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4886worker1 # [ 56.373943] nix-daemon[842]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4887worker1 # [ 56.386899] nix-daemon[842]: reaped child process 1077, status = succeeded4888worker1 # [ 56.541586] nix-daemon[842]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4889worker1 # [ 56.554165] nix-daemon[842]: reaped child process 1082, status = succeeded4890lb # [ 56.549372] niks3-server[1054]: 2026/09/20 09:33:51 INFO Received uploads request method=POST path=/api/pending_closures4891worker1 # [ 56.573874] nix-grpc-daemon[853]: time=2026-09-20T09:33:52.614Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4892worker1 # [ 56.575250] nix-grpc-daemon[853]: time=2026-09-20T09:33:52.614Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4893lb # [ 56.575245] niks3-server[1054]: 2026/09/20 09:33:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4894lb # [ 56.580062] niks3-server[1054]: 2026/09/20 09:33:51 INFO Signed narinfos id=13 count=14895worker1 # [ 56.599898] nix-grpc-daemon[853]: time=2026-09-20T09:33:52.640Z level=INFO msg="Uploading 1 narinfos"4896lb # [ 56.587372] niks3-server[1054]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4897lb # [ 56.599545] niks3-server[1054]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4898lb # [ 56.605143] niks3-server[1054]: 2026/09/20 09:33:51 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4899worker1 # [ 56.626903] nix-grpc-daemon[853]: time=2026-09-20T09:33:52.667Z level=INFO msg="Upload complete. (410ms)"4900lb # [ 56.610103] niks3-server[1054]: 2026/09/20 09:33:51 INFO Completed upload id=134901worker1 # [ 56.629084] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_in=6244902lb # [ 56.614857] niks3-server[1054]: 2026/09/20 09:33:51 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4903worker1 # [ 56.635746] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384904worker1 # [ 56.642878] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4905worker1 # [ 56.645098] nix-daemon[842]: reaped child process 1072, status = succeeded4906worker1 # [ 56.650220] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4907worker1 # [ 57.677189] nix-daemon[842]: reaped child process 1086, status = succeeded4908worker1 # [ 57.827564] nix-daemon[842]: accepted connection from pid 1101, user nix-grpc-daemon (trusted)4909worker1 # [ 57.839718] nix-daemon[842]: reaped child process 1104, status = succeeded4910lb # [ 57.839169] niks3-server[1054]: 2026/09/20 09:33:53 INFO Received uploads request method=POST path=/api/pending_closures4911worker1 # [ 57.863499] nix-grpc-daemon[853]: time=2026-09-20T09:33:53.903Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4912worker1 # [ 57.865516] nix-grpc-daemon[853]: time=2026-09-20T09:33:53.905Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4913lb # [ 57.877916] niks3-server[1054]: 2026/09/20 09:33:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4914lb # [ 57.884651] niks3-server[1054]: 2026/09/20 09:33:53 INFO Signed narinfos id=14 count=14915worker1 # [ 57.904596] nix-grpc-daemon[853]: time=2026-09-20T09:33:53.945Z level=INFO msg="Uploading 1 narinfos"4916lb # [ 57.896148] niks3-server[1054]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4917lb # [ 57.900327] niks3-server[1054]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4918lb # [ 57.910464] niks3-server[1054]: 2026/09/20 09:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4919lb # [ 57.918049] niks3-server[1054]: 2026/09/20 09:33:53 INFO Completed upload id=144920worker1 # [ 57.933941] nix-grpc-daemon[853]: time=2026-09-20T09:33:53.974Z level=INFO msg="Upload complete. (276ms)"4921lb # [ 57.919792] niks3-server[1054]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4922lb # [ 57.922228] niks3-server[1054]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4923worker1 # [ 57.938922] nix-grpc-daemon[784]: ts=2026-09-20T09:33:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14924worker1 # [ 57.949862] nix-daemon[842]: reaped child process 1084, status = succeeded4925worker1 # [ 57.983957] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380384926worker2 # [ 57.991440] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329744927worker2 # [ 58.003322] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4928worker2 # [ 58.017211] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=0 paths=1 nar_bytes_out=1764929worker2 # [ 58.260788] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329744930worker2 # [ 58.266985] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329744931worker2 # [ 58.272513] nix-grpc-daemon[900]: time=2026-09-20T09:33:54.371Z level=INFO msg="All 2 paths already cached"4932worker2 # [ 58.281920] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04933worker2 # [ 58.286182] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14934worker1 # [ 58.335929] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384935worker1 # [ 58.348515] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380384936worker1 # [ 58.361088] nix-grpc-daemon[853]: time=2026-09-20T09:33:54.401Z level=INFO msg="All 2 paths already cached"4937worker1 # [ 58.368200] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380384938worker1 # [ 58.376472] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4939worker1 # [ 58.535197] nix-daemon[842]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)4940worker1 # [ 58.547066] nix-daemon[842]: reaped child process 1111, status = succeeded4941worker1 # [ 58.703296] nix-daemon[842]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4942worker1 # [ 58.714947] nix-daemon[842]: reaped child process 1116, status = succeeded4943lb # [ 58.714171] niks3-server[1054]: 2026/09/20 09:33:53 INFO Received uploads request method=POST path=/api/pending_closures4944worker1 # [ 58.739408] nix-grpc-daemon[853]: time=2026-09-20T09:33:54.780Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4945worker1 # [ 58.740867] nix-grpc-daemon[853]: time=2026-09-20T09:33:54.781Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4946lb # [ 58.742912] niks3-server[1054]: 2026/09/20 09:33:54 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4947lb # [ 58.757385] niks3-server[1054]: 2026/09/20 09:33:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4948worker1 # [ 58.780457] nix-grpc-daemon[853]: time=2026-09-20T09:33:54.820Z level=INFO msg="Uploading 1 narinfos"4949lb # [ 58.763352] niks3-server[1054]: 2026/09/20 09:33:54 INFO Signed narinfos id=15 count=14950lb # [ 58.766915] niks3-server[1054]: 2026/09/20 09:33:54 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4951lb # [ 58.777132] niks3-server[1054]: 2026/09/20 09:33:54 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4952lb # [ 58.783717] niks3-server[1054]: 2026/09/20 09:33:54 INFO Completed upload id=154953worker1 # [ 58.800588] nix-grpc-daemon[853]: time=2026-09-20T09:33:54.841Z level=INFO msg="Upload complete. (410ms)"4954lb # [ 58.785331] niks3-server[1054]: 2026/09/20 09:33:54 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4955worker1 # [ 58.802624] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_in=7524956lb # [ 58.787830] envoy[926]: conn peer=[2001:db8:1::1]:38516 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1355 ms=46554957lb # [ 58.789954] envoy[926]: conn peer=[2001:db8:1::1]:38524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=35414958worker1 # [ 58.807112] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380384959lb # [ 58.791460] envoy[926]: conn peer=[2001:db8:1::1]:55188 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=334960lb # [ 58.793211] envoy[926]: conn peer=[2001:db8:1::1]:55172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3378 tx=1727 ms=19124961lb # [ 58.794701] envoy[926]: conn peer=[2001:db8:1::1]:55202 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3880 tx=1851 ms=19494962lb # [ 58.796266] envoy[926]: conn peer=[2001:db8:1::1]:55214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=774963worker1 # [ 58.813400] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4964worker1 # [ 58.814668] nix-daemon[842]: reaped child process 1106, status = succeeded4965worker1 # [ 58.819906] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4966worker1 # [ 60.446471] nix-daemon[842]: reaped child process 1120, status = succeeded4967worker1 # [ 60.584923] nix-daemon[842]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)4968worker1 # [ 60.595592] nix-daemon[842]: reaped child process 1138, status = succeeded4969lb # [ 60.595818] niks3-server[1054]: 2026/09/20 09:33:55 INFO Received uploads request method=POST path=/api/pending_closures4970worker1 # [ 60.620954] nix-grpc-daemon[853]: time=2026-09-20T09:33:56.661Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4971worker1 # [ 60.622510] nix-grpc-daemon[853]: time=2026-09-20T09:33:56.663Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4972lb # [ 60.637971] niks3-server[1054]: 2026/09/20 09:33:55 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4973lb # [ 60.647474] niks3-server[1054]: 2026/09/20 09:33:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4974lb # [ 60.655418] niks3-server[1054]: 2026/09/20 09:33:55 INFO Signed narinfos id=16 count=14975worker1 # [ 60.675626] nix-grpc-daemon[853]: time=2026-09-20T09:33:56.716Z level=INFO msg="Uploading 1 narinfos"4976lb # [ 60.660599] niks3-server[1054]: 2026/09/20 09:33:55 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4977lb # [ 60.667087] niks3-server[1054]: 2026/09/20 09:33:55 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4978lb # [ 60.676701] niks3-server[1054]: 2026/09/20 09:33:55 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4979lb # [ 60.682845] niks3-server[1054]: 2026/09/20 09:33:55 INFO Completed upload id=164980worker1 # [ 60.699572] nix-grpc-daemon[853]: time=2026-09-20T09:33:56.740Z level=INFO msg="Upload complete. (268ms)"4981lb # [ 60.685097] niks3-server[1054]: 2026/09/20 09:33:55 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4982worker1 # [ 60.704765] nix-grpc-daemon[784]: ts=2026-09-20T09:33:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14983worker1 # [ 60.712403] nix-grpc-daemon[784]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380384984worker1 # [ 60.722220] nix-daemon[842]: reaped child process 1118, status = succeeded4985worker1 # [ 60.725661] nix-grpc-daemon[784]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:380384986worker1 # [ 60.728687] nix-grpc-daemon[784]: ts=2026-09-20T09:33:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 paths=1 nar_bytes_out=2324987client # [ 60.826797] nix-daemon[944]: reaped child process 946, status = succeeded4988client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.33 seconds)4989client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4990client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4991client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4992client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4993client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4994client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4995(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.43 seconds)4996subtest: an input only one worker has still reaches the builder4997client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4998client # this derivation will be built:4999client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5000client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5001client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.41 seconds)5002client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5003client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.15 seconds)5004worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5005worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)5006worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5007worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5008client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25009client # [ 61.856359] nix-daemon[944]: accepted connection from pid 1023, user root (trusted)5010client # this derivation will be built:5011client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5012worker1 # [ 62.035394] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:380385013worker2 # [ 62.028672] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329745014client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5015client # copying 1 paths...5016client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5017worker2 # [ 62.036269] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329745018worker2 # [ 62.039993] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5019worker2 # [ 62.171239] nix-daemon[847]: accepted connection from pid 985, user nix-grpc-daemon (trusted)5020worker2 # [ 62.183724] nix-daemon[847]: reaped child process 988, status = succeeded5021lb # [ 62.187491] niks3-server[1054]: 2026/09/20 09:33:57 INFO Received uploads request method=POST path=/api/pending_closures5022worker2 # [ 62.205178] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.305Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5023worker2 # [ 62.206991] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.306Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5024lb # [ 62.220315] niks3-server[1054]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5025lb # [ 62.230359] niks3-server[1054]: 2026/09/20 09:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5026lb # [ 62.236692] niks3-server[1054]: 2026/09/20 09:33:57 INFO Signed narinfos id=17 count=15027worker2 # [ 62.247212] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.347Z level=INFO msg="Uploading 1 narinfos"5028lb # [ 62.241761] niks3-server[1054]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5029lb # [ 62.251164] niks3-server[1054]: 2026/09/20 09:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5030worker2 # [ 62.265020] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.365Z level=INFO msg="Upload complete. (219ms)"5031lb # [ 62.258068] niks3-server[1054]: 2026/09/20 09:33:57 INFO Completed upload id=175032worker2 # [ 62.266969] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=0 paths=1 nar_bytes_in=1205033lb # [ 62.261345] niks3-server[1054]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5034worker2 # [ 62.273085] nix-daemon[847]: reaped child process 983, status = succeeded5035worker2 # [ 62.278162] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05036worker2 # [ 62.279906] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15037worker2 # [ 62.325506] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329745038worker2 # [ 62.334108] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329745039client # copying 2 paths to 'lb:50051'...5040worker2 # [ 62.345390] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329745041client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5042client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5043worker2 # [ 62.359165] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5044worker2 # [ 62.510939] nix-daemon[847]: accepted connection from pid 994, user nix-grpc-daemon (trusted)5045worker2 # [ 62.523902] nix-daemon[847]: reaped child process 997, status = succeeded5046worker2 # [ 62.660702] nix-daemon[847]: accepted connection from pid 999, user nix-grpc-daemon (trusted)5047worker2 # [ 62.672878] nix-daemon[847]: reaped child process 1002, status = succeeded5048lb # [ 62.678920] niks3-server[1054]: 2026/09/20 09:33:57 INFO Received uploads request method=POST path=/api/pending_closures5049lb # [ 62.687096] niks3-server[1054]: 2026/09/20 09:33:57 INFO Received uploads request method=POST path=/api/pending_closures5050worker2 # [ 62.701860] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.802Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5051worker2 # [ 62.703396] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.803Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5052worker2 # [ 62.705155] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.805Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5053lb # [ 62.718445] niks3-server[1054]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5054lb # [ 62.740502] niks3-server[1054]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5055lb # [ 62.745633] niks3-server[1054]: 2026/09/20 09:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5056lb # [ 62.749308] niks3-server[1054]: 2026/09/20 09:33:58 INFO Signed narinfos id=18 count=15057lb # [ 62.753479] niks3-server[1054]: 2026/09/20 09:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5058lb # [ 62.758578] niks3-server[1054]: 2026/09/20 09:33:58 INFO Signed narinfos id=19 count=25059lb # [ 62.759867] niks3-server[1054]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5060worker2 # [ 62.767372] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.867Z level=INFO msg="Uploading 3 narinfos"5061lb # [ 62.762457] niks3-server[1054]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5062lb # [ 62.772618] niks3-server[1054]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5063lb # [ 62.775973] niks3-server[1054]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5064lb # [ 62.781292] niks3-server[1054]: 2026/09/20 09:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5065lb # [ 62.783829] niks3-server[1054]: 2026/09/20 09:33:58 INFO Completed upload id=185066lb # [ 62.786192] niks3-server[1054]: 2026/09/20 09:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5067lb # [ 62.788192] niks3-server[1054]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5068lb # [ 62.790400] niks3-server[1054]: 2026/09/20 09:33:58 INFO Completed upload id=195069worker2 # [ 62.799112] nix-grpc-daemon[900]: time=2026-09-20T09:33:58.899Z level=INFO msg="Upload complete. (425ms)"5070worker2 # [ 62.803927] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=0 paths=2 nar_bytes_in=9765071worker2 # [ 62.810352] nix-grpc-daemon[784]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329745072worker2 # [ 62.819499] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5073worker2 # [ 62.828470] nix-daemon[847]: reaped child process 992, status = succeeded5074worker2 # [ 62.836555] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5075client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5076client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5077worker2 # [ 64.048848] nix-daemon[847]: reaped child process 1006, status = succeeded5078worker2 # [ 64.186175] nix-daemon[847]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)5079worker2 # [ 64.198840] nix-daemon[847]: reaped child process 1024, status = succeeded5080lb # [ 64.205599] niks3-server[1054]: 2026/09/20 09:33:59 INFO Received uploads request method=POST path=/api/pending_closures5081worker2 # [ 64.222248] nix-grpc-daemon[900]: time=2026-09-20T09:34:00.322Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5082lb # [ 64.236584] niks3-server[1054]: 2026/09/20 09:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5083lb # [ 64.245719] niks3-server[1054]: 2026/09/20 09:33:59 INFO Signed narinfos id=20 count=15084worker2 # [ 64.256253] nix-grpc-daemon[900]: time=2026-09-20T09:34:00.356Z level=INFO msg="Uploading 1 narinfos"5085lb # [ 64.251426] niks3-server[1054]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5086lb # [ 64.256765] niks3-server[1054]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5087lb # [ 64.269284] niks3-server[1054]: 2026/09/20 09:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5088lb # [ 64.274190] niks3-server[1054]: 2026/09/20 09:33:59 INFO Completed upload id=205089worker2 # [ 64.283996] nix-grpc-daemon[900]: time=2026-09-20T09:34:00.384Z level=INFO msg="Upload complete. (241ms)"5090lb # [ 64.276985] niks3-server[1054]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5091worker2 # [ 64.289354] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15092worker2 # [ 64.300525] nix-daemon[847]: reaped child process 1004, status = succeeded5093client # copying 1 paths...5094worker2 # [ 64.334142] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329745095client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5096worker2 # [ 64.348997] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329745097worker2 # [ 64.352684] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=0 paths=1 nar_bytes_out=1205098client # [ 64.456776] nix-daemon[944]: reaped child process 1026, status = succeeded5099client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.77 seconds)5100client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25101client # [ 64.635666] nix-daemon[944]: accepted connection from pid 1044, user root (trusted)5102client # this derivation will be built:5103client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5104worker2 # [ 64.810147] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329745105client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5106worker1 # [ 64.823983] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380385107worker1 # [ 64.830937] nix-grpc-daemon[853]: time=2026-09-20T09:34:00.870Z level=INFO msg="All 1 paths already cached"5108worker2 # [ 64.830544] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05109worker1 # [ 64.842351] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380385110worker2 # [ 64.835075] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15111worker1 # [ 64.854079] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380385112client # copying 1 paths to 'lb:50051'...5113client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5114worker1 # [ 64.869994] nix-grpc-daemon[784]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380385115worker1 # [ 64.881145] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5116worker1 # [ 64.896174] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5)5117worker1 # [ 64.940732] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 108 ms (attempt 2/5)5118worker1 # [ 65.051317] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 3/5)5119worker1 # [ 65.391205] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 529 ms (attempt 4/5)5120worker1 # [ 65.926988] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5121worker1 # [ 65.932747] nix-grpc-daemon[784]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5122worker1 # [ 66.121467] nix-daemon[842]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)5123worker1 # [ 66.133102] nix-daemon[842]: reaped child process 1154, status = succeeded5124worker1 # [ 66.295630] nix-daemon[842]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)5125worker1 # [ 66.308300] nix-daemon[842]: reaped child process 1159, status = succeeded5126lb # [ 66.306812] niks3-server[1054]: 2026/09/20 09:34:01 INFO Received uploads request method=POST path=/api/pending_closures5127worker1 # [ 66.331594] nix-grpc-daemon[853]: time=2026-09-20T09:34:02.371Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5128worker1 # [ 66.332933] nix-grpc-daemon[853]: time=2026-09-20T09:34:02.371Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5129lb # [ 66.332522] niks3-server[1054]: 2026/09/20 09:34:01 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5130lb # [ 66.340313] niks3-server[1054]: 2026/09/20 09:34:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5131worker1 # [ 66.362406] nix-grpc-daemon[853]: time=2026-09-20T09:34:02.402Z level=INFO msg="Uploading 1 narinfos"5132lb # [ 66.345812] niks3-server[1054]: 2026/09/20 09:34:01 INFO Signed narinfos id=21 count=15133lb # [ 66.354669] niks3-server[1054]: 2026/09/20 09:34:01 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5134lb # [ 66.363890] niks3-server[1054]: 2026/09/20 09:34:01 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5135worker1 # [ 66.386390] nix-grpc-daemon[853]: time=2026-09-20T09:34:02.426Z level=INFO msg="Upload complete. (429ms)"5136lb # [ 66.369774] niks3-server[1054]: 2026/09/20 09:34:01 INFO Completed upload id=215137worker1 # [ 66.388376] nix-grpc-daemon[784]: ts=2026-09-20T09:34:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=1 paths=1 nar_bytes_in=5605138lb # [ 66.373581] niks3-server[1054]: 2026/09/20 09:34:01 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5139worker1 # [ 66.394423] nix-grpc-daemon[784]: ts=2026-09-20T09:34:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380385140worker1 # [ 66.399299] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5141worker1 # [ 66.401500] nix-daemon[842]: reaped child process 1145, status = succeeded5142client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5143worker1 # [ 66.406843] nix-daemon[842]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5144client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5145worker1 # [ 67.030583] nix-daemon[842]: reaped child process 1163, status = succeeded5146worker1 # [ 67.182451] nix-daemon[842]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5147worker1 # [ 67.193160] nix-daemon[842]: reaped child process 1181, status = succeeded5148lb # [ 67.192341] niks3-server[1054]: 2026/09/20 09:34:02 INFO Received uploads request method=POST path=/api/pending_closures5149worker1 # [ 67.229903] nix-grpc-daemon[853]: time=2026-09-20T09:34:03.270Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5150lb # [ 67.234199] niks3-server[1054]: 2026/09/20 09:34:02 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5151lb # [ 67.241488] niks3-server[1054]: 2026/09/20 09:34:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5152lb # [ 67.245710] niks3-server[1054]: 2026/09/20 09:34:02 INFO Signed narinfos id=22 count=15153worker1 # [ 67.265559] nix-grpc-daemon[853]: time=2026-09-20T09:34:03.305Z level=INFO msg="Uploading 1 narinfos"5154lb # [ 67.256073] niks3-server[1054]: 2026/09/20 09:34:02 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5155lb # [ 67.263341] niks3-server[1054]: 2026/09/20 09:34:02 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5156lb # [ 67.269563] niks3-server[1054]: 2026/09/20 09:34:02 INFO Completed upload id=225157worker1 # [ 67.286552] nix-grpc-daemon[853]: time=2026-09-20T09:34:03.327Z level=INFO msg="Upload complete. (273ms)"5158lb # [ 67.274075] niks3-server[1054]: 2026/09/20 09:34:02 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5159worker1 # [ 67.291649] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38038 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15160worker1 # [ 67.304577] nix-daemon[842]: reaped child process 1161, status = succeeded5161client # copying 1 paths...5162worker2 # [ 67.327863] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329745163worker2 # [ 67.340303] nix-grpc-daemon[784]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5164client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5165worker2 # [ 67.368812] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329745166worker2 # [ 67.373799] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=0 paths=1 nar_bytes_out=1205167client # [ 67.468735] nix-daemon[944]: reaped child process 1047, status = succeeded5168client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.01 seconds)5169client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25170client # [ 67.643976] nix-daemon[944]: accepted connection from pid 1065, user root (trusted)5171client # this derivation will be built:5172client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5173worker2 # [ 67.820923] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:329745174client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5175worker2 # [ 67.827078] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329745176worker2 # [ 67.833470] nix-grpc-daemon[900]: time=2026-09-20T09:34:03.931Z level=INFO msg="All 1 paths already cached"5177worker2 # [ 67.842121] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05178worker2 # [ 67.847584] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15179worker2 # [ 67.885543] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329745180worker2 # [ 67.896283] nix-grpc-daemon[784]: ts=2026-09-20T09:34:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329745181worker2 # [ 67.906320] nix-grpc-daemon[900]: time=2026-09-20T09:34:04.006Z level=INFO msg="All 1 paths already cached"5182client # copying 1 paths to 'lb:50051'...5183client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5184worker2 # [ 67.913909] nix-grpc-daemon[784]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329745185worker2 # [ 67.925097] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5186worker2 # [ 68.097422] nix-daemon[847]: accepted connection from pid 1033, user nix-grpc-daemon (trusted)5187worker2 # [ 68.110502] nix-daemon[847]: reaped child process 1036, status = succeeded5188worker2 # [ 68.268740] nix-daemon[847]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5189worker2 # [ 68.280745] nix-daemon[847]: reaped child process 1041, status = succeeded5190lb # [ 68.288956] niks3-server[1054]: 2026/09/20 09:34:03 INFO Received uploads request method=POST path=/api/pending_closures5191worker2 # [ 68.305629] nix-grpc-daemon[900]: time=2026-09-20T09:34:04.405Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5192worker2 # [ 68.306983] nix-grpc-daemon[900]: time=2026-09-20T09:34:04.405Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5193lb # [ 68.314070] niks3-server[1054]: 2026/09/20 09:34:03 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5194lb # [ 68.326088] niks3-server[1054]: 2026/09/20 09:34:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5195lb # [ 68.330984] niks3-server[1054]: 2026/09/20 09:34:03 INFO Signed narinfos id=23 count=15196worker2 # [ 68.341335] nix-grpc-daemon[900]: time=2026-09-20T09:34:04.441Z level=INFO msg="Uploading 1 narinfos"5197lb # [ 68.336920] niks3-server[1054]: 2026/09/20 09:34:03 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5198lb # [ 68.341143] niks3-server[1054]: 2026/09/20 09:34:03 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5199lb # [ 68.344727] niks3-server[1054]: 2026/09/20 09:34:03 INFO Completed upload id=235200worker2 # [ 68.352187] nix-grpc-daemon[900]: time=2026-09-20T09:34:04.452Z level=INFO msg="Upload complete. (415ms)"5201worker2 # [ 68.354408] nix-grpc-daemon[784]: ts=2026-09-20T09:34:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=0 paths=1 nar_bytes_in=5605202lb # [ 68.347310] niks3-server[1054]: 2026/09/20 09:34:03 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5203worker2 # [ 68.358168] nix-grpc-daemon[784]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329745204worker2 # [ 68.364481] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5205worker2 # [ 68.367073] nix-daemon[847]: reaped child process 1031, status = succeeded5206client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5207worker2 # [ 68.372187] nix-daemon[847]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5208client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5209lb # [ 68.787198] envoy[926]: conn peer=[2001:db8:1::1]:55232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=515210lb # [ 68.788653] envoy[926]: conn peer=[2001:db8:1::1]:55222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4298 tx=2067 ms=25075211lb # [ 68.790109] envoy[926]: conn peer=[2001:db8:1::1]:55240 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4203 tx=1725 ms=23785212lb # [ 68.791562] envoy[926]: conn peer=[2001:db8:1::1]:48888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=495213lb # [ 68.793140] envoy[926]: conn peer=[2001:db8:1::1]:48902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3711 tx=1940 ms=26155214lb # [ 68.794690] envoy[926]: conn peer=[2001:db8:1::1]:48910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555215worker2 # [ 69.397646] nix-daemon[847]: reaped child process 1045, status = succeeded5216worker2 # [ 69.556357] nix-daemon[847]: accepted connection from pid 1060, user nix-grpc-daemon (trusted)5217worker2 # [ 69.566968] nix-daemon[847]: reaped child process 1063, status = succeeded5218lb # [ 69.575867] niks3-server[1054]: 2026/09/20 09:34:04 INFO Received uploads request method=POST path=/api/pending_closures5219worker2 # [ 69.593874] nix-grpc-daemon[900]: time=2026-09-20T09:34:05.693Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5220lb # [ 69.609846] niks3-server[1054]: 2026/09/20 09:34:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5221worker2 # [ 69.623926] nix-grpc-daemon[900]: time=2026-09-20T09:34:05.724Z level=INFO msg="Uploading 1 narinfos"5222lb # [ 69.616452] niks3-server[1054]: 2026/09/20 09:34:04 INFO Signed narinfos id=24 count=15223lb # [ 69.628260] niks3-server[1054]: 2026/09/20 09:34:04 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5224lb # [ 69.640853] niks3-server[1054]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5225lb # [ 69.655088] niks3-server[1054]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5226lb # [ 69.669837] niks3-server[1054]: 2026/09/20 09:34:04 INFO Completed upload id=245227worker2 # [ 69.677332] nix-grpc-daemon[900]: time=2026-09-20T09:34:05.777Z level=INFO msg="Upload complete. (297ms)"5228lb # [ 69.672602] niks3-server[1054]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5229worker2 # [ 69.682859] nix-grpc-daemon[784]: ts=2026-09-20T09:34:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15230worker2 # [ 69.697838] nix-daemon[847]: reaped child process 1043, status = succeeded5231client # copying 1 paths...5232worker2 # [ 69.731192] nix-grpc-daemon[784]: ts=2026-09-20T09:34:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:329745233client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5234worker2 # [ 69.754412] nix-grpc-daemon[784]: ts=2026-09-20T09:34:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:329745235worker2 # [ 69.758790] nix-grpc-daemon[784]: ts=2026-09-20T09:34:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:32974 duration_s=0 paths=1 nar_bytes_out=1205236client # [ 69.871166] nix-daemon[944]: reaped child process 1068, status = succeeded5237client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.40 seconds)5238(finished: subtest: an input only one worker has still reaches the builder, in 8.95 seconds)5239subtest: upload whose reference the worker lacks is completed from the cache5240client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5241client # this derivation will be built:5242client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5243client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5244client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.38 seconds)5245client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5246client # this derivation will be built:5247client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5248client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5249client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.36 seconds)5250client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5251client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5252worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5253worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5254worker2 # [ 71.056369] systemd[1]: Stopping Nix worker-protocol over gRPC...5255worker2 # [ 71.079194] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=signal_exit5256worker2 # [ 71.349841] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_yield5257lb # [ 71.346570] niks3-server[1054]: 2026/09/20 09:34:06 INFO lead: released remote=192.168.1.4:496865258worker2 # [ 71.354526] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_restarting addr=lb:500515259worker1 # [ 71.370228] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_restarting addr=lb:500515260worker2 # [ 71.360269] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:329705261worker2 # [ 71.365675] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=worker_gone addr=192.168.1.4:500515262worker2 # [ 71.368741] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:329705263worker2 # [ 71.372065] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=worker_gone addr=192.168.1.3:500515264worker2 # [ 71.555373] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_connected addr=lb:500515265worker2 # [ 71.559299] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5266worker1 # [ 71.570971] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_connected addr=lb:500515267worker1 # [ 71.574907] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5268worker2 # [ 71.760185] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_connected addr=lb:500515269worker2 # [ 71.764773] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5270worker1 # [ 71.775793] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_connected addr=lb:500515271worker1 # [ 71.779731] nix-grpc-daemon[784]: ts=2026-09-20T09:34:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5272worker2 # [ 71.965760] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_connected addr=lb:500515273worker1 # [ 71.980539] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_connected addr=lb:500515274worker2 # [ 71.970539] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5275worker1 # [ 71.984399] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5276worker2 # [ 72.172522] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_connected addr=lb:500515277worker1 # [ 72.185143] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_connected addr=lb:500515278worker2 # [ 72.176639] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5279worker1 # [ 72.189114] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5280worker2 # [ 72.354615] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_restarting5281worker1 # [ 72.390811] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_connected addr=lb:500515282worker1 # [ 72.393839] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker2 # [ 72.389332] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5284worker2 # [ 72.393814] nix-daemon[847]: reaped child process 849, status = succeeded5285worker1 # [ 72.591635] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_connected addr=lb:500515286worker1 # [ 72.595509] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5287worker1 # [ 72.796318] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_connected addr=lb:500515288worker1 # [ 72.800205] nix-grpc-daemon[784]: ts=2026-09-20T09:34:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5289worker1 # [ 73.000574] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_connected addr=lb:500515290worker1 # [ 73.002348] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5291worker1 # [ 73.202833] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_connected addr=lb:500515292worker1 # [ 73.204604] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5293worker2 # [ 73.204149] nix-grpc-daemon[784]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5294worker2 # [ 73.207449] nix-grpc-daemon[784]: E0000 00:00:1789896849.302966 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5295worker2 # [ 73.212090] nix-grpc-daemon[784]: E0000 00:00:1789896849.303595 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5296worker1 # [ 73.406263] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_connected addr=lb:500515297worker1 # [ 73.409955] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5298worker2 # [ 73.541763] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5299worker2 # [ 73.543892] systemd[1]: Stopped Nix worker-protocol over gRPC.5300worker2 # [ 73.544870] systemd[1]: nix-grpc-daemon.service: Consumed 1.449s CPU time over 54.750s wall clock time, 52.7M memory peak, 55.6K incoming IP traffic, 109K outgoing IP traffic.5301worker2 # [ 73.549096] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5302worker2 # [ 73.550775] systemd[1]: Closed Nix worker-protocol over gRPC.5303worker2 # [ 73.552108] systemd[1]: nix-grpc-daemon.socket: Consumed 285us CPU time over 1min 3.170s wall clock time, 256K memory peak, 74.7K incoming IP traffic, 66K outgoing IP traffic.5304lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5305lb: must succeed: curl -sf localhost:9901/clusters5306worker1 # [ 73.610687] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_connected addr=lb:500515307worker1 # [ 73.614542] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5308lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5309lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5310client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25311worker1 # [ 73.815602] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_connected addr=lb:500515312worker1 # [ 73.819611] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker1 # [ 73.846287] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380385314worker1 # [ 73.853072] nix-grpc-daemon[784]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380385315client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5316client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.25 seconds)5317worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5318worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5319worker2 # [ 73.942131] systemd[1]: Listening on Nix worker-protocol over gRPC.5320worker1 # [ 74.011295] systemd[1]: Stopping Nix worker-protocol over gRPC...5321worker1 # [ 74.021117] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_connected addr=lb:500515322worker1 # [ 74.024147] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker1 # [ 74.091538] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=signal_exit5324worker1 # [ 74.224979] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_connected addr=lb:500515325worker1 # [ 74.228825] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5326worker1 # [ 74.431085] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_connected addr=lb:500515327worker1 # [ 74.434376] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5328worker2 # [ 74.536423] systemd[1]: Starting Nix worker-protocol over gRPC...5329worker1 # [ 74.631209] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_connected addr=lb:500515330worker1 # [ 74.633136] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5331worker2 # [ 74.715808] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5332worker2 # [ 74.719500] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15333worker2 # [ 74.725515] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5334worker2 # [ 74.729305] systemd[1]: Started Nix worker-protocol over gRPC.5335worker2 # [ 74.739117] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:10Z level=info event=scheduler_take_over5336lb # [ 74.732984] niks3-server[1054]: 2026/09/20 09:34:09 INFO lead: acquired remote=192.168.1.4:452065337worker2 # [ 74.757136] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:10Z level=info event=scheduler_connected addr=lb:500515338worker2 # [ 74.760173] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:10Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05339worker1 # [ 74.833619] nix-grpc-daemon[784]: ts=2026-09-20T09:34:10Z level=info event=scheduler_connected addr=lb:500515340worker2 # [ 74.825922] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05341worker1 # [ 75.359848] nix-grpc-daemon[784]: ts=2026-09-20T09:34:11Z level=info event=scheduler_restarting5342worker2 # [ 75.359631] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:340845343worker2 # [ 75.363183] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:11Z level=info event=worker_gone addr=192.168.1.3:500515344worker1 # [ 75.385627] nix-grpc-daemon[784]: ts=2026-09-20T09:34:11Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5345worker1 # [ 75.402995] nix-daemon[842]: reaped child process 844, status = succeeded5346worker1 # [ 75.495778] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5347worker1 # [ 75.500455] systemd[1]: Stopped Nix worker-protocol over gRPC.5348worker1 # [ 75.503355] systemd[1]: nix-grpc-daemon.service: Consumed 2.608s CPU time over 56.700s wall clock time, 61.3M memory peak, 84.2K incoming IP traffic, 161.1K outgoing IP traffic.5349worker1 # [ 75.514434] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5350worker1 # [ 75.517713] systemd[1]: Closed Nix worker-protocol over gRPC.5351worker1 # [ 75.520358] systemd[1]: nix-grpc-daemon.socket: Consumed 298us CPU time over 1min 6.088s wall clock time, 256K memory peak, 73.6K incoming IP traffic, 60.3K outgoing IP traffic.5352lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5353lb: must succeed: curl -sf localhost:9901/clusters5354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5355lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5356client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25357worker2 # [ 75.798637] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340905358worker2 # [ 75.810159] nix-daemon[847]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5359worker2 # [ 75.826208] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5)5360worker2 # [ 75.837903] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 2/5)5361worker2 # [ 75.862606] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 211 ms (attempt 3/5)5362worker2 # [ 76.079788] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 535 ms (attempt 4/5)5363worker2 # [ 76.615604] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5364client # copying 2 paths...5365client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5366worker2 # [ 76.638135] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340905367client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5368worker2 # [ 76.650343] nix-daemon[847]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5369worker2 # [ 76.672566] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:12Z level=info event=niks3_push_started pid=11405370worker2 # [ 77.028350] nix-daemon[847]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5371worker2 # [ 77.040626] nix-daemon[847]: reaped child process 1156, status = succeeded5372lb # [ 77.050932] niks3-server[1054]: 2026/09/20 09:34:12 INFO Received uploads request method=POST path=/api/pending_closures5373lb # [ 77.055355] niks3-server[1054]: 2026/09/20 09:34:12 INFO Received uploads request method=POST path=/api/pending_closures5374worker2 # [ 77.066067] nix-grpc-daemon[1140]: time=2026-09-20T09:34:13.166Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5375worker2 # [ 77.068134] nix-grpc-daemon[1140]: time=2026-09-20T09:34:13.167Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5376worker2 # [ 77.069756] nix-grpc-daemon[1140]: time=2026-09-20T09:34:13.168Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5377lb # [ 77.093287] niks3-server[1054]: 2026/09/20 09:34:12 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5378lb # [ 77.104840] niks3-server[1054]: 2026/09/20 09:34:12 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5379lb # [ 77.111468] niks3-server[1054]: 2026/09/20 09:34:12 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5380lb # [ 77.120755] niks3-server[1054]: 2026/09/20 09:34:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5381lb # [ 77.124418] niks3-server[1054]: 2026/09/20 09:34:12 INFO Signed narinfos id=25 count=15382lb # [ 77.126731] niks3-server[1054]: 2026/09/20 09:34:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5383lb # [ 77.128599] niks3-server[1054]: 2026/09/20 09:34:12 INFO Signed narinfos id=26 count=25384lb # [ 77.129780] niks3-server[1054]: 2026/09/20 09:34:12 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5385worker2 # [ 77.138143] nix-grpc-daemon[1140]: time=2026-09-20T09:34:13.237Z level=INFO msg="Uploading 3 narinfos"5386lb # [ 77.141770] niks3-server[1054]: 2026/09/20 09:34:12 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5387lb # [ 77.147965] niks3-server[1054]: 2026/09/20 09:34:12 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5388lb # [ 77.149971] niks3-server[1054]: 2026/09/20 09:34:12 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5389lb # [ 77.153935] niks3-server[1054]: 2026/09/20 09:34:12 INFO Completed upload id=265390lb # [ 77.155185] niks3-server[1054]: 2026/09/20 09:34:12 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5391lb # [ 77.157451] niks3-server[1054]: 2026/09/20 09:34:12 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5392lb # [ 77.159696] niks3-server[1054]: 2026/09/20 09:34:12 INFO Completed upload id=255393worker2 # [ 77.168483] nix-grpc-daemon[1140]: time=2026-09-20T09:34:13.268Z level=INFO msg="Upload complete. (301ms)"5394worker2 # [ 77.173934] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34090 duration_s=0 paths=2 nar_bytes_in=3125395worker2 # [ 77.185440] nix-daemon[847]: reaped child process 1138, status = succeeded5396client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.62 seconds)5397worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5398worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5399worker1 # [ 77.301685] systemd[1]: Listening on Nix worker-protocol over gRPC.5400worker1 # [ 77.306878] systemd[1]: Starting Nix worker-protocol over gRPC...5401worker1 # [ 77.470920] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5402worker1 # [ 77.473211] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15403worker1 # [ 77.478898] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5404worker1 # [ 77.480466] systemd[1]: Started Nix worker-protocol over gRPC.5405lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5406lb: must succeed: curl -sf localhost:9901/clusters5407worker1 # [ 77.508516] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:13Z level=info event=scheduler_connected addr=lb:500515408worker2 # [ 77.501732] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05409lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5410lb: must succeed: curl -sf localhost:9901/clusters5411lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5412lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)5413(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 8.75 seconds)5414subtest: scheduler down: the other node takes the lock and keeps it5415lb: must succeed: curl -sf localhost:9901/clusters5416lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5417lb: must succeed: curl -sf localhost:9901/clusters5418lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5419lb: must succeed: curl -sf localhost:9901/clusters5420lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5421lb: must succeed: curl -sf localhost:9901/clusters5422lb # [ 78.788185] envoy[926]: conn peer=[2001:db8:1::1]:48932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575423lb # [ 78.789672] envoy[926]: conn peer=[2001:db8:1::1]:48918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3712 tx=1798 ms=19915424lb # [ 78.791060] envoy[926]: conn peer=192.168.1.4:40946 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13397 tx=2597 ms=532385425lb # [ 78.792405] envoy[926]: conn peer=[2001:db8:1::1]:53180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=205426lb # [ 78.793914] envoy[926]: conn peer=192.168.1.3:39042 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=17253 tx=3379 ms=562325427lb # [ 78.795761] envoy[926]: conn peer=[2001:db8:1::1]:53190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=305 ms=13885428lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5429worker2 # [ 78.863624] systemd[1]: Stopping Nix worker-protocol over gRPC...5430worker2 # [ 79.733132] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:15Z level=info event=signal_exit5431worker2 # [ 79.739375] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:15Z level=info event=scheduler_yield5432lb # [ 79.733791] niks3-server[1054]: 2026/09/20 09:34:14 INFO lead: released remote=192.168.1.4:452065433worker2 # [ 79.746257] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:15Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:340845434worker1 # [ 79.755885] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:15Z level=info event=scheduler_restarting addr=lb:500515435worker2 # [ 79.748675] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:15Z level=info event=worker_gone addr=192.168.1.3:500515436worker2 # [ 79.750793] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:15Z level=info event=scheduler_restarting addr=lb:500515437worker2 # [ 79.753454] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:15Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:340845438worker2 # [ 79.756160] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:15Z level=info event=worker_gone addr=192.168.1.4:500515439worker1 # [ 79.956142] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:15Z level=info event=scheduler_connected addr=lb:500515440worker2 # [ 79.949777] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500515441worker1 # [ 79.960233] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5442worker2 # [ 79.953949] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5443worker1 # [ 80.160941] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500515444worker2 # [ 80.154939] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500515445worker1 # [ 80.164961] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5446worker2 # [ 80.159103] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5447worker1 # [ 80.365725] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500515448worker2 # [ 80.359902] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500515449worker1 # [ 80.369593] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5450worker2 # [ 80.363859] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5451worker1 # [ 80.570430] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500515452worker2 # [ 80.564708] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500515453worker1 # [ 80.574623] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5454worker2 # [ 80.568646] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5455worker2 # [ 80.743609] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=scheduler_restarting5456worker2 # [ 80.754752] nix-grpc-daemon[1084]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5457worker2 # [ 80.759246] nix-grpc-daemon[1084]: E0000 00:00:1789896856.854680 1127 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5458worker2 # [ 80.763935] nix-grpc-daemon[1084]: E0000 00:00:1789896856.854967 1127 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5459worker1 # [ 80.776630] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_connected addr=lb:500515460worker1 # [ 80.779738] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5461worker2 # [ 80.781126] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5462worker2 # [ 80.789592] nix-daemon[847]: reaped child process 1133, status = succeeded5463worker1 # [ 80.977583] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500515464worker1 # [ 80.981397] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5465worker1 # [ 81.182164] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500515466worker1 # [ 81.186208] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5467worker1 # [ 81.386948] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500515468worker1 # [ 81.390878] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5469worker1 # [ 81.591670] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500515470worker1 # [ 81.595597] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5471worker1 # [ 81.797561] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_connected addr=lb:500515472worker1 # [ 81.800936] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5473worker1 # [ 82.001785] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_connected addr=lb:500515474worker1 # [ 82.005667] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5475worker1 # [ 82.206513] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_connected addr=lb:500515476worker1 # [ 82.210295] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5477worker1 # [ 82.411100] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_connected addr=lb:500515478worker1 # [ 82.414962] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5479lb # [ 82.472175] niks3-server[1054]: 2026/09/20 09:34:17 INFO lead: acquired remote=192.168.1.3:381505480worker1 # [ 82.492350] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_take_over5481worker1 # [ 82.615880] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_connected addr=lb:500515482worker1 # [ 82.619807] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5483worker2 # [ 82.760583] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5484worker2 # [ 82.764597] systemd[1]: Stopped Nix worker-protocol over gRPC.5485worker2 # [ 82.768340] systemd[1]: nix-grpc-daemon.service: Consumed 340ms CPU time over 8.223s wall clock time, 32.6M memory peak, 17.2K incoming IP traffic, 21.7K outgoing IP traffic.5486worker2 # [ 82.779236] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5487worker2 # [ 82.783623] systemd[1]: Closed Nix worker-protocol over gRPC.5488worker2 # [ 82.788168] systemd[1]: nix-grpc-daemon.socket: Consumed 282us CPU time over 8.844s wall clock time, 256K memory peak, 18.8K incoming IP traffic, 14K outgoing IP traffic.5489worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5490worker1 # [ 82.823648] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_connected addr=lb:500515491worker1 # [ 82.827192] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5492worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5493worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5494worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5495worker1 # [ 83.023391] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:19Z level=info event=scheduler_connected addr=lb:500515496worker1 # [ 83.027236] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker1 # [ 83.228058] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:19Z level=info event=scheduler_connected addr=lb:500515498worker1 # [ 83.231950] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker1 # [ 83.432745] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:19Z level=info event=scheduler_connected addr=lb:500515500worker1 # [ 83.440533] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05501worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5502worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5503client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25504worker1 # [ 84.194819] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333585505worker1 # [ 84.207393] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5506client # don't know how to build these paths:5507client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5508worker1 # [ 84.222494] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333585509worker1 # [ 84.233997] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)5510worker1 # [ 84.254112] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 104 ms (attempt 2/5)5511worker1 # [ 84.364077] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 103 ms (attempt 3/5)5512worker1 # [ 84.467502] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 4/5)5513worker1 # [ 84.535199] nix-grpc-daemon[1204]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5514worker1 # [ 84.553250] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:20Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05515worker1 # [ 84.555810] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:20Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15516worker1 # [ 84.596981] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333585517worker1 # [ 84.603821] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333585518client # copying 1 paths to 'lb:50051'...5519worker1 # [ 84.610768] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333585520client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5521worker1 # [ 84.616254] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5522worker1 # [ 84.627778] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:20Z level=info event=niks3_push_started pid=12785523worker1 # [ 84.980250] nix-daemon[842]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5524worker1 # [ 84.994513] nix-daemon[842]: reaped child process 1294, status = succeeded5525worker1 # [ 85.155808] nix-daemon[842]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5526worker1 # [ 85.169785] nix-daemon[842]: reaped child process 1299, status = succeeded5527lb # [ 85.164490] niks3-server[1054]: 2026/09/20 09:34:20 INFO Received uploads request method=POST path=/api/pending_closures5528worker1 # [ 85.189266] nix-grpc-daemon[1278]: time=2026-09-20T09:34:21.229Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5529worker1 # [ 85.192570] nix-grpc-daemon[1278]: time=2026-09-20T09:34:21.231Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5530lb # [ 85.199226] niks3-server[1054]: 2026/09/20 09:34:20 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5531lb # [ 85.212920] niks3-server[1054]: 2026/09/20 09:34:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5532worker1 # [ 85.235170] nix-grpc-daemon[1278]: time=2026-09-20T09:34:21.275Z level=INFO msg="Uploading 1 narinfos"5533lb # [ 85.218149] niks3-server[1054]: 2026/09/20 09:34:20 INFO Signed narinfos id=27 count=15534lb # [ 85.231615] niks3-server[1054]: 2026/09/20 09:34:20 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5535worker1 # [ 85.254048] nix-grpc-daemon[1278]: time=2026-09-20T09:34:21.294Z level=INFO msg="Upload complete. (438ms)"5536lb # [ 85.238304] niks3-server[1054]: 2026/09/20 09:34:20 INFO Completed upload id=275537worker1 # [ 85.256607] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33358 duration_s=0 paths=1 nar_bytes_in=5205538lb # [ 85.243686] niks3-server[1054]: 2026/09/20 09:34:20 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5539worker1 # [ 85.264311] nix-daemon[842]: reaped child process 1276, status = succeeded5540worker1 # [ 85.266350] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333585541worker1 # [ 85.268679] nix-grpc-daemon[1278]: time=2026-09-20T09:34:21.308Z level=WARN msg="Failed to register uploaded object" key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo error="server returned 404: object is not pending upload\n"5542worker1 # [ 85.275551] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5543worker1 # [ 85.281513] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5544client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5545client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5546client # farm-a> LOG-farm-a-on-standby5547worker1 # [ 86.699761] nix-daemon[842]: reaped child process 1303, status = succeeded5548worker1 # [ 86.874230] nix-daemon[842]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5549worker1 # [ 86.888782] nix-daemon[842]: reaped child process 1321, status = succeeded5550lb # [ 86.889263] niks3-server[1054]: 2026/09/20 09:34:22 INFO Received uploads request method=POST path=/api/pending_closures5551worker1 # [ 86.919204] nix-grpc-daemon[1278]: time=2026-09-20T09:34:22.959Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5552worker1 # [ 86.920833] nix-grpc-daemon[1278]: time=2026-09-20T09:34:22.961Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5553lb # [ 86.938782] niks3-server[1054]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5554lb # [ 86.944142] niks3-server[1054]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5555lb # [ 86.955144] niks3-server[1054]: 2026/09/20 09:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5556worker1 # [ 86.977903] nix-grpc-daemon[1278]: time=2026-09-20T09:34:23.017Z level=INFO msg="Uploading 1 narinfos"5557lb # [ 86.960676] niks3-server[1054]: 2026/09/20 09:34:22 INFO Signed narinfos id=28 count=15558lb # [ 86.965939] niks3-server[1054]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5559lb # [ 86.973875] niks3-server[1054]: 2026/09/20 09:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5560lb # [ 86.978967] niks3-server[1054]: 2026/09/20 09:34:22 INFO Completed upload id=285561lb # [ 86.980669] niks3-server[1054]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5562worker1 # [ 86.996129] nix-grpc-daemon[1278]: time=2026-09-20T09:34:23.036Z level=INFO msg="Upload complete. (306ms)"5563worker1 # [ 87.007824] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33358 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15564worker1 # [ 87.021527] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:23Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05565worker1 # [ 87.023606] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:23Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15566worker1 # [ 87.028772] nix-daemon[842]: reaped child process 1301, status = succeeded5567worker1 # [ 87.066206] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333585568worker1 # [ 87.077585] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333585569worker1 # [ 87.088779] nix-grpc-daemon[1278]: time=2026-09-20T09:34:23.129Z level=INFO msg="All 1 paths already cached"5570client # copying 1 paths to 'lb:50051'...5571client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5572worker1 # [ 87.097140] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333585573worker1 # [ 87.106684] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5574worker1 # [ 87.295224] nix-daemon[842]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)5575worker1 # [ 87.309445] nix-daemon[842]: reaped child process 1330, status = succeeded5576worker1 # [ 87.490199] nix-daemon[842]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5577worker1 # [ 87.503963] nix-daemon[842]: reaped child process 1335, status = succeeded5578lb # [ 87.504889] niks3-server[1054]: 2026/09/20 09:34:22 INFO Received uploads request method=POST path=/api/pending_closures5579worker1 # [ 87.529536] nix-grpc-daemon[1278]: time=2026-09-20T09:34:23.570Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5580worker1 # [ 87.531348] nix-grpc-daemon[1278]: time=2026-09-20T09:34:23.571Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5581lb # [ 87.530840] niks3-server[1054]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5582lb # [ 87.543799] niks3-server[1054]: 2026/09/20 09:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5583worker1 # [ 87.565945] nix-grpc-daemon[1278]: time=2026-09-20T09:34:23.606Z level=INFO msg="Uploading 1 narinfos"5584lb # [ 87.549052] niks3-server[1054]: 2026/09/20 09:34:22 INFO Signed narinfos id=29 count=15585lb # [ 87.554117] niks3-server[1054]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5586lb # [ 87.565755] niks3-server[1054]: 2026/09/20 09:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5587lb # [ 87.571526] niks3-server[1054]: 2026/09/20 09:34:22 INFO Completed upload id=295588worker1 # [ 87.588370] nix-grpc-daemon[1278]: time=2026-09-20T09:34:23.628Z level=INFO msg="Upload complete. (468ms)"5589worker1 # [ 87.591537] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33358 duration_s=0 paths=1 nar_bytes_in=6405590lb # [ 87.575293] niks3-server[1054]: 2026/09/20 09:34:22 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5591worker1 # [ 87.597279] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333585592worker1 # [ 87.604615] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5593worker1 # [ 87.610965] nix-daemon[842]: reaped child process 1325, status = succeeded5594worker1 # [ 87.616423] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5595client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5596client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5597client # farm-b> LOG-farm-b-on-standby5598worker1 # [ 88.644372] nix-daemon[842]: reaped child process 1339, status = succeeded5599worker1 # [ 88.803212] nix-daemon[842]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5600lb # [ 88.788504] envoy[926]: conn peer=[2001:db8:1::4]:54378 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1323 tx=349 ms=60185601worker1 # [ 88.817442] nix-daemon[842]: reaped child process 1357, status = succeeded5602lb # [ 88.817096] niks3-server[1054]: 2026/09/20 09:34:24 INFO Received uploads request method=POST path=/api/pending_closures5603worker1 # [ 88.842457] nix-grpc-daemon[1278]: time=2026-09-20T09:34:24.882Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5604worker1 # [ 88.843911] nix-grpc-daemon[1278]: time=2026-09-20T09:34:24.882Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5605lb # [ 88.849399] niks3-server[1054]: 2026/09/20 09:34:24 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5606lb # [ 88.860601] niks3-server[1054]: 2026/09/20 09:34:24 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5607lb # [ 88.869245] niks3-server[1054]: 2026/09/20 09:34:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5608lb # [ 88.873078] niks3-server[1054]: 2026/09/20 09:34:24 INFO Signed narinfos id=30 count=15609worker1 # [ 88.892499] nix-grpc-daemon[1278]: time=2026-09-20T09:34:24.932Z level=INFO msg="Uploading 1 narinfos"5610lb # [ 88.878601] niks3-server[1054]: 2026/09/20 09:34:24 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5611lb # [ 88.887030] niks3-server[1054]: 2026/09/20 09:34:24 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5612lb # [ 88.889466] niks3-server[1054]: 2026/09/20 09:34:24 INFO Completed upload id=305613worker1 # [ 88.906497] nix-grpc-daemon[1278]: time=2026-09-20T09:34:24.947Z level=INFO msg="Upload complete. (282ms)"5614lb # [ 88.892593] niks3-server[1054]: 2026/09/20 09:34:24 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5615worker1 # [ 88.914462] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33358 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15616worker1 # [ 88.923698] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:24Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05617worker1 # [ 88.926834] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:24Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15618worker1 # [ 88.933340] nix-daemon[842]: reaped child process 1337, status = succeeded5619worker1 # [ 88.967800] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333585620worker1 # [ 88.979298] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333585621worker1 # [ 88.992159] nix-grpc-daemon[1278]: time=2026-09-20T09:34:25.032Z level=INFO msg="All 2 paths already cached"5622client # copying 1 paths to 'lb:50051'...5623client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5624worker1 # [ 88.999115] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333585625worker1 # [ 89.007899] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5626worker1 # [ 89.182879] nix-daemon[842]: accepted connection from pid 1363, user nix-grpc-daemon (trusted)5627worker1 # [ 89.197094] nix-daemon[842]: reaped child process 1366, status = succeeded5628worker1 # [ 89.342127] nix-daemon[842]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)5629worker1 # [ 89.355789] nix-daemon[842]: reaped child process 1371, status = succeeded5630lb # [ 89.355473] niks3-server[1054]: 2026/09/20 09:34:24 INFO Received uploads request method=POST path=/api/pending_closures5631worker1 # [ 89.379812] nix-grpc-daemon[1278]: time=2026-09-20T09:34:25.420Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5632worker1 # [ 89.381309] nix-grpc-daemon[1278]: time=2026-09-20T09:34:25.421Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5633lb # [ 89.382332] niks3-server[1054]: 2026/09/20 09:34:24 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5634lb # [ 89.392807] niks3-server[1054]: 2026/09/20 09:34:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5635lb # [ 89.396964] niks3-server[1054]: 2026/09/20 09:34:24 INFO Signed narinfos id=31 count=15636worker1 # [ 89.416677] nix-grpc-daemon[1278]: time=2026-09-20T09:34:25.457Z level=INFO msg="Uploading 1 narinfos"5637lb # [ 89.405104] niks3-server[1054]: 2026/09/20 09:34:24 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5638lb # [ 89.417118] niks3-server[1054]: 2026/09/20 09:34:24 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5639lb # [ 89.421764] niks3-server[1054]: 2026/09/20 09:34:24 INFO Completed upload id=315640worker1 # [ 89.441069] nix-grpc-daemon[1278]: time=2026-09-20T09:34:25.481Z level=INFO msg="Upload complete. (420ms)"5641lb # [ 89.424602] niks3-server[1054]: 2026/09/20 09:34:24 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5642worker1 # [ 89.444080] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33358 duration_s=0 paths=1 nar_bytes_in=7685643worker1 # [ 89.447693] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333585644worker1 # [ 89.454500] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5645worker1 # [ 89.457250] nix-daemon[842]: reaped child process 1361, status = succeeded5646worker1 # [ 89.462341] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5647client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5648client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5649client # farm-top> LOG-farm-top-on-standby5650worker1 # [ 90.277660] nix-daemon[842]: reaped child process 1375, status = succeeded5651worker1 # [ 90.432200] nix-daemon[842]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5652worker1 # [ 90.446622] nix-daemon[842]: reaped child process 1393, status = succeeded5653lb # [ 90.445822] niks3-server[1054]: 2026/09/20 09:34:25 INFO Received uploads request method=POST path=/api/pending_closures5654worker1 # [ 90.467290] nix-grpc-daemon[1278]: time=2026-09-20T09:34:26.507Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5655worker1 # [ 90.468927] nix-grpc-daemon[1278]: time=2026-09-20T09:34:26.509Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5656lb # [ 90.463971] niks3-server[1054]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5657lb # [ 90.470491] niks3-server[1054]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5658lb # [ 90.473665] niks3-server[1054]: 2026/09/20 09:34:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5659lb # [ 90.475488] niks3-server[1054]: 2026/09/20 09:34:25 INFO Signed narinfos id=32 count=15660worker1 # [ 90.493884] nix-grpc-daemon[1278]: time=2026-09-20T09:34:26.534Z level=INFO msg="Uploading 1 narinfos"5661lb # [ 90.480760] niks3-server[1054]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5662lb # [ 90.484153] niks3-server[1054]: 2026/09/20 09:34:25 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5663lb # [ 90.487636] niks3-server[1054]: 2026/09/20 09:34:25 INFO Completed upload id=325664worker1 # [ 90.504538] nix-grpc-daemon[1278]: time=2026-09-20T09:34:26.545Z level=INFO msg="Upload complete. (235ms)"5665lb # [ 90.490721] niks3-server[1054]: 2026/09/20 09:34:25 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5666worker1 # [ 90.507496] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33358 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15667worker1 # [ 90.515664] nix-daemon[842]: reaped child process 1373, status = succeeded5668client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.55 seconds)5669client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5670client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5671worker2 # [ 90.790838] systemd[1]: Listening on Nix worker-protocol over gRPC.5672worker2 # [ 90.798665] systemd[1]: Starting Nix worker-protocol over gRPC...5673worker2 # [ 90.957166] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5674worker2 # [ 90.961051] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15675worker2 # [ 90.966954] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5676worker2 # [ 90.968541] systemd[1]: Started Nix worker-protocol over gRPC.5677lb: must succeed: curl -sf localhost:9901/clusters5678worker2 # [ 90.997731] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:27Z level=info event=scheduler_connected addr=lb:500515679worker1 # [ 91.010518] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05680lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5681worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5682worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5683lb: must succeed: curl -sf localhost:9901/clusters5684lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5685lb: must succeed: curl -sf localhost:9901/clusters5686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5687worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5688worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5689(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.68 seconds)5690subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5691client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5692client # [ 91.361981] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5693client # Running as unit: rs.service; invocation ID: fcf67f3f99bf4be4909037223831ec315694client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5695client # [ 91.576554] nix[1167]: don't know how to build these paths:5696client # [ 91.577587] nix[1167]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5697worker1 # [ 91.580844] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333585698worker1 # [ 91.593581] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:27Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05699worker1 # [ 91.598558] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:27Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15700worker1 # [ 91.637884] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333585701worker1 # [ 91.655255] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333585702client # [ 91.664181] nix[1167]: copying 1 paths to 'lb:50051'...5703worker1 # [ 91.671223] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333585704client # [ 91.667132] nix[1167]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5705worker1 # [ 91.684256] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5706worker1 # [ 91.853430] nix-daemon[842]: accepted connection from pid 1408, user nix-grpc-daemon (trusted)5707worker1 # [ 91.867183] nix-daemon[842]: reaped child process 1411, status = succeeded5708worker1 # [ 92.050298] nix-daemon[842]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)5709worker1 # [ 92.064109] nix-daemon[842]: reaped child process 1416, status = succeeded5710lb # [ 92.064843] niks3-server[1054]: 2026/09/20 09:34:27 INFO Received uploads request method=POST path=/api/pending_closures5711worker1 # [ 92.090232] nix-grpc-daemon[1278]: time=2026-09-20T09:34:28.130Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5712worker1 # [ 92.091737] nix-grpc-daemon[1278]: time=2026-09-20T09:34:28.132Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5713lb # [ 92.099590] niks3-server[1054]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5714lb # [ 92.106639] niks3-server[1054]: 2026/09/20 09:34:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5715lb # [ 92.111841] niks3-server[1054]: 2026/09/20 09:34:27 INFO Signed narinfos id=33 count=15716worker1 # [ 92.131816] nix-grpc-daemon[1278]: time=2026-09-20T09:34:28.172Z level=INFO msg="Uploading 1 narinfos"5717lb # [ 92.120938] niks3-server[1054]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5718lb # [ 92.132719] niks3-server[1054]: 2026/09/20 09:34:27 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5719lb # [ 92.138410] niks3-server[1054]: 2026/09/20 09:34:27 INFO Completed upload id=335720worker1 # [ 92.155229] nix-grpc-daemon[1278]: time=2026-09-20T09:34:28.195Z level=INFO msg="Upload complete. (457ms)"5721lb # [ 92.141235] niks3-server[1054]: 2026/09/20 09:34:27 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5722worker1 # [ 92.159379] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33358 duration_s=0 paths=1 nar_bytes_in=4165723worker1 # [ 92.165937] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333585724worker1 # [ 92.176133] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5725worker1 # [ 92.183890] nix-daemon[842]: reaped child process 1406, status = succeeded5726worker1 # [ 92.189154] nix-daemon[842]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5727client # [ 92.182755] nix[1167]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5728lb: must succeed: curl -sf localhost:9901/clusters5729lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5730lb: must succeed: curl -sf localhost:9901/clusters5731lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5732lb: must succeed: curl -sf localhost:9901/clusters5733lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5734lb: must succeed: curl -sf localhost:9901/clusters5735lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5736worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5737worker2 # [ 94.238266] systemd-journald[287]: Received client request to rotate journal, rotating.5738worker2 # [ 94.255671] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/2cc425d102f84ab5a62ad5f6317abc92.5739worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5740worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5741worker1 # [ 94.186385] systemd[1]: Stopping Nix worker-protocol over gRPC...5742worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5743??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5744 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735745client: must succeed: sleep 5.05746worker1 # [ 94.486885] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:30Z level=info event=draining5747lb # [ 98.790542] envoy[926]: conn peer=[2001:db8:1::1]:39658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9577 tx=3818 ms=63315748client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5749worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5750worker1 # [ 99.620767] nix-daemon[842]: reaped child process 1420, status = succeeded5751worker1 # [ 99.783766] nix-daemon[842]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)5752worker1 # [ 99.794530] nix-daemon[842]: reaped child process 1463, status = succeeded5753lb # [ 99.790880] niks3-server[1054]: 2026/09/20 09:34:35 INFO Received uploads request method=POST path=/api/pending_closures5754worker1 # [ 99.812855] nix-grpc-daemon[1278]: time=2026-09-20T09:34:35.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5755worker1 # [ 99.814340] nix-grpc-daemon[1278]: time=2026-09-20T09:34:35.854Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5756lb # [ 99.814608] niks3-server[1054]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5757lb # [ 99.817836] niks3-server[1054]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5758lb # [ 99.822056] niks3-server[1054]: 2026/09/20 09:34:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5759lb # [ 99.824455] niks3-server[1054]: 2026/09/20 09:34:35 INFO Signed narinfos id=34 count=15760worker1 # [ 99.841458] nix-grpc-daemon[1278]: time=2026-09-20T09:34:35.881Z level=INFO msg="Uploading 1 narinfos"5761lb # [ 99.830513] niks3-server[1054]: 2026/09/20 09:34:35 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5762lb # [ 99.833939] niks3-server[1054]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5763lb # [ 99.835923] niks3-server[1054]: 2026/09/20 09:34:35 INFO Completed upload id=345764worker1 # [ 99.853897] nix-grpc-daemon[1278]: time=2026-09-20T09:34:35.894Z level=INFO msg="Upload complete. (250ms)"5765worker1 # [ 99.856406] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33358 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15766worker1 # [ 99.860394] nix-grpc-daemon[1278]: time=2026-09-20T09:34:35.899Z level=WARN msg="Failed to register uploaded object" key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo error="server returned 404: object is not pending upload\n"5767worker1 # [ 99.866299] nix-daemon[842]: reaped child process 1418, status = succeeded5768client # [ 99.891319] systemd[1]: rs.service: Deactivated successfully.5769client # [ 99.894215] systemd[1]: rs.service: Consumed 116ms CPU time over 8.528s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5770worker1 # [ 100.487585] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=signal_exit5771worker1 # [ 100.711378] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=scheduler_yield5772lb # [ 100.696735] niks3-server[1054]: 2026/09/20 09:34:35 INFO lead: released remote=192.168.1.3:381505773worker1 # [ 100.715818] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=scheduler_restarting addr=lb:500515774worker1 # [ 100.717700] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:333445775worker1 # [ 100.719238] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=worker_gone addr=192.168.1.3:500515776worker2 # [ 100.709072] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:36Z level=info event=scheduler_restarting addr=lb:500515777worker1 # [ 100.720465] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:333445778worker1 # [ 100.721757] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=worker_gone addr=192.168.1.4:500515779worker1 # [ 100.916810] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=scheduler_connected addr=lb:500515780worker2 # [ 100.908112] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_connected addr=lb:500515781worker1 # [ 100.919852] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5782worker2 # [ 100.910415] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5783lb # [ 100.970714] niks3-server[1054]: 2026/09/20 09:34:36 INFO lead: acquired remote=192.168.1.4:365145784worker2 # [ 100.981273] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_take_over5785worker1 # [ 101.120532] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:37Z level=info event=scheduler_connected addr=lb:500515786worker2 # [ 101.111094] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_connected addr=lb:500515787worker2 # [ 101.114089] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5788worker1 # [ 101.124948] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5789worker2 # [ 101.314692] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_connected addr=lb:500515790worker1 # [ 101.325512] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:37Z level=info event=scheduler_connected addr=lb:500515791worker2 # [ 101.316595] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5792worker1 # [ 101.327233] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5793worker1 # [ 101.527802] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:37Z level=info event=scheduler_connected addr=lb:500515794worker2 # [ 101.517681] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_connected addr=lb:500515795worker1 # [ 101.529683] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5796worker2 # [ 101.522500] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5797worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.15 seconds)5798client: waiting for success: ! systemctl is-active rs5799client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5800client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5801client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5802client: must fail: journalctl -u rs | grep -q 'reconnecting'5803client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5804lb: must succeed: curl -sf localhost:9901/clusters5805worker1 # [ 101.715053] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:37Z level=info event=scheduler_restarting5806worker1 # [ 101.726756] nix-daemon[842]: reaped child process 1269, status = succeeded5807worker1 # [ 101.734992] nix-grpc-daemon[1204]: ts=2026-09-20T09:34:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5808worker2 # [ 101.724260] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_connected addr=lb:500515809worker2 # [ 101.727905] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5811worker2 # [ 101.924885] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_connected addr=lb:500515812worker2 # [ 101.928899] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5813worker2 # [ 102.129729] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_connected addr=lb:500515814worker2 # [ 102.133726] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5815worker1 # [ 102.299678] nix-grpc-daemon[1204]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5816worker1 # [ 102.301090] nix-grpc-daemon[1204]: E0000 00:00:1789896878.339567 1247 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5817worker1 # [ 102.302978] nix-grpc-daemon[1204]: E0000 00:00:1789896878.339882 1247 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5818worker2 # [ 102.334615] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_connected addr=lb:500515819worker2 # [ 102.338542] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker2 # [ 102.539417] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_connected addr=lb:500515821worker2 # [ 102.543303] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822worker2 # [ 102.745479] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 102.748531] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5824lb: must succeed: curl -sf localhost:9901/clusters5825lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5826worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5827worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.08 seconds)5828worker2 # [ 102.946095] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:39Z level=info event=scheduler_connected addr=lb:500515829worker2 # [ 102.952147] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05830worker1 # [ 103.167180] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5831worker1 # [ 103.171190] systemd[1]: Stopped Nix worker-protocol over gRPC.5832worker1 # [ 103.173267] systemd[1]: nix-grpc-daemon.service: Consumed 1.381s CPU time over 25.859s wall clock time, 35.4M memory peak, 47.7K incoming IP traffic, 89.9K outgoing IP traffic.5833worker1 # [ 103.188901] systemd[1]: Starting Nix worker-protocol over gRPC...5834worker1 # [ 103.352293] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5835worker1 # [ 103.356209] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15836worker1 # [ 103.362316] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5837worker1 # [ 103.365905] systemd[1]: Started Nix worker-protocol over gRPC.5838worker1 # [ 103.386382] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:39Z level=info event=scheduler_connected addr=lb:500515839worker2 # [ 103.378785] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05840lb: must succeed: curl -sf localhost:9901/clusters5841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5842worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5843worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5844(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.73 seconds)5845subtest: scheduler restart mid-build: build finishes, no double build5846client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5847client # [ 104.093463] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5848client # Running as unit: mid.service; invocation ID: d4b238a13bcb482eac3df846a7926cfa5849client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5850worker2 # [ 104.300763] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507645851worker2 # [ 104.311928] nix-daemon[847]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5852client # [ 104.327863] nix[1202]: don't know how to build these paths:5853client # [ 104.328807] nix[1202]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5854worker2 # [ 104.330682] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:40Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05855worker2 # [ 104.334749] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:40Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15856worker2 # [ 104.383599] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507645857worker2 # [ 104.394349] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5)5858worker2 # [ 104.463087] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 2/5)5859worker2 # [ 104.521625] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 282 ms (attempt 3/5)5860worker2 # [ 104.809462] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 375 ms (attempt 4/5)5861worker2 # [ 105.191191] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5862worker2 # [ 105.209557] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507645863client # [ 105.222964] nix[1202]: copying 1 paths to 'lb:50051'...5864client # [ 105.227194] nix[1202]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5865worker2 # [ 105.223688] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507645866worker2 # [ 105.232731] nix-daemon[847]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5867worker2 # [ 105.254966] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:41Z level=info event=niks3_push_started pid=13015868worker2 # [ 105.569107] nix-daemon[847]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5869worker2 # [ 105.579391] nix-daemon[847]: reaped child process 1321, status = succeeded5870worker2 # [ 105.719492] nix-daemon[847]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5871worker2 # [ 105.730886] nix-daemon[847]: reaped child process 1326, status = succeeded5872lb # [ 105.736616] niks3-server[1054]: 2026/09/20 09:34:40 INFO Received uploads request method=POST path=/api/pending_closures5873worker2 # [ 105.750462] nix-grpc-daemon[1301]: time=2026-09-20T09:34:41.850Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5874worker2 # [ 105.751981] nix-grpc-daemon[1301]: time=2026-09-20T09:34:41.852Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5875lb # [ 105.758424] niks3-server[1054]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5876lb # [ 105.760872] niks3-server[1054]: 2026/09/20 09:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5877lb # [ 105.763725] niks3-server[1054]: 2026/09/20 09:34:41 INFO Signed narinfos id=35 count=15878worker2 # [ 105.771162] nix-grpc-daemon[1301]: time=2026-09-20T09:34:41.871Z level=INFO msg="Uploading 1 narinfos"5879lb # [ 105.767579] niks3-server[1054]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5880lb # [ 105.772844] niks3-server[1054]: 2026/09/20 09:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5881lb # [ 105.775827] niks3-server[1054]: 2026/09/20 09:34:41 INFO Completed upload id=355882worker2 # [ 105.784402] nix-grpc-daemon[1301]: time=2026-09-20T09:34:41.884Z level=INFO msg="Upload complete. (355ms)"5883lb # [ 105.777308] niks3-server[1054]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5884worker2 # [ 105.787451] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50764 duration_s=0 paths=1 nar_bytes_in=4245885worker2 # [ 105.790138] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507645886worker2 # [ 105.796291] nix-daemon[847]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5887worker2 # [ 105.799651] nix-daemon[847]: reaped child process 1299, status = succeeded5888worker2 # [ 105.804071] nix-daemon[847]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5889client # [ 105.808129] nix[1202]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5890lb: must succeed: curl -sf localhost:9901/clusters5891lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5892lb: must succeed: curl -sf localhost:9901/clusters5893lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5894worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5895worker2 # [ 106.697580] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1177 (nix-grpc-daemon) on client request.5896worker2 # [ 106.699380] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5897lb # [ 106.702480] niks3-server[1054]: 2026/09/20 09:34:41 INFO lead: released remote=192.168.1.4:365145898worker1 # [ 106.726552] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:42Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5899client # [ 106.721790] nix[1202]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5900worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5901client: must succeed: sleep 5.05902client # [ 106.726724] nix[1202]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5903worker2 # [ 106.735601] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5904worker2 # [ 106.737221] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5905worker2 # [ 106.738933] systemd[1]: nix-grpc-daemon.service: Consumed 425ms CPU time over 15.936s wall clock time, 33.1M memory peak, 15.1K incoming IP traffic, 23.3K outgoing IP traffic.5906worker2 # [ 106.742708] nix-daemon[847]: reaped child process 1328, status = succeeded5907worker2 # [ 106.744119] nix-daemon[847]: reaped child process 1292, status = succeeded5908worker2 # [ 106.750238] nix-daemon[1330]: unexpected Nix daemon error: error: interrupted by the user5909worker2 # [ 106.754759] nix-daemon[847]: reaped child process 1330, status = failed with exit code 15910worker1 # [ 106.926818] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:42Z level=info event=scheduler_connected addr=lb:500515911worker2 # [ 107.023594] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5912worker2 # [ 107.029077] systemd[1]: Starting Nix worker-protocol over gRPC...5913worker2 # [ 107.170237] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5914worker2 # [ 107.172351] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15915worker2 # [ 107.177948] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5916worker2 # [ 107.179562] systemd[1]: Started Nix worker-protocol over gRPC.5917worker2 # [ 107.189236] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=info event=scheduler_take_over5918lb # [ 107.182750] niks3-server[1054]: 2026/09/20 09:34:42 INFO lead: acquired remote=192.168.1.4:343905919worker2 # [ 107.199425] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05920worker2 # [ 107.213407] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=info event=scheduler_connected addr=lb:500515921worker2 # [ 107.215090] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05922worker2 # [ 107.229264] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05923worker2 # [ 107.231165] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:43Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15924worker1 # [ 107.289349] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358725925worker1 # [ 107.294723] nix-daemon[842]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5926worker1 # [ 107.302723] nix-grpc-daemon[1470]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)5927worker1 # [ 107.385254] nix-grpc-daemon[1470]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5)5928worker1 # [ 107.418379] nix-grpc-daemon[1470]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 385 ms (attempt 3/5)5929worker1 # [ 107.806860] nix-grpc-daemon[1470]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 531 ms (attempt 4/5)5930worker1 # [ 108.343691] nix-grpc-daemon[1470]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5931worker1 # [ 108.354497] nix-grpc-daemon[1470]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5932worker1 # [ 108.367697] nix-daemon[842]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5933worker1 # [ 108.374540] nix-daemon[842]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)5934client # [ 108.369293] nix[1202]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5935lb # [ 108.790510] envoy[926]: conn peer=[2001:db8:1::1]:39662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2934 tx=1349 ms=82945936lb # [ 108.794360] envoy[926]: conn peer=[2001:db8:1::3]:47612 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5853 tx=1512 ms=242285937lb # [ 108.797960] envoy[926]: conn peer=[2001:db8:1::4]:56590 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2724 tx=703 ms=157255938client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5939worker1 # [ 111.985479] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:48Z level=info event=niks3_push_started pid=15675940worker1 # [ 111.996133] nix-daemon[842]: reaped child process 1543, status = succeeded5941client: waiting for success: ! systemctl is-active mid5942worker1 # [ 112.318469] nix-daemon[842]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)5943worker1 # [ 112.329123] nix-daemon[842]: reaped child process 1583, status = succeeded5944lb # [ 112.330548] niks3-server[1054]: 2026/09/20 09:34:47 INFO Received uploads request method=POST path=/api/pending_closures5945worker1 # [ 112.356559] nix-grpc-daemon[1567]: time=2026-09-20T09:34:48.396Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5946worker1 # [ 112.358129] nix-grpc-daemon[1567]: time=2026-09-20T09:34:48.398Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5947lb # [ 112.375356] niks3-server[1054]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5948lb # [ 112.383277] niks3-server[1054]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5949lb # [ 112.392389] niks3-server[1054]: 2026/09/20 09:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5950worker1 # [ 112.416568] nix-grpc-daemon[1567]: time=2026-09-20T09:34:48.456Z level=INFO msg="Uploading 1 narinfos"5951lb # [ 112.400353] niks3-server[1054]: 2026/09/20 09:34:47 INFO Signed narinfos id=36 count=15952lb # [ 112.412283] niks3-server[1054]: 2026/09/20 09:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5953lb # [ 112.418318] niks3-server[1054]: 2026/09/20 09:34:47 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5954lb # [ 112.421264] niks3-server[1054]: 2026/09/20 09:34:47 INFO Completed upload id=365955worker1 # [ 112.438376] nix-grpc-daemon[1567]: time=2026-09-20T09:34:48.478Z level=INFO msg="Upload complete. (275ms)"5956worker1 # [ 112.442061] nix-grpc-daemon[1567]: time=2026-09-20T09:34:48.481Z level=WARN msg="Failed to register uploaded object" key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo error="server returned 404: object is not pending upload\n"5957worker1 # [ 112.444598] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35872 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15958worker1 # [ 112.450720] nix-daemon[842]: reaped child process 1541, status = succeeded5959client # [ 112.470601] systemd[1]: mid.service: Deactivated successfully.5960client # [ 112.474343] systemd[1]: mid.service: Consumed 129ms CPU time over 8.380s wall clock time, 37.1M memory peak, 9.7K incoming IP traffic, 10.6K outgoing IP traffic.5961client: (finished: waiting for success: ! systemctl is-active mid, in 1.10 seconds)5962client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5963client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5964(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.11 seconds)5965subtest: interrupting the client stops the build on the worker5966client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5967client # [ 113.178737] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5968client # Running as unit: intr.service; invocation ID: f9b05fbc67a748deb68ee5e524bc82e85969client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.03 seconds)5970worker2 # [ 113.356792] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457665971worker2 # [ 113.365630] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)5972client # [ 113.383292] nix[1237]: don't know how to build these paths:5973client # [ 113.384181] nix[1237]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5974worker2 # [ 113.387543] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:49Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05975worker2 # [ 113.392800] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:49Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=15976worker2 # [ 113.436090] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457665977worker2 # [ 113.447099] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)5978worker2 # [ 113.532922] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 2/5)5979worker2 # [ 113.672722] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 3/5)5980worker2 # [ 113.735626] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 631 ms (attempt 4/5)5981worker2 # [ 114.373035] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5982worker2 # [ 114.387066] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457665983client # [ 114.396513] nix[1237]: copying 1 paths to 'lb:50051'...5984worker2 # [ 114.393211] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457665985client # [ 114.397430] nix[1237]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5986worker2 # [ 114.399922] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)5987worker2 # [ 114.410142] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:50Z level=info event=niks3_push_started pid=14385988worker2 # [ 114.710076] nix-daemon[847]: accepted connection from pid 1451, user nix-grpc-daemon (trusted)5989worker2 # [ 114.721517] nix-daemon[847]: reaped child process 1454, status = succeeded5990worker2 # [ 114.863432] nix-daemon[847]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5991worker2 # [ 114.874948] nix-daemon[847]: reaped child process 1459, status = succeeded5992lb # [ 114.881443] niks3-server[1054]: 2026/09/20 09:34:50 INFO Received uploads request method=POST path=/api/pending_closures5993worker2 # [ 114.901279] nix-grpc-daemon[1438]: time=2026-09-20T09:34:51.000Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5994worker2 # [ 114.902607] nix-grpc-daemon[1438]: time=2026-09-20T09:34:51.000Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5995lb # [ 114.926818] niks3-server[1054]: 2026/09/20 09:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5996lb # [ 114.933364] niks3-server[1054]: 2026/09/20 09:34:50 INFO Signed narinfos id=37 count=15997worker2 # [ 114.944136] nix-grpc-daemon[1438]: time=2026-09-20T09:34:51.044Z level=INFO msg="Uploading 1 narinfos"5998lb # [ 114.937100] niks3-server[1054]: 2026/09/20 09:34:50 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5999lb # [ 114.949670] niks3-server[1054]: 2026/09/20 09:34:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6000lb # [ 114.957275] niks3-server[1054]: 2026/09/20 09:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6001lb # [ 114.965351] niks3-server[1054]: 2026/09/20 09:34:50 INFO Completed upload id=376002lb # [ 114.966646] niks3-server[1054]: 2026/09/20 09:34:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6003worker2 # [ 114.973115] nix-grpc-daemon[1438]: time=2026-09-20T09:34:51.073Z level=INFO msg="Upload complete. (404ms)"6004worker2 # [ 114.977162] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=0 paths=1 nar_bytes_in=4246005worker2 # [ 114.982275] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666006worker2 # [ 114.990744] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6007worker2 # [ 115.000129] nix-daemon[847]: reaped child process 1436, status = succeeded6008worker2 # [ 115.003745] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6009client # [ 115.007332] nix[1237]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6010client: must succeed: systemctl kill -s INT intr6011client # [ 116.667460] systemd[1]: intr.service: Sent signal SIGINT to main process 1237 (nix) on client request.6012client # [ 116.673739] nix[1237]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6013client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6014client # [ 116.680935] nix[1237]: error: interrupted by the user6015client # [ 116.710509] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6016client # [ 116.711953] systemd[1]: intr.service: Failed with result 'exit-code'.6017client # [ 116.713929] systemd[1]: intr.service: Consumed 106ms CPU time over 3.535s wall clock time, 39.1M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.6018worker2 # [ 116.821304] nix-daemon[847]: reaped child process 1461, status = succeeded6019worker2 # [ 116.823229] nix-daemon[1463]: unexpected Nix daemon error: error: interrupted by the user6020worker2 # [ 116.827483] nix-daemon[847]: reaped child process 1463, status = failed with exit code 16021(finished: subtest: interrupting the client stops the build on the worker, in 4.75 seconds)6022subtest: a deploy mid-build drains: the build finishes, the new generation takes over6023client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6024client # [ 117.957687] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6025client # Running as unit: sw.service; invocation ID: 7a37afc340e74e56b0abe547203b1b336026client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6027client # [ 118.146074] nix[1257]: don't know how to build these paths:6028client # [ 118.146929] nix[1257]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6029worker2 # [ 118.141451] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457666030worker2 # [ 118.153150] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:54Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06031worker2 # [ 118.157149] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:54Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16032worker1 # [ 118.211207] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358726033worker1 # [ 118.224257] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358726034client # [ 118.229285] nix[1257]: copying 1 paths to 'lb:50051'...6035worker1 # [ 118.235732] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358726036client # [ 118.232142] nix[1257]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6037worker1 # [ 118.244766] nix-daemon[842]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)6038worker1 # [ 118.407036] nix-daemon[842]: accepted connection from pid 1615, user nix-grpc-daemon (trusted)6039worker1 # [ 118.419104] nix-daemon[842]: reaped child process 1618, status = succeeded6040worker1 # [ 118.582084] nix-daemon[842]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6041worker1 # [ 118.594164] nix-daemon[842]: reaped child process 1623, status = succeeded6042lb # [ 118.593574] niks3-server[1054]: 2026/09/20 09:34:53 INFO Received uploads request method=POST path=/api/pending_closures6043worker1 # [ 118.619041] nix-grpc-daemon[1567]: time=2026-09-20T09:34:54.659Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6044worker1 # [ 118.620539] nix-grpc-daemon[1567]: time=2026-09-20T09:34:54.661Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6045lb # [ 118.622786] niks3-server[1054]: 2026/09/20 09:34:53 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6046lb # [ 118.636198] niks3-server[1054]: 2026/09/20 09:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6047worker1 # [ 118.660262] nix-grpc-daemon[1567]: time=2026-09-20T09:34:54.700Z level=INFO msg="Uploading 1 narinfos"6048lb # [ 118.642885] niks3-server[1054]: 2026/09/20 09:34:53 INFO Signed narinfos id=38 count=16049lb # [ 118.650084] niks3-server[1054]: 2026/09/20 09:34:53 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6050lb # [ 118.661239] niks3-server[1054]: 2026/09/20 09:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6051worker1 # [ 118.683582] nix-grpc-daemon[1567]: time=2026-09-20T09:34:54.724Z level=INFO msg="Upload complete. (425ms)"6052lb # [ 118.666798] niks3-server[1054]: 2026/09/20 09:34:53 INFO Completed upload id=386053worker1 # [ 118.685479] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35872 duration_s=0 paths=1 nar_bytes_in=4166054lb # [ 118.669672] niks3-server[1054]: 2026/09/20 09:34:53 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6055worker1 # [ 118.691424] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358726056worker1 # [ 118.695978] nix-daemon[842]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)6057worker1 # [ 118.698226] nix-daemon[842]: reaped child process 1613, status = succeeded6058worker1 # [ 118.703446] nix-daemon[842]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)6059client # [ 118.697313] nix[1257]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6060lb # [ 118.791103] envoy[926]: conn peer=[2001:db8:1::1]:33286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3835 tx=1660 ms=81506061lb # [ 118.794991] envoy[926]: conn peer=[2001:db8:1::1]:54728 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=757 ms=33376062worker1: must succeed: systemctl reload nix-grpc-daemon.service6063worker1 # [ 120.496283] systemd[1]: Reloading Nix worker-protocol over gRPC...6064worker1 # [ 120.532274] systemd[1]: Reloaded Nix worker-protocol over gRPC.6065worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6066worker1: must succeed: pgrep -f 'read -t [9]0 x'6067worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6068client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26069worker2 # [ 120.791374] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457666070client # don't know how to build these paths:6071client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6072worker2 # [ 120.799489] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457666073worker2 # [ 120.821092] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:56Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06074worker2 # [ 120.826227] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:56Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16075worker2 # [ 120.864460] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666076worker2 # [ 120.876669] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457666077client # copying 1 paths to 'lb:50051'...6078worker2 # [ 120.888884] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457666079client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6080worker2 # [ 120.910400] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6081worker2 # [ 121.077372] nix-daemon[847]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6082worker2 # [ 121.089142] nix-daemon[847]: reaped child process 1520, status = succeeded6083worker2 # [ 121.238303] nix-daemon[847]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)6084worker2 # [ 121.249859] nix-daemon[847]: reaped child process 1525, status = succeeded6085lb # [ 121.258242] niks3-server[1054]: 2026/09/20 09:34:56 INFO Received uploads request method=POST path=/api/pending_closures6086worker2 # [ 121.270905] nix-grpc-daemon[1438]: time=2026-09-20T09:34:57.370Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6087worker2 # [ 121.272374] nix-grpc-daemon[1438]: time=2026-09-20T09:34:57.372Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6088lb # [ 121.274605] niks3-server[1054]: 2026/09/20 09:34:56 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6089lb # [ 121.279050] niks3-server[1054]: 2026/09/20 09:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6090lb # [ 121.282038] niks3-server[1054]: 2026/09/20 09:34:56 INFO Signed narinfos id=39 count=16091worker2 # [ 121.289324] nix-grpc-daemon[1438]: time=2026-09-20T09:34:57.389Z level=INFO msg="Uploading 1 narinfos"6092lb # [ 121.284303] niks3-server[1054]: 2026/09/20 09:34:56 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6093lb # [ 121.289160] niks3-server[1054]: 2026/09/20 09:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6094lb # [ 121.292526] niks3-server[1054]: 2026/09/20 09:34:56 INFO Completed upload id=396095worker2 # [ 121.299891] nix-grpc-daemon[1438]: time=2026-09-20T09:34:57.400Z level=INFO msg="Upload complete. (378ms)"6096lb # [ 121.294439] niks3-server[1054]: 2026/09/20 09:34:56 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6097worker2 # [ 121.301897] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=0 paths=1 nar_bytes_in=5286098worker2 # [ 121.306159] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666099worker2 # [ 121.312661] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6100worker2 # [ 121.314039] nix-daemon[847]: reaped child process 1515, status = succeeded6101client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6102client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6103worker2 # [ 121.319518] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6104worker1 # [ 121.370372] nix-grpc-daemon[1470]: ts=2026-09-20T09:34:57Z level=info event=draining6105client # farm-a> LOG-farm-a-during-drain6106worker2 # [ 122.548429] nix-daemon[847]: reaped child process 1529, status = succeeded6107worker2 # [ 122.706440] nix-daemon[847]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6108worker2 # [ 122.718288] nix-daemon[847]: reaped child process 1547, status = succeeded6109lb # [ 122.726410] niks3-server[1054]: 2026/09/20 09:34:57 INFO Received uploads request method=POST path=/api/pending_closures6110worker2 # [ 122.742034] nix-grpc-daemon[1438]: time=2026-09-20T09:34:58.842Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6111worker2 # [ 122.743682] nix-grpc-daemon[1438]: time=2026-09-20T09:34:58.843Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6112lb # [ 122.765315] niks3-server[1054]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6113lb # [ 122.775310] niks3-server[1054]: 2026/09/20 09:34:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6114lb # [ 122.779979] niks3-server[1054]: 2026/09/20 09:34:58 INFO Signed narinfos id=40 count=16115worker2 # [ 122.790873] nix-grpc-daemon[1438]: time=2026-09-20T09:34:58.890Z level=INFO msg="Uploading 1 narinfos"6116lb # [ 122.786195] niks3-server[1054]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6117lb # [ 122.793570] niks3-server[1054]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6118lb # [ 122.801380] niks3-server[1054]: 2026/09/20 09:34:58 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6119lb # [ 122.810692] niks3-server[1054]: 2026/09/20 09:34:58 INFO Completed upload id=406120lb # [ 122.812378] niks3-server[1054]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6121worker2 # [ 122.818457] nix-grpc-daemon[1438]: time=2026-09-20T09:34:58.918Z level=INFO msg="Upload complete. (290ms)"6122worker2 # [ 122.827374] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16123worker2 # [ 122.834893] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:58Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06124worker2 # [ 122.837930] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:58Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16125worker2 # [ 122.846253] nix-daemon[847]: reaped child process 1527, status = succeeded6126worker2 # [ 122.883974] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666127worker2 # [ 122.896447] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457666128worker2 # [ 122.909613] nix-grpc-daemon[1438]: time=2026-09-20T09:34:59.009Z level=INFO msg="All 1 paths already cached"6129client # copying 1 paths to 'lb:50051'...6130worker2 # [ 122.917660] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457666131client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6132worker2 # [ 122.938175] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6133worker2 # [ 123.090190] nix-daemon[847]: accepted connection from pid 1553, user nix-grpc-daemon (trusted)6134worker2 # [ 123.102259] nix-daemon[847]: reaped child process 1556, status = succeeded6135worker2 # [ 123.254080] nix-daemon[847]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6136worker2 # [ 123.266191] nix-daemon[847]: reaped child process 1561, status = succeeded6137lb # [ 123.271712] niks3-server[1054]: 2026/09/20 09:34:58 INFO Received uploads request method=POST path=/api/pending_closures6138worker2 # [ 123.288441] nix-grpc-daemon[1438]: time=2026-09-20T09:34:59.388Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6139worker2 # [ 123.290549] nix-grpc-daemon[1438]: time=2026-09-20T09:34:59.390Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6140lb # [ 123.301794] niks3-server[1054]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6141lb # [ 123.313337] niks3-server[1054]: 2026/09/20 09:34:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6142worker2 # [ 123.326109] nix-grpc-daemon[1438]: time=2026-09-20T09:34:59.426Z level=INFO msg="Uploading 1 narinfos"6143lb # [ 123.318598] niks3-server[1054]: 2026/09/20 09:34:58 INFO Signed narinfos id=41 count=16144lb # [ 123.322755] niks3-server[1054]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6145lb # [ 123.333462] niks3-server[1054]: 2026/09/20 09:34:58 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6146worker2 # [ 123.348624] nix-grpc-daemon[1438]: time=2026-09-20T09:34:59.448Z level=INFO msg="Upload complete. (398ms)"6147lb # [ 123.341288] niks3-server[1054]: 2026/09/20 09:34:58 INFO Completed upload id=416148lb # [ 123.344432] niks3-server[1054]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6149worker2 # [ 123.351858] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=0 paths=1 nar_bytes_in=6406150worker2 # [ 123.356360] nix-grpc-daemon[1362]: ts=2026-09-20T09:34:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666151worker2 # [ 123.364497] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6152worker2 # [ 123.373143] nix-daemon[847]: reaped child process 1551, status = succeeded6153worker2 # [ 123.376353] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6154client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6155client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6156client # farm-b> LOG-farm-b-during-drain6157worker2 # [ 124.789921] nix-daemon[847]: reaped child process 1565, status = succeeded6158worker2 # [ 124.916273] nix-daemon[847]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)6159worker2 # [ 124.928272] nix-daemon[847]: reaped child process 1583, status = succeeded6160lb # [ 124.935056] niks3-server[1054]: 2026/09/20 09:35:00 INFO Received uploads request method=POST path=/api/pending_closures6161worker2 # [ 124.949212] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.049Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6162worker2 # [ 124.950650] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.050Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6163lb # [ 124.966149] niks3-server[1054]: 2026/09/20 09:35:00 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6164lb # [ 124.968939] niks3-server[1054]: 2026/09/20 09:35:00 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6165lb # [ 124.973424] niks3-server[1054]: 2026/09/20 09:35:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6166lb # [ 124.975183] niks3-server[1054]: 2026/09/20 09:35:00 INFO Signed narinfos id=42 count=16167worker2 # [ 124.983900] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.083Z level=INFO msg="Uploading 1 narinfos"6168lb # [ 124.979266] niks3-server[1054]: 2026/09/20 09:35:00 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6169lb # [ 124.984156] niks3-server[1054]: 2026/09/20 09:35:00 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6170lb # [ 124.987541] niks3-server[1054]: 2026/09/20 09:35:00 INFO Completed upload id=426171worker2 # [ 124.994733] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.094Z level=INFO msg="Upload complete. (210ms)"6172lb # [ 124.990425] niks3-server[1054]: 2026/09/20 09:35:00 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6173worker2 # [ 124.997308] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16174worker2 # [ 125.003532] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:01Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06175worker2 # [ 125.005582] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:01Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16176worker2 # [ 125.009783] nix-daemon[847]: reaped child process 1563, status = succeeded6177worker2 # [ 125.048360] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666178worker2 # [ 125.058441] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457666179worker2 # [ 125.069880] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.170Z level=INFO msg="All 2 paths already cached"6180client # copying 1 paths to 'lb:50051'...6181client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6182worker2 # [ 125.076385] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457666183worker2 # [ 125.086829] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6184worker2 # [ 125.240877] nix-daemon[847]: accepted connection from pid 1589, user nix-grpc-daemon (trusted)6185worker2 # [ 125.253563] nix-daemon[847]: reaped child process 1592, status = succeeded6186worker2 # [ 125.407639] nix-daemon[847]: accepted connection from pid 1594, user nix-grpc-daemon (trusted)6187worker2 # [ 125.418756] nix-daemon[847]: reaped child process 1597, status = succeeded6188lb # [ 125.423922] niks3-server[1054]: 2026/09/20 09:35:00 INFO Received uploads request method=POST path=/api/pending_closures6189worker2 # [ 125.435368] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.535Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6190worker2 # [ 125.436924] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.537Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6191lb # [ 125.441509] niks3-server[1054]: 2026/09/20 09:35:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6192lb # [ 125.443715] niks3-server[1054]: 2026/09/20 09:35:00 INFO Signed narinfos id=43 count=16193worker2 # [ 125.451179] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.551Z level=INFO msg="Uploading 1 narinfos"6194lb # [ 125.446964] niks3-server[1054]: 2026/09/20 09:35:00 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6195lb # [ 125.450159] niks3-server[1054]: 2026/09/20 09:35:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6196lb # [ 125.454469] niks3-server[1054]: 2026/09/20 09:35:00 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6197lb # [ 125.457848] niks3-server[1054]: 2026/09/20 09:35:00 INFO Completed upload id=436198worker2 # [ 125.466334] nix-grpc-daemon[1438]: time=2026-09-20T09:35:01.566Z level=INFO msg="Upload complete. (367ms)"6199lb # [ 125.459530] niks3-server[1054]: 2026/09/20 09:35:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6200worker2 # [ 125.468994] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=0 paths=1 nar_bytes_in=7686201worker2 # [ 125.472148] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666202worker2 # [ 125.476816] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6203worker2 # [ 125.482278] nix-daemon[847]: reaped child process 1587, status = succeeded6204client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6205worker2 # [ 125.487865] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6206client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6207client # farm-top> LOG-farm-top-during-drain6208worker2 # [ 126.515474] nix-daemon[847]: reaped child process 1601, status = succeeded6209worker2 # [ 126.662031] nix-daemon[847]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6210worker2 # [ 126.672572] nix-daemon[847]: reaped child process 1619, status = succeeded6211lb # [ 126.679385] niks3-server[1054]: 2026/09/20 09:35:01 INFO Received uploads request method=POST path=/api/pending_closures6212worker2 # [ 126.692350] nix-grpc-daemon[1438]: time=2026-09-20T09:35:02.792Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6213worker2 # [ 126.694275] nix-grpc-daemon[1438]: time=2026-09-20T09:35:02.794Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6214lb # [ 126.701702] niks3-server[1054]: 2026/09/20 09:35:01 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6215lb # [ 126.704383] niks3-server[1054]: 2026/09/20 09:35:01 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6216lb # [ 126.707572] niks3-server[1054]: 2026/09/20 09:35:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6217lb # [ 126.709379] niks3-server[1054]: 2026/09/20 09:35:01 INFO Signed narinfos id=44 count=16218worker2 # [ 126.717120] nix-grpc-daemon[1438]: time=2026-09-20T09:35:02.817Z level=INFO msg="Uploading 1 narinfos"6219lb # [ 126.714096] niks3-server[1054]: 2026/09/20 09:35:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6220lb # [ 126.718484] niks3-server[1054]: 2026/09/20 09:35:01 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6221lb # [ 126.721452] niks3-server[1054]: 2026/09/20 09:35:01 INFO Completed upload id=446222worker2 # [ 126.730043] nix-grpc-daemon[1438]: time=2026-09-20T09:35:02.830Z level=INFO msg="Upload complete. (235ms)"6223lb # [ 126.723229] niks3-server[1054]: 2026/09/20 09:35:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6224worker2 # [ 126.734218] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16225worker2 # [ 126.741023] nix-daemon[847]: reaped child process 1599, status = succeeded6226client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.20 seconds)6227client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6228client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.21 seconds)6229worker1: must succeed: pkill -f 'read -t [9]0 x'6230worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6231client: waiting for success: ! systemctl is-active sw6232worker1 # [ 127.124416] nix-daemon[842]: reaped child process 1627, status = succeeded6233worker1 # [ 127.278301] nix-daemon[842]: accepted connection from pid 1667, user nix-grpc-daemon (trusted)6234worker1 # [ 127.289147] nix-daemon[842]: reaped child process 1670, status = succeeded6235lb # [ 127.289368] niks3-server[1054]: 2026/09/20 09:35:02 INFO Received uploads request method=POST path=/api/pending_closures6236worker1 # [ 127.314696] nix-grpc-daemon[1567]: time=2026-09-20T09:35:03.355Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6237worker1 # [ 127.316184] nix-grpc-daemon[1567]: time=2026-09-20T09:35:03.356Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6238lb # [ 127.324622] niks3-server[1054]: 2026/09/20 09:35:02 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6239lb # [ 127.337460] niks3-server[1054]: 2026/09/20 09:35:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6240worker1 # [ 127.361883] nix-grpc-daemon[1567]: time=2026-09-20T09:35:03.401Z level=INFO msg="Uploading 1 narinfos"6241lb # [ 127.345677] niks3-server[1054]: 2026/09/20 09:35:02 INFO Signed narinfos id=45 count=16242lb # [ 127.350285] niks3-server[1054]: 2026/09/20 09:35:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6243lb # [ 127.358898] niks3-server[1054]: 2026/09/20 09:35:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6244lb # [ 127.366781] niks3-server[1054]: 2026/09/20 09:35:02 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6245lb # [ 127.371233] niks3-server[1054]: 2026/09/20 09:35:02 INFO Completed upload id=456246worker1 # [ 127.388178] nix-grpc-daemon[1567]: time=2026-09-20T09:35:03.428Z level=INFO msg="Upload complete. (271ms)"6247lb # [ 127.373180] niks3-server[1054]: 2026/09/20 09:35:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6248worker1 # [ 127.391282] nix-grpc-daemon[1470]: ts=2026-09-20T09:35:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35872 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16249worker1 # [ 127.397778] nix-daemon[842]: reaped child process 1625, status = succeeded6250client # [ 127.425809] systemd[1]: sw.service: Deactivated successfully.6251client # [ 127.430269] systemd[1]: sw.service: Consumed 115ms CPU time over 9.470s wall clock time, 37.1M memory peak, 8.4K incoming IP traffic, 9.1K outgoing IP traffic.6252client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6253client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6254client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6255worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6256worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.03 seconds)6257lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6258lb: must succeed: curl -sf localhost:9901/clusters6259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6260worker1 # [ 128.372193] nix-grpc-daemon[1470]: ts=2026-09-20T09:35:04Z level=info event=signal_exit6261lb # [ 128.791874] envoy[926]: conn peer=[2001:db8:1::1]:54750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9555 tx=3815 ms=59576262lb # [ 128.795546] envoy[926]: conn peer=[2001:db8:1::1]:54734 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2932 tx=1308 ms=92556263lb: must succeed: curl -sf localhost:9901/clusters6264lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6265worker1 # [ 129.594193] nix-grpc-daemon[1470]: ts=2026-09-20T09:35:05Z level=info event=scheduler_restarting6266worker2 # [ 129.589432] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:508066267worker1 # [ 129.601937] nix-grpc-daemon[1470]: ts=2026-09-20T09:35:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6268worker2 # [ 129.593181] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:05Z level=info event=worker_gone addr=192.168.1.3:500516269worker1 # [ 129.613739] nix-daemon[842]: reaped child process 1535, status = succeeded6270worker1 # [ 130.203726] nix-grpc-daemon[1470]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6271worker1 # [ 130.207156] nix-grpc-daemon[1470]: E0000 00:00:1789896906.244097 1513 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6272worker1 # [ 130.211845] nix-grpc-daemon[1470]: E0000 00:00:1789896906.244242 1513 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6273lb: must succeed: curl -sf localhost:9901/clusters6274lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6275lb: must succeed: curl -sf localhost:9901/clusters6276worker1 # [ 131.409890] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6277worker1 # [ 131.413856] systemd[1]: nix-grpc-daemon.service: Consumed 589ms CPU time over 28.220s wall clock time, 32.6M memory peak, 20.9K incoming IP traffic, 27.3K outgoing IP traffic.6278lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6279worker1 # [ 132.392761] systemd[1]: Starting Nix worker-protocol over gRPC...6280lb: must succeed: curl -sf localhost:9901/clusters6281lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6282worker1 # [ 132.557155] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6283worker1 # [ 132.561069] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16284worker1 # [ 132.567060] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:08Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6285worker1 # [ 132.571273] systemd[1]: Started Nix worker-protocol over gRPC.6286worker1 # [ 132.598971] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:08Z level=info event=scheduler_connected addr=lb:500516287worker2 # [ 132.592469] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06288lb: must succeed: curl -sf localhost:9901/clusters6289lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6290lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 5.39 seconds)6291lb: must succeed: curl -sf localhost:9901/clusters6292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6293worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6294worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)6295(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.80 seconds)6296subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6297client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16298worker2 # [ 133.926632] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:457666299worker1 # [ 133.947788] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626300worker1 # [ 133.959132] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6301worker1 # [ 133.975135] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5)6302worker1 # [ 133.986720] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 2/5)6303worker1 # [ 134.130581] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 384 ms (attempt 3/5)6304worker1 # [ 134.520676] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 793 ms (attempt 4/5)6305worker1 # [ 135.320753] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6306worker2 # [ 135.345619] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:11Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06307worker2 # [ 135.349611] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:11Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16308worker2 # [ 135.353862] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:11Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06309worker2 # [ 135.357676] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:11Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06310worker1 # [ 135.398993] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626311worker1 # [ 135.409746] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626312worker1 # [ 135.419328] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626313worker1 # [ 135.430162] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6314worker1 # [ 135.447575] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:11Z level=info event=niks3_push_started pid=17386315worker1 # [ 135.774423] nix-daemon[842]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)6316worker1 # [ 135.785292] nix-daemon[842]: reaped child process 1754, status = succeeded6317worker1 # [ 135.934294] nix-daemon[842]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6318worker1 # [ 135.946104] nix-daemon[842]: reaped child process 1759, status = succeeded6319lb # [ 135.947824] niks3-server[1054]: 2026/09/20 09:35:11 INFO Received uploads request method=POST path=/api/pending_closures6320worker1 # [ 135.972729] nix-grpc-daemon[1738]: time=2026-09-20T09:35:12.012Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6321worker1 # [ 135.975883] nix-grpc-daemon[1738]: time=2026-09-20T09:35:12.013Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6322lb # [ 135.980472] niks3-server[1054]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6323lb # [ 135.993368] niks3-server[1054]: 2026/09/20 09:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6324worker1 # [ 136.014539] nix-grpc-daemon[1738]: time=2026-09-20T09:35:12.055Z level=INFO msg="Uploading 1 narinfos"6325lb # [ 135.998345] niks3-server[1054]: 2026/09/20 09:35:11 INFO Signed narinfos id=46 count=16326lb # [ 136.011511] niks3-server[1054]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6327lb # [ 136.018333] niks3-server[1054]: 2026/09/20 09:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6328worker1 # [ 136.040722] nix-grpc-daemon[1738]: time=2026-09-20T09:35:12.081Z level=INFO msg="Upload complete. (409ms)"6329lb # [ 136.023791] niks3-server[1054]: 2026/09/20 09:35:11 INFO Completed upload id=466330worker1 # [ 136.043066] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=5046331lb # [ 136.029085] niks3-server[1054]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6332worker1 # [ 136.049395] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626333worker1 # [ 136.051303] nix-daemon[842]: reaped child process 1736, status = succeeded6334worker1 # [ 136.054354] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6335worker1 # [ 136.060985] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6336worker1 # [ 137.073852] nix-daemon[842]: reaped child process 1763, status = succeeded6337worker1 # [ 137.243436] nix-daemon[842]: accepted connection from pid 1778, user nix-grpc-daemon (trusted)6338worker1 # [ 137.254277] nix-daemon[842]: reaped child process 1781, status = succeeded6339lb # [ 137.257850] niks3-server[1054]: 2026/09/20 09:35:12 INFO Received uploads request method=POST path=/api/pending_closures6340worker1 # [ 137.283061] nix-grpc-daemon[1738]: time=2026-09-20T09:35:13.323Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6341worker1 # [ 137.284571] nix-grpc-daemon[1738]: time=2026-09-20T09:35:13.325Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6342lb # [ 137.301515] niks3-server[1054]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6343lb # [ 137.305978] niks3-server[1054]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6344lb # [ 137.317380] niks3-server[1054]: 2026/09/20 09:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6345lb # [ 137.321318] niks3-server[1054]: 2026/09/20 09:35:12 INFO Signed narinfos id=47 count=16346worker1 # [ 137.341482] nix-grpc-daemon[1738]: time=2026-09-20T09:35:13.381Z level=INFO msg="Uploading 1 narinfos"6347lb # [ 137.326221] niks3-server[1054]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6348lb # [ 137.336512] niks3-server[1054]: 2026/09/20 09:35:12 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6349lb # [ 137.341641] niks3-server[1054]: 2026/09/20 09:35:12 INFO Completed upload id=476350worker1 # [ 137.359768] nix-grpc-daemon[1738]: time=2026-09-20T09:35:13.400Z level=INFO msg="Upload complete. (292ms)"6351lb # [ 137.343248] niks3-server[1054]: 2026/09/20 09:35:12 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6352worker1 # [ 137.362360] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16353worker1 # [ 137.366369] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626354worker2 # [ 137.354784] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:13Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16355worker1 # [ 137.372861] nix-daemon[842]: reaped child process 1761, status = succeeded6356worker1 # [ 137.374617] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626357worker2 # [ 137.364273] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:13Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06358worker1 # [ 137.379822] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626359worker1 # [ 137.384595] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6360worker1 # [ 137.543400] nix-daemon[842]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6361worker1 # [ 137.555438] nix-daemon[842]: reaped child process 1788, status = succeeded6362worker1 # [ 137.732582] nix-daemon[842]: accepted connection from pid 1790, user nix-grpc-daemon (trusted)6363worker1 # [ 137.744232] nix-daemon[842]: reaped child process 1793, status = succeeded6364lb # [ 137.744565] niks3-server[1054]: 2026/09/20 09:35:13 INFO Received uploads request method=POST path=/api/pending_closures6365worker1 # [ 137.765934] nix-grpc-daemon[1738]: time=2026-09-20T09:35:13.806Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6366worker1 # [ 137.767554] nix-grpc-daemon[1738]: time=2026-09-20T09:35:13.808Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6367lb # [ 137.761195] niks3-server[1054]: 2026/09/20 09:35:13 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6368lb # [ 137.763409] niks3-server[1054]: 2026/09/20 09:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6369lb # [ 137.765118] niks3-server[1054]: 2026/09/20 09:35:13 INFO Signed narinfos id=48 count=16370worker1 # [ 137.783141] nix-grpc-daemon[1738]: time=2026-09-20T09:35:13.823Z level=INFO msg="Uploading 1 narinfos"6371lb # [ 137.769034] niks3-server[1054]: 2026/09/20 09:35:13 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6372lb # [ 137.773364] niks3-server[1054]: 2026/09/20 09:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6373lb # [ 137.776684] niks3-server[1054]: 2026/09/20 09:35:13 INFO Completed upload id=486374worker1 # [ 137.793552] nix-grpc-daemon[1738]: time=2026-09-20T09:35:13.834Z level=INFO msg="Upload complete. (404ms)"6375lb # [ 137.778348] niks3-server[1054]: 2026/09/20 09:35:13 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6376worker1 # [ 137.795430] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=5046377worker1 # [ 137.799392] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626378worker1 # [ 137.805453] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6379worker1 # [ 137.807719] nix-daemon[842]: reaped child process 1783, status = succeeded6380worker1 # [ 137.812743] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6381lb # [ 138.792302] envoy[926]: conn peer=[2001:db8:1::3]:50770 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1114 tx=448 ms=262156382worker1 # [ 139.226679] nix-daemon[842]: reaped child process 1797, status = succeeded6383worker1 # [ 139.374207] nix-daemon[842]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6384worker1 # [ 139.385782] nix-daemon[842]: reaped child process 1815, status = succeeded6385lb # [ 139.385887] niks3-server[1054]: 2026/09/20 09:35:14 INFO Received uploads request method=POST path=/api/pending_closures6386worker1 # [ 139.407101] nix-grpc-daemon[1738]: time=2026-09-20T09:35:15.447Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6387worker1 # [ 139.408563] nix-grpc-daemon[1738]: time=2026-09-20T09:35:15.449Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6388lb # [ 139.405465] niks3-server[1054]: 2026/09/20 09:35:14 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6389lb # [ 139.408032] niks3-server[1054]: 2026/09/20 09:35:14 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6390lb # [ 139.411649] niks3-server[1054]: 2026/09/20 09:35:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6391lb # [ 139.413902] niks3-server[1054]: 2026/09/20 09:35:14 INFO Signed narinfos id=49 count=16392worker1 # [ 139.430739] nix-grpc-daemon[1738]: time=2026-09-20T09:35:15.471Z level=INFO msg="Uploading 1 narinfos"6393lb # [ 139.416345] niks3-server[1054]: 2026/09/20 09:35:14 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6394lb # [ 139.421588] niks3-server[1054]: 2026/09/20 09:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6395lb # [ 139.424388] niks3-server[1054]: 2026/09/20 09:35:14 INFO Completed upload id=496396worker1 # [ 139.441299] nix-grpc-daemon[1738]: time=2026-09-20T09:35:15.481Z level=INFO msg="Upload complete. (221ms)"6397lb # [ 139.426111] niks3-server[1054]: 2026/09/20 09:35:14 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6398worker1 # [ 139.443588] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16399worker2 # [ 139.436594] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:15Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16400worker1 # [ 139.448072] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626401worker2 # [ 139.441943] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:15Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06402worker1 # [ 139.454105] nix-daemon[842]: reaped child process 1795, status = succeeded6403worker1 # [ 139.455517] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626404worker1 # [ 139.460295] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626405worker1 # [ 139.464897] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6406worker1 # [ 139.622980] nix-daemon[842]: accepted connection from pid 1819, user nix-grpc-daemon (trusted)6407worker1 # [ 139.635254] nix-daemon[842]: reaped child process 1822, status = succeeded6408worker1 # [ 139.795349] nix-daemon[842]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6409worker1 # [ 139.807032] nix-daemon[842]: reaped child process 1827, status = succeeded6410lb # [ 139.806868] niks3-server[1054]: 2026/09/20 09:35:15 INFO Received uploads request method=POST path=/api/pending_closures6411worker1 # [ 139.827785] nix-grpc-daemon[1738]: time=2026-09-20T09:35:15.868Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6412worker1 # [ 139.829272] nix-grpc-daemon[1738]: time=2026-09-20T09:35:15.869Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6413lb # [ 139.823285] niks3-server[1054]: 2026/09/20 09:35:15 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6414lb # [ 139.826169] niks3-server[1054]: 2026/09/20 09:35:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6415lb # [ 139.828733] niks3-server[1054]: 2026/09/20 09:35:15 INFO Signed narinfos id=50 count=16416worker1 # [ 139.845679] nix-grpc-daemon[1738]: time=2026-09-20T09:35:15.886Z level=INFO msg="Uploading 1 narinfos"6417lb # [ 139.831093] niks3-server[1054]: 2026/09/20 09:35:15 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6418lb # [ 139.835572] niks3-server[1054]: 2026/09/20 09:35:15 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6419lb # [ 139.839032] niks3-server[1054]: 2026/09/20 09:35:15 INFO Completed upload id=506420worker1 # [ 139.855879] nix-grpc-daemon[1738]: time=2026-09-20T09:35:15.896Z level=INFO msg="Upload complete. (386ms)"6421lb # [ 139.840693] niks3-server[1054]: 2026/09/20 09:35:15 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6422worker1 # [ 139.857913] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=5046423worker1 # [ 139.861382] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626424worker1 # [ 139.866535] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6425worker1 # [ 139.869396] nix-daemon[842]: reaped child process 1817, status = succeeded6426worker1 # [ 139.874772] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6427worker1 # [ 140.487249] nix-daemon[842]: reaped child process 1831, status = succeeded6428worker1 # [ 140.635817] nix-daemon[842]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6429worker1 # [ 140.646600] nix-daemon[842]: reaped child process 1849, status = succeeded6430lb # [ 140.647210] niks3-server[1054]: 2026/09/20 09:35:15 INFO Received uploads request method=POST path=/api/pending_closures6431worker1 # [ 140.671790] nix-grpc-daemon[1738]: time=2026-09-20T09:35:16.712Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6432worker1 # [ 140.673246] nix-grpc-daemon[1738]: time=2026-09-20T09:35:16.713Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6433lb # [ 140.682530] niks3-server[1054]: 2026/09/20 09:35:15 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6434lb # [ 140.690113] niks3-server[1054]: 2026/09/20 09:35:15 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6435lb # [ 140.700194] niks3-server[1054]: 2026/09/20 09:35:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6436worker1 # [ 140.721140] nix-grpc-daemon[1738]: time=2026-09-20T09:35:16.761Z level=INFO msg="Uploading 1 narinfos"6437lb # [ 140.704160] niks3-server[1054]: 2026/09/20 09:35:15 INFO Signed narinfos id=51 count=16438lb # [ 140.716232] niks3-server[1054]: 2026/09/20 09:35:15 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6439lb # [ 140.719669] niks3-server[1054]: 2026/09/20 09:35:15 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6440lb # [ 140.726350] niks3-server[1054]: 2026/09/20 09:35:15 INFO Completed upload id=516441lb # [ 140.728268] niks3-server[1054]: 2026/09/20 09:35:15 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6442worker1 # [ 140.743706] nix-grpc-daemon[1738]: time=2026-09-20T09:35:16.784Z level=INFO msg="Upload complete. (263ms)"6443worker2 # [ 140.738594] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:16Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16444worker2 # [ 140.744269] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:16Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06445worker1 # [ 140.749868] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16446worker1 # [ 140.757859] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626447worker1 # [ 140.767519] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626448worker1 # [ 140.773390] nix-daemon[842]: reaped child process 1829, status = succeeded6449worker1 # [ 140.779600] nix-grpc-daemon[1738]: time=2026-09-20T09:35:16.820Z level=INFO msg="All 1 paths already cached"6450worker1 # [ 140.783229] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626451worker1 # [ 140.787131] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6452worker1 # [ 140.949829] nix-daemon[842]: accepted connection from pid 1853, user nix-grpc-daemon (trusted)6453worker1 # [ 140.961950] nix-daemon[842]: reaped child process 1856, status = succeeded6454worker1 # [ 141.140203] nix-daemon[842]: accepted connection from pid 1858, user nix-grpc-daemon (trusted)6455worker1 # [ 141.151920] nix-daemon[842]: reaped child process 1861, status = succeeded6456lb # [ 141.153322] niks3-server[1054]: 2026/09/20 09:35:16 INFO Received uploads request method=POST path=/api/pending_closures6457worker1 # [ 141.177690] nix-grpc-daemon[1738]: time=2026-09-20T09:35:17.218Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6458worker1 # [ 141.179187] nix-grpc-daemon[1738]: time=2026-09-20T09:35:17.219Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6459lb # [ 141.181112] niks3-server[1054]: 2026/09/20 09:35:16 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6460lb # [ 141.191909] niks3-server[1054]: 2026/09/20 09:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6461worker1 # [ 141.213622] nix-grpc-daemon[1738]: time=2026-09-20T09:35:17.254Z level=INFO msg="Uploading 1 narinfos"6462lb # [ 141.197370] niks3-server[1054]: 2026/09/20 09:35:16 INFO Signed narinfos id=52 count=16463lb # [ 141.202350] niks3-server[1054]: 2026/09/20 09:35:16 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6464lb # [ 141.214271] niks3-server[1054]: 2026/09/20 09:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6465worker1 # [ 141.236916] nix-grpc-daemon[1738]: time=2026-09-20T09:35:17.277Z level=INFO msg="Upload complete. (444ms)"6466lb # [ 141.220304] niks3-server[1054]: 2026/09/20 09:35:16 INFO Completed upload id=526467lb # [ 141.224084] niks3-server[1054]: 2026/09/20 09:35:16 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6468worker1 # [ 141.239176] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=6246469worker1 # [ 141.245055] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626470worker1 # [ 141.253977] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6471worker1 # [ 141.262038] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6472worker1 # [ 141.264348] nix-daemon[842]: reaped child process 1851, status = succeeded6473worker1 # [ 142.877821] nix-daemon[842]: reaped child process 1865, status = succeeded6474worker1 # [ 143.017659] nix-daemon[842]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6475worker1 # [ 143.029645] nix-daemon[842]: reaped child process 1883, status = succeeded6476lb # [ 143.026996] niks3-server[1054]: 2026/09/20 09:35:18 INFO Received uploads request method=POST path=/api/pending_closures6477worker1 # [ 143.052957] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.093Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6478worker1 # [ 143.056144] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.095Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6479lb # [ 143.063405] niks3-server[1054]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6480lb # [ 143.072666] niks3-server[1054]: 2026/09/20 09:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6481lb # [ 143.076792] niks3-server[1054]: 2026/09/20 09:35:18 INFO Signed narinfos id=53 count=16482worker1 # [ 143.096063] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.136Z level=INFO msg="Uploading 1 narinfos"6483lb # [ 143.084176] niks3-server[1054]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6484lb # [ 143.090443] niks3-server[1054]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6485lb # [ 143.095040] niks3-server[1054]: 2026/09/20 09:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6486lb # [ 143.098426] niks3-server[1054]: 2026/09/20 09:35:18 INFO Completed upload id=536487worker1 # [ 143.115109] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.155Z level=INFO msg="Upload complete. (244ms)"6488lb # [ 143.101171] niks3-server[1054]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6489worker1 # [ 143.117590] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16490worker2 # [ 143.109575] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:19Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16491worker1 # [ 143.124592] nix-daemon[842]: reaped child process 1863, status = succeeded6492worker1 # [ 143.126237] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626493worker2 # [ 143.119239] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:19Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06494worker1 # [ 143.132220] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626495worker1 # [ 143.137857] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.178Z level=INFO msg="All 1 paths already cached"6496worker1 # [ 143.141419] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626497worker1 # [ 143.150950] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6498worker1 # [ 143.312360] nix-daemon[842]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6499worker1 # [ 143.324375] nix-daemon[842]: reaped child process 1890, status = succeeded6500worker1 # [ 143.505945] nix-daemon[842]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6501worker1 # [ 143.517914] nix-daemon[842]: reaped child process 1895, status = succeeded6502lb # [ 143.516351] niks3-server[1054]: 2026/09/20 09:35:18 INFO Received uploads request method=POST path=/api/pending_closures6503worker1 # [ 143.540722] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.581Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6504worker1 # [ 143.542166] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.581Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6505lb # [ 143.539512] niks3-server[1054]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6506lb # [ 143.551224] niks3-server[1054]: 2026/09/20 09:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6507worker1 # [ 143.572747] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.613Z level=INFO msg="Uploading 1 narinfos"6508lb # [ 143.555953] niks3-server[1054]: 2026/09/20 09:35:18 INFO Signed narinfos id=54 count=16509lb # [ 143.562130] niks3-server[1054]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6510lb # [ 143.575091] niks3-server[1054]: 2026/09/20 09:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6511worker1 # [ 143.596940] nix-grpc-daemon[1738]: time=2026-09-20T09:35:19.637Z level=INFO msg="Upload complete. (441ms)"6512lb # [ 143.580584] niks3-server[1054]: 2026/09/20 09:35:18 INFO Completed upload id=546513lb # [ 143.584073] niks3-server[1054]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6514worker1 # [ 143.598979] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=6246515worker1 # [ 143.604994] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626516worker1 # [ 143.613254] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6517worker1 # [ 143.616104] nix-daemon[842]: reaped child process 1885, status = succeeded6518worker1 # [ 143.621155] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6519worker1 # [ 145.246659] nix-daemon[842]: reaped child process 1899, status = succeeded6520worker1 # [ 145.401089] nix-daemon[842]: accepted connection from pid 1914, user nix-grpc-daemon (trusted)6521worker1 # [ 145.413166] nix-daemon[842]: reaped child process 1917, status = succeeded6522lb # [ 145.413331] niks3-server[1054]: 2026/09/20 09:35:20 INFO Received uploads request method=POST path=/api/pending_closures6523worker1 # [ 145.438503] nix-grpc-daemon[1738]: time=2026-09-20T09:35:21.478Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6524worker1 # [ 145.440048] nix-grpc-daemon[1738]: time=2026-09-20T09:35:21.480Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6525lb # [ 145.448144] niks3-server[1054]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6526lb # [ 145.461134] niks3-server[1054]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6527lb # [ 145.473499] niks3-server[1054]: 2026/09/20 09:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6528lb # [ 145.479527] niks3-server[1054]: 2026/09/20 09:35:20 INFO Signed narinfos id=55 count=16529worker1 # [ 145.498947] nix-grpc-daemon[1738]: time=2026-09-20T09:35:21.539Z level=INFO msg="Uploading 1 narinfos"6530lb # [ 145.483459] niks3-server[1054]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6531lb # [ 145.491760] niks3-server[1054]: 2026/09/20 09:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6532lb # [ 145.497519] niks3-server[1054]: 2026/09/20 09:35:20 INFO Completed upload id=556533worker1 # [ 145.514340] nix-grpc-daemon[1738]: time=2026-09-20T09:35:21.554Z level=INFO msg="Upload complete. (285ms)"6534lb # [ 145.500283] niks3-server[1054]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6535worker2 # [ 145.510611] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:21Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16536worker1 # [ 145.518875] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16537worker2 # [ 145.515343] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:21Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06538worker1 # [ 145.528322] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626539worker1 # [ 145.537543] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626540worker1 # [ 145.544432] nix-daemon[842]: reaped child process 1897, status = succeeded6541worker1 # [ 145.549518] nix-grpc-daemon[1738]: time=2026-09-20T09:35:21.590Z level=INFO msg="All 1 paths already cached"6542worker1 # [ 145.553622] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626543worker1 # [ 145.558478] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6544worker1 # [ 145.695207] nix-daemon[842]: accepted connection from pid 1921, user nix-grpc-daemon (trusted)6545worker1 # [ 145.707546] nix-daemon[842]: reaped child process 1924, status = succeeded6546worker1 # [ 145.874775] nix-daemon[842]: accepted connection from pid 1926, user nix-grpc-daemon (trusted)6547worker1 # [ 145.886722] nix-daemon[842]: reaped child process 1929, status = succeeded6548lb # [ 145.886452] niks3-server[1054]: 2026/09/20 09:35:21 INFO Received uploads request method=POST path=/api/pending_closures6549worker1 # [ 145.910513] nix-grpc-daemon[1738]: time=2026-09-20T09:35:21.951Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6550worker1 # [ 145.912028] nix-grpc-daemon[1738]: time=2026-09-20T09:35:21.952Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6551lb # [ 145.925594] niks3-server[1054]: 2026/09/20 09:35:21 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6552lb # [ 145.933245] niks3-server[1054]: 2026/09/20 09:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6553worker1 # [ 145.954703] nix-grpc-daemon[1738]: time=2026-09-20T09:35:21.995Z level=INFO msg="Uploading 1 narinfos"6554lb # [ 145.937858] niks3-server[1054]: 2026/09/20 09:35:21 INFO Signed narinfos id=56 count=16555lb # [ 145.944222] niks3-server[1054]: 2026/09/20 09:35:21 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6556lb # [ 145.953789] niks3-server[1054]: 2026/09/20 09:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6557worker1 # [ 145.976422] nix-grpc-daemon[1738]: time=2026-09-20T09:35:22.017Z level=INFO msg="Upload complete. (413ms)"6558lb # [ 145.959617] niks3-server[1054]: 2026/09/20 09:35:21 INFO Completed upload id=566559worker1 # [ 145.978354] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=6246560lb # [ 145.962110] niks3-server[1054]: 2026/09/20 09:35:21 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6561worker1 # [ 145.984691] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626562worker1 # [ 145.988611] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6563worker1 # [ 145.990835] nix-daemon[842]: reaped child process 1919, status = succeeded6564worker1 # [ 145.996167] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6565worker1 # [ 147.210474] nix-daemon[842]: reaped child process 1933, status = succeeded6566worker1 # [ 147.368814] nix-daemon[842]: accepted connection from pid 1948, user nix-grpc-daemon (trusted)6567worker1 # [ 147.380893] nix-daemon[842]: reaped child process 1951, status = succeeded6568lb # [ 147.382734] niks3-server[1054]: 2026/09/20 09:35:22 INFO Received uploads request method=POST path=/api/pending_closures6569worker1 # [ 147.409547] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.449Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6570worker1 # [ 147.412306] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.451Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6571lb # [ 147.416302] niks3-server[1054]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6572lb # [ 147.428845] niks3-server[1054]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6573lb # [ 147.439517] niks3-server[1054]: 2026/09/20 09:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6574worker1 # [ 147.461286] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.501Z level=INFO msg="Uploading 1 narinfos"6575lb # [ 147.444141] niks3-server[1054]: 2026/09/20 09:35:22 INFO Signed narinfos id=57 count=16576lb # [ 147.451451] niks3-server[1054]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6577lb # [ 147.459774] niks3-server[1054]: 2026/09/20 09:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6578lb # [ 147.465631] niks3-server[1054]: 2026/09/20 09:35:22 INFO Completed upload id=576579worker1 # [ 147.482421] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.522Z level=INFO msg="Upload complete. (279ms)"6580lb # [ 147.467147] niks3-server[1054]: 2026/09/20 09:35:22 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6581worker1 # [ 147.485884] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16582worker1 # [ 147.489954] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626583worker2 # [ 147.478611] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:23Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16584worker1 # [ 147.497715] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626585worker1 # [ 147.500159] nix-daemon[842]: reaped child process 1931, status = succeeded6586worker2 # [ 147.488273] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:23Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06587worker1 # [ 147.503767] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.544Z level=INFO msg="All 2 paths already cached"6588worker1 # [ 147.507661] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626589worker1 # [ 147.512152] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6590worker1 # [ 147.675304] nix-daemon[842]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)6591worker1 # [ 147.687537] nix-daemon[842]: reaped child process 1958, status = succeeded6592worker1 # [ 147.854113] nix-daemon[842]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)6593worker1 # [ 147.866446] nix-daemon[842]: reaped child process 1963, status = succeeded6594lb # [ 147.865976] niks3-server[1054]: 2026/09/20 09:35:23 INFO Received uploads request method=POST path=/api/pending_closures6595worker1 # [ 147.890739] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.931Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6596worker1 # [ 147.892146] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.931Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6597lb # [ 147.890943] niks3-server[1054]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6598lb # [ 147.903875] niks3-server[1054]: 2026/09/20 09:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6599lb # [ 147.908925] niks3-server[1054]: 2026/09/20 09:35:23 INFO Signed narinfos id=58 count=16600worker1 # [ 147.928998] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.969Z level=INFO msg="Uploading 1 narinfos"6601lb # [ 147.916256] niks3-server[1054]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6602lb # [ 147.925216] niks3-server[1054]: 2026/09/20 09:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6603lb # [ 147.931279] niks3-server[1054]: 2026/09/20 09:35:23 INFO Completed upload id=586604worker1 # [ 147.948309] nix-grpc-daemon[1738]: time=2026-09-20T09:35:23.988Z level=INFO msg="Upload complete. (429ms)"6605lb # [ 147.935696] niks3-server[1054]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6606worker1 # [ 147.953134] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=7526607worker1 # [ 147.958058] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626608worker1 # [ 147.968308] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6609worker1 # [ 147.970563] nix-daemon[842]: reaped child process 1953, status = succeeded6610worker1 # [ 147.976099] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6611worker1 # [ 149.189953] nix-daemon[842]: reaped child process 1967, status = succeeded6612worker1 # [ 149.343339] nix-daemon[842]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6613worker1 # [ 149.353961] nix-daemon[842]: reaped child process 1985, status = succeeded6614lb # [ 149.355635] niks3-server[1054]: 2026/09/20 09:35:24 INFO Received uploads request method=POST path=/api/pending_closures6615worker1 # [ 149.381993] nix-grpc-daemon[1738]: time=2026-09-20T09:35:25.422Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6616worker1 # [ 149.383659] nix-grpc-daemon[1738]: time=2026-09-20T09:35:25.424Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6617lb # [ 149.390674] niks3-server[1054]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6618lb # [ 149.402847] niks3-server[1054]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6619lb # [ 149.414647] niks3-server[1054]: 2026/09/20 09:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6620worker1 # [ 149.436825] nix-grpc-daemon[1738]: time=2026-09-20T09:35:25.477Z level=INFO msg="Uploading 1 narinfos"6621lb # [ 149.419882] niks3-server[1054]: 2026/09/20 09:35:24 INFO Signed narinfos id=59 count=16622lb # [ 149.425297] niks3-server[1054]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6623lb # [ 149.433680] niks3-server[1054]: 2026/09/20 09:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6624lb # [ 149.438675] niks3-server[1054]: 2026/09/20 09:35:24 INFO Completed upload id=596625worker1 # [ 149.455893] nix-grpc-daemon[1738]: time=2026-09-20T09:35:25.496Z level=INFO msg="Upload complete. (272ms)"6626lb # [ 149.441598] niks3-server[1054]: 2026/09/20 09:35:24 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6627worker2 # [ 149.452753] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:25Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16628worker1 # [ 149.463690] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16629worker1 # [ 149.476545] nix-daemon[842]: reaped child process 1965, status = succeeded6630worker1 # [ 149.505345] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626631worker1 # [ 149.517704] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626632worker1 # [ 149.529730] nix-grpc-daemon[1738]: time=2026-09-20T09:35:25.570Z level=INFO msg="All 2 paths already cached"6633worker1 # [ 149.536789] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626634worker1 # [ 149.546159] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6635worker1 # [ 149.727172] nix-daemon[842]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6636worker1 # [ 149.739827] nix-daemon[842]: reaped child process 1992, status = succeeded6637worker1 # [ 149.904398] nix-daemon[842]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6638worker1 # [ 149.915948] nix-daemon[842]: reaped child process 1997, status = succeeded6639lb # [ 149.916756] niks3-server[1054]: 2026/09/20 09:35:25 INFO Received uploads request method=POST path=/api/pending_closures6640worker1 # [ 149.941661] nix-grpc-daemon[1738]: time=2026-09-20T09:35:25.981Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6641worker1 # [ 149.943349] nix-grpc-daemon[1738]: time=2026-09-20T09:35:25.982Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6642lb # [ 149.942397] niks3-server[1054]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6643lb # [ 149.952512] niks3-server[1054]: 2026/09/20 09:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6644worker1 # [ 149.974456] nix-grpc-daemon[1738]: time=2026-09-20T09:35:26.014Z level=INFO msg="Uploading 1 narinfos"6645lb # [ 149.958147] niks3-server[1054]: 2026/09/20 09:35:25 INFO Signed narinfos id=60 count=16646lb # [ 149.962881] niks3-server[1054]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6647lb # [ 149.974324] niks3-server[1054]: 2026/09/20 09:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6648worker1 # [ 149.997034] nix-grpc-daemon[1738]: time=2026-09-20T09:35:26.037Z level=INFO msg="Upload complete. (439ms)"6649lb # [ 149.980403] niks3-server[1054]: 2026/09/20 09:35:25 INFO Completed upload id=606650worker1 # [ 150.000082] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=7526651lb # [ 149.983559] niks3-server[1054]: 2026/09/20 09:35:25 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6652worker1 # [ 150.004389] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626653worker1 # [ 150.009313] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6654worker1 # [ 150.011536] nix-daemon[842]: reaped child process 1987, status = succeeded6655worker1 # [ 150.016824] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6656worker1 # [ 151.046845] nix-daemon[842]: reaped child process 2001, status = succeeded6657worker1 # [ 151.188233] nix-daemon[842]: accepted connection from pid 2016, user nix-grpc-daemon (trusted)6658worker1 # [ 151.200951] nix-daemon[842]: reaped child process 2019, status = succeeded6659lb # [ 151.200849] niks3-server[1054]: 2026/09/20 09:35:26 INFO Received uploads request method=POST path=/api/pending_closures6660worker1 # [ 151.222560] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.262Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6661worker1 # [ 151.224182] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.264Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6662lb # [ 151.221560] niks3-server[1054]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6663lb # [ 151.223524] niks3-server[1054]: 2026/09/20 09:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6664lb # [ 151.225263] niks3-server[1054]: 2026/09/20 09:35:26 INFO Signed narinfos id=61 count=16665worker1 # [ 151.243486] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.283Z level=INFO msg="Uploading 1 narinfos"6666lb # [ 151.228761] niks3-server[1054]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6667lb # [ 151.233315] niks3-server[1054]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6668lb # [ 151.235910] niks3-server[1054]: 2026/09/20 09:35:26 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6669lb # [ 151.239323] niks3-server[1054]: 2026/09/20 09:35:26 INFO Completed upload id=616670worker1 # [ 151.257449] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.298Z level=INFO msg="Upload complete. (233ms)"6671lb # [ 151.241118] niks3-server[1054]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6672worker1 # [ 151.259898] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16673worker2 # [ 151.252214] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:27Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16674worker1 # [ 151.266833] nix-daemon[842]: reaped child process 1999, status = succeeded6675worker1 # [ 151.304887] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626676worker1 # [ 151.314127] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626677worker1 # [ 151.322949] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.363Z level=INFO msg="All 2 paths already cached"6678worker1 # [ 151.328816] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572626679worker1 # [ 151.335755] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6680worker1 # [ 151.501327] nix-daemon[842]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)6681worker1 # [ 151.513356] nix-daemon[842]: reaped child process 2026, status = succeeded6682worker1 # [ 151.673665] nix-daemon[842]: accepted connection from pid 2028, user nix-grpc-daemon (trusted)6683worker1 # [ 151.685508] nix-daemon[842]: reaped child process 2031, status = succeeded6684lb # [ 151.685954] niks3-server[1054]: 2026/09/20 09:35:26 INFO Received uploads request method=POST path=/api/pending_closures6685worker1 # [ 151.710431] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.751Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6686worker1 # [ 151.712058] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.752Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6687lb # [ 151.712491] niks3-server[1054]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6688lb # [ 151.724112] niks3-server[1054]: 2026/09/20 09:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6689worker1 # [ 151.745655] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.786Z level=INFO msg="Uploading 1 narinfos"6690lb # [ 151.728906] niks3-server[1054]: 2026/09/20 09:35:26 INFO Signed narinfos id=62 count=16691lb # [ 151.735211] niks3-server[1054]: 2026/09/20 09:35:26 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6692lb # [ 151.748143] niks3-server[1054]: 2026/09/20 09:35:27 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6693lb # [ 151.752460] niks3-server[1054]: 2026/09/20 09:35:27 INFO Completed upload id=626694worker1 # [ 151.771715] nix-grpc-daemon[1738]: time=2026-09-20T09:35:27.812Z level=INFO msg="Upload complete. (427ms)"6695lb # [ 151.755216] niks3-server[1054]: 2026/09/20 09:35:27 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6696worker1 # [ 151.775141] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=0 paths=1 nar_bytes_in=7526697worker1 # [ 151.781317] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572626698worker1 # [ 151.791636] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6699worker1 # [ 151.796945] nix-daemon[842]: reaped child process 2021, status = succeeded6700worker1 # [ 151.801949] nix-daemon[842]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6701worker1 # [ 152.816665] nix-daemon[842]: reaped child process 2035, status = succeeded6702worker1 # [ 152.955382] nix-daemon[842]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6703worker1 # [ 152.967459] nix-daemon[842]: reaped child process 2053, status = succeeded6704lb # [ 152.961915] niks3-server[1054]: 2026/09/20 09:35:28 INFO Received uploads request method=POST path=/api/pending_closures6705worker1 # [ 152.983241] nix-grpc-daemon[1738]: time=2026-09-20T09:35:29.023Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6706worker1 # [ 152.984891] nix-grpc-daemon[1738]: time=2026-09-20T09:35:29.025Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6707lb # [ 152.983103] niks3-server[1054]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6708lb # [ 152.986295] niks3-server[1054]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6709lb # [ 152.993062] niks3-server[1054]: 2026/09/20 09:35:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6710worker1 # [ 153.014663] nix-grpc-daemon[1738]: time=2026-09-20T09:35:29.055Z level=INFO msg="Uploading 1 narinfos"6711lb # [ 152.998222] niks3-server[1054]: 2026/09/20 09:35:28 INFO Signed narinfos id=63 count=16712lb # [ 153.000367] niks3-server[1054]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6713lb # [ 153.007747] niks3-server[1054]: 2026/09/20 09:35:28 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6714lb # [ 153.011057] niks3-server[1054]: 2026/09/20 09:35:28 INFO Completed upload id=636715worker1 # [ 153.027722] nix-grpc-daemon[1738]: time=2026-09-20T09:35:29.068Z level=INFO msg="Upload complete. (218ms)"6716lb # [ 153.013458] niks3-server[1054]: 2026/09/20 09:35:28 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6717worker1 # [ 153.030359] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57262 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16718worker1 # [ 153.036820] nix-daemon[842]: reaped child process 2033, status = succeeded6719client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 19.38 seconds)6720worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6721worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.05 seconds)6722client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16723worker1 # [ 153.356849] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:572626724worker1 # [ 153.361291] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572626725worker2 # [ 153.369467] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:29Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06726client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.27 seconds)6727(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 19.70 seconds)6728subtest: low disk drains a worker and builds go to the other6729worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6730worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)6731worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6732worker1 # [ 154.576082] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:30Z level=info event=unhealthy reason=min_free6733worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6734client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26735worker1 # [ 155.725813] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:572626736client # don't know how to build these paths:6737client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6738worker2 # [ 155.723592] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457666739worker2 # [ 155.744936] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:31Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06740worker2 # [ 155.750316] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:31Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16741worker2 # [ 155.788862] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666742worker2 # [ 155.800710] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457666743client # copying 1 paths to 'lb:50051'...6744client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6745worker2 # [ 155.812925] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457666746worker2 # [ 155.821670] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6747worker2 # [ 155.998239] nix-daemon[847]: accepted connection from pid 1646, user nix-grpc-daemon (trusted)6748worker2 # [ 156.010772] nix-daemon[847]: reaped child process 1649, status = succeeded6749worker2 # [ 156.173107] nix-daemon[847]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)6750worker2 # [ 156.184683] nix-daemon[847]: reaped child process 1654, status = succeeded6751lb # [ 156.195146] niks3-server[1054]: 2026/09/20 09:35:31 INFO Received uploads request method=POST path=/api/pending_closures6752worker2 # [ 156.207482] nix-grpc-daemon[1438]: time=2026-09-20T09:35:32.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6753worker2 # [ 156.208870] nix-grpc-daemon[1438]: time=2026-09-20T09:35:32.307Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6754lb # [ 156.209232] niks3-server[1054]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6755lb # [ 156.213980] niks3-server[1054]: 2026/09/20 09:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6756lb # [ 156.215701] niks3-server[1054]: 2026/09/20 09:35:31 INFO Signed narinfos id=64 count=16757worker2 # [ 156.224387] nix-grpc-daemon[1438]: time=2026-09-20T09:35:32.324Z level=INFO msg="Uploading 1 narinfos"6758lb # [ 156.218469] niks3-server[1054]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6759lb # [ 156.223149] niks3-server[1054]: 2026/09/20 09:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6760lb # [ 156.230066] niks3-server[1054]: 2026/09/20 09:35:31 INFO Completed upload id=646761worker2 # [ 156.237074] nix-grpc-daemon[1438]: time=2026-09-20T09:35:32.337Z level=INFO msg="Upload complete. (401ms)"6762lb # [ 156.232427] niks3-server[1054]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6763worker2 # [ 156.239240] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=0 paths=1 nar_bytes_in=5126764worker2 # [ 156.242054] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666765worker2 # [ 156.248450] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6766worker2 # [ 156.251554] nix-daemon[847]: reaped child process 1644, status = succeeded6767worker2 # [ 156.256472] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6768client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6769client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6770client # farm-a> LOG-farm-a-drain6771worker2 # [ 157.276959] nix-daemon[847]: reaped child process 1658, status = succeeded6772worker2 # [ 157.403640] nix-daemon[847]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6773worker2 # [ 157.415633] nix-daemon[847]: reaped child process 1676, status = succeeded6774lb # [ 157.419197] niks3-server[1054]: 2026/09/20 09:35:32 INFO Received uploads request method=POST path=/api/pending_closures6775worker2 # [ 157.437771] nix-grpc-daemon[1438]: time=2026-09-20T09:35:33.537Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6776worker2 # [ 157.439430] nix-grpc-daemon[1438]: time=2026-09-20T09:35:33.539Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6777lb # [ 157.460808] niks3-server[1054]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6778lb # [ 157.470280] niks3-server[1054]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6779lb # [ 157.476400] niks3-server[1054]: 2026/09/20 09:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6780lb # [ 157.481508] niks3-server[1054]: 2026/09/20 09:35:32 INFO Signed narinfos id=65 count=16781worker2 # [ 157.491488] nix-grpc-daemon[1438]: time=2026-09-20T09:35:33.591Z level=INFO msg="Uploading 1 narinfos"6782lb # [ 157.484307] niks3-server[1054]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6783lb # [ 157.491062] niks3-server[1054]: 2026/09/20 09:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6784lb # [ 157.494664] niks3-server[1054]: 2026/09/20 09:35:32 INFO Completed upload id=656785worker2 # [ 157.502149] nix-grpc-daemon[1438]: time=2026-09-20T09:35:33.602Z level=INFO msg="Upload complete. (237ms)"6786lb # [ 157.496023] niks3-server[1054]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6787worker2 # [ 157.506917] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16788worker2 # [ 157.516735] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:33Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06789worker2 # [ 157.520702] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:33Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16790worker2 # [ 157.527276] nix-daemon[847]: reaped child process 1656, status = succeeded6791worker2 # [ 157.560592] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666792worker2 # [ 157.572058] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457666793worker2 # [ 157.585273] nix-grpc-daemon[1438]: time=2026-09-20T09:35:33.685Z level=INFO msg="All 1 paths already cached"6794client # copying 1 paths to 'lb:50051'...6795client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6796worker2 # [ 157.592842] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457666797worker2 # [ 157.603121] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6798worker2 # [ 157.742077] nix-daemon[847]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6799worker2 # [ 157.754677] nix-daemon[847]: reaped child process 1685, status = succeeded6800worker2 # [ 157.883845] nix-daemon[847]: accepted connection from pid 1687, user nix-grpc-daemon (trusted)6801worker2 # [ 157.896405] nix-daemon[847]: reaped child process 1690, status = succeeded6802lb # [ 157.899322] niks3-server[1054]: 2026/09/20 09:35:33 INFO Received uploads request method=POST path=/api/pending_closures6803worker2 # [ 157.914099] nix-grpc-daemon[1438]: time=2026-09-20T09:35:34.014Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6804worker2 # [ 157.916047] nix-grpc-daemon[1438]: time=2026-09-20T09:35:34.015Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6805lb # [ 157.926260] niks3-server[1054]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6806lb # [ 157.938931] niks3-server[1054]: 2026/09/20 09:35:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6807worker2 # [ 157.951963] nix-grpc-daemon[1438]: time=2026-09-20T09:35:34.052Z level=INFO msg="Uploading 1 narinfos"6808lb # [ 157.944065] niks3-server[1054]: 2026/09/20 09:35:33 INFO Signed narinfos id=66 count=16809lb # [ 157.949975] niks3-server[1054]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6810lb # [ 157.961361] niks3-server[1054]: 2026/09/20 09:35:33 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6811lb # [ 157.968469] niks3-server[1054]: 2026/09/20 09:35:33 INFO Completed upload id=666812worker2 # [ 157.975956] nix-grpc-daemon[1438]: time=2026-09-20T09:35:34.076Z level=INFO msg="Upload complete. (363ms)"6813lb # [ 157.972050] niks3-server[1054]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6814worker2 # [ 157.980145] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=0 paths=1 nar_bytes_in=6246815worker2 # [ 157.985630] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666816worker2 # [ 157.995912] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6817worker2 # [ 157.997530] nix-daemon[847]: reaped child process 1680, status = succeeded6818worker2 # [ 158.002558] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6819client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6820client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6821lb # [ 158.794561] envoy[926]: conn peer=[2001:db8:1::1]:33106 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27111 tx=10690 ms=191156822lb # [ 158.798425] envoy[926]: conn peer=[2001:db8:1::1]:48122 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=878 tx=480 ms=386823client # farm-b> LOG-farm-b-drain6824worker2 # [ 159.428860] nix-daemon[847]: reaped child process 1694, status = succeeded6825worker2 # [ 159.591103] nix-daemon[847]: accepted connection from pid 1709, user nix-grpc-daemon (trusted)6826worker2 # [ 159.602463] nix-daemon[847]: reaped child process 1712, status = succeeded6827lb # [ 159.610715] niks3-server[1054]: 2026/09/20 09:35:34 INFO Received uploads request method=POST path=/api/pending_closures6828worker2 # [ 159.626049] nix-grpc-daemon[1438]: time=2026-09-20T09:35:35.726Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6829worker2 # [ 159.629058] nix-grpc-daemon[1438]: time=2026-09-20T09:35:35.727Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6830lb # [ 159.642810] niks3-server[1054]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6831lb # [ 159.656725] niks3-server[1054]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6832lb # [ 159.668488] niks3-server[1054]: 2026/09/20 09:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6833lb # [ 159.672655] niks3-server[1054]: 2026/09/20 09:35:34 INFO Signed narinfos id=67 count=16834worker2 # [ 159.682767] nix-grpc-daemon[1438]: time=2026-09-20T09:35:35.782Z level=INFO msg="Uploading 1 narinfos"6835lb # [ 159.677685] niks3-server[1054]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6836lb # [ 159.685654] niks3-server[1054]: 2026/09/20 09:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6837lb # [ 159.691410] niks3-server[1054]: 2026/09/20 09:35:34 INFO Completed upload id=676838worker2 # [ 159.698952] nix-grpc-daemon[1438]: time=2026-09-20T09:35:35.799Z level=INFO msg="Upload complete. (288ms)"6839lb # [ 159.694027] niks3-server[1054]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6840worker2 # [ 159.706341] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16841worker2 # [ 159.718214] nix-daemon[847]: reaped child process 1692, status = succeeded6842worker2 # [ 159.753158] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:35Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06843worker2 # [ 159.758536] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:35Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16844worker2 # [ 159.796531] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666845worker2 # [ 159.805028] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:457666846worker2 # [ 159.814117] nix-grpc-daemon[1438]: time=2026-09-20T09:35:35.914Z level=INFO msg="All 2 paths already cached"6847client # copying 1 paths to 'lb:50051'...6848client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6849worker2 # [ 159.819902] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:457666850worker2 # [ 159.826091] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6851worker2 # [ 159.982224] nix-daemon[847]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6852worker2 # [ 159.995606] nix-daemon[847]: reaped child process 1721, status = succeeded6853worker2 # [ 160.149851] nix-daemon[847]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6854worker2 # [ 160.161931] nix-daemon[847]: reaped child process 1726, status = succeeded6855lb # [ 160.169160] niks3-server[1054]: 2026/09/20 09:35:35 INFO Received uploads request method=POST path=/api/pending_closures6856worker2 # [ 160.184070] nix-grpc-daemon[1438]: time=2026-09-20T09:35:36.284Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6857worker2 # [ 160.185445] nix-grpc-daemon[1438]: time=2026-09-20T09:35:36.284Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6858lb # [ 160.194294] niks3-server[1054]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6859lb # [ 160.202649] niks3-server[1054]: 2026/09/20 09:35:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6860worker2 # [ 160.215967] nix-grpc-daemon[1438]: time=2026-09-20T09:35:36.316Z level=INFO msg="Uploading 1 narinfos"6861lb # [ 160.208484] niks3-server[1054]: 2026/09/20 09:35:35 INFO Signed narinfos id=68 count=16862lb # [ 160.215584] niks3-server[1054]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6863lb # [ 160.224569] niks3-server[1054]: 2026/09/20 09:35:35 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6864lb # [ 160.230764] niks3-server[1054]: 2026/09/20 09:35:35 INFO Completed upload id=686865worker2 # [ 160.238377] nix-grpc-daemon[1438]: time=2026-09-20T09:35:36.338Z level=INFO msg="Upload complete. (406ms)"6866worker2 # [ 160.242163] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=0 paths=1 nar_bytes_in=7526867lb # [ 160.235380] niks3-server[1054]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6868worker2 # [ 160.247337] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:457666869worker2 # [ 160.249164] nix-daemon[847]: reaped child process 1716, status = succeeded6870worker2 # [ 160.252150] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6871client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6872worker2 # [ 160.258764] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6873client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6874client # farm-top> LOG-farm-top-drain6875worker2 # [ 161.684190] nix-daemon[847]: reaped child process 1730, status = succeeded6876worker2 # [ 161.837315] nix-daemon[847]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6877worker2 # [ 161.848100] nix-daemon[847]: reaped child process 1748, status = succeeded6878lb # [ 161.856563] niks3-server[1054]: 2026/09/20 09:35:37 INFO Received uploads request method=POST path=/api/pending_closures6879worker2 # [ 161.873198] nix-grpc-daemon[1438]: time=2026-09-20T09:35:37.973Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6880worker2 # [ 161.874727] nix-grpc-daemon[1438]: time=2026-09-20T09:35:37.974Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6881lb # [ 161.895273] niks3-server[1054]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6882lb # [ 161.903514] niks3-server[1054]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6883lb # [ 161.913364] niks3-server[1054]: 2026/09/20 09:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6884worker2 # [ 161.925226] nix-grpc-daemon[1438]: time=2026-09-20T09:35:38.025Z level=INFO msg="Uploading 1 narinfos"6885lb # [ 161.918167] niks3-server[1054]: 2026/09/20 09:35:37 INFO Signed narinfos id=69 count=16886lb # [ 161.924483] niks3-server[1054]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6887lb # [ 161.932168] niks3-server[1054]: 2026/09/20 09:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6888lb # [ 161.937651] niks3-server[1054]: 2026/09/20 09:35:37 INFO Completed upload id=696889worker2 # [ 161.945260] nix-grpc-daemon[1438]: time=2026-09-20T09:35:38.045Z level=INFO msg="Upload complete. (277ms)"6890lb # [ 161.941916] niks3-server[1054]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6891worker2 # [ 161.950857] nix-grpc-daemon[1362]: ts=2026-09-20T09:35:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45766 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16892worker2 # [ 161.964487] nix-daemon[847]: reaped child process 1728, status = succeeded6893client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.49 seconds)6894client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6895client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.19 seconds)6896worker1: must succeed: rm /nix/.rw-store/fill6897worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)6898worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6899worker1 # [ 162.583637] nix-grpc-daemon[1682]: ts=2026-09-20T09:35:38Z level=info event=healthy reason=min_free6900worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)6901(finished: subtest: low disk drains a worker and builds go to the other, in 10.08 seconds)6902(finished: run the VM test script, in 164.32 seconds)6903test script finished in 164.55s6904cleanup6905kill QemuMachine (pid 49)6906client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6907kill QemuMachine (pid 50)6908lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6909kill QemuMachine (pid 51)6910worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6911kill QemuMachine (pid 52)6912worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6913(finished: cleanup, in 0.93 seconds)6914additionally exposed symbols:6915 client, lb, worker1, worker2,6916 vlan1,6917 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh