vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #280
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client: QEMU running (pid 49)13client # Disk image does not exist, creating the virtualisation disk image...14client # Formatting '/build/vm-state-client/tmp.DHtPNBgrfa', fmt=raw size=107374182415client # mke2fs 1.47.4 (6-Mar-2025)16client # Discarding device blocks: 0/262144 done17worker2 # Disk image does not exist, creating the virtualisation disk image...18worker2 # Formatting '/build/vm-state-worker2/tmp.eecsacH6yr', fmt=raw size=107374182419client # Creating filesystem with 262144 4k blocks and 65536 inodes20lb: QEMU running (pid 50)21worker2 # mke2fs 1.47.4 (6-Mar-2025)22client # Filesystem UUID: 18a54aae-1d80-4e51-919f-5612ae09168623lb # Disk image does not exist, creating the virtualisation disk image...24worker2: QEMU running (pid 52)25worker2 # Discarding device blocks: 0/262144 done26client # Superblock backups stored on blocks:27worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes28client # 32768, 98304, 163840, 22937629worker2 # Filesystem UUID: e90090fb-f6a1-4ae3-b72e-6f3dc0fcf65e30client # 31worker2 # Superblock backups stored on blocks:32client # Allocating group tables: 0/8 done33worker2 # 32768, 98304, 163840, 22937634client # Writing inode tables: 0/8 done35worker2 # 36client # Creating journal (8192 blocks): done37worker2 # Allocating group tables: 0/8 done38client # Writing superblocks and filesystem accounting information: 0/8 done39worker2 # Writing inode tables: 0/8 done40client # 41worker2 # Creating journal (8192 blocks): done42client # Virtualisation disk image created.43worker2 # Writing superblocks and filesystem accounting information: 0/8 done44lb # Formatting '/build/vm-state-lb/tmp.TnUnr2PDxV', fmt=raw size=107374182445worker2 # 46lb # mke2fs 1.47.4 (6-Mar-2025)47worker2 # Virtualisation disk image created.48lb # Discarding device blocks: 0/262144 done49lb # Creating filesystem with 262144 4k blocks and 65536 inodes50lb # Filesystem UUID: 53900604-cd18-4a29-9f85-ba7d3f58be5e51lb # Superblock backups stored on blocks:52lb # 32768, 98304, 163840, 22937653lb # 54lb # Allocating group tables: 0/8 done55lb # Writing inode tables: 0/8 done56lb # Creating journal (8192 blocks): done57lb # Writing superblocks and filesystem accounting information: 0/8 done58lb # 59lb # Virtualisation disk image created.60worker1 # Disk image does not exist, creating the virtualisation disk image...61worker1: QEMU running (pid 51)62worker1 # Formatting '/build/vm-state-worker1/tmp.o7K5zDLvtz', fmt=raw size=107374182463worker1 # mke2fs 1.47.4 (6-Mar-2025)64worker1 # Discarding device blocks: 0/262144 done65worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes66worker1 # Filesystem UUID: d04c5f41-bd4f-4061-a09e-b709a2137f8567(finished: start all VMs, in 0.25 seconds)68worker1 # Superblock backups stored on blocks:69client: waiting for unit network-addresses-eth1.service70worker1 # 32768, 98304, 163840, 22937671client: waiting for the VM to finish booting72worker1 # 73worker1 # Allocating group tables: 0/8 done74worker1 # Writing inode tables: 0/8 done75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83lb # 84lb # 85lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker2 # 87worker2 # 88worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90lb # 91lb # 92lb # 93lb # 94lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0095worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...96worker2 # 97worker2 # 98worker2 # 99worker2 # 100worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00101lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...102lb # 103lb # 104worker1 # 105worker1 # 106worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00107worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...108worker2 # 109worker2 # 110worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...111worker1 # 112worker1 # 113worker1 # 114worker1 # 115worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116client # 117client # 118client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00119worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...120worker1 # 121worker1 # 122client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123client # 124client # 125client # 126client # 127client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129client # 130client # 131lb # Booting from ROM...132worker2 # Booting from ROM...133worker1 # Booting from ROM...134client # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136lb # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026137worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g6bjvn1fpihxjl3rkk64li3qwyfil5lr-nixos-system-worker2-test/init regInfo=/nix/store/mddi5sa1bwc5cd417i20xm8sk5njknhg-closure-info/registration console=ttyS0,115200n8 console=tty0138worker2 # [ 0.000000] BIOS-provided physical RAM map:139lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r7i2qny6zxqjgksiw8bnyjrpmwmq2adf-nixos-system-lb-test/init regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration console=ttyS0,115200n8 console=tty0140lb # [ 0.000000] BIOS-provided physical RAM map:141worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable142lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable143worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved144lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved145worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved146lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved147worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable148lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable149worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved150lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved151worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved152lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved153worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved154lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved155worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved156worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026157lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved158worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved159worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved160worker2 # [ 0.000000] NX (Execute Disable) protection: active161lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved162worker2 # [ 0.000000] APIC: Static calls initialized163worker2 # [ 0.000000] SMBIOS 2.8 present.164lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved165lb # [ 0.000000] NX (Execute Disable) protection: active166worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9vpb52xhb70b1gg4gv26lpf9869r95q7-nixos-system-worker1-test/init regInfo=/nix/store/az75mwmf4lqycv6f4hmfwc0nayrgxhwf-closure-info/registration console=ttyS0,115200n8 console=tty0167lb # [ 0.000000] APIC: Static calls initialized168worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014169lb # [ 0.000000] SMBIOS 2.8 present.170worker1 # [ 0.000000] BIOS-provided physical RAM map:171worker2 # [ 0.000000] DMI: Memory slots populated: 1/1172worker2 # [ 0.000000] Hypervisor detected: KVM173worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable174lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014175worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000176worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved177lb # [ 0.000000] DMI: Memory slots populated: 1/1178worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00179lb # [ 0.000000] Hypervisor detected: KVM180worker2 # [ 0.000000] kvm-clock: using sched offset of 474653318 cycles181worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved182lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000183worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable184lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00185worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns186lb # [ 0.000000] kvm-clock: using sched offset of 475962240 cycles187worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved188worker2 # [ 0.000006] tsc: Detected 2400.012 MHz processor189worker2 # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000190worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved191lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns192worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved193lb # [ 0.000006] tsc: Detected 2400.012 MHz processor194worker2 # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs195lb # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000196worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved197worker2 # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT198lb # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs199worker2 # [ 0.002745] found SMP MP-table at [mem 0x000f5450-0x000f545f]200worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved201worker2 # [ 0.002759] Using GB pages for direct mapping202lb # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT203worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved204worker2 # [ 0.002831] RAMDISK: [mem 0x5e367000-0x5ffcffff]205client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026206worker1 # [ 0.000000] NX (Execute Disable) protection: active207lb # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f]208worker2 # [ 0.002839] ACPI: Early table checksum verification disabled209worker1 # [ 0.000000] APIC: Static calls initialized210lb # [ 0.002756] Using GB pages for direct mapping211worker1 # [ 0.000000] SMBIOS 2.8 present.212worker2 # [ 0.002843] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )213lb # [ 0.002869] RAMDISK: [mem 0x5e367000-0x5ffcffff]214lb # [ 0.002877] ACPI: Early table checksum verification disabled215worker2 # [ 0.002847] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)216worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014217lb # [ 0.002880] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )218worker1 # [ 0.000000] DMI: Memory slots populated: 1/1219worker1 # [ 0.000000] Hypervisor detected: KVM220worker2 # [ 0.002852] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)221client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test/init regInfo=/nix/store/941m2hwqpcjqiysnxdfsqwdw595s87vk-closure-info/registration console=ttyS0,115200n8 console=tty0222lb # [ 0.002885] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)223worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000224client # [ 0.000000] BIOS-provided physical RAM map:225worker2 # [ 0.002859] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)226worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00227lb # [ 0.002890] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)228worker2 # [ 0.002862] ACPI: FACS 0x000000005FFE0000 000040229client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable230worker1 # [ 0.000000] kvm-clock: using sched offset of 474885437 cycles231lb # [ 0.002897] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)232client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved233worker2 # [ 0.002863] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)234lb # [ 0.002899] ACPI: FACS 0x000000005FFE0000 000040235worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns236client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved237worker2 # [ 0.002865] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor239lb # [ 0.002901] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)240client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable241worker1 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000242worker2 # [ 0.002866] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)243client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved244lb # [ 0.002902] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)245worker1 # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs246worker2 # [ 0.002868] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)247client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved248lb # [ 0.002904] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)249worker1 # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT250worker2 # [ 0.002869] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]251client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved252worker1 # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f]253lb # [ 0.002905] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)254worker2 # [ 0.002870] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]255client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved256worker1 # [ 0.002752] Using GB pages for direct mapping257lb # [ 0.002906] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]258worker1 # [ 0.002882] RAMDISK: [mem 0x5e367000-0x5ffcffff]259worker2 # [ 0.002871] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]260client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved261worker1 # [ 0.002890] ACPI: Early table checksum verification disabled262lb # [ 0.002908] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]263worker2 # [ 0.002871] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]264client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved265worker1 # [ 0.002894] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )266lb # [ 0.002908] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]267client # [ 0.000000] NX (Execute Disable) protection: active268worker2 # [ 0.002872] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]269client # [ 0.000000] APIC: Static calls initialized270worker1 # [ 0.002898] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)271lb # [ 0.002909] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]272client # [ 0.000000] SMBIOS 2.8 present.273worker2 # [ 0.002872] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]274lb # [ 0.002909] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]275worker1 # [ 0.002902] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)276worker2 # [ 0.002873] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]277client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014278worker2 # [ 0.003101] No NUMA configuration found279lb # [ 0.002910] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]280client # [ 0.000000] DMI: Memory slots populated: 1/1281worker1 # [ 0.002909] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)282worker2 # [ 0.003102] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]283client # [ 0.000000] Hypervisor detected: KVM284lb # [ 0.002910] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]285worker1 # [ 0.002911] ACPI: FACS 0x000000005FFE0000 000040286lb # [ 0.003133] No NUMA configuration found287worker2 # [ 0.003104] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]288client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000289worker2 # [ 0.005385] Zone ranges:290worker1 # [ 0.002913] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)291client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00292lb # [ 0.003134] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]293worker2 # [ 0.005386] DMA [mem 0x0000000000001000-0x0000000000ffffff]294client # [ 0.000000] kvm-clock: using sched offset of 503091746 cycles295lb # [ 0.003137] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]296worker1 # [ 0.002914] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)297lb # [ 0.005421] Zone ranges:298worker2 # [ 0.005388] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]299worker2 # [ 0.005389] Normal empty300lb # [ 0.005422] DMA [mem 0x0000000000001000-0x0000000000ffffff]301worker2 # [ 0.005389] Device empty302client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns303worker1 # [ 0.002916] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)304worker2 # [ 0.005390] Movable zone start for each node305client # [ 0.000005] tsc: Detected 2400.012 MHz processor306lb # [ 0.005423] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]307worker2 # [ 0.005391] Early memory node ranges308lb # [ 0.005424] Normal empty309worker1 # [ 0.002918] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)310client # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000311lb # [ 0.005425] Device empty312worker2 # [ 0.005391] node 0: [mem 0x0000000000001000-0x000000000009efff]313lb # [ 0.005426] Movable zone start for each node314worker1 # [ 0.002919] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]315client # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs316lb # [ 0.005427] Early memory node ranges317worker2 # [ 0.005392] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]318worker1 # [ 0.002920] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]319client # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT320lb # [ 0.005427] node 0: [mem 0x0000000000001000-0x000000000009efff]321worker2 # [ 0.005393] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]322worker1 # [ 0.002920] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]323client # [ 0.002750] found SMP MP-table at [mem 0x000f5450-0x000f545f]324lb # [ 0.005428] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]325worker2 # [ 0.005423] On node 0, zone DMA: 1 pages in unavailable ranges326client # [ 0.002762] Using GB pages for direct mapping327worker1 # [ 0.002921] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]328worker2 # [ 0.005670] On node 0, zone DMA: 97 pages in unavailable ranges329lb # [ 0.005429] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]330client # [ 0.002871] RAMDISK: [mem 0x5e366000-0x5ffcffff]331worker1 # [ 0.002921] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]332worker2 # [ 0.030506] On node 0, zone DMA32: 40 pages in unavailable ranges333lb # [ 0.005448] On node 0, zone DMA: 1 pages in unavailable ranges334client # [ 0.002879] ACPI: Early table checksum verification disabled335worker2 # [ 0.030964] ACPI: PM-Timer IO Port: 0x608336lb # [ 0.005694] On node 0, zone DMA: 97 pages in unavailable ranges337worker1 # [ 0.002922] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]338client # [ 0.002882] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )339worker2 # [ 0.030977] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])340lb # [ 0.030938] On node 0, zone DMA32: 40 pages in unavailable ranges341worker1 # [ 0.002922] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]342client # [ 0.002886] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)343lb # [ 0.031406] ACPI: PM-Timer IO Port: 0x608344worker2 # [ 0.031003] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23345worker1 # [ 0.003143] No NUMA configuration found346lb # [ 0.031419] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])347worker2 # [ 0.031006] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)348client # [ 0.002891] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)349worker1 # [ 0.003144] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]350lb # [ 0.031444] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23351worker2 # [ 0.031008] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)352worker1 # [ 0.003146] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]353client # [ 0.002898] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)354worker1 # [ 0.005248] Zone ranges:355lb # [ 0.031447] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)356worker2 # [ 0.031009] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)357client # [ 0.002901] ACPI: FACS 0x000000005FFE0000 000040358worker1 # [ 0.005249] DMA [mem 0x0000000000001000-0x0000000000ffffff]359lb # [ 0.031448] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)360worker2 # [ 0.031010] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)361worker1 # [ 0.005250] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]362client # [ 0.002903] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)363lb # [ 0.031450] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)364worker1 # [ 0.005251] Normal empty365worker2 # [ 0.031011] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)366worker1 # [ 0.005252] Device empty367client # [ 0.002904] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)368lb # [ 0.031451] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)369worker1 # [ 0.005252] Movable zone start for each node370worker2 # [ 0.031013] ACPI: Using ACPI (MADT) for SMP configuration information371worker1 # [ 0.005253] Early memory node ranges372worker2 # [ 0.031014] ACPI: HPET id: 0x8086a201 base: 0xfed00000373lb # [ 0.031451] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)374client # [ 0.002906] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)375worker1 # [ 0.005254] node 0: [mem 0x0000000000001000-0x000000000009efff]376worker2 # [ 0.031019] TSC deadline timer available377lb # [ 0.031454] ACPI: Using ACPI (MADT) for SMP configuration information378worker2 # [ 0.031023] CPU topo: Max. logical packages: 1379client # [ 0.002907] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)380worker1 # [ 0.005255] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]381lb # [ 0.031455] ACPI: HPET id: 0x8086a201 base: 0xfed00000382worker2 # [ 0.031023] CPU topo: Max. logical dies: 1383lb # [ 0.031459] TSC deadline timer available384client # [ 0.002909] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]385worker1 # [ 0.005256] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]386worker2 # [ 0.031024] CPU topo: Max. dies per package: 1387lb # [ 0.031463] CPU topo: Max. logical packages: 1388worker2 # [ 0.031027] CPU topo: Max. threads per core: 1389worker1 # [ 0.005274] On node 0, zone DMA: 1 pages in unavailable ranges390client # [ 0.002910] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]391lb # [ 0.031464] CPU topo: Max. logical dies: 1392worker2 # [ 0.031028] CPU topo: Num. cores per package: 1393worker1 # [ 0.005524] On node 0, zone DMA: 97 pages in unavailable ranges394lb # [ 0.031465] CPU topo: Max. dies per package: 1395client # [ 0.002910] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]396worker2 # [ 0.031028] CPU topo: Num. threads per package: 1397lb # [ 0.031468] CPU topo: Max. threads per core: 1398worker1 # [ 0.030489] On node 0, zone DMA32: 40 pages in unavailable ranges399worker2 # [ 0.031029] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs400lb # [ 0.031468] CPU topo: Num. cores per package: 1401client # [ 0.002911] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]402worker1 # [ 0.030951] ACPI: PM-Timer IO Port: 0x608403lb # [ 0.031469] CPU topo: Num. threads per package: 1404worker1 # [ 0.030966] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])405client # [ 0.002911] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]406worker2 # [ 0.031051] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()407lb # [ 0.031469] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs408worker1 # [ 0.030994] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23409client # [ 0.002912] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]410worker2 # [ 0.031084] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]411lb # [ 0.031490] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()412worker1 # [ 0.030997] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)413client # [ 0.002912] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]414worker2 # [ 0.031085] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]415client # [ 0.003147] No NUMA configuration found416worker1 # [ 0.030999] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)417lb # [ 0.031524] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]418worker2 # [ 0.031087] [mem 0x60000000-0xafffffff] available for PCI devices419client # [ 0.003147] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]420worker1 # [ 0.031000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)421lb # [ 0.031526] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]422worker2 # [ 0.031089] Booting paravirtualized kernel on KVM423client # [ 0.003150] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]424lb # [ 0.031527] [mem 0x60000000-0xafffffff] available for PCI devices425worker1 # [ 0.031001] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)426client # [ 0.005309] Zone ranges:427worker2 # [ 0.031092] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns428lb # [ 0.031529] Booting paravirtualized kernel on KVM429worker1 # [ 0.031002] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)430client # [ 0.005311] DMA [mem 0x0000000000001000-0x0000000000ffffff]431worker2 # [ 0.035529] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1432worker1 # [ 0.031004] ACPI: Using ACPI (MADT) for SMP configuration information433client # [ 0.005312] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]434lb # [ 0.031533] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns435client # [ 0.005313] Normal empty436worker2 # [ 0.037577] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152437worker1 # [ 0.031005] ACPI: HPET id: 0x8086a201 base: 0xfed00000438client # [ 0.005314] Device empty439lb # [ 0.035955] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1440worker1 # [ 0.031010] TSC deadline timer available441client # [ 0.005314] Movable zone start for each node442worker2 # [ 0.037618] kvm-guest: PV spinlocks disabled, single CPU443client # [ 0.005315] Early memory node ranges444worker1 # [ 0.031015] CPU topo: Max. logical packages: 1445lb # [ 0.038013] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152446worker1 # [ 0.031016] CPU topo: Max. logical dies: 1447client # [ 0.005316] node 0: [mem 0x0000000000001000-0x000000000009efff]448lb # [ 0.038055] kvm-guest: PV spinlocks disabled, single CPU449worker1 # [ 0.031016] CPU topo: Max. dies per package: 1450client # [ 0.005317] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]451worker1 # [ 0.031020] CPU topo: Max. threads per core: 1452worker1 # [ 0.031020] CPU topo: Num. cores per package: 1453client # [ 0.005318] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]454worker1 # [ 0.031021] CPU topo: Num. threads per package: 1455client # [ 0.005340] On node 0, zone DMA: 1 pages in unavailable ranges456worker2 # [ 0.037619] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g6bjvn1fpihxjl3rkk64li3qwyfil5lr-nixos-system-worker2-test/init regInfo=/nix/store/mddi5sa1bwc5cd417i20xm8sk5njknhg-closure-info/registration console=ttyS0,115200n8 console=tty0457worker1 # [ 0.031021] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs458client # [ 0.005587] On node 0, zone DMA: 97 pages in unavailable ranges459worker1 # [ 0.031045] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()460client # [ 0.031011] On node 0, zone DMA32: 40 pages in unavailable ranges461lb # [ 0.038057] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r7i2qny6zxqjgksiw8bnyjrpmwmq2adf-nixos-system-lb-test/init regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration console=ttyS0,115200n8 console=tty0462client # [ 0.031484] ACPI: PM-Timer IO Port: 0x608463worker2 # [ 0.037713] Unknown kernel command line parameters "regInfo=/nix/store/mddi5sa1bwc5cd417i20xm8sk5njknhg-closure-info/registration", will be passed to user space.464worker1 # [ 0.031080] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]465worker2 # [ 0.037726] random: crng init done466client # [ 0.031496] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])467worker1 # [ 0.031082] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]468worker2 # [ 0.037727] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes469lb # [ 0.038149] Unknown kernel command line parameters "regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration", will be passed to user space.470client # [ 0.031527] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23471lb # [ 0.038162] random: crng init done472worker1 # [ 0.031083] [mem 0x60000000-0xafffffff] available for PCI devices473client # [ 0.031530] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)474worker2 # [ 0.039750] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)475lb # [ 0.038163] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes476worker1 # [ 0.031085] Booting paravirtualized kernel on KVM477client # [ 0.031532] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)478worker2 # [ 0.040757] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)479lb # [ 0.040185] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)480worker2 # [ 0.040789] Fallback order for Node 0: 0481client # [ 0.031533] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)482worker1 # [ 0.031089] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns483lb # [ 0.041322] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)484worker2 # [ 0.040791] Built 1 zonelists, mobility grouping on. Total pages: 393078485client # [ 0.031534] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)486worker1 # [ 0.035575] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1487worker2 # [ 0.040792] Policy zone: DMA32488lb # [ 0.041355] Fallback order for Node 0: 0489client # [ 0.031535] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)490worker1 # [ 0.037641] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152491worker2 # [ 0.043283] mem auto-init: stack:all(zero), heap alloc:on, heap free:off492lb # [ 0.041358] Built 1 zonelists, mobility grouping on. Total pages: 393078493worker1 # [ 0.037684] kvm-guest: PV spinlocks disabled, single CPU494client # [ 0.031537] ACPI: Using ACPI (MADT) for SMP configuration information495lb # [ 0.041367] Policy zone: DMA32496worker2 # [ 0.046385] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1497client # [ 0.031538] ACPI: HPET id: 0x8086a201 base: 0xfed00000498lb # [ 0.043743] mem auto-init: stack:all(zero), heap alloc:on, heap free:off499worker2 # [ 0.049549] allocated 3145728 bytes of page_ext500client # [ 0.031542] TSC deadline timer available501worker2 # [ 0.059331] ftrace: allocating 48731 entries in 192 pages502lb # [ 0.046887] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1503client # [ 0.031546] CPU topo: Max. logical packages: 1504lb # [ 0.050036] allocated 3145728 bytes of page_ext505worker2 # [ 0.059334] ftrace: allocated 192 pages with 2 groups506client # [ 0.031547] CPU topo: Max. logical dies: 1507worker2 # [ 0.060201] Dynamic Preempt: lazy508lb # [ 0.059893] ftrace: allocating 48731 entries in 192 pages509client # [ 0.031547] CPU topo: Max. dies per package: 1510worker2 # [ 0.060340] rcu: Preemptible hierarchical RCU implementation.511lb # [ 0.059895] ftrace: allocated 192 pages with 2 groups512client # [ 0.031551] CPU topo: Max. threads per core: 1513lb # [ 0.060763] Dynamic Preempt: lazy514worker1 # [ 0.037686] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9vpb52xhb70b1gg4gv26lpf9869r95q7-nixos-system-worker1-test/init regInfo=/nix/store/az75mwmf4lqycv6f4hmfwc0nayrgxhwf-closure-info/registration console=ttyS0,115200n8 console=tty0515worker2 # [ 0.060341] rcu: RCU event tracing is enabled.516client # [ 0.031551] CPU topo: Num. cores per package: 1517lb # [ 0.060885] rcu: Preemptible hierarchical RCU implementation.518worker2 # [ 0.060342] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.519client # [ 0.031552] CPU topo: Num. threads per package: 1520lb # [ 0.060886] rcu: RCU event tracing is enabled.521worker2 # [ 0.060343] Trampoline variant of Tasks RCU enabled.522client # [ 0.031552] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs523worker1 # [ 0.037783] Unknown kernel command line parameters "regInfo=/nix/store/az75mwmf4lqycv6f4hmfwc0nayrgxhwf-closure-info/registration", will be passed to user space.524worker2 # [ 0.060344] Rude variant of Tasks RCU enabled.525lb # [ 0.060887] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.526worker1 # [ 0.037796] random: crng init done527client # [ 0.031571] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()528worker2 # [ 0.060344] Tracing variant of Tasks RCU enabled.529lb # [ 0.060888] Trampoline variant of Tasks RCU enabled.530worker1 # [ 0.037797] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes531lb # [ 0.060889] Rude variant of Tasks RCU enabled.532client # [ 0.031603] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]533worker2 # [ 0.060345] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.534lb # [ 0.060889] Tracing variant of Tasks RCU enabled.535worker1 # [ 0.039826] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)536client # [ 0.031605] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]537worker2 # [ 0.060345] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1538lb # [ 0.060890] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.539worker1 # [ 0.040847] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)540client # [ 0.031606] [mem 0x60000000-0xafffffff] available for PCI devices541worker2 # [ 0.060400] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.542worker1 # [ 0.040882] Fallback order for Node 0: 0543client # [ 0.031608] Booting paravirtualized kernel on KVM544lb # [ 0.060890] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1545worker1 # [ 0.040885] Built 1 zonelists, mobility grouping on. Total pages: 393078546worker2 # [ 0.060402] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.547lb # [ 0.060944] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.548worker1 # [ 0.040886] Policy zone: DMA32549client # [ 0.031610] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns550worker1 # [ 0.043390] mem auto-init: stack:all(zero), heap alloc:on, heap free:off551worker2 # [ 0.060403] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.552lb # [ 0.060946] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.553client # [ 0.036052] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1554worker1 # [ 0.046467] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1555worker2 # [ 0.064726] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16556client # [ 0.038128] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152557lb # [ 0.060947] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.558worker1 # [ 0.049616] allocated 3145728 bytes of page_ext559client # [ 0.038172] kvm-guest: PV spinlocks disabled, single CPU560worker2 # [ 0.065023] rcu: srcu_init: Setting srcu_struct sizes based on contention.561lb # [ 0.065373] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16562worker1 # [ 0.059302] ftrace: allocating 48731 entries in 192 pages563worker1 # [ 0.059304] ftrace: allocated 192 pages with 2 groups564lb # [ 0.065643] rcu: srcu_init: Setting srcu_struct sizes based on contention.565worker2 # [ 0.065029] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns566worker1 # [ 0.060120] Dynamic Preempt: lazy567worker1 # [ 0.060278] rcu: Preemptible hierarchical RCU implementation.568lb # [ 0.065650] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns569worker2 # [ 0.065136] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)570worker1 # [ 0.060279] rcu: RCU event tracing is enabled.571worker2 # [ 0.068620] Console: colour VGA+ 80x25572worker2 # [ 0.068623] printk: legacy console [tty0] enabled573worker1 # [ 0.060279] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.574lb # [ 0.065755] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)575client # [ 0.038174] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test/init regInfo=/nix/store/941m2hwqpcjqiysnxdfsqwdw595s87vk-closure-info/registration console=ttyS0,115200n8 console=tty0576lb # [ 0.069273] Console: colour VGA+ 80x25577worker2 # [ 0.098959] printk: legacy console [ttyS0] enabled578worker1 # [ 0.060281] Trampoline variant of Tasks RCU enabled.579worker1 # [ 0.060282] Rude variant of Tasks RCU enabled.580lb # [ 0.069277] printk: legacy console [tty0] enabled581worker2 # [ 0.195609] ACPI: Core revision 20250807582lb # [ 0.098764] printk: legacy console [ttyS0] enabled583worker1 # [ 0.060282] Tracing variant of Tasks RCU enabled.584client # [ 0.038269] Unknown kernel command line parameters "regInfo=/nix/store/941m2hwqpcjqiysnxdfsqwdw595s87vk-closure-info/registration", will be passed to user space.585lb # [ 0.195680] ACPI: Core revision 20250807586client # [ 0.038281] random: crng init done587worker1 # [ 0.060283] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.588worker2 # [ 0.196465] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns589client # [ 0.038282] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes590worker1 # [ 0.060284] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1591worker2 # [ 0.197939] APIC: Switch to symmetric I/O mode setup592lb # [ 0.196560] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns593client # [ 0.040331] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)594worker2 # [ 0.198867] x2apic enabled595worker1 # [ 0.060304] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.596lb # [ 0.198098] APIC: Switch to symmetric I/O mode setup597client # [ 0.041496] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)598worker2 # [ 0.199601] APIC: Switched APIC routing to: physical x2apic599lb # [ 0.199028] x2apic enabled600worker1 # [ 0.060306] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.601client # [ 0.041527] Fallback order for Node 0: 0602client # [ 0.041530] Built 1 zonelists, mobility grouping on. Total pages: 393078603worker1 # [ 0.060307] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.604lb # [ 0.199757] APIC: Switched APIC routing to: physical x2apic605client # [ 0.041531] Policy zone: DMA32606worker1 # [ 0.064666] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16607worker2 # [ 0.201397] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1608client # [ 0.043913] mem auto-init: stack:all(zero), heap alloc:on, heap free:off609worker1 # [ 0.064936] rcu: srcu_init: Setting srcu_struct sizes based on contention.610client # [ 0.046938] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1611lb # [ 0.201523] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1612client # [ 0.050181] allocated 3145728 bytes of page_ext613worker2 # [ 0.202386] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns614worker1 # [ 0.064941] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns615client # [ 0.059762] ftrace: allocating 48731 entries in 192 pages616client # [ 0.059764] ftrace: allocated 192 pages with 2 groups617lb # [ 0.202458] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns618worker2 # [ 0.204025] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)619worker1 # [ 0.065046] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)620client # [ 0.060606] Dynamic Preempt: lazy621worker1 # [ 0.068553] Console: colour VGA+ 80x25622client # [ 0.060768] rcu: Preemptible hierarchical RCU implementation.623worker1 # [ 0.068556] printk: legacy console [tty0] enabled624worker2 # [ 0.205107] x86/cpu: User Mode Instruction Prevention (UMIP) activated625lb # [ 0.204022] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)626client # [ 0.060768] rcu: RCU event tracing is enabled.627worker1 # [ 0.097992] printk: legacy console [ttyS0] enabled628client # [ 0.060769] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.629lb # [ 0.206111] x86/cpu: User Mode Instruction Prevention (UMIP) activated630worker2 # [ 0.206155] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127631worker1 # [ 0.193736] ACPI: Core revision 20250807632client # [ 0.060770] Trampoline variant of Tasks RCU enabled.633lb # [ 0.207181] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127634client # [ 0.060771] Rude variant of Tasks RCU enabled.635worker2 # [ 0.207022] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0636client # [ 0.060771] Tracing variant of Tasks RCU enabled.637worker1 # [ 0.194604] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns638lb # [ 0.208020] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0639client # [ 0.060772] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.640worker2 # [ 0.208026] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto641worker1 # [ 0.196165] APIC: Switch to symmetric I/O mode setup642client # [ 0.060773] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1643worker1 # [ 0.197094] x2apic enabled644lb # [ 0.209023] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto645worker2 # [ 0.209022] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl646client # [ 0.060789] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.647worker1 # [ 0.197828] APIC: Switched APIC routing to: physical x2apic648lb # [ 0.210020] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl649worker2 # [ 0.210022] Transient Scheduler Attacks: Vulnerable: No microcode650client # [ 0.060790] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.651lb # [ 0.212020] Transient Scheduler Attacks: Vulnerable: No microcode652worker2 # [ 0.211021] Spectre V2 : Mitigation: Enhanced / Automatic IBRS653worker1 # [ 0.199629] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1654client # [ 0.060791] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.655lb # [ 0.213020] Spectre V2 : Mitigation: Enhanced / Automatic IBRS656worker2 # [ 0.212021] Speculative Return Stack Overflow: Mitigation: Safe RET657client # [ 0.065304] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16658lb # [ 0.214020] Speculative Return Stack Overflow: Mitigation: Safe RET659client # [ 0.065589] rcu: srcu_init: Setting srcu_struct sizes based on contention.660worker1 # [ 0.200600] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns661client # [ 0.065597] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns662worker1 # [ 0.202317] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)663client # [ 0.065698] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)664worker1 # [ 0.203398] x86/cpu: User Mode Instruction Prevention (UMIP) activated665client # [ 0.069575] Console: colour VGA+ 80x25666worker2 # [ 0.213022] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization667client # [ 0.069579] printk: legacy console [tty0] enabled668lb # [ 0.215019] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization669worker1 # [ 0.204442] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127670worker2 # [ 0.215029] Spectre V2 : Enabling IBPB for BPF671client # [ 0.099541] printk: legacy console [ttyS0] enabled672lb # [ 0.217027] Spectre V2 : Enabling IBPB for BPF673client # [ 0.195011] ACPI: Core revision 20250807674worker1 # [ 0.205315] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0675worker2 # [ 0.216022] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier676lb # [ 0.218533] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier677worker2 # [ 0.217022] active return thunk: srso_alias_return_thunk678client # [ 0.195934] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns679worker1 # [ 0.206318] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto680lb # [ 0.219020] active return thunk: srso_alias_return_thunk681worker2 # [ 0.218043] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'682client # [ 0.197475] APIC: Switch to symmetric I/O mode setup683lb # [ 0.220653] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'684worker1 # [ 0.208315] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl685client # [ 0.198431] x2apic enabled686worker2 # [ 0.219022] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'687lb # [ 0.221880] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'688worker1 # [ 0.209315] Transient Scheduler Attacks: Vulnerable: No microcode689client # [ 0.199209] APIC: Switched APIC routing to: physical x2apic690worker2 # [ 0.220022] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'691lb # [ 0.222739] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'692worker1 # [ 0.210314] Spectre V2 : Mitigation: Enhanced / Automatic IBRS693worker2 # [ 0.221021] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'694lb # [ 0.223742] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'695worker1 # [ 0.211314] Speculative Return Stack Overflow: Mitigation: Safe RET696client # [ 0.201060] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1697worker2 # [ 0.222021] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'698lb # [ 0.224755] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'699worker2 # [ 0.223021] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'700lb # [ 0.225019] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'701client # [ 0.202024] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns702worker2 # [ 0.224021] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'703lb # [ 0.226019] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'704client # [ 0.203676] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)705worker2 # [ 0.226022] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256706worker1 # [ 0.212314] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization707lb # [ 0.227904] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256708client # [ 0.204765] x86/cpu: User Mode Instruction Prevention (UMIP) activated709worker1 # [ 0.214322] Spectre V2 : Enabling IBPB for BPF710worker2 # [ 0.226934] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64711lb # [ 0.228718] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64712client # [ 0.205816] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127713lb # [ 0.229724] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512714worker2 # [ 0.227732] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512715worker1 # [ 0.215316] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier716client # [ 0.207346] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0717lb # [ 0.230020] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024718worker2 # [ 0.228722] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024719worker1 # [ 0.216315] active return thunk: srso_alias_return_thunk720worker2 # [ 0.229731] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8721lb # [ 0.231020] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8722worker1 # [ 0.217336] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'723client # [ 0.207678] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto724lb # [ 0.232019] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.725worker1 # [ 0.218315] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'726worker2 # [ 0.230725] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.727client # [ 0.209673] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl728worker1 # [ 0.220034] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'729client # [ 0.210673] Transient Scheduler Attacks: Vulnerable: No microcode730worker1 # [ 0.220315] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'731client # [ 0.211673] Spectre V2 : Mitigation: Enhanced / Automatic IBRS732worker1 # [ 0.221314] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'733client # [ 0.212673] Speculative Return Stack Overflow: Mitigation: Safe RET734worker1 # [ 0.223065] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'735worker1 # [ 0.224106] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'736worker1 # [ 0.225315] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256737client # [ 0.213673] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization738worker1 # [ 0.226314] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64739client # [ 0.215687] Spectre V2 : Enabling IBPB for BPF740worker1 # [ 0.227315] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512741worker1 # [ 0.228314] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024742client # [ 0.216674] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier743worker1 # [ 0.229314] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8744client # [ 0.217673] active return thunk: srso_alias_return_thunk745client # [ 0.218694] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'746worker1 # [ 0.230314] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.747client # [ 0.219673] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748client # [ 0.221401] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749client # [ 0.222437] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750client # [ 0.223433] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751client # [ 0.224399] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752client # [ 0.225445] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753client # [ 0.226673] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754client # [ 0.227608] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755client # [ 0.228391] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756client # [ 0.229397] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757client # [ 0.230405] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758client # [ 0.231447] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker2 # [ 0.264558] Freeing SMP alternatives memory: 44K760lb # [ 0.266708] Freeing SMP alternatives memory: 44K761worker2 # [ 0.265025] pid_max: default: 32768 minimum: 301762lb # [ 0.267023] pid_max: default: 32768 minimum: 301763worker2 # [ 0.266102] LSM: initializing lsm=capability,landlock,yama,bpf,ima764lb # [ 0.268118] LSM: initializing lsm=capability,landlock,yama,bpf,ima765worker2 # [ 0.267113] landlock: Up and running.766lb # [ 0.269139] landlock: Up and running.767worker2 # [ 0.267751] Yama: becoming mindful.768lb # [ 0.270020] Yama: becoming mindful.769worker2 # [ 0.268667] LSM support for eBPF active770lb # [ 0.270806] LSM support for eBPF active771worker2 # [ 0.269611] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)772lb # [ 0.271141] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)773lb # [ 0.272059] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)774worker2 # [ 0.270058] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)775lb # [ 0.274918] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)776worker2 # [ 0.273320] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)777lb # [ 0.275499] Performance Events: Fam17h+ core perfctr, AMD PMU driver.778worker2 # [ 0.274555] Performance Events: Fam17h+ core perfctr, AMD PMU driver.779lb # [ 0.276024] ... version: 2780worker2 # [ 0.275031] ... version: 2781lb # [ 0.276715] ... bit width: 48782worker2 # [ 0.275708] ... bit width: 48783worker1 # [ 0.264895] Freeing SMP alternatives memory: 44K784lb # [ 0.277022] ... generic counters: 6785worker2 # [ 0.276024] ... generic counters: 6786worker1 # [ 0.265317] pid_max: default: 32768 minimum: 301787lb # [ 0.277699] ... generic bitmap: 000000000000003f788worker2 # [ 0.276723] ... generic bitmap: 000000000000003f789lb # [ 0.278067] ... fixed-purpose counters: 0790worker1 # [ 0.266401] LSM: initializing lsm=capability,landlock,yama,bpf,ima791worker2 # [ 0.277024] ... fixed-purpose counters: 0792worker1 # [ 0.267399] landlock: Up and running.793lb # [ 0.278758] ... fixed-purpose bitmap: 0000000000000000794worker2 # [ 0.277698] ... fixed-purpose bitmap: 0000000000000000795worker1 # [ 0.268015] Yama: becoming mindful.796lb # [ 0.279022] ... value mask: 0000ffffffffffff797worker2 # [ 0.278024] ... value mask: 0000ffffffffffff798worker1 # [ 0.268946] LSM support for eBPF active799lb # [ 0.279874] ... max period: 00007fffffffffff800worker2 # [ 0.278880] ... max period: 00007fffffffffff801lb # [ 0.280676] ... global_ctrl mask: 000000000000003f802worker1 # [ 0.269580] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)803worker2 # [ 0.279685] ... global_ctrl mask: 000000000000003f804lb # [ 0.281133] signal: max sigframe size: 3376805worker2 # [ 0.280139] signal: max sigframe size: 3376806worker1 # [ 0.271219] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)807lb # [ 0.281879] rcu: Hierarchical SRCU implementation.808worker2 # [ 0.280909] rcu: Hierarchical SRCU implementation.809lb # [ 0.282582] rcu: Max phase no-delay instances is 400.810worker2 # [ 0.281602] rcu: Max phase no-delay instances is 400.811worker1 # [ 0.273477] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)812worker1 # [ 0.274870] Performance Events: Fam17h+ core perfctr, AMD PMU driver.813worker1 # [ 0.275325] ... version: 2814client # [ 0.266128] Freeing SMP alternatives memory: 44K815worker1 # [ 0.276023] ... bit width: 48816lb # [ 0.287502] smp: Bringing up secondary CPUs ...817client # [ 0.266676] pid_max: default: 32768 minimum: 301818worker1 # [ 0.276318] ... generic counters: 6819worker2 # [ 0.286660] smp: Bringing up secondary CPUs ...820lb # [ 0.288035] smp: Brought up 1 node, 1 CPU821worker1 # [ 0.277019] ... generic bitmap: 000000000000003f822worker2 # [ 0.287039] smp: Brought up 1 node, 1 CPU823client # [ 0.267772] LSM: initializing lsm=capability,landlock,yama,bpf,ima824worker1 # [ 0.277317] ... fixed-purpose counters: 0825lb # [ 0.288689] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)826client # [ 0.268783] landlock: Up and running.827worker2 # [ 0.287678] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)828client # [ 0.269382] Yama: becoming mindful.829worker1 # [ 0.277992] ... fixed-purpose bitmap: 0000000000000000830client # [ 0.269899] LSM support for eBPF active831worker1 # [ 0.278317] ... value mask: 0000ffffffffffff832lb # [ 0.289252] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)833worker1 # [ 0.279171] ... max period: 00007fffffffffff834worker2 # [ 0.288215] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)835client # [ 0.270799] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)836lb # [ 0.290246] devtmpfs: initialized837worker1 # [ 0.279983] ... global_ctrl mask: 000000000000003f838worker2 # [ 0.289247] devtmpfs: initialized839lb # [ 0.291027] x86/mm: Memory block size: 128MB840worker1 # [ 0.280424] signal: max sigframe size: 3376841client # [ 0.271716] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)842worker2 # [ 0.290063] x86/mm: Memory block size: 128MB843worker1 # [ 0.281241] rcu: Hierarchical SRCU implementation.844worker1 # [ 0.282035] rcu: Max phase no-delay instances is 400.845lb # [ 0.292781] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)846worker2 # [ 0.291790] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)847client # [ 0.274672] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)848lb # [ 0.293904] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).849worker2 # [ 0.292894] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).850lb # [ 0.295100] pinctrl core: initialized pinctrl subsystem851client # [ 0.275252] Performance Events: Fam17h+ core perfctr, AMD PMU driver.852worker2 # [ 0.294102] pinctrl core: initialized pinctrl subsystem853client # [ 0.275678] ... version: 2854lb # [ 0.296187] PM: RTC time: 09:32:39, date: 2026-09-20855client # [ 0.276366] ... bit width: 48856worker2 # [ 0.295226] PM: RTC time: 09:32:39, date: 2026-09-20857client # [ 0.276675] ... generic counters: 6858worker1 # [ 0.286774] smp: Bringing up secondary CPUs ...859client # [ 0.277426] ... generic bitmap: 000000000000003f860worker1 # [ 0.287331] smp: Brought up 1 node, 1 CPU861client # [ 0.277686] ... fixed-purpose counters: 0862lb # [ 0.299618] NET: Registered PF_NETLINK/PF_ROUTE protocol family863worker1 # [ 0.288048] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)864client # [ 0.278397] ... fixed-purpose bitmap: 0000000000000000865worker2 # [ 0.298634] NET: Registered PF_NETLINK/PF_ROUTE protocol family866client # [ 0.278676] ... value mask: 0000ffffffffffff867lb # [ 0.300482] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations868client # [ 0.279574] ... max period: 00007fffffffffff869worker2 # [ 0.299512] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations870worker1 # [ 0.288513] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)871lb # [ 0.301046] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations872client # [ 0.280310] ... global_ctrl mask: 000000000000003f873worker1 # [ 0.289536] devtmpfs: initialized874worker2 # [ 0.300044] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations875client # [ 0.280796] signal: max sigframe size: 3376876worker1 # [ 0.290343] x86/mm: Memory block size: 128MB877lb # [ 0.302287] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations878client # [ 0.281593] rcu: Hierarchical SRCU implementation.879worker2 # [ 0.301281] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations880lb # [ 0.303031] audit: initializing netlink subsys (disabled)881client # [ 0.282256] rcu: Max phase no-delay instances is 400.882worker2 # [ 0.302034] audit: initializing netlink subsys (disabled)883worker1 # [ 0.292122] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)884lb # [ 0.304024] audit: type=2000 audit(1789896760.161:1): state=initialized audit_enabled=0 res=1885worker2 # [ 0.303119] thermal_sys: Registered thermal governor 'fair_share'886lb # [ 0.305122] thermal_sys: Registered thermal governor 'fair_share'887worker1 # [ 0.293226] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).888worker2 # [ 0.303121] thermal_sys: Registered thermal governor 'bang_bang'889lb # [ 0.305124] thermal_sys: Registered thermal governor 'bang_bang'890worker1 # [ 0.294411] pinctrl core: initialized pinctrl subsystem891worker2 # [ 0.304025] thermal_sys: Registered thermal governor 'step_wise'892lb # [ 0.306023] thermal_sys: Registered thermal governor 'step_wise'893worker1 # [ 0.295522] PM: RTC time: 09:32:39, date: 2026-09-20894worker2 # [ 0.304951] thermal_sys: Registered thermal governor 'user_space'895lb # [ 0.306966] thermal_sys: Registered thermal governor 'user_space'896client # [ 0.287153] smp: Bringing up secondary CPUs ...897worker2 # [ 0.305734] audit: type=2000 audit(1789896760.159:1): state=initialized audit_enabled=0 res=1898lb # [ 0.307717] thermal_sys: Registered thermal governor 'power_allocator'899client # [ 0.287689] smp: Brought up 1 node, 1 CPU900lb # [ 0.308045] cpuidle: using governor menu901worker2 # [ 0.307027] thermal_sys: Registered thermal governor 'power_allocator'902client # [ 0.288392] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)903worker2 # [ 0.307050] cpuidle: using governor menu904worker1 # [ 0.298930] NET: Registered PF_NETLINK/PF_ROUTE protocol family905lb # [ 0.310022] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5906worker2 # [ 0.309235] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5907client # [ 0.288868] Memory: 1454580K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)908worker1 # [ 0.299819] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations909lb # [ 0.311230] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]910client # [ 0.289949] devtmpfs: initialized911worker2 # [ 0.310256] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]912lb # [ 0.312025] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry913worker1 # [ 0.300337] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations914client # [ 0.290738] x86/mm: Memory block size: 128MB915worker2 # [ 0.311027] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry916lb # [ 0.313034] PCI: Using configuration type 1 for base access917worker1 # [ 0.301575] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations918worker2 # [ 0.312035] PCI: Using configuration type 1 for base access919lb # [ 0.314092] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.920worker1 # [ 0.302327] audit: initializing netlink subsys (disabled)921client # [ 0.292505] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)922worker2 # [ 0.313094] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.923worker1 # [ 0.303480] thermal_sys: Registered thermal governor 'fair_share'924client # [ 0.293544] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).925worker1 # [ 0.303482] thermal_sys: Registered thermal governor 'bang_bang'926client # [ 0.294762] pinctrl core: initialized pinctrl subsystem927worker1 # [ 0.304319] thermal_sys: Registered thermal governor 'step_wise'928client # [ 0.295979] PM: RTC time: 09:32:39, date: 2026-09-20929worker1 # [ 0.305284] thermal_sys: Registered thermal governor 'user_space'930worker1 # [ 0.306077] audit: type=2000 audit(1789896760.175:1): state=initialized audit_enabled=0 res=1931worker2 # [ 0.318287] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages932worker1 # [ 0.307321] thermal_sys: Registered thermal governor 'power_allocator'933lb # [ 0.321039] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages934worker2 # [ 0.319026] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page935client # [ 0.299361] NET: Registered PF_NETLINK/PF_ROUTE protocol family936worker1 # [ 0.307335] cpuidle: using governor menu937lb # [ 0.322023] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page938worker1 # [ 0.309590] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5939client # [ 0.300171] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations940client # [ 0.300696] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations941worker1 # [ 0.310574] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]942worker1 # [ 0.311320] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry943client # [ 0.301945] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations944worker2 # [ 0.324026] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages945worker1 # [ 0.312330] PCI: Using configuration type 1 for base access946client # [ 0.302688] audit: initializing netlink subsys (disabled)947lb # [ 0.327023] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages948worker2 # [ 0.325024] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page949client # [ 0.303783] thermal_sys: Registered thermal governor 'fair_share'950lb # [ 0.328023] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page951worker1 # [ 0.313442] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.952client # [ 0.303785] thermal_sys: Registered thermal governor 'bang_bang'953client # [ 0.304677] thermal_sys: Registered thermal governor 'step_wise'954client # [ 0.305679] audit: type=2000 audit(1789896760.185:1): state=initialized audit_enabled=0 res=1955client # [ 0.307680] thermal_sys: Registered thermal governor 'user_space'956client # [ 0.307682] thermal_sys: Registered thermal governor 'power_allocator'957worker1 # [ 0.318571] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages958client # [ 0.308689] cpuidle: using governor menu959worker1 # [ 0.319319] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page960client # [ 0.310884] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5961worker2 # [ 0.335413] ACPI: Added _OSI(Module Device)962client # [ 0.311911] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]963worker2 # [ 0.336026] ACPI: Added _OSI(Processor Device)964client # [ 0.312679] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry965worker2 # [ 0.336745] ACPI: Added _OSI(Processor Aggregator Device)966lb # [ 0.339270] ACPI: Added _OSI(Module Device)967client # [ 0.313687] PCI: Using configuration type 1 for base access968worker1 # [ 0.324318] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages969lb # [ 0.339967] ACPI: Added _OSI(Processor Device)970lb # [ 0.340535] ACPI: Added _OSI(Processor Aggregator Device)971worker1 # [ 0.325317] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page972client # [ 0.314773] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.973lb # [ 0.346659] ACPI: 1 ACPI AML tables successfully acquired and loaded974client # [ 0.321951] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages975worker2 # [ 0.345652] ACPI: 1 ACPI AML tables successfully acquired and loaded976client # [ 0.322678] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page977worker1 # [ 0.335757] ACPI: Added _OSI(Module Device)978lb # [ 0.350417] ACPI: Interpreter enabled979worker1 # [ 0.336319] ACPI: Added _OSI(Processor Device)980lb # [ 0.351039] ACPI: PM: (supports S0 S3 S4 S5)981worker1 # [ 0.337064] ACPI: Added _OSI(Processor Aggregator Device)982worker2 # [ 0.349440] ACPI: Interpreter enabled983lb # [ 0.351729] ACPI: Using IOAPIC for interrupt routing984worker2 # [ 0.350041] ACPI: PM: (supports S0 S3 S4 S5)985worker2 # [ 0.350768] ACPI: Using IOAPIC for interrupt routing986client # [ 0.327677] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages987client # [ 0.328676] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page988worker2 # [ 0.351094] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug989lb # [ 0.354071] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug990worker2 # [ 0.352025] PCI: Using E820 reservations for host bridge windows991lb # [ 0.355024] PCI: Using E820 reservations for host bridge windows992worker2 # [ 0.355166] ACPI: Enabled 2 GPEs in block 00 to 3F993lb # [ 0.358087] ACPI: Enabled 2 GPEs in block 00 to 3F994worker1 # [ 0.345865] ACPI: 1 ACPI AML tables successfully acquired and loaded995client # [ 0.337267] ACPI: Added _OSI(Module Device)996client # [ 0.337678] ACPI: Added _OSI(Processor Device)997client # [ 0.338455] ACPI: Added _OSI(Processor Aggregator Device)998worker1 # [ 0.349588] ACPI: Interpreter enabled999worker1 # [ 0.350234] ACPI: PM: (supports S0 S3 S4 S5)1000lb # [ 0.365389] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1001worker1 # [ 0.350826] ACPI: Using IOAPIC for interrupt routing1002worker2 # [ 0.363953] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1003lb # [ 0.366029] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1004worker1 # [ 0.351385] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1005worker1 # [ 0.352318] PCI: Using E820 reservations for host bridge windows1006worker2 # [ 0.364788] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1007lb # [ 0.367101] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1008lb # [ 0.368145] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1009worker2 # [ 0.366495] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1010lb # [ 0.369364] PCI host bridge to bus 0000:001011worker2 # [ 0.367150] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1012worker1 # [ 0.355382] ACPI: Enabled 2 GPEs in block 00 to 3F1013client # [ 0.345222] ACPI: 1 ACPI AML tables successfully acquired and loaded1014lb # [ 0.370026] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1015worker2 # [ 0.368387] PCI host bridge to bus 0000:001016lb # [ 0.371022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1017lb # [ 0.372022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1018worker2 # [ 0.369042] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1019lb # [ 0.373022] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1020client # [ 0.348653] ACPI: Interpreter enabled1021worker2 # [ 0.370040] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1022client # [ 0.349146] ACPI: PM: (supports S0 S3 S4 S5)1023lb # [ 0.374022] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1024worker2 # [ 0.371025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1025client # [ 0.349676] ACPI: Using IOAPIC for interrupt routing1026lb # [ 0.375022] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1027worker2 # [ 0.372025] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1028client # [ 0.350547] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1029lb # [ 0.376023] pci_bus 0000:00: root bus resource [bus 00-ff]1030worker2 # [ 0.373024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1031lb # [ 0.376944] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1032worker2 # [ 0.374025] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1033worker1 # [ 0.364012] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1034worker2 # [ 0.375025] pci_bus 0000:00: root bus resource [bus 00-ff]1035lb # [ 0.378448] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1036worker1 # [ 0.364325] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1037worker2 # [ 0.376087] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1038client # [ 0.355677] PCI: Using E820 reservations for host bridge windows1039worker1 # [ 0.365968] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1040client # [ 0.356787] ACPI: Enabled 2 GPEs in block 00 to 3F1041worker2 # [ 0.377527] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1042worker1 # [ 0.366440] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1043worker1 # [ 0.367691] PCI host bridge to bus 0000:001044worker1 # [ 0.368322] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1045worker1 # [ 0.369318] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1046lb # [ 0.381060] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1047worker1 # [ 0.370318] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1048lb # [ 0.382034] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1049lb # [ 0.382970] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1050worker1 # [ 0.371318] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1051worker1 # [ 0.372318] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1052lb # [ 0.383961] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1053worker2 # [ 0.380082] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1054worker2 # [ 0.381039] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1055client # [ 0.365528] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1056worker1 # [ 0.373318] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1057worker1 # [ 0.374319] pci_bus 0000:00: root bus resource [bus 00-ff]1058worker2 # [ 0.382030] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1059lb # [ 0.386068] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1060client # [ 0.367010] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1061worker1 # [ 0.375290] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1062worker2 # [ 0.383312] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1063client # [ 0.367754] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1064worker1 # [ 0.376744] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1065lb # [ 0.388053] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1066client # [ 0.368806] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1067worker2 # [ 0.384779] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1068lb # [ 0.388890] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1069client # [ 0.370042] PCI host bridge to bus 0000:001070lb # [ 0.389748] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1071client # [ 0.370680] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1072lb # [ 0.390028] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1073client # [ 0.371676] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1074worker2 # [ 0.387058] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1075worker2 # [ 0.387898] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1076client # [ 0.372675] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1077lb # [ 0.391687] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1078worker2 # [ 0.388745] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1079client # [ 0.373675] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1080worker2 # [ 0.389031] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1081client # [ 0.374676] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1082worker1 # [ 0.379750] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1083lb # [ 0.394056] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1084client # [ 0.375676] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1085worker2 # [ 0.390592] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1086worker1 # [ 0.380330] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1087lb # [ 0.394889] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1088client # [ 0.376676] pci_bus 0000:00: root bus resource [bus 00-ff]1089worker1 # [ 0.381276] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1090lb # [ 0.395745] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1091client # [ 0.377607] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1092worker2 # [ 0.392057] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1093worker1 # [ 0.382273] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1094worker2 # [ 0.392934] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1095lb # [ 0.396588] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1096client # [ 0.379122] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1097worker2 # [ 0.393750] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1098worker1 # [ 0.384240] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1099lb # [ 0.398929] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1100worker2 # [ 0.394615] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1101lb # [ 0.399659] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1102worker1 # [ 0.387352] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1103lb # [ 0.400066] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1104worker1 # [ 0.388296] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1105worker2 # [ 0.397059] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1106worker1 # [ 0.389074] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1107lb # [ 0.401591] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1108client # [ 0.381729] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1109worker2 # [ 0.397898] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1110worker1 # [ 0.389324] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1111client # [ 0.382688] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1112worker2 # [ 0.398777] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1113client # [ 0.383635] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1114lb # [ 0.403052] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1115worker1 # [ 0.391307] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1116lb # [ 0.403896] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1117worker2 # [ 0.399598] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118client # [ 0.384655] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1119lb # [ 0.404737] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1120worker2 # [ 0.401032] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1121worker1 # [ 0.394267] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1122lb # [ 0.405568] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1123client # [ 0.386594] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1124worker2 # [ 0.401911] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1125worker1 # [ 0.394955] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1126worker2 # [ 0.402752] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1127worker1 # [ 0.395339] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1128client # [ 0.389712] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1129worker2 # [ 0.403576] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1130lb # [ 0.407056] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1131worker1 # [ 0.396969] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1132client # [ 0.390541] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1133lb # [ 0.408648] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1134client # [ 0.391401] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1135lb # [ 0.409044] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1136client # [ 0.391682] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1137worker2 # [ 0.405893] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1138worker2 # [ 0.406711] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1139lb # [ 0.410583] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1140worker1 # [ 0.400574] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1141worker2 # [ 0.407048] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1142client # [ 0.393229] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1143worker1 # [ 0.401957] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144lb # [ 0.412030] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1145worker1 # [ 0.402339] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1146worker2 # [ 0.408596] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1147lb # [ 0.412865] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1148client # [ 0.395683] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1149worker1 # [ 0.403876] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150lb # [ 0.413728] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1151client # [ 0.396524] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1152worker2 # [ 0.410047] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1153client # [ 0.397377] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1154worker2 # [ 0.410907] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1155lb # [ 0.414625] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1156worker2 # [ 0.411744] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1157worker1 # [ 0.406299] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1158client # [ 0.398253] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1159worker1 # [ 0.407004] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1160lb # [ 0.416031] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1161worker1 # [ 0.407340] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1162worker2 # [ 0.413173] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1163lb # [ 0.416867] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1164lb # [ 0.417753] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1165worker2 # [ 0.415031] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1166worker1 # [ 0.409411] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1167lb # [ 0.418029] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1168client # [ 0.400709] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1169worker2 # [ 0.415944] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1170client # [ 0.401684] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1171worker2 # [ 0.416797] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1172lb # [ 0.419557] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1173worker1 # [ 0.412148] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1174client # [ 0.402626] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1175worker2 # [ 0.417873] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1176worker1 # [ 0.412932] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1177client # [ 0.404062] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1178worker1 # [ 0.413339] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1179lb # [ 0.421035] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1180worker2 # [ 0.419372] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1181lb # [ 0.422067] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1182worker1 # [ 0.414914] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1183client # [ 0.406412] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1184worker2 # [ 0.421063] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1185lb # [ 0.423555] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1186client # [ 0.406684] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1187worker2 # [ 0.422046] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1188client # [ 0.407659] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1189lb # [ 0.425069] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1190worker2 # [ 0.423569] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1191worker1 # [ 0.417572] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1192lb # [ 0.425912] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1193worker1 # [ 0.418954] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1194client # [ 0.409163] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1195lb # [ 0.426737] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1196worker2 # [ 0.425066] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1197worker1 # [ 0.419342] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1198worker2 # [ 0.425895] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1199lb # [ 0.427563] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1200worker1 # [ 0.420920] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1201worker2 # [ 0.426808] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1202client # [ 0.411412] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1203client # [ 0.411684] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1204worker2 # [ 0.428432] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1205client # [ 0.412697] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1206lb # [ 0.429328] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1207worker1 # [ 0.423300] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1208lb # [ 0.430664] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1209worker1 # [ 0.424012] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1210client # [ 0.414305] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1211worker2 # [ 0.430614] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1212lb # [ 0.431044] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1213worker1 # [ 0.424339] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1214worker2 # [ 0.431032] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1215worker1 # [ 0.425323] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1216lb # [ 0.432612] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1217worker2 # [ 0.432047] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1218lb # [ 0.433709] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1219client # [ 0.416707] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1220worker1 # [ 0.426859] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1221worker2 # [ 0.433609] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1222client # [ 0.417684] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1223lb # [ 0.434244] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1224worker2 # [ 0.434587] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1225client # [ 0.418662] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1226worker1 # [ 0.428351] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1227lb # [ 0.435610] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1228worker2 # [ 0.435224] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1229worker1 # [ 0.429305] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1230client # [ 0.420147] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1231worker2 # [ 0.436621] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1232worker1 # [ 0.430756] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1233client # [ 0.422443] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1234client # [ 0.423388] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1235worker1 # [ 0.432325] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1236client # [ 0.423698] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1237worker1 # [ 0.433324] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1238worker1 # [ 0.434286] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1239client # [ 0.424682] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1240worker1 # [ 0.435706] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1241client # [ 0.426248] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1242worker1 # [ 0.437324] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1243client # [ 0.427688] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1244lb # [ 0.436212] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1245worker1 # [ 0.438327] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1246client # [ 0.428691] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1247worker1 # [ 0.439344] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1248client # [ 0.430229] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1249worker2 # [ 0.437686] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1250lb # [ 0.439600] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1251worker1 # [ 0.440873] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1252lb # [ 0.440269] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1253worker2 # [ 0.440532] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1254worker1 # [ 0.442140] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1255client # [ 0.432442] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1256lb # [ 0.441934] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1257worker2 # [ 0.441263] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1258client # [ 0.433710] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1259worker1 # [ 0.443166] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1260lb # [ 0.442988] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1261client # [ 0.434698] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1262worker2 # [ 0.443056] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1263worker1 # [ 0.445088] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1264lb # [ 0.444283] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1265client # [ 0.436246] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1266worker2 # [ 0.444327] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1267lb # [ 0.445279] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1268worker2 # [ 0.445313] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1269client # [ 0.438378] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1270lb # [ 0.447640] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1271client # [ 0.438683] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1272worker2 # [ 0.446289] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1273lb # [ 0.448029] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1274client # [ 0.439650] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1275lb # [ 0.449268] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1276worker2 # [ 0.448077] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1277worker2 # [ 0.448985] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1278client # [ 0.441153] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1279lb # [ 0.450908] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1280worker2 # [ 0.450091] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1281client # [ 0.442534] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1282client # [ 0.443556] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1283worker2 # [ 0.452053] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1284lb # [ 0.454634] ACPI: PCI: Interrupt link LNKA configured for IRQ 101285worker1 # [ 0.446113] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1286client # [ 0.445351] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1287lb # [ 0.455128] ACPI: PCI: Interrupt link LNKB configured for IRQ 101288lb # [ 0.456122] ACPI: PCI: Interrupt link LNKC configured for IRQ 111289worker1 # [ 0.448154] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1290lb # [ 0.457120] ACPI: PCI: Interrupt link LNKD configured for IRQ 111291worker1 # [ 0.449563] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1292worker2 # [ 0.456350] ACPI: PCI: Interrupt link LNKA configured for IRQ 101293lb # [ 0.458125] ACPI: PCI: Interrupt link LNKE configured for IRQ 101294worker2 # [ 0.457136] ACPI: PCI: Interrupt link LNKB configured for IRQ 101295lb # [ 0.459122] ACPI: PCI: Interrupt link LNKF configured for IRQ 101296worker1 # [ 0.451018] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1297worker2 # [ 0.458132] ACPI: PCI: Interrupt link LNKC configured for IRQ 111298lb # [ 0.460132] ACPI: PCI: Interrupt link LNKG configured for IRQ 111299worker1 # [ 0.451589] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1300lb # [ 0.461129] ACPI: PCI: Interrupt link LNKH configured for IRQ 111301worker2 # [ 0.459133] ACPI: PCI: Interrupt link LNKD configured for IRQ 111302lb # [ 0.462058] ACPI: PCI: Interrupt link GSIA configured for IRQ 161303worker2 # [ 0.460124] ACPI: PCI: Interrupt link LNKE configured for IRQ 101304worker1 # [ 0.452603] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1305lb # [ 0.462992] ACPI: PCI: Interrupt link GSIB configured for IRQ 171306worker2 # [ 0.461121] ACPI: PCI: Interrupt link LNKF configured for IRQ 101307worker1 # [ 0.453568] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1308lb # [ 0.463752] ACPI: PCI: Interrupt link GSIC configured for IRQ 181309worker2 # [ 0.462136] ACPI: PCI: Interrupt link LNKG configured for IRQ 111310lb # [ 0.464034] ACPI: PCI: Interrupt link GSID configured for IRQ 191311worker2 # [ 0.463122] ACPI: PCI: Interrupt link LNKH configured for IRQ 111312lb # [ 0.465038] ACPI: PCI: Interrupt link GSIE configured for IRQ 201313worker1 # [ 0.455371] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1314worker2 # [ 0.464076] ACPI: PCI: Interrupt link GSIA configured for IRQ 161315lb # [ 0.465990] ACPI: PCI: Interrupt link GSIF configured for IRQ 211316client # [ 0.445897] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1317worker1 # [ 0.456325] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1318worker2 # [ 0.465040] ACPI: PCI: Interrupt link GSIB configured for IRQ 171319lb # [ 0.466744] ACPI: PCI: Interrupt link GSIG configured for IRQ 221320worker2 # [ 0.466042] ACPI: PCI: Interrupt link GSIC configured for IRQ 181321lb # [ 0.467038] ACPI: PCI: Interrupt link GSIH configured for IRQ 231322client # [ 0.448428] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1323worker1 # [ 0.457555] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1324worker2 # [ 0.467038] ACPI: PCI: Interrupt link GSID configured for IRQ 191325lb # [ 0.469088] iommu: Default domain type: Translated1326worker2 # [ 0.468028] ACPI: PCI: Interrupt link GSIE configured for IRQ 201327worker1 # [ 0.459146] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1328client # [ 0.449110] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1329lb # [ 0.469838] iommu: DMA domain TLB invalidation policy: lazy mode1330worker2 # [ 0.468981] ACPI: PCI: Interrupt link GSIF configured for IRQ 211331client # [ 0.450399] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1332lb # [ 0.470950] ACPI: bus type USB registered1333worker2 # [ 0.469781] ACPI: PCI: Interrupt link GSIG configured for IRQ 221334lb # [ 0.471549] usbcore: registered new interface driver usbfs1335worker2 # [ 0.470039] ACPI: PCI: Interrupt link GSIH configured for IRQ 231336client # [ 0.450963] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1337lb # [ 0.472038] usbcore: registered new interface driver hub1338worker1 # [ 0.462852] ACPI: PCI: Interrupt link LNKA configured for IRQ 101339lb # [ 0.472891] usbcore: registered new device driver usb1340worker2 # [ 0.472056] iommu: Default domain type: Translated1341client # [ 0.451988] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1342worker1 # [ 0.463422] ACPI: PCI: Interrupt link LNKB configured for IRQ 101343worker2 # [ 0.472824] iommu: DMA domain TLB invalidation policy: lazy mode1344lb # [ 0.474530] NetLabel: Initializing1345worker1 # [ 0.464427] ACPI: PCI: Interrupt link LNKC configured for IRQ 111346lb # [ 0.475023] NetLabel: domain hash size = 1281347client # [ 0.452943] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1348worker2 # [ 0.474037] ACPI: bus type USB registered1349lb # [ 0.475709] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1350worker1 # [ 0.465418] ACPI: PCI: Interrupt link LNKD configured for IRQ 111351worker2 # [ 0.474757] usbcore: registered new interface driver usbfs1352lb # [ 0.476089] NetLabel: unlabeled traffic allowed by default1353client # [ 0.454730] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1354worker2 # [ 0.475043] usbcore: registered new interface driver hub1355worker1 # [ 0.466414] ACPI: PCI: Interrupt link LNKE configured for IRQ 101356lb # [ 0.476962] PCI: Using ACPI for IRQ routing1357worker2 # [ 0.475878] usbcore: registered new device driver usb1358worker1 # [ 0.467415] ACPI: PCI: Interrupt link LNKF configured for IRQ 101359client # [ 0.455551] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1360worker1 # [ 0.468418] ACPI: PCI: Interrupt link LNKG configured for IRQ 111361worker2 # [ 0.477581] NetLabel: Initializing1362worker2 # [ 0.478026] NetLabel: domain hash size = 1281363worker1 # [ 0.469414] ACPI: PCI: Interrupt link LNKH configured for IRQ 111364client # [ 0.456782] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1365worker2 # [ 0.478740] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1366worker1 # [ 0.470355] ACPI: PCI: Interrupt link GSIA configured for IRQ 161367worker2 # [ 0.479058] NetLabel: unlabeled traffic allowed by default1368client # [ 0.458483] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1369worker1 # [ 0.471333] ACPI: PCI: Interrupt link GSIB configured for IRQ 171370worker2 # [ 0.479960] PCI: Using ACPI for IRQ routing1371worker1 # [ 0.472272] ACPI: PCI: Interrupt link GSIC configured for IRQ 181372worker1 # [ 0.473105] ACPI: PCI: Interrupt link GSID configured for IRQ 191373worker1 # [ 0.474133] ACPI: PCI: Interrupt link GSIE configured for IRQ 201374client # [ 0.462301] ACPI: PCI: Interrupt link LNKA configured for IRQ 101375worker1 # [ 0.475293] ACPI: PCI: Interrupt link GSIF configured for IRQ 211376client # [ 0.462787] ACPI: PCI: Interrupt link LNKB configured for IRQ 101377worker1 # [ 0.476046] ACPI: PCI: Interrupt link GSIG configured for IRQ 221378worker1 # [ 0.476330] ACPI: PCI: Interrupt link GSIH configured for IRQ 231379client # [ 0.463802] ACPI: PCI: Interrupt link LNKC configured for IRQ 111380client # [ 0.464979] ACPI: PCI: Interrupt link LNKD configured for IRQ 111381worker1 # [ 0.478419] iommu: Default domain type: Translated1382worker1 # [ 0.479191] iommu: DMA domain TLB invalidation policy: lazy mode1383client # [ 0.465789] ACPI: PCI: Interrupt link LNKE configured for IRQ 101384worker1 # [ 0.480388] ACPI: bus type USB registered1385client # [ 0.466778] ACPI: PCI: Interrupt link LNKF configured for IRQ 101386worker1 # [ 0.481162] usbcore: registered new interface driver usbfs1387client # [ 0.467780] ACPI: PCI: Interrupt link LNKG configured for IRQ 111388worker1 # [ 0.482035] usbcore: registered new interface driver hub1389client # [ 0.468785] ACPI: PCI: Interrupt link LNKH configured for IRQ 111390worker1 # [ 0.482330] usbcore: registered new device driver usb1391client # [ 0.469720] ACPI: PCI: Interrupt link GSIA configured for IRQ 161392client # [ 0.470692] ACPI: PCI: Interrupt link GSIB configured for IRQ 171393worker1 # [ 0.484268] NetLabel: Initializing1394worker1 # [ 0.484791] NetLabel: domain hash size = 1281395client # [ 0.471692] ACPI: PCI: Interrupt link GSIC configured for IRQ 181396worker1 # [ 0.485323] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1397client # [ 0.472688] ACPI: PCI: Interrupt link GSID configured for IRQ 191398worker1 # [ 0.486383] NetLabel: unlabeled traffic allowed by default1399client # [ 0.473688] ACPI: PCI: Interrupt link GSIE configured for IRQ 201400worker1 # [ 0.487330] PCI: Using ACPI for IRQ routing1401client # [ 0.474690] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402client # [ 0.475628] ACPI: PCI: Interrupt link GSIG configured for IRQ 221403client # [ 0.476419] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404client # [ 0.477769] iommu: Default domain type: Translated1405client # [ 0.478560] iommu: DMA domain TLB invalidation policy: lazy mode1406client # [ 0.479720] ACPI: bus type USB registered1407client # [ 0.480440] usbcore: registered new interface driver usbfs1408client # [ 0.480691] usbcore: registered new interface driver hub1409client # [ 0.481689] usbcore: registered new device driver usb1410client # [ 0.483554] NetLabel: Initializing1411client # [ 0.484162] NetLabel: domain hash size = 1281412client # [ 0.484675] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413client # [ 0.485679] NetLabel: unlabeled traffic allowed by default1414client # [ 0.486599] PCI: Using ACPI for IRQ routing1415lb # [ 0.520998] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416lb # [ 0.521018] pci 0000:00:01.0: vgaarb: bridge control possible1417lb # [ 0.521018] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418lb # [ 0.521027] vgaarb: loaded1419lb # [ 0.521741] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420lb # [ 0.522022] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421worker2 # [ 0.525373] pci 0000:00:01.0: vgaarb: setting as boot VGA device1422worker2 # [ 0.526020] pci 0000:00:01.0: vgaarb: bridge control possible1423worker2 # [ 0.526020] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1424worker2 # [ 0.526029] vgaarb: loaded1425worker2 # [ 0.526767] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01426lb # [ 0.528104] clocksource: Switched to clocksource kvm-clock1427worker2 # [ 0.527027] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1428lb # [ 0.531590] VFS: Disk quotas dquot_6.6.01429worker2 # [ 0.530119] clocksource: Switched to clocksource kvm-clock1430lb # [ 0.532302] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1431lb # [ 0.533618] pnp: PnP ACPI init1432worker2 # [ 0.531784] VFS: Disk quotas dquot_6.6.01433lb # [ 0.534412] ACPI: IRQ 4 override to edge(!), high(!)1434lb # [ 0.535350] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1435worker2 # [ 0.534256] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1436lb # [ 0.536847] pnp: PnP ACPI: found 6 devices1437worker2 # [ 0.535600] pnp: PnP ACPI init1438worker2 # [ 0.536443] ACPI: IRQ 4 override to edge(!), high(!)1439worker1 # [ 0.531530] pci 0000:00:01.0: vgaarb: setting as boot VGA device1440worker2 # [ 0.537430] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1441worker1 # [ 0.532313] pci 0000:00:01.0: vgaarb: bridge control possible1442worker2 # [ 0.539070] pnp: PnP ACPI: found 6 devices1443worker1 # [ 0.532313] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1444worker1 # [ 0.532321] vgaarb: loaded1445worker1 # [ 0.533065] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01446worker1 # [ 0.533319] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1447lb # [ 0.544336] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1448lb # [ 0.545740] clocksource: Switched to clocksource acpi_pm1449lb # [ 0.546657] NET: Registered PF_INET protocol family1450lb # [ 0.547800] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1451worker1 # [ 0.536397] clocksource: Switched to clocksource kvm-clock1452worker2 # [ 0.546969] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1453worker1 # [ 0.538114] VFS: Disk quotas dquot_6.6.01454worker2 # [ 0.548390] clocksource: Switched to clocksource acpi_pm1455worker2 # [ 0.549312] NET: Registered PF_INET protocol family1456worker2 # [ 0.550455] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1457worker1 # [ 0.540609] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1458worker1 # [ 0.541956] pnp: PnP ACPI init1459client # [ 0.530672] pci 0000:00:01.0: vgaarb: setting as boot VGA device1460worker1 # [ 0.542752] ACPI: IRQ 4 override to edge(!), high(!)1461client # [ 0.530672] pci 0000:00:01.0: vgaarb: bridge control possible1462worker1 # [ 0.543780] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1463client # [ 0.530672] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1464client # [ 0.530682] vgaarb: loaded1465worker1 # [ 0.545323] pnp: PnP ACPI: found 6 devices1466client # [ 0.531410] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01467client # [ 0.531676] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1468lb # [ 0.563657] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1469client # [ 0.535771] clocksource: Switched to clocksource kvm-clock1470lb # [ 0.565127] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1471worker1 # [ 0.552919] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1472worker1 # [ 0.554347] clocksource: Switched to clocksource acpi_pm1473lb # [ 0.566396] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1474client # [ 0.539248] VFS: Disk quotas dquot_6.6.01475worker1 # [ 0.555337] NET: Registered PF_INET protocol family1476lb # [ 0.567674] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1477client # [ 0.539921] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1478lb # [ 0.568862] TCP: Hash tables configured (established 16384 bind 16384)1479worker2 # [ 0.566595] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1480worker1 # [ 0.556538] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1481client # [ 0.541215] pnp: PnP ACPI init1482lb # [ 0.569983] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1483client # [ 0.541969] ACPI: IRQ 4 override to edge(!), high(!)1484worker2 # [ 0.568081] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1485lb # [ 0.571260] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1486client # [ 0.542923] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1487worker2 # [ 0.569355] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1488lb # [ 0.572353] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1489client # [ 0.544467] pnp: PnP ACPI: found 6 devices1490worker2 # [ 0.570688] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1491lb # [ 0.573521] NET: Registered PF_UNIX/PF_LOCAL protocol family1492worker2 # [ 0.571929] TCP: Hash tables configured (established 16384 bind 16384)1493lb # [ 0.574444] NET: Registered PF_XDP protocol family1494lb # [ 0.575236] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1495worker2 # [ 0.573101] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1496lb # [ 0.576198] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1497worker2 # [ 0.574375] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1498lb # [ 0.577159] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1499worker2 # [ 0.575460] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1500lb # [ 0.578206] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1501worker2 # [ 0.576673] NET: Registered PF_UNIX/PF_LOCAL protocol family1502lb # [ 0.579252] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1503worker2 # [ 0.577654] NET: Registered PF_XDP protocol family1504lb # [ 0.580296] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1505worker2 # [ 0.578502] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1506client # [ 0.552065] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1507worker2 # [ 0.579525] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1508lb # [ 0.581920] ACPI: \_SB_.GSIA: Enabled at IRQ 161509client # [ 0.553514] clocksource: Switched to clocksource acpi_pm1510worker2 # [ 0.580501] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1511client # [ 0.554468] NET: Registered PF_INET protocol family1512worker2 # [ 0.581623] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1513lb # [ 0.584008] ACPI: \_SB_.GSIB: Enabled at IRQ 171514client # [ 0.555647] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1515worker2 # [ 0.582715] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1516lb # [ 0.585937] ACPI: \_SB_.GSIC: Enabled at IRQ 181517worker1 # [ 0.572999] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1518worker2 # [ 0.583825] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1519worker1 # [ 0.574643] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1520worker2 # [ 0.585526] ACPI: \_SB_.GSIA: Enabled at IRQ 161521lb # [ 0.587820] ACPI: \_SB_.GSID: Enabled at IRQ 191522worker1 # [ 0.576060] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1523lb # [ 0.589409] PCI: CLS 0 bytes, default 641524worker1 # [ 0.577487] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1525worker2 # [ 0.587867] ACPI: \_SB_.GSIB: Enabled at IRQ 171526lb # [ 0.590278] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1527worker1 # [ 0.578756] TCP: Hash tables configured (established 16384 bind 16384)1528lb # [ 0.592084] Trying to unpack rootfs image as initramfs...1529worker2 # [ 0.590176] ACPI: \_SB_.GSIC: Enabled at IRQ 181530worker1 # [ 0.580011] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1531worker1 # [ 0.581303] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1532worker2 # [ 0.592311] ACPI: \_SB_.GSID: Enabled at IRQ 191533worker1 # [ 0.582464] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1534worker2 # [ 0.593952] PCI: CLS 0 bytes, default 641535worker1 # [ 0.583757] NET: Registered PF_UNIX/PF_LOCAL protocol family1536worker1 # [ 0.584756] NET: Registered PF_XDP protocol family1537worker2 # [ 0.594863] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1538worker1 # [ 0.585614] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1539worker2 # [ 0.596638] Trying to unpack rootfs image as initramfs...1540worker1 # [ 0.586624] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1541worker1 # [ 0.587643] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1542client # [ 0.571944] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1543worker1 # [ 0.588799] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1544client # [ 0.573467] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1545worker1 # [ 0.589904] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1546worker1 # [ 0.591000] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1547client # [ 0.574798] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1548client # [ 0.576112] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1549worker1 # [ 0.592718] ACPI: \_SB_.GSIA: Enabled at IRQ 161550client # [ 0.577294] TCP: Hash tables configured (established 16384 bind 16384)1551client # [ 0.578416] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1552worker1 # [ 0.595030] ACPI: \_SB_.GSIB: Enabled at IRQ 171553client # [ 0.579680] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1554worker1 # [ 0.597041] ACPI: \_SB_.GSIC: Enabled at IRQ 181555client # [ 0.580751] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1556client # [ 0.582014] NET: Registered PF_UNIX/PF_LOCAL protocol family1557worker1 # [ 0.599137] ACPI: \_SB_.GSID: Enabled at IRQ 191558client # [ 0.582989] NET: Registered PF_XDP protocol family1559client # [ 0.583834] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1560worker1 # [ 0.600814] PCI: CLS 0 bytes, default 641561client # [ 0.584805] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1562client # [ 0.585797] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1563worker1 # [ 0.601730] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1564client # [ 0.586916] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1565worker1 # [ 0.603471] Trying to unpack rootfs image as initramfs...1566client # [ 0.587971] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1567client # [ 0.589113] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1568client # [ 0.590821] ACPI: \_SB_.GSIA: Enabled at IRQ 161569client # [ 0.593554] ACPI: \_SB_.GSIB: Enabled at IRQ 171570client # [ 0.595452] ACPI: \_SB_.GSIC: Enabled at IRQ 181571client # [ 0.597375] ACPI: \_SB_.GSID: Enabled at IRQ 191572client # [ 0.599011] PCI: CLS 0 bytes, default 641573client # [ 0.599886] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574client # [ 0.601610] Trying to unpack rootfs image as initramfs...1575lb # [ 0.633723] Initialise system trusted keyrings1576lb # [ 0.636541] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.645037] Initialise system trusted keyrings1578worker2 # [ 0.648068] workingset: timestamp_bits=40 max_order=19 bucket_order=01579lb # [ 0.658483] Key type asymmetric registered1580lb # [ 0.659190] Asymmetric key parser 'x509' registered1581lb # [ 0.660033] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582worker1 # [ 0.649801] Initialise system trusted keyrings1583worker1 # [ 0.650699] workingset: timestamp_bits=40 max_order=19 bucket_order=01584lb # [ 0.665521] io scheduler mq-deadline registered1585lb # [ 0.666298] io scheduler kyber registered1586worker2 # [ 0.667908] Key type asymmetric registered1587lb # [ 0.669876] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1588lb # [ 0.671122] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1589client # [ 0.644206] Initialise system trusted keyrings1590worker2 # [ 0.671067] Asymmetric key parser 'x509' registered1591lb # [ 0.674185] Linux agpgart interface v0.1031592worker2 # [ 0.671944] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1593lb # [ 0.674949] ACPI: bus type drm_connector registered1594client # [ 0.647139] workingset: timestamp_bits=40 max_order=19 bucket_order=01595worker2 # [ 0.675157] io scheduler mq-deadline registered1596worker2 # [ 0.675936] io scheduler kyber registered1597lb # [ 0.679853] usbcore: registered new interface driver usbserial_generic1598lb # [ 0.680906] usbserial: USB Serial support registered for generic1599worker2 # [ 0.680182] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1600worker2 # [ 0.681519] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1601worker1 # [ 0.672757] Key type asymmetric registered1602lb # [ 0.684407] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1603lb # [ 0.684407] Please enable it if your BIOS has the CPPC option.1604worker1 # [ 0.673477] Asymmetric key parser 'x509' registered1605lb # [ 0.686525] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1606worker1 # [ 0.674357] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1607lb # [ 0.687862] drop_monitor: Initializing network drop monitor service1608worker2 # [ 0.686795] Linux agpgart interface v0.1031609lb # [ 0.688988] NET: Registered PF_INET6 protocol family1610worker2 # [ 0.687591] ACPI: bus type drm_connector registered1611worker1 # [ 0.679493] io scheduler mq-deadline registered1612worker1 # [ 0.680259] io scheduler kyber registered1613worker2 # [ 0.690470] usbcore: registered new interface driver usbserial_generic1614lb # [ 0.693694] Segment Routing with IPv61615worker2 # [ 0.691549] usbserial: USB Serial support registered for generic1616lb # [ 0.694351] In-situ OAM (IOAM) with IPv61617client # [ 0.666584] Key type asymmetric registered1618worker1 # [ 0.683957] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1619lb # [ 0.696682] IPI shorthand broadcast: enabled1620worker2 # [ 0.695034] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1621worker1 # [ 0.685249] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1622worker2 # [ 0.695034] Please enable it if your BIOS has the CPPC option.1623client # [ 0.671128] Asymmetric key parser 'x509' registered1624worker2 # [ 0.697280] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1625client # [ 0.671945] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1626worker1 # [ 0.690475] Linux agpgart interface v0.1031627worker2 # [ 0.701150] drop_monitor: Initializing network drop monitor service1628worker1 # [ 0.691259] ACPI: bus type drm_connector registered1629client # [ 0.675267] io scheduler mq-deadline registered1630client # [ 0.675993] io scheduler kyber registered1631worker2 # [ 0.702330] NET: Registered PF_INET6 protocol family1632lb # [ 0.704918] sched_clock: Marking stable (570014684, 134369698)->(775642232, -71257850)1633worker1 # [ 0.694515] usbcore: registered new interface driver usbserial_generic1634worker2 # [ 0.705470] Segment Routing with IPv61635worker1 # [ 0.695627] usbserial: USB Serial support registered for generic1636client # [ 0.679695] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1637lb # [ 0.709485] registered taskstats version 11638client # [ 0.680975] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1639worker2 # [ 0.708070] In-situ OAM (IOAM) with IPv61640lb # [ 0.710470] Loading compiled-in X.509 certificates1641worker2 # [ 0.709143] IPI shorthand broadcast: enabled1642worker1 # [ 0.698408] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1643worker1 # [ 0.698408] Please enable it if your BIOS has the CPPC option.1644client # [ 0.684861] Linux agpgart interface v0.1031645worker1 # [ 0.700645] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1646client # [ 0.685608] ACPI: bus type drm_connector registered1647worker1 # [ 0.704540] drop_monitor: Initializing network drop monitor service1648client # [ 0.688550] usbcore: registered new interface driver usbserial_generic1649worker1 # [ 0.705726] NET: Registered PF_INET6 protocol family1650client # [ 0.689589] usbserial: USB Serial support registered for generic1651worker2 # [ 0.717388] sched_clock: Marking stable (582020297, 134991845)->(791430462, -74418320)1652worker1 # [ 0.708875] Segment Routing with IPv61653client # [ 0.692117] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1654client # [ 0.692117] Please enable it if your BIOS has the CPPC option.1655client # [ 0.694329] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1656worker1 # [ 0.711412] In-situ OAM (IOAM) with IPv61657worker1 # [ 0.712493] IPI shorthand broadcast: enabled1658worker2 # [ 0.723226] registered taskstats version 11659worker2 # [ 0.724238] Loading compiled-in X.509 certificates1660lb # [ 0.726876] Demotion targets for Node 0: null1661client # [ 0.698268] drop_monitor: Initializing network drop monitor service1662client # [ 0.699471] NET: Registered PF_INET6 protocol family1663lb # [ 0.729442] Key type .fscrypt registered1664lb # [ 0.730116] Key type fscrypt-provisioning registered1665lb # [ 0.731035] ima: No TPM chip found, activating TPM-bypass!1666client # [ 0.704215] Segment Routing with IPv61667client # [ 0.704881] In-situ OAM (IOAM) with IPv61668lb # [ 0.733400] ima: Allocated hash algorithm: sha11669worker1 # [ 0.720697] sched_clock: Marking stable (588016974, 132372373)->(791766821, -71377474)1670lb # [ 0.734175] ima: No architecture policies found1671client # [ 0.705913] IPI shorthand broadcast: enabled1672worker1 # [ 0.725485] registered taskstats version 11673lb # [ 0.738399] PM: Magic number: 10:951:5251674worker1 # [ 0.726462] Loading compiled-in X.509 certificates1675lb # [ 0.739110] PM: hash matches drivers/base/power/main.c:8931676worker2 # [ 0.740715] Demotion targets for Node 0: null1677lb # [ 0.742879] RAS: Correctable Errors collector initialized.1678client # [ 0.714059] sched_clock: Marking stable (579014119, 134075529)->(789532613, -76442965)1679worker2 # [ 0.743068] Key type .fscrypt registered1680client # [ 0.717295] registered taskstats version 11681worker2 # [ 0.743764] Key type fscrypt-provisioning registered1682client # [ 0.718256] Loading compiled-in X.509 certificates1683worker2 # [ 0.744749] ima: No TPM chip found, activating TPM-bypass!1684worker2 # [ 0.749034] ima: Allocated hash algorithm: sha11685lb # [ 0.751451] clk: Disabling unused clocks1686worker2 # [ 0.749868] ima: No architecture policies found1687lb # [ 0.752139] PM: genpd: Disabling unused power domains1688worker2 # [ 0.752041] PM: Magic number: 10:951:5251689worker2 # [ 0.752770] PM: hash matches drivers/base/power/main.c:8931690worker1 # [ 0.744408] Demotion targets for Node 0: null1691worker1 # [ 0.745312] Key type .fscrypt registered1692worker1 # [ 0.746054] Key type fscrypt-provisioning registered1693worker2 # [ 0.756613] RAS: Correctable Errors collector initialized.1694worker1 # [ 0.747022] ima: No TPM chip found, activating TPM-bypass!1695worker1 # [ 0.750409] ima: Allocated hash algorithm: sha11696worker1 # [ 0.751245] ima: No architecture policies found1697client # [ 0.736109] Demotion targets for Node 0: null1698worker2 # [ 0.765277] clk: Disabling unused clocks1699client # [ 0.739145] Key type .fscrypt registered1700worker1 # [ 0.755406] PM: Magic number: 10:951:5251701worker2 # [ 0.765951] PM: genpd: Disabling unused power domains1702client # [ 0.739787] Key type fscrypt-provisioning registered1703worker1 # [ 0.756128] PM: hash matches drivers/base/power/main.c:8931704client # [ 0.740694] ima: No TPM chip found, activating TPM-bypass!1705client # [ 0.743108] ima: Allocated hash algorithm: sha11706client # [ 0.743888] ima: No architecture policies found1707worker1 # [ 0.759970] RAS: Correctable Errors collector initialized.1708client # [ 0.746104] PM: Magic number: 10:951:5251709client # [ 0.746801] PM: hash matches drivers/base/power/main.c:8931710client # [ 0.750637] RAS: Correctable Errors collector initialized.1711worker1 # [ 0.768631] clk: Disabling unused clocks1712worker1 # [ 0.771396] PM: genpd: Disabling unused power domains1713client # [ 0.759268] clk: Disabling unused clocks1714client # [ 0.759945] PM: genpd: Disabling unused power domains1715lb # [ 0.909862] Freeing initrd memory: 29092K1716lb # [ 0.913013] Freeing unused decrypted memory: 2028K1717lb # [ 0.915627] Freeing unused kernel image (initmem) memory: 3644K1718lb # [ 0.916658] Write protecting the kernel read-only data: 32768k1719lb # [ 0.918447] Freeing unused kernel image (text/rodata gap) memory: 1208K1720lb # [ 0.919890] Freeing unused kernel image (rodata/data gap) memory: 752K1721worker2 # [ 0.925279] Freeing initrd memory: 29092K1722worker2 # [ 0.928414] Freeing unused decrypted memory: 2028K1723worker2 # [ 0.931076] Freeing unused kernel image (initmem) memory: 3644K1724worker2 # [ 0.932176] Write protecting the kernel read-only data: 32768k1725worker2 # [ 0.934066] Freeing unused kernel image (text/rodata gap) memory: 1208K1726worker1 # [ 0.924982] Freeing initrd memory: 29092K1727worker2 # [ 0.935585] Freeing unused kernel image (rodata/data gap) memory: 752K1728worker1 # [ 0.928089] Freeing unused decrypted memory: 2028K1729worker1 # [ 0.930705] Freeing unused kernel image (initmem) memory: 3644K1730worker1 # [ 0.931806] Write protecting the kernel read-only data: 32768k1731worker1 # [ 0.933670] Freeing unused kernel image (text/rodata gap) memory: 1208K1732worker1 # [ 0.935200] Freeing unused kernel image (rodata/data gap) memory: 752K1733client # [ 0.924975] Freeing initrd memory: 29096K1734client # [ 0.928161] Freeing unused decrypted memory: 2028K1735client # [ 0.930781] Freeing unused kernel image (initmem) memory: 3644K1736client # [ 0.931869] Write protecting the kernel read-only data: 32768k1737client # [ 0.933710] Freeing unused kernel image (text/rodata gap) memory: 1208K1738client # [ 0.935193] Freeing unused kernel image (rodata/data gap) memory: 752K1739lb # [ 0.971011] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740lb # [ 0.972063] Run /init as init process1741lb # [ 0.982086] systemd[1]: Inserted module 'autofs4'1742worker2 # [ 0.986576] x86/mm: Checked W+X mappings: passed, no W+X pages found.1743worker2 # [ 0.987714] Run /init as init process1744lb # [ 0.998418] fuse: init (API version 7.45)1745worker1 # [ 0.986500] x86/mm: Checked W+X mappings: passed, no W+X pages found.1746worker1 # [ 0.987602] Run /init as init process1747worker2 # [ 0.997767] systemd[1]: Inserted module 'autofs4'1748lb # [ 1.004195] ACPI: \_SB_.GSIG: Enabled at IRQ 221749lb # [ 1.006545] ACPI: \_SB_.GSIH: Enabled at IRQ 231750lb # [ 1.009430] ACPI: \_SB_.GSIE: Enabled at IRQ 201751worker1 # [ 0.997504] systemd[1]: Inserted module 'autofs4'1752lb # [ 1.011522] ACPI: \_SB_.GSIF: Enabled at IRQ 211753client # [ 0.986334] x86/mm: Checked W+X mappings: passed, no W+X pages found.1754client # [ 0.987424] Run /init as init process1755worker2 # [ 1.014208] fuse: init (API version 7.45)1756worker2 # [ 1.020143] ACPI: \_SB_.GSIG: Enabled at IRQ 221757worker2 # [ 1.022478] ACPI: \_SB_.GSIH: Enabled at IRQ 231758worker1 # [ 1.013978] fuse: init (API version 7.45)1759client # [ 0.997761] systemd[1]: Inserted module 'autofs4'1760worker2 # [ 1.025601] ACPI: \_SB_.GSIE: Enabled at IRQ 201761worker2 # [ 1.027880] ACPI: \_SB_.GSIF: Enabled at IRQ 211762worker1 # [ 1.019845] ACPI: \_SB_.GSIG: Enabled at IRQ 221763worker1 # [ 1.022271] ACPI: \_SB_.GSIH: Enabled at IRQ 231764worker1 # [ 1.025988] ACPI: \_SB_.GSIE: Enabled at IRQ 201765lb # [ 1.039585] systemd[1]: Successfully made /usr/ read-only.1766worker1 # [ 1.028716] ACPI: \_SB_.GSIF: Enabled at IRQ 211767client # [ 1.014068] fuse: init (API version 7.45)1768client # [ 1.019858] ACPI: \_SB_.GSIG: Enabled at IRQ 221769client # [ 1.022432] ACPI: \_SB_.GSIH: Enabled at IRQ 231770client # [ 1.025590] ACPI: \_SB_.GSIE: Enabled at IRQ 201771client # [ 1.027977] ACPI: \_SB_.GSIF: Enabled at IRQ 211772worker2 # [ 1.058153] systemd[1]: Successfully made /usr/ read-only.1773worker1 # [ 1.060517] systemd[1]: Successfully made /usr/ read-only.1774client # [ 1.055918] systemd[1]: Successfully made /usr/ read-only.1775lb # [ 1.376285] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776lb # [ 1.388252] systemd[1]: Detected virtualization kvm.1777lb # [ 1.390438] systemd[1]: Detected architecture x86-64.1778lb # [ 1.392613] systemd[1]: Running in initrd.1779lb # [ 1.395155] systemd[1]: Initializing machine ID from random generator.1780lb # [ 1.398020] systemd[1]: Hostname set to <lb>.1781worker2 # [ 1.394956] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1782worker2 # [ 1.407048] systemd[1]: Detected virtualization kvm.1783worker2 # [ 1.409171] systemd[1]: Detected architecture x86-64.1784worker1 # [ 1.396559] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1785worker1 # [ 1.401753] systemd[1]: Detected virtualization kvm.1786worker2 # [ 1.411346] systemd[1]: Running in initrd.1787worker1 # [ 1.402634] systemd[1]: Detected architecture x86-64.1788worker1 # [ 1.403546] systemd[1]: Running in initrd.1789worker1 # [ 1.404801] systemd[1]: Initializing machine ID from random generator.1790worker2 # [ 1.413890] systemd[1]: Initializing machine ID from random generator.1791worker1 # [ 1.406051] systemd[1]: Hostname set to <worker1>.1792worker2 # [ 1.416831] systemd[1]: Hostname set to <worker2>.1793client # [ 1.392495] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1794client # [ 1.404767] systemd[1]: Detected virtualization kvm.1795client # [ 1.406953] systemd[1]: Detected architecture x86-64.1796client # [ 1.409219] systemd[1]: Running in initrd.1797client # [ 1.411812] systemd[1]: Initializing machine ID from random generator.1798client # [ 1.414747] systemd[1]: Hostname set to <client>.1799worker2 # [ 1.611103] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker1 # [ 1.601761] systemd[1]: bpf-restrict-fs: LSM BPF program attached1801lb # [ 1.624770] systemd[1]: bpf-restrict-fs: LSM BPF program attached1802worker2 # [ 1.651281] systemd[1]: Queued start job for default target Initrd Default Target.1803worker2 # [ 1.655304] systemd[1]: Created slice Slice /system/modprobe.1804worker2 # [ 1.656536] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1805worker2 # [ 1.657979] systemd[1]: Expecting device /dev/disk/by-label/nixos...1806worker2 # [ 1.659171] systemd[1]: Reached target Path Units.1807worker1 # [ 1.648685] systemd[1]: Queued start job for default target Initrd Default Target.1808worker2 # [ 1.660069] systemd[1]: Reached target Slice Units.1809worker2 # [ 1.660967] systemd[1]: Reached target Swaps.1810worker2 # [ 1.661835] systemd[1]: Reached target Timer Units.1811worker2 # [ 1.662849] systemd[1]: Listening on D-Bus System Message Bus Socket.1812worker1 # [ 1.652832] systemd[1]: Created slice Slice /system/modprobe.1813worker2 # [ 1.664102] systemd[1]: Listening on Journal Socket (/dev/log).1814worker1 # [ 1.654011] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1815worker2 # [ 1.665283] systemd[1]: Listening on Journal Sockets.1816worker1 # [ 1.655493] systemd[1]: Expecting device /dev/disk/by-label/nixos...1817worker2 # [ 1.666313] systemd[1]: Listening on udev Control Socket.1818worker1 # [ 1.656645] systemd[1]: Reached target Path Units.1819worker2 # [ 1.667401] systemd[1]: Listening on udev Kernel Socket.1820worker1 # [ 1.657505] systemd[1]: Reached target Slice Units.1821worker2 # [ 1.668349] systemd[1]: Reached target Socket Units.1822client # [ 1.640294] systemd[1]: bpf-restrict-fs: LSM BPF program attached1823worker1 # [ 1.658411] systemd[1]: Reached target Swaps.1824worker1 # [ 1.659195] systemd[1]: Reached target Timer Units.1825worker2 # [ 1.670080] systemd[1]: Starting Create List of Static Device Nodes...1826worker1 # [ 1.660227] systemd[1]: Listening on D-Bus System Message Bus Socket.1827worker1 # [ 1.661489] systemd[1]: Listening on Journal Socket (/dev/log).1828worker1 # [ 1.662685] systemd[1]: Listening on Journal Sockets.1829worker1 # [ 1.663702] systemd[1]: Listening on udev Control Socket.1830worker2 # [ 1.673821] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1831worker1 # [ 1.664704] systemd[1]: Listening on udev Kernel Socket.1832worker1 # [ 1.665672] systemd[1]: Reached target Socket Units.1833worker1 # [ 1.667658] systemd[1]: Starting Create List of Static Device Nodes...1834lb # [ 1.680613] systemd[1]: Queued start job for default target Initrd Default Target.1835worker2 # [ 1.680581] systemd[1]: Starting Load Kernel Module configfs...1836worker1 # [ 1.671460] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1837lb # [ 1.684836] systemd[1]: Created slice Slice /system/modprobe.1838lb # [ 1.686057] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1839lb # [ 1.687508] systemd[1]: Expecting device /dev/disk/by-label/nixos...1840lb # [ 1.688676] systemd[1]: Reached target Path Units.1841lb # [ 1.689569] systemd[1]: Reached target Slice Units.1842lb # [ 1.690545] systemd[1]: Reached target Swaps.1843lb # [ 1.691501] systemd[1]: Reached target Timer Units.1844worker1 # [ 1.679623] systemd[1]: Starting Load Kernel Module configfs...1845lb # [ 1.692512] systemd[1]: Listening on D-Bus System Message Bus Socket.1846lb # [ 1.693754] systemd[1]: Listening on Journal Socket (/dev/log).1847lb # [ 1.694975] systemd[1]: Listening on Journal Sockets.1848lb # [ 1.696041] systemd[1]: Listening on udev Control Socket.1849worker2 # [ 1.694197] systemd[1]: Starting Journal Service...1850lb # [ 1.697150] systemd[1]: Listening on udev Kernel Socket.1851lb # [ 1.698151] systemd[1]: Reached target Socket Units.1852lb # [ 1.700111] systemd[1]: Starting Create List of Static Device Nodes...1853lb # [ 1.704243] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1854worker1 # [ 1.696638] systemd[1]: Starting Journal Service...1855lb # [ 1.710421] systemd[1]: Starting Load Kernel Module configfs...1856worker2 # [ 1.708126] systemd[1]: Starting Load Kernel Modules...1857client # [ 1.695284] systemd[1]: Queued start job for default target Initrd Default Target.1858lb # [ 1.724474] systemd[1]: Starting Journal Service...1859client # [ 1.699409] systemd[1]: Created slice Slice /system/modprobe.1860worker2 # [ 1.727373] netfs: FS-Cache loaded1861client # [ 1.700586] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1862client # [ 1.701940] systemd[1]: Expecting device /dev/disk/by-label/nixos...1863client # [ 1.702988] systemd[1]: Reached target Path Units.1864client # [ 1.703839] systemd[1]: Reached target Slice Units.1865client # [ 1.704689] systemd[1]: Reached target Swaps.1866worker1 # [ 1.709487] systemd[1]: Starting Load Kernel Modules...1867client # [ 1.705475] systemd[1]: Reached target Timer Units.1868worker2 # [ 1.731982] 9pnet: Installing 9P2000 support1869client # [ 1.706430] systemd[1]: Listening on D-Bus System Message Bus Socket.1870client # [ 1.707614] systemd[1]: Listening on Journal Socket (/dev/log).1871client # [ 1.708736] systemd[1]: Listening on Journal Sockets.1872client # [ 1.709721] systemd[1]: Listening on udev Control Socket.1873client # [ 1.710714] systemd[1]: Listening on udev Kernel Socket.1874client # [ 1.711690] systemd[1]: Reached target Socket Units.1875worker1 # [ 1.728745] netfs: FS-Cache loaded1876worker2 # [ 1.739116] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1877client # [ 1.713435] systemd[1]: Starting Create List of Static Device Nodes...1878worker1 # [ 1.733570] 9pnet: Installing 9P2000 support1879client # [ 1.717159] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1880lb # [ 1.738474] systemd[1]: Starting Load Kernel Modules...1881worker2 # [ 1.748346] systemd[1]: Starting Coldplug All udev Devices...1882client # [ 1.723791] systemd[1]: Starting Load Kernel Module configfs...1883worker1 # [ 1.744538] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1884lb # [ 1.760624] netfs: FS-Cache loaded1885worker2 # [ 1.761328] systemd[1]: Finished Create List of Static Device Nodes.1886client # [ 1.737167] systemd[1]: Starting Journal Service...1887worker1 # [ 1.753468] systemd[1]: Starting Coldplug All udev Devices...1888lb # [ 1.766858] 9pnet: Installing 9P2000 support1889worker2 # [ 1.766611] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1890lb # [ 1.768527] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1891worker2 # [ 1.772568] systemd-journald[67]: Collecting audit messages is disabled.1892worker2 # [ 1.776375] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1893worker1 # [ 1.766534] systemd[1]: Finished Create List of Static Device Nodes.1894lb # [ 1.779607] systemd[1]: Starting Coldplug All udev Devices...1895worker2 # [ 1.781426] systemd[1]: modprobe@configfs.service: Deactivated successfully.1896worker1 # [ 1.774013] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897worker2 # [ 1.784453] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1898client # [ 1.750176] systemd[1]: Starting Load Kernel Modules...1899worker2 # [ 1.788377] systemd[1]: Finished Load Kernel Module configfs.1900worker1 # [ 1.778267] systemd-journald[67]: Collecting audit messages is disabled.1901worker1 # [ 1.781773] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1902worker2 # [ 1.792434] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1903lb # [ 1.797457] systemd[1]: Finished Create List of Static Device Nodes.1904client # [ 1.769695] netfs: FS-Cache loaded1905worker2 # [ 1.797149] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1906worker1 # [ 1.787814] systemd[1]: modprobe@configfs.service: Deactivated successfully.1907lb # [ 1.803068] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1908client # [ 1.775393] 9pnet: Installing 9P2000 support1909worker1 # [ 1.792785] systemd[1]: Finished Load Kernel Module configfs.1910worker2 # [ 1.805326] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1911worker1 # [ 1.795303] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1912client # [ 1.781111] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1913lb # [ 1.810238] systemd-journald[67]: Collecting audit messages is disabled.1914lb # [ 1.812791] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1915worker1 # [ 1.799887] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1916lb # [ 1.814800] systemd[1]: modprobe@configfs.service: Deactivated successfully.1917worker1 # [ 1.806558] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1918client # [ 1.791184] systemd[1]: Starting Coldplug All udev Devices...1919lb # [ 1.822437] systemd[1]: Finished Load Kernel Module configfs.1920worker1 # [ 1.811470] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1921lb # [ 1.825263] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1922worker2 # [ 1.826511] systemd[1]: Finished Load Kernel Modules.1923lb # [ 1.829918] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1924client # [ 1.804643] systemd[1]: Finished Create List of Static Device Nodes.1925lb # [ 1.835585] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1926worker2 # [ 1.835222] systemd[1]: Starting Apply Kernel Variables...1927client # [ 1.810718] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1928lb # [ 1.841707] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1929client # [ 1.817821] systemd-journald[67]: Collecting audit messages is disabled.1930client # [ 1.819425] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1931worker1 # [ 1.836917] systemd[1]: Finished Load Kernel Modules.1932worker2 # [ 1.848106] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1933client # [ 1.824507] systemd[1]: modprobe@configfs.service: Deactivated successfully.1934worker1 # [ 1.845594] systemd[1]: Starting Apply Kernel Variables...1935worker2 # [ 1.857211] systemd[1]: Starting Create Static Device Nodes in /dev...1936client # [ 1.830987] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1937client # [ 1.834458] systemd[1]: Finished Load Kernel Module configfs.1938lb # [ 1.866111] systemd[1]: Finished Load Kernel Modules.1939client # [ 1.838553] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1940worker1 # [ 1.855530] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1941client # [ 1.844227] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1942lb # [ 1.874610] systemd[1]: Starting Apply Kernel Variables...1943worker2 # [ 1.873168] systemd[1]: Finished Apply Kernel Variables.1944worker1 # [ 1.865047] systemd[1]: Starting Create Static Device Nodes in /dev...1945client # [ 1.849218] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1946lb # [ 1.891131] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1947worker2 # [ 1.889538] systemd[1]: Finished Create Static Device Nodes in /dev.1948worker2 # [ 1.756165] systemd-modules-load[68]: Inserted module 'dm_mod'1949worker2 # [ 1.758275] systemd-modules-load[68]: Inserted module 'virtio_balloon'1950worker1 # [ 1.883710] systemd[1]: Finished Apply Kernel Variables.1951worker2 # [ 1.760323] systemd-modules-load[68]: Inserted module 'virtio_gpu'1952worker2 # [ 1.896313] systemd[1]: Started Journal Service.1953lb # [ 1.902590] systemd[1]: Starting Create Static Device Nodes in /dev...1954client # [ 1.876622] systemd[1]: Finished Load Kernel Modules.1955worker2 # [ 1.771126] systemd[1]: Reached target Preparation for Local File Systems.1956worker1 # [ 1.896000] systemd[1]: Started Journal Service.1957worker2 # [ 1.772151] systemd[1]: Reached target Local File Systems.1958worker2 # [ 1.775070] systemd[1]: Starting Create System Files and Directories...1959worker1 # [ 1.765209] systemd-modules-load[68]: Inserted module 'dm_mod'1960client # [ 1.884286] systemd[1]: Starting Apply Kernel Variables...1961worker1 # [ 1.769210] systemd-modules-load[68]: Inserted module 'virtio_balloon'1962lb # [ 1.914036] systemd[1]: Finished Apply Kernel Variables.1963worker1 # [ 1.770335] systemd-modules-load[68]: Inserted module 'virtio_gpu'1964worker1 # [ 1.772243] systemd[1]: Finished Create Static Device Nodes in /dev.1965worker2 # [ 1.780084] systemd[1]: Starting Rule-based Manager for Device Events and Files...1966worker1 # [ 1.773329] systemd[1]: Reached target Preparation for Local File Systems.1967worker1 # [ 1.775324] systemd[1]: Reached target Local File Systems.1968worker1 # [ 1.778261] systemd[1]: Starting Create System Files and Directories...1969worker1 # [ 1.780355] systemd[1]: Starting Rule-based Manager for Device Events and Files...1970client # [ 1.897531] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1971lb # [ 1.931009] systemd[1]: Started Journal Service.1972lb # [ 1.797738] systemd-modules-load[68]: Inserted module 'dm_mod'1973lb # [ 1.801083] systemd-modules-load[68]: Inserted module 'virtio_balloon'1974lb # [ 1.802147] systemd-modules-load[68]: Inserted module 'virtio_gpu'1975client # [ 1.908548] systemd[1]: Starting Create Static Device Nodes in /dev...1976worker2 # [ 1.803550] systemd[1]: Finished Create System Files and Directories.1977lb # [ 1.809083] systemd[1]: Finished Create Static Device Nodes in /dev.1978lb # [ 1.815849] systemd[1]: Reached target Preparation for Local File Systems.1979lb # [ 1.816871] systemd[1]: Reached target Local File Systems.1980lb # [ 1.818254] systemd[1]: Starting Create System Files and Directories...1981worker1 # [ 1.808590] systemd[1]: Finished Create System Files and Directories.1982client # [ 1.926227] systemd[1]: Finished Apply Kernel Variables.1983lb # [ 1.827078] systemd[1]: Starting Rule-based Manager for Device Events and Files...1984worker2 # [ 1.826917] systemd-udevd[83]: Using default interface naming scheme 'v261'.1985client # [ 1.942432] systemd[1]: Started Journal Service.1986client # [ 1.811224] systemd-modules-load[68]: Inserted module 'dm_mod'1987client # [ 1.812920] systemd-modules-load[68]: Inserted module 'virtio_balloon'1988client # [ 1.816543] systemd-modules-load[68]: Inserted module 'virtio_gpu'1989worker1 # [ 1.834849] systemd-udevd[76]: Using default interface naming scheme 'v261'.1990lb # [ 1.850941] systemd[1]: Finished Create System Files and Directories.1991client # [ 1.823068] systemd[1]: Finished Create Static Device Nodes in /dev.1992worker2 # [ 1.852338] systemd[1]: Started Rule-based Manager for Device Events and Files.1993client # [ 1.827920] systemd[1]: Reached target Preparation for Local File Systems.1994client # [ 1.830898] systemd[1]: Reached target Local File Systems.1995client # [ 1.834069] systemd[1]: Starting Create System Files and Directories...1996client # [ 1.839403] systemd[1]: Starting Rule-based Manager for Device Events and Files...1997worker1 # [ 1.859288] systemd[1]: Started Rule-based Manager for Device Events and Files.1998lb # [ 1.875225] systemd-udevd[83]: Using default interface naming scheme 'v261'.1999client # [ 1.865303] systemd[1]: Finished Create System Files and Directories.2000worker2 # [ 1.898391] systemd[1]: Finished Coldplug All udev Devices.
loading 1,925 hidden lines…
3926client # [ 12.786105] dhcpcd[569]: eth0: adding address fec0::5054:ff:fe12:3456/643927client # [ 12.789328] dhcpcd[569]: eth0: adding route to fec0::/643928client # [ 12.791236] dhcpcd[569]: eth0: adding default route via fe80::23929client # [ 12.794312] dhcpcd[569]: eth0: probing address 10.0.2.15/243930lb # [ 12.940241] postgresql-pre-start[729]: running bootstrap script ... ok3931lb # [ 12.941234] dhcpcd[710]: eth0: soliciting an IPv6 router3932lb # [ 12.942422] dhcpcd[710]: eth0: Router Advertisement from fe80::23933lb # [ 12.943398] dhcpcd[710]: eth0: adding address fec0::5054:ff:fe12:3456/643934lb # [ 12.944428] dhcpcd[710]: eth0: adding route to fec0::/643935lb # [ 12.945301] dhcpcd[710]: eth0: adding default route via fe80::23936lb # [ 13.469635] postgresql-pre-start[729]: performing post-bootstrap initialization ... ok3937lb # [ 13.582804] postgresql-pre-start[729]: syncing data to disk ... ok3938lb # [ 13.583850] postgresql-pre-start[729]: initdb: warning: enabling "trust" authentication for local connections3939lb # [ 13.585126] postgresql-pre-start[729]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.3940lb # [ 13.587760] postgresql-pre-start[729]: Success. You can now start the database server using:3941lb # [ 13.588855] postgresql-pre-start[729]: pg_ctl -D /var/lib/postgresql/18 -l logfile start3942lb # [ 13.707960] postgres[791]: [791] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3943lb # [ 13.713498] postgres[791]: [791] LOG: listening on IPv6 address "::1", port 54323944lb # [ 13.714507] postgres[791]: [791] LOG: listening on IPv4 address "127.0.0.1", port 54323945lb # [ 13.717061] postgres[791]: [791] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3946lb # [ 13.730352] postgres[800]: [800] LOG: database system was shut down at 2026-09-20 09:32:52 GMT3947lb # [ 13.735466] postgres[791]: [791] LOG: database system is ready to accept connections3948lb # [ 13.739860] systemd[1]: Started PostgreSQL Server.3949lb # [ 13.746840] systemd[1]: Starting PostgreSQL Setup Scripts...3950lb # [ 13.939702] postgresql-setup-start[811]: CREATE DATABASE3951lb # [ 13.995350] postgresql-setup-start[822]: CREATE ROLE3952lb # [ 14.018466] postgresql-setup-start[826]: ALTER DATABASE3953lb # [ 14.023740] systemd[1]: Finished PostgreSQL Setup Scripts.3954lb # [ 14.025461] systemd[1]: Reached target PostgreSQL.3955worker1 # [ 15.060580] dhcpcd[693]: eth0: leased 10.0.2.15 for 86400 seconds3956worker1 # [ 15.061741] dhcpcd[693]: eth0: adding route to 10.0.2.0/243957worker1 # [ 15.062690] dhcpcd[693]: eth0: adding default route via 10.0.2.23958worker1 # [ 15.157639] systemd[1]: Started DHCP Client.3959worker1 # [ 15.159639] systemd[1]: Reached target Multi-User System.3960worker1 # [ 15.160554] systemd[1]: Startup finished in 858ms (kernel) + 4.116s (initrd) + 10.184s (userspace) = 15.159s.3961worker2 # [ 16.481917] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds3962worker2 # [ 16.484992] dhcpcd[692]: eth0: adding route to 10.0.2.0/243963worker2 # [ 16.486893] dhcpcd[692]: eth0: adding default route via 10.0.2.23964worker2 # [ 16.575640] systemd[1]: Started DHCP Client.3965worker2 # [ 16.577509] systemd[1]: Reached target Multi-User System.3966worker2 # [ 16.578564] systemd[1]: Startup finished in 856ms (kernel) + 4.149s (initrd) + 11.572s (userspace) = 16.578s.3967lb # [ 16.796132] dhcpcd[710]: eth0: leased 10.0.2.15 for 86400 seconds3968lb # [ 16.798880] dhcpcd[710]: eth0: adding route to 10.0.2.0/243969lb # [ 16.801363] dhcpcd[710]: eth0: adding default route via 10.0.2.23970lb # [ 16.932808] systemd[1]: Started DHCP Client.3971lb # [ 16.935188] systemd[1]: Reached target Network is Online.3972lb # [ 16.939174] systemd[1]: Started Envoy reverse proxy.3973lb # [ 16.942627] systemd[1]: Started mock-oidc.service.3974lb # [ 17.040336] mock-oidc-server[909]: Mock OIDC Server running3975lb # [ 17.041222] mock-oidc-server[909]: OIDC Address: 192.168.1.2:80803976lb # [ 17.042412] mock-oidc-server[909]: Issue Address: 0.0.0.0:80813977lb # [ 17.043362] mock-oidc-server[909]: Issuer: http://192.168.1.2:8080/oidc3978lb # [ 17.044390] mock-oidc-server[909]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3979lb # [ 17.045711] mock-oidc-server[909]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3980lb # [ 17.047091] mock-oidc-server[909]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3981lb # [ 17.518809] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3982lb # [ 17.520684] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:469] statically linked extensions:3983lb # [ 17.522456] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3984lb # [ 17.525089] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3985lb # [ 17.527786] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3986lb # [ 17.530412] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3987lb # [ 17.533596] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3988lb # [ 17.535114] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3989lb # [ 17.536719] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23990lb # [ 17.538390] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3991lb # [ 17.540081] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3992lb # [ 17.542493] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3993lb # [ 17.544988] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3994lb # [ 17.546866] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3995lb # [ 17.548606] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3996lb # [ 17.567218] envoy[908]: [2026-09-20 09:32:56.837][908][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3997lb # [ 17.568889] envoy[908]: [2026-09-20 09:32:56.838][908][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default3998lb # [ 17.570551] envoy[908]: [2026-09-20 09:32:56.838][908][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3999lb # [ 17.572242] envoy[908]: [2026-09-20 09:32:56.838][908][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4000lb # [ 17.578144] envoy[908]: [2026-09-20 09:32:56.838][908][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4001lb # [ 17.579839] envoy[908]: [2026-09-20 09:32:56.838][908][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4002lb # [ 17.582171] envoy[908]: [2026-09-20 09:32:56.838][908][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4003lb # [ 17.584302] envoy[908]: [2026-09-20 09:32:56.838][908][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4004lb # [ 17.586550] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4005lb # [ 17.588477] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4006lb # [ 17.590447] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4007lb # [ 17.595126] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4008lb # [ 17.596848] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4009lb # [ 17.598578] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4010lb # [ 17.601035] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4011lb # [ 17.602671] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4012lb # [ 17.604312] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4013lb # [ 17.608103] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4014lb # [ 17.609870] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4015lb # [ 17.612118] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4016lb # [ 17.614278] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4017lb # [ 17.616386] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4018lb # [ 17.618482] envoy[908]: [2026-09-20 09:32:56.841][908][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4019lb # [ 17.620154] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4020lb # [ 17.622990] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4021lb # [ 17.626970] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4022lb # [ 17.630255] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4023lb # [ 17.632789] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4024lb # [ 17.635058] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4025lb # [ 17.638193] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4026lb # [ 17.640036] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4027lb # [ 17.641871] envoy[908]: [2026-09-20 09:32:56.844][908][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4028lb # [ 17.648363] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4029lb # [ 17.650756] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4030lb # [ 17.657771] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4031lb # [ 17.660370] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4032lb # [ 17.664963] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4033lb # [ 17.667039] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4034lb # [ 17.669483] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4035lb # [ 17.674336] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4036lb # [ 17.676265] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4037lb # [ 17.677990] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4038lb # [ 17.680036] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4039lb # [ 17.681928] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4040lb # [ 17.685072] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4041lb # [ 17.687933] envoy[908]: [2026-09-20 09:32:56.846][908][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4042lb # [ 17.689847] envoy[908]: [2026-09-20 09:32:56.847][908][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4043lb # [ 17.695691] envoy[908]: [2026-09-20 09:32:56.847][908][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4044lb # [ 17.697264] envoy[908]: [2026-09-20 09:32:56.847][908][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4045lb # [ 17.701230] envoy[908]: [2026-09-20 09:32:56.847][908][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4046lb # [ 17.703326] envoy[908]: [2026-09-20 09:32:56.847][908][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4047lb # [ 17.706716] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4048lb # [ 17.712687] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4049lb # [ 17.714849] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4050lb # [ 17.716607] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4051lb # [ 17.720774] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4052lb # [ 17.722367] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4053lb # [ 17.724589] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4054lb # [ 17.726429] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4055lb # [ 17.728057] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4056lb # [ 17.729942] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4057lb # [ 17.732335] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4058lb # [ 17.737784] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84059lb # [ 17.739625] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4060lb # [ 17.742943] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4061lb # [ 17.768904] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4062lb # [ 17.771293] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4063lb # [ 17.773821] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4064lb # [ 17.784920] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24065lb # [ 17.786594] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4066lb # [ 17.788255] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4067lb # [ 17.790285] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4068lb # [ 17.792655] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4069lb # [ 17.794942] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14070lb # [ 17.797067] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4071lb # [ 17.798806] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4072lb # [ 17.800704] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4073lb # [ 17.803249] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4074lb # [ 17.806615] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4075lb # [ 17.811132] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24076lb # [ 17.813319] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4077lb # [ 17.815520] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4078lb # [ 17.817460] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4079lb # [ 17.819311] envoy[908]: [2026-09-20 09:32:56.852][908][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4080lb # [ 17.821656] envoy[908]: [2026-09-20 09:32:56.857][908][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4081lb # [ 17.824266] envoy[908]: [2026-09-20 09:32:56.857][908][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4082lb # [ 17.828470] envoy[908]: [2026-09-20 09:32:56.857][908][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4083lb # [ 17.831329] envoy[908]: [2026-09-20 09:32:56.857][908][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4084lb # [ 17.837116] envoy[908]: [2026-09-20 09:32:56.857][908][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4085lb # [ 17.838746] envoy[908]: [2026-09-20 09:32:56.857][908][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4086lb # [ 17.840719] envoy[908]: [2026-09-20 09:32:56.859][908][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4087lb # [ 17.842116] envoy[908]: [2026-09-20 09:32:56.959][908][info][main] [source/server/server.cc:531] HTTP header map info:4088lb # [ 17.843369] envoy[908]: [2026-09-20 09:32:57.004][908][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4089lb # [ 17.856187] envoy[908]: [2026-09-20 09:32:57.004][908][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4090lb # [ 17.857919] envoy[908]: [2026-09-20 09:32:57.007][908][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4091lb # [ 17.865411] envoy[908]: [2026-09-20 09:32:57.007][908][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4092lb # [ 17.897859] envoy[908]: [2026-09-20 09:32:57.217][908][info][main] [source/server/server.cc:968] runtime: {}4093lb # [ 17.904328] envoy[908]: [2026-09-20 09:32:57.223][908][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4094lb # [ 17.906640] envoy[908]: [2026-09-20 09:32:57.225][908][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014095lb # [ 17.909381] envoy[908]: [2026-09-20 09:32:57.228][908][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014096lb # [ 17.912354] envoy[908]: [2026-09-20 09:32:57.231][908][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4097lb # [ 17.914167] envoy[908]: [2026-09-20 09:32:57.233][908][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4098lb # [ 17.915706] envoy[908]: [2026-09-20 09:32:57.233][908][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4099worker1 # [ 17.947433] systemd[1]: Starting Nix worker-protocol over gRPC...4100lb # [ 17.959027] envoy[908]: [2026-09-20 09:32:57.278][908][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4101worker2 # [ 17.964202] systemd[1]: Starting Nix worker-protocol over gRPC...4102lb # [ 17.980842] envoy[908]: [2026-09-20 09:32:57.300][908][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4103lb # [ 17.983304] envoy[908]: [2026-09-20 09:32:57.302][908][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4104lb # [ 17.986543] envoy[908]: [2026-09-20 09:32:57.305][908][info][main] [source/server/server.cc:1077] starting main dispatch loop4105worker1 # [ 18.146754] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4106worker2 # [ 18.160906] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4107worker1 # [ 18.160864] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14108worker2 # [ 18.174375] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14109worker1 # [ 18.200392] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4110worker1 # [ 18.203798] systemd[1]: Started Nix worker-protocol over gRPC.4111worker2 # [ 18.211393] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4112worker2 # [ 18.215825] systemd[1]: Started Nix worker-protocol over gRPC.4113client # [ 18.198469] dhcpcd[569]: eth0: leased 10.0.2.15 for 86400 seconds4114client # [ 18.201183] dhcpcd[569]: eth0: adding route to 10.0.2.0/244115client # [ 18.203647] dhcpcd[569]: eth0: adding default route via 10.0.2.24116worker1 # [ 18.262739] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4117worker2 # [ 18.274968] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4118lb # [ 18.276505] envoy[908]: [2026-09-20 09:32:57.595][908][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4119lb # [ 18.281458] envoy[908]: [2026-09-20 09:32:57.595][908][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4120lb # [ 18.285419] envoy[908]: [2026-09-20 09:32:57.596][908][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4121lb # [ 18.290179] envoy[908]: [2026-09-20 09:32:57.607][908][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4122client # [ 18.320511] systemd[1]: Started DHCP Client.4123client # [ 18.322994] systemd[1]: Reached target Multi-User System.4124client # [ 18.324079] systemd[1]: Startup finished in 856ms (kernel) + 4.956s (initrd) + 12.510s (userspace) = 18.323s.4125worker1 # [ 18.463615] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4126worker2 # [ 18.476197] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4127worker1 # [ 18.864071] nix-grpc-daemon[784]: ts=2026-09-20T09:32:59Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4128worker2 # [ 18.876390] nix-grpc-daemon[784]: ts=2026-09-20T09:32:58Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4129worker1 # [ 19.666595] nix-grpc-daemon[784]: ts=2026-09-20T09:32:59Z level=info event=scheduler_connected addr=lb:500514130worker2 # [ 19.679731] nix-grpc-daemon[784]: ts=2026-09-20T09:32:59Z level=info event=scheduler_connected addr=lb:500514131worker1 # [ 19.680578] nix-grpc-daemon[784]: ts=2026-09-20T09:32:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4132worker2 # [ 19.688591] nix-grpc-daemon[784]: ts=2026-09-20T09:32:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker1 # [ 19.881282] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514134worker1 # [ 19.883222] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker2 # [ 19.889712] nix-grpc-daemon[784]: ts=2026-09-20T09:32:59Z level=info event=scheduler_connected addr=lb:500514136worker2 # [ 19.893680] nix-grpc-daemon[784]: ts=2026-09-20T09:32:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker1 # [ 20.084292] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514138worker2 # [ 20.095861] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514139worker1 # [ 20.088358] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker2 # [ 20.099165] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4141worker1 # [ 20.288875] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514142worker2 # [ 20.297091] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514143worker1 # [ 20.292959] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker2 # [ 20.301186] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 20.498902] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514146worker1 # [ 20.493514] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514147worker1 # [ 20.495504] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4148worker2 # [ 20.502845] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker1 # [ 20.696355] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514150worker1 # [ 20.698172] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker2 # [ 20.704429] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_connected addr=lb:500514152worker2 # [ 20.708354] nix-grpc-daemon[784]: ts=2026-09-20T09:33:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker1 # [ 20.899170] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 20.909529] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514155worker1 # [ 20.903229] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4156worker2 # [ 20.913227] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 21.111233] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514158worker1 # [ 21.103979] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514159worker2 # [ 21.115229] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4160worker1 # [ 21.108118] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 21.308736] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 21.315935] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514163worker2 # [ 21.320118] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164worker1 # [ 21.312646] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 21.520719] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514166worker1 # [ 21.513485] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514167worker2 # [ 21.524939] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4168worker1 # [ 21.517502] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker2 # [ 21.726747] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514170worker1 # [ 21.718971] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_connected addr=lb:500514171worker1 # [ 21.722943] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker2 # [ 21.730701] nix-grpc-daemon[784]: ts=2026-09-20T09:33:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker2 # [ 21.931575] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 21.923913] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514175worker2 # [ 21.935460] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker1 # [ 21.927868] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 22.136143] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 22.128707] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 22.140225] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 22.132979] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 22.333345] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 22.335065] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 22.340992] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 22.344896] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 22.535963] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 22.545927] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514187worker1 # [ 22.539815] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker2 # [ 22.549795] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker1 # [ 22.740622] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 22.742273] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 22.751579] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_connected addr=lb:500514192worker2 # [ 22.755299] nix-grpc-daemon[784]: ts=2026-09-20T09:33:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker1 # [ 22.943543] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 22.947271] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker2 # [ 22.956118] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514196worker2 # [ 22.959333] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker1 # [ 23.148092] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514198worker1 # [ 23.152110] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker2 # [ 23.160403] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514200worker2 # [ 23.164146] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker1 # [ 23.352770] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514202worker1 # [ 23.356643] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker2 # [ 23.364789] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514204worker2 # [ 23.368626] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 23.557369] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514206worker1 # [ 23.561243] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker2 # [ 23.569560] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 23.573373] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker1 # [ 23.763413] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 23.774525] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_connected addr=lb:500514211worker2 # [ 23.775781] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker1 # [ 23.766601] nix-grpc-daemon[784]: ts=2026-09-20T09:33:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker1 # [ 23.963885] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 23.974889] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514215worker1 # [ 23.967817] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker2 # [ 23.977383] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 24.177847] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 24.168662] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514219worker2 # [ 24.179676] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker1 # [ 24.172630] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 24.380529] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 24.373322] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514223worker2 # [ 24.384428] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker1 # [ 24.377237] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 24.585376] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514226worker1 # [ 24.578117] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 24.589207] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker1 # [ 24.582087] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 24.790931] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514230worker1 # [ 24.783709] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_connected addr=lb:500514231worker2 # [ 24.794778] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker1 # [ 24.787563] nix-grpc-daemon[784]: ts=2026-09-20T09:33:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 24.985508] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 24.995387] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514235worker2 # [ 24.997545] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker1 # [ 24.989369] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 25.198198] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514238worker1 # [ 25.190342] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514239worker2 # [ 25.199872] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker1 # [ 25.194300] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker2 # [ 25.400403] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 25.402171] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker1 # [ 25.394926] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514244worker1 # [ 25.398738] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker2 # [ 25.602706] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514246worker2 # [ 25.604507] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker1 # [ 25.599600] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 25.603346] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker2 # [ 25.806501] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514250worker2 # [ 25.810207] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker1 # [ 25.805377] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_connected addr=lb:500514252worker1 # [ 25.808481] nix-grpc-daemon[784]: ts=2026-09-20T09:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker2 # [ 26.007679] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 26.009427] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker1 # [ 26.005553] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514256worker1 # [ 26.007216] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker2 # [ 26.209983] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514258worker2 # [ 26.211658] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker1 # [ 26.208204] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514260worker1 # [ 26.212091] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker2 # [ 26.412178] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514262worker2 # [ 26.413860] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker1 # [ 26.412911] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514264worker1 # [ 26.416837] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker2 # [ 26.614820] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514266worker2 # [ 26.618674] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker1 # [ 26.617690] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514268worker1 # [ 26.621624] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker2 # [ 26.820311] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514270worker2 # [ 26.823961] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker1 # [ 26.823807] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_connected addr=lb:500514272worker1 # [ 26.826875] nix-grpc-daemon[784]: ts=2026-09-20T09:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker2 # [ 27.024731] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514274worker2 # [ 27.028362] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker1 # [ 27.024684] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514276worker1 # [ 27.028429] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker2 # [ 27.228660] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514278worker2 # [ 27.230390] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker1 # [ 27.229144] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514280worker1 # [ 27.233099] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker2 # [ 27.431536] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514282worker2 # [ 27.435280] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker1 # [ 27.434470] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 27.437739] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker2 # [ 27.636354] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514286worker2 # [ 27.640102] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker1 # [ 27.638766] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514288worker1 # [ 27.642561] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker2 # [ 27.842472] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_connected addr=lb:500514290worker2 # [ 27.845493] nix-grpc-daemon[784]: ts=2026-09-20T09:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker1 # [ 27.843798] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514292worker1 # [ 27.845263] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker2 # [ 28.043153] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 28.046986] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker1 # [ 28.044779] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514296worker1 # [ 28.048722] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker2 # [ 28.247775] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514298worker2 # [ 28.251660] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker1 # [ 28.250178] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514300worker1 # [ 28.253334] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker2 # [ 28.452415] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514302worker2 # [ 28.456149] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker1 # [ 28.451809] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514304worker1 # [ 28.454968] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker2 # [ 28.656980] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 28.653669] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514307worker2 # [ 28.660766] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4308worker1 # [ 28.656775] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker2 # [ 28.862924] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 28.856154] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514311worker2 # [ 28.866118] nix-grpc-daemon[784]: ts=2026-09-20T09:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4312worker1 # [ 28.859270] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker2 # [ 29.063383] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514314worker1 # [ 29.056083] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514315worker2 # [ 29.065094] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4316worker1 # [ 29.057772] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker1 # [ 29.258126] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514318worker1 # [ 29.259856] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker2 # [ 29.266352] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514320worker2 # [ 29.270892] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker1 # [ 29.460519] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514322worker1 # [ 29.462263] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker2 # [ 29.472392] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514324worker2 # [ 29.475492] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker1 # [ 29.663264] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514326worker2 # [ 29.673151] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514327worker2 # [ 29.674850] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4328worker1 # [ 29.667142] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker1 # [ 29.868519] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514330worker2 # [ 29.876623] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_connected addr=lb:500514331worker1 # [ 29.871334] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4332worker2 # [ 29.880530] nix-grpc-daemon[784]: ts=2026-09-20T09:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker1 # [ 30.072309] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514334worker2 # [ 30.081448] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514335worker1 # [ 30.076434] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4336worker2 # [ 30.085373] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker1 # [ 30.277190] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514338worker2 # [ 30.286804] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514339worker1 # [ 30.282452] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4340worker2 # [ 30.290121] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker2 # [ 30.487376] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514342worker2 # [ 30.489178] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker1 # [ 30.482316] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514344worker1 # [ 30.486197] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4345worker2 # [ 30.690761] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514346worker1 # [ 30.686614] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514347worker1 # [ 30.688325] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4348worker2 # [ 30.694705] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349worker2 # [ 30.896321] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_connected addr=lb:500514350worker1 # [ 30.890707] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514351worker2 # [ 30.900158] nix-grpc-daemon[784]: ts=2026-09-20T09:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4352worker1 # [ 30.893785] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4353worker1 # [ 31.091301] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514354worker2 # [ 31.101175] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514355worker1 # [ 31.095174] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4356worker2 # [ 31.104920] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4357worker1 # [ 31.295653] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514358worker2 # [ 31.305172] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514359worker2 # [ 31.307099] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4360worker1 # [ 31.299505] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4361worker2 # [ 31.507642] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514362worker2 # [ 31.509416] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4363worker1 # [ 31.500203] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514364worker1 # [ 31.504241] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4365lb # [ 31.668585] rustfs-bucket-start[1057]: mb s3://farm4366lb # [ 31.672187] systemd[1]: Finished rustfs-bucket.service.4367lb # [ 31.678596] systemd[1]: Starting niks3 server...4368worker2 # [ 31.710367] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514369worker1 # [ 31.704366] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_connected addr=lb:500514370worker1 # [ 31.706430] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4371worker2 # [ 31.714352] nix-grpc-daemon[784]: ts=2026-09-20T09:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4372lb # [ 31.876316] postgres[1072]: [1072] ERROR: relation "goose_db_version" does not exist at character 364373lb # [ 31.877756] postgres[1072]: [1072] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4374lb # [ 31.912509] niks3-server[1067]: 2026/09/20 09:33:11 OK 20241026095416_initial_model.sql (19.67ms)4375lb # [ 31.915906] niks3-server[1067]: 2026/09/20 09:33:11 OK 20251210153512_drop_unused_gin_index.sql (3.37ms)4376worker1 # [ 31.906344] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_connected addr=lb:500514377worker1 # [ 31.908816] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4378worker2 # [ 31.916656] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_connected addr=lb:500514379lb # [ 31.921665] niks3-server[1067]: 2026/09/20 09:33:11 OK 20251218171726_add_pins.sql (5.67ms)4380worker2 # [ 31.919716] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4381lb # [ 31.926281] niks3-server[1067]: 2026/09/20 09:33:11 OK 20260628120000_add_object_size_and_stats.sql (4.74ms)4382lb # [ 31.930536] niks3-server[1067]: 2026/09/20 09:33:11 OK 20260905000000_add_claims.sql (2.81ms)4383lb # [ 31.934908] niks3-server[1067]: 2026/09/20 09:33:11 OK 20260920000000_drop_claims.sql (4.27ms)4384lb # [ 31.936115] niks3-server[1067]: 2026/09/20 09:33:11 goose: successfully migrated database to version: 202609200000004385lb # [ 31.941416] niks3-server[1067]: 2026/09/20 09:33:11 OK 1_commit_pending_closure.sql (6.57ms)4386lb # [ 31.943910] niks3-server[1067]: 2026/09/20 09:33:11 OK 2_object_stats_trigger.sql (1.3ms)4387lb # [ 31.945091] niks3-server[1067]: 2026/09/20 09:33:11 goose: up to current file version: 24388lb # [ 31.951178] niks3-server[1067]: 2026/09/20 09:33:11 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4389lb # [ 32.018755] niks3-server[1067]: 2026/09/20 09:33:11 INFO Read proxy enabled — serving cache objects from S34390lb # [ 32.020560] niks3-server[1067]: 2026/09/20 09:33:11 INFO Using socket-activated listener address=0.0.0.0:57514391lb # [ 32.025108] niks3-server[1067]: 2026/09/20 09:33:11 INFO systemd watchdog enabled interval=15s4392lb # [ 32.027261] systemd[1]: Started niks3 server.4393lb # [ 32.027902] systemd[1]: Reached target Multi-User System.4394lb # [ 32.030075] niks3-server[1067]: 2026/09/20 09:33:11 INFO Starting HTTP server address=0.0.0.0:57514395lb # [ 32.031643] systemd[1]: Startup finished in 841ms (kernel) + 5.035s (initrd) + 26.153s (userspace) = 32.029s.4396lb # [ 32.036808] niks3-server[1067]: 2026/09/20 09:33:11 INFO lead: acquired remote=192.168.1.4:378464397worker2 # [ 32.040382] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_take_over4398worker1 # [ 32.109625] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_connected addr=lb:500514399worker2 # [ 32.117716] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_connected addr=lb:500514400worker1 # [ 32.111682] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4401worker2 # [ 32.121598] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4402worker1 # [ 32.312092] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_connected addr=lb:500514403worker2 # [ 32.322395] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=scheduler_connected addr=lb:500514404worker2 # [ 32.333533] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04405worker2 # [ 32.339830] nix-grpc-daemon[784]: ts=2026-09-20T09:33:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04406lb: (finished: waiting for unit niks3.service, in 20.74 seconds)4407lb: waiting for TCP port 5751 on localhost4408lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4409lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4410lb: waiting for unit envoy.service4411lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4412lb: waiting for TCP port 50051 on localhost4413lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4414lb # [ 32.615595] envoy[908]: conn peer=[::1]:40398 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24415lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4416lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4417??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4418 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284419lb: must succeed: curl -sf localhost:9901/clusters4420lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4421lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.05 seconds)4422subtest: one node schedules and both builders hold a WorkerSession on it4423lb: must succeed: curl -sf localhost:9901/clusters4424lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4425worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4426worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.05 seconds)4427(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.09 seconds)4428subtest: balancer auth: client cert, bearer token, nothing, unknown CN4429worker2 # [ 33.192371] nix-grpc-daemon[784]: ts=2026-09-20T09:33:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409724430worker2 # [ 33.204175] systemd[1]: Started Nix Daemon.4431worker2 # [ 33.394343] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4432worker2 # [ 33.436746] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5)4433worker2 # [ 33.442280] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 2/5)4434worker2 # [ 33.462613] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 329 ms (attempt 3/5)4435worker2 # [ 33.794907] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 227 ms (attempt 4/5)4436worker2 # [ 34.025272] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4437worker1 # [ 34.051777] nix-grpc-daemon[784]: ts=2026-09-20T09:33:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:547764438worker1 # [ 34.065377] systemd[1]: Started Nix Daemon.4439worker1 # [ 34.270348] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4440worker1 # [ 34.303873] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 1/5)4441worker1 # [ 34.333323] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 2/5)4442worker1 # [ 34.375957] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 3/5)4443worker1 # [ 34.564579] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 327 ms (attempt 4/5)4444worker1 # [ 34.902596] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4445worker1 # [ 34.920905] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=debug event=path_info_miss path=bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test error="error: path '/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test' is required, but there is no substituter that can build it"4446worker1 # [ 34.931618] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:547764447worker1 # [ 34.940120] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=debug event=path_info_miss path=bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test error="error: path '/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test' is required, but there is no substituter that can build it"4448client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4449client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)4450client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4451client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4452worker1 # [ 35.352857] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=oidc_jwks provider=mock keys=14453worker1 # [ 35.356567] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:547764454worker2 # [ 35.393925] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=oidc_jwks provider=mock keys=14455worker2 # [ 35.397779] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:409724456worker2 # [ 35.412790] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=debug event=path_info_miss path=bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test error="error: path '/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test' is required, but there is no substituter that can build it"4457worker2 # [ 35.421861] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:409724458worker2 # [ 35.429527] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=debug event=path_info_miss path=bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test error="error: path '/nix/store/bgb9qwzhklrxfjy4awx25fzniv721g1y-nixos-system-client-test' is required, but there is no substituter that can build it"4459client: must succeed: rm -r /var/lib/nix-grpc-store4460client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4461worker2 # [ 35.693241] nix-grpc-daemon[784]: ts=2026-09-20T09:33:15Z level=info event=denied method=QueryMissing cn=- role=none4462worker1 # [ 35.887833] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=denied method=QueryMissing cn=stranger role=none4463(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.06 seconds)4464subtest: DAG build is scheduled across workers and lands in the cache4465client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24466worker1 # [ 36.099826] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:547764467client # don't know how to build these paths:4468client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4469worker2 # [ 36.121608] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724470worker2 # [ 36.153666] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04471worker2 # [ 36.158353] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14472worker2 # [ 36.204474] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724473worker2 # [ 36.231072] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724474client # copying 1 paths to 'lb:50051'...4475worker2 # [ 36.244144] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409724476client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4477worker2 # [ 36.271206] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4478worker2 # [ 36.298487] nix-grpc-daemon[784]: ts=2026-09-20T09:33:16Z level=info event=niks3_push_started pid=8604479worker2 # [ 36.743879] nix-daemon[848]: accepted connection from pid 872, user nix-grpc-daemon (trusted)4480worker2 # [ 36.761568] nix-daemon[848]: reaped child process 875, status = succeeded4481worker2 # [ 36.916863] nix-daemon[848]: accepted connection from pid 878, user nix-grpc-daemon (trusted)4482worker2 # [ 36.930744] nix-daemon[848]: reaped child process 881, status = succeeded4483lb # [ 36.945215] niks3-server[1067]: 2026/09/20 09:33:16 INFO Received uploads request method=POST path=/api/pending_closures4484worker2 # [ 36.956283] nix-grpc-daemon[860]: time=2026-09-20T09:33:17.049Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4485worker2 # [ 36.958950] nix-grpc-daemon[860]: time=2026-09-20T09:33:17.052Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4486lb # [ 36.986132] niks3-server[1067]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4487lb # [ 36.990345] niks3-server[1067]: 2026/09/20 09:33:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4488lb # [ 36.996797] niks3-server[1067]: 2026/09/20 09:33:16 INFO Signed narinfos id=1 count=14489worker2 # [ 36.995819] nix-grpc-daemon[860]: time=2026-09-20T09:33:17.090Z level=INFO msg="Uploading 1 narinfos"4490lb # [ 37.011265] niks3-server[1067]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4491lb # [ 37.014461] niks3-server[1067]: 2026/09/20 09:33:16 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4492lb # [ 37.022117] niks3-server[1067]: 2026/09/20 09:33:16 INFO Completed upload id=14493worker2 # [ 37.019673] nix-grpc-daemon[860]: time=2026-09-20T09:33:17.113Z level=INFO msg="Upload complete. (455ms)"4494lb # [ 37.024695] niks3-server[1067]: 2026/09/20 09:33:16 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4495worker2 # [ 37.022508] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_in=5044496worker2 # [ 37.027671] nix-grpc-daemon[784]: ts=2026-09-20T09:33:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724497worker2 # [ 37.034672] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4498worker2 # [ 37.037260] nix-daemon[848]: reaped child process 858, status = succeeded4499client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4500worker2 # [ 37.043205] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4501client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4502lb # [ 37.967461] envoy[908]: conn peer=[2001:db8:1::1]:60486 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=383 ms=18124503lb # [ 37.969300] envoy[908]: conn peer=[2001:db8:1::1]:60492 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=984504lb # [ 37.970968] envoy[908]: conn peer=[2001:db8:1::1]:60500 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=84505lb # [ 37.972616] envoy[908]: conn peer=[2001:db8:1::1]:60504 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=94506client # farm-a> LOG-farm-a-t14507worker2 # [ 38.059818] nix-daemon[848]: reaped child process 885, status = succeeded4508worker2 # [ 38.208648] nix-daemon[848]: accepted connection from pid 900, user nix-grpc-daemon (trusted)4509worker2 # [ 38.225843] nix-daemon[848]: reaped child process 903, status = succeeded4510lb # [ 38.242629] niks3-server[1067]: 2026/09/20 09:33:17 INFO Received uploads request method=POST path=/api/pending_closures4511worker2 # [ 38.245226] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.339Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4512worker2 # [ 38.247062] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.341Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4513lb # [ 38.267671] niks3-server[1067]: 2026/09/20 09:33:17 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4514lb # [ 38.270832] niks3-server[1067]: 2026/09/20 09:33:17 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4515lb # [ 38.274869] niks3-server[1067]: 2026/09/20 09:33:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4516lb # [ 38.276426] niks3-server[1067]: 2026/09/20 09:33:17 INFO Signed narinfos id=2 count=14517worker2 # [ 38.274637] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.368Z level=INFO msg="Uploading 1 narinfos"4518lb # [ 38.280625] niks3-server[1067]: 2026/09/20 09:33:17 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4519lb # [ 38.285312] niks3-server[1067]: 2026/09/20 09:33:17 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4520lb # [ 38.289353] niks3-server[1067]: 2026/09/20 09:33:17 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4521lb # [ 38.292220] niks3-server[1067]: 2026/09/20 09:33:17 INFO Completed upload id=24522worker2 # [ 38.289725] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.384Z level=INFO msg="Upload complete. (238ms)"4523worker2 # [ 38.294207] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14524worker2 # [ 38.306426] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04525worker2 # [ 38.308613] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14526worker2 # [ 38.312403] nix-daemon[848]: reaped child process 883, status = succeeded4527worker2 # [ 38.350526] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724528worker2 # [ 38.359384] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724529worker2 # [ 38.371394] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.465Z level=INFO msg="All 1 paths already cached"4530client # copying 1 paths to 'lb:50051'...4531client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4532worker2 # [ 38.378596] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409724533worker2 # [ 38.393112] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4534worker2 # [ 38.551309] nix-daemon[848]: accepted connection from pid 909, user nix-grpc-daemon (trusted)4535worker2 # [ 38.567351] nix-daemon[848]: reaped child process 912, status = succeeded4536worker2 # [ 38.728730] nix-daemon[848]: accepted connection from pid 914, user nix-grpc-daemon (trusted)4537worker2 # [ 38.741103] nix-daemon[848]: reaped child process 917, status = succeeded4538lb # [ 38.758937] niks3-server[1067]: 2026/09/20 09:33:18 INFO Received uploads request method=POST path=/api/pending_closures4539worker2 # [ 38.798943] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.893Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4540worker2 # [ 38.801096] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.894Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4541lb # [ 38.821450] niks3-server[1067]: 2026/09/20 09:33:18 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4542lb # [ 38.830424] niks3-server[1067]: 2026/09/20 09:33:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4543lb # [ 38.836427] niks3-server[1067]: 2026/09/20 09:33:18 INFO Signed narinfos id=3 count=14544worker2 # [ 38.836911] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.930Z level=INFO msg="Uploading 1 narinfos"4545lb # [ 38.842688] niks3-server[1067]: 2026/09/20 09:33:18 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4546lb # [ 38.854366] niks3-server[1067]: 2026/09/20 09:33:18 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4547worker2 # [ 38.858070] nix-grpc-daemon[860]: time=2026-09-20T09:33:18.951Z level=INFO msg="Upload complete. (454ms)"4548lb # [ 38.860250] niks3-server[1067]: 2026/09/20 09:33:18 INFO Completed upload id=34549worker2 # [ 38.859799] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_in=6244550worker2 # [ 38.863094] nix-grpc-daemon[784]: ts=2026-09-20T09:33:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724551lb # [ 38.865193] niks3-server[1067]: 2026/09/20 09:33:18 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4552worker2 # [ 38.873387] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4553worker2 # [ 38.877705] nix-daemon[848]: reaped child process 907, status = succeeded4554client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4555worker2 # [ 38.883413] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4556client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4557client # farm-b> LOG-farm-b-t14558worker2 # [ 39.706252] nix-daemon[848]: reaped child process 921, status = succeeded4559worker2 # [ 39.843896] nix-daemon[848]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4560worker2 # [ 39.855240] nix-daemon[848]: reaped child process 940, status = succeeded4561lb # [ 39.873218] niks3-server[1067]: 2026/09/20 09:33:19 INFO Received uploads request method=POST path=/api/pending_closures4562worker2 # [ 39.880664] nix-grpc-daemon[860]: time=2026-09-20T09:33:19.974Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4563worker2 # [ 39.882811] nix-grpc-daemon[860]: time=2026-09-20T09:33:19.976Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4564lb # [ 39.910407] niks3-server[1067]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4565lb # [ 39.922146] niks3-server[1067]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4566lb # [ 39.927672] niks3-server[1067]: 2026/09/20 09:33:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4567lb # [ 39.931932] niks3-server[1067]: 2026/09/20 09:33:19 INFO Signed narinfos id=4 count=14568worker2 # [ 39.932504] nix-grpc-daemon[860]: time=2026-09-20T09:33:20.026Z level=INFO msg="Uploading 1 narinfos"4569lb # [ 39.942511] niks3-server[1067]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4570lb # [ 39.950371] niks3-server[1067]: 2026/09/20 09:33:19 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4571lb # [ 39.955844] niks3-server[1067]: 2026/09/20 09:33:19 INFO Completed upload id=44572worker2 # [ 39.953339] nix-grpc-daemon[860]: time=2026-09-20T09:33:20.047Z level=INFO msg="Upload complete. (261ms)"4573lb # [ 39.958634] niks3-server[1067]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4574worker2 # [ 39.964913] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14575worker2 # [ 39.970867] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04576worker2 # [ 39.975231] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14577worker2 # [ 39.983469] nix-daemon[848]: reaped child process 919, status = succeeded4578worker2 # [ 40.009786] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724579worker2 # [ 40.021956] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724580worker2 # [ 40.035066] nix-grpc-daemon[860]: time=2026-09-20T09:33:20.129Z level=INFO msg="All 2 paths already cached"4581client # copying 1 paths to 'lb:50051'...4582worker2 # [ 40.042388] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409724583client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4584worker2 # [ 40.060169] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4585worker2 # [ 40.222833] nix-daemon[848]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4586worker2 # [ 40.234966] nix-daemon[848]: reaped child process 949, status = succeeded4587worker2 # [ 40.390549] nix-daemon[848]: accepted connection from pid 951, user nix-grpc-daemon (trusted)4588worker2 # [ 40.402907] nix-daemon[848]: reaped child process 954, status = succeeded4589lb # [ 40.419155] niks3-server[1067]: 2026/09/20 09:33:19 INFO Received uploads request method=POST path=/api/pending_closures4590worker2 # [ 40.424482] nix-grpc-daemon[860]: time=2026-09-20T09:33:20.518Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4591worker2 # [ 40.426051] nix-grpc-daemon[860]: time=2026-09-20T09:33:20.520Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4592lb # [ 40.449954] niks3-server[1067]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4593lb # [ 40.457865] niks3-server[1067]: 2026/09/20 09:33:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4594lb # [ 40.460222] niks3-server[1067]: 2026/09/20 09:33:19 INFO Signed narinfos id=5 count=14595worker2 # [ 40.457677] nix-grpc-daemon[860]: time=2026-09-20T09:33:20.552Z level=INFO msg="Uploading 1 narinfos"4596lb # [ 40.462964] niks3-server[1067]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4597lb # [ 40.468150] niks3-server[1067]: 2026/09/20 09:33:19 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4598lb # [ 40.470765] niks3-server[1067]: 2026/09/20 09:33:19 INFO Completed upload id=54599worker2 # [ 40.468185] nix-grpc-daemon[860]: time=2026-09-20T09:33:20.562Z level=INFO msg="Upload complete. (395ms)"4600worker2 # [ 40.470074] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_in=7524601lb # [ 40.473797] niks3-server[1067]: 2026/09/20 09:33:19 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4602worker2 # [ 40.473462] nix-grpc-daemon[784]: ts=2026-09-20T09:33:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724603worker2 # [ 40.478155] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4604worker2 # [ 40.482376] nix-daemon[848]: reaped child process 944, status = succeeded4605client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4606worker2 # [ 40.488365] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4607client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4608client # farm-top> LOG-farm-top-t14609worker2 # [ 41.701473] nix-daemon[848]: reaped child process 958, status = succeeded4610worker2 # [ 41.835053] nix-daemon[848]: accepted connection from pid 973, user nix-grpc-daemon (trusted)4611worker2 # [ 41.847388] nix-daemon[848]: reaped child process 976, status = succeeded4612lb # [ 41.861429] niks3-server[1067]: 2026/09/20 09:33:21 INFO Received uploads request method=POST path=/api/pending_closures4613worker2 # [ 41.869296] nix-grpc-daemon[860]: time=2026-09-20T09:33:21.962Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4614worker2 # [ 41.870653] nix-grpc-daemon[860]: time=2026-09-20T09:33:21.962Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4615lb # [ 41.885457] niks3-server[1067]: 2026/09/20 09:33:21 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4616lb # [ 41.889926] niks3-server[1067]: 2026/09/20 09:33:21 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4617lb # [ 41.899880] niks3-server[1067]: 2026/09/20 09:33:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4618lb # [ 41.901830] niks3-server[1067]: 2026/09/20 09:33:21 INFO Signed narinfos id=6 count=14619worker2 # [ 41.900620] nix-grpc-daemon[860]: time=2026-09-20T09:33:21.994Z level=INFO msg="Uploading 1 narinfos"4620lb # [ 41.909409] niks3-server[1067]: 2026/09/20 09:33:21 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4621lb # [ 41.912280] niks3-server[1067]: 2026/09/20 09:33:21 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4622lb # [ 41.917373] niks3-server[1067]: 2026/09/20 09:33:21 INFO Completed upload id=64623worker2 # [ 41.914763] nix-grpc-daemon[860]: time=2026-09-20T09:33:22.009Z level=INFO msg="Upload complete. (220ms)"4624worker2 # [ 41.917265] nix-grpc-daemon[784]: ts=2026-09-20T09:33:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14625worker2 # [ 41.922684] nix-grpc-daemon[860]: time=2026-09-20T09:33:22.017Z level=WARN msg="Failed to register uploaded object" key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo error="server returned 404: object is not pending upload\n"4626worker2 # [ 41.926890] nix-daemon[848]: reaped child process 956, status = succeeded4627client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.04 seconds)4628client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4629client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4630client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4631client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4632client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4633client # copying 3 paths...4634client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4635client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4636client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4637client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4638(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.58 seconds)4639subtest: repeat is answered Cached by the scheduler without building4640worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4641worker1 # finding garbage collector roots...4642worker1 # deleting specified paths...4643worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4644worker1 # deleting unused links...4645worker1 # note: hard linking is currently saving 0.0 KiB4646worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.28 seconds)4647worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4648worker2 # finding garbage collector roots...4649worker2 # deleting specified paths...4650worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4651worker2 # deleting unused links...4652worker2 # note: hard linking is currently saving 0.0 KiB4653worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4654client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24655worker2 # [ 43.228860] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409724656client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4657client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4658worker1 # [ 43.232434] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:547764659worker2 # [ 43.257581] nix-grpc-daemon[784]: ts=2026-09-20T09:33:23Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14660client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.27 seconds)4661client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4662client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4663lb: must succeed: curl -sf localhost:9901/clusters4664lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4665lb: must succeed: curl -sf localhost:9901/clusters4666lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4667worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4668worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.07 seconds)4669(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.24 seconds)4670subtest: inputs already in the cache are not sent to the scheduler4671lb: must succeed: curl -sf localhost:9901/clusters4672lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4673lb: must succeed: curl -sf localhost:9901/clusters4674lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4675worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4676worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4677client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24678worker1 # [ 44.091913] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:547764679client # don't know how to build these paths:4680client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4681worker1 # [ 44.100879] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:547764682worker2 # [ 44.121911] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04683worker2 # [ 44.127877] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14684worker2 # [ 44.171130] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724685worker2 # [ 44.183796] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724686worker2 # [ 44.196380] nix-grpc-daemon[860]: time=2026-09-20T09:33:24.290Z level=INFO msg="All 2 paths already cached"4687client # copying 1 paths to 'lb:50051'...4688worker2 # [ 44.204531] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409724689client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4690worker2 # [ 44.220573] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4691worker2 # [ 44.402752] nix-daemon[848]: accepted connection from pid 1008, user nix-grpc-daemon (trusted)4692worker2 # [ 44.416656] nix-daemon[848]: reaped child process 1011, status = succeeded4693worker2 # [ 44.577034] nix-daemon[848]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)4694worker2 # [ 44.588877] nix-daemon[848]: reaped child process 1016, status = succeeded4695lb # [ 44.608926] niks3-server[1067]: 2026/09/20 09:33:23 INFO Received uploads request method=POST path=/api/pending_closures4696worker2 # [ 44.616890] nix-grpc-daemon[860]: time=2026-09-20T09:33:24.711Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4697worker2 # [ 44.618402] nix-grpc-daemon[860]: time=2026-09-20T09:33:24.712Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4698lb # [ 44.647582] niks3-server[1067]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4699lb # [ 44.655380] niks3-server[1067]: 2026/09/20 09:33:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4700lb # [ 44.661359] niks3-server[1067]: 2026/09/20 09:33:23 INFO Signed narinfos id=7 count=14701worker2 # [ 44.661868] nix-grpc-daemon[860]: time=2026-09-20T09:33:24.756Z level=INFO msg="Uploading 1 narinfos"4702lb # [ 44.671556] niks3-server[1067]: 2026/09/20 09:33:23 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4703lb # [ 44.678409] niks3-server[1067]: 2026/09/20 09:33:23 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4704lb # [ 44.685241] niks3-server[1067]: 2026/09/20 09:33:24 INFO Completed upload id=74705worker2 # [ 44.682178] nix-grpc-daemon[860]: time=2026-09-20T09:33:24.776Z level=INFO msg="Upload complete. (448ms)"4706lb # [ 44.687151] niks3-server[1067]: 2026/09/20 09:33:24 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4707worker2 # [ 44.688217] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_in=7524708worker2 # [ 44.693745] nix-grpc-daemon[784]: ts=2026-09-20T09:33:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724709worker2 # [ 44.703265] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4710worker2 # [ 44.710463] nix-daemon[848]: reaped child process 1006, status = succeeded4711worker2 # [ 44.716053] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4712client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4713client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4714client # farm-top> LOG-farm-top-t1b4715worker2 # [ 45.942350] nix-daemon[848]: reaped child process 1020, status = succeeded4716worker2 # [ 46.099822] nix-daemon[848]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4717worker2 # [ 46.112236] nix-daemon[848]: reaped child process 1038, status = succeeded4718lb # [ 46.131798] niks3-server[1067]: 2026/09/20 09:33:25 INFO Received uploads request method=POST path=/api/pending_closures4719worker2 # [ 46.137653] nix-grpc-daemon[860]: time=2026-09-20T09:33:26.231Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4720worker2 # [ 46.140257] nix-grpc-daemon[860]: time=2026-09-20T09:33:26.233Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4721lb # [ 46.159490] niks3-server[1067]: 2026/09/20 09:33:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4722lb # [ 46.161547] niks3-server[1067]: 2026/09/20 09:33:25 INFO Signed narinfos id=8 count=14723worker2 # [ 46.160487] nix-grpc-daemon[860]: time=2026-09-20T09:33:26.254Z level=INFO msg="Uploading 1 narinfos"4724lb # [ 46.164206] niks3-server[1067]: 2026/09/20 09:33:25 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4725lb # [ 46.169408] niks3-server[1067]: 2026/09/20 09:33:25 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4726lb # [ 46.173041] niks3-server[1067]: 2026/09/20 09:33:25 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4727lb # [ 46.175750] niks3-server[1067]: 2026/09/20 09:33:25 INFO Completed upload id=84728worker2 # [ 46.174679] nix-grpc-daemon[860]: time=2026-09-20T09:33:26.268Z level=INFO msg="Upload complete. (250ms)"4729lb # [ 46.177764] niks3-server[1067]: 2026/09/20 09:33:25 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4730lb # [ 46.180388] niks3-server[1067]: 2026/09/20 09:33:25 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4731worker2 # [ 46.178279] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14732worker2 # [ 46.185889] nix-daemon[848]: reaped child process 1018, status = succeeded4733client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.32 seconds)4734client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4735client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.20 seconds)4736lb: must succeed: curl -sf localhost:9901/clusters4737lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4738lb: must succeed: curl -sf localhost:9901/clusters4739lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4740worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4741worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4742(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.81 seconds)4743subtest: two clients wanting the same drv build it once4744client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4745client # [ 46.560698] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4746client # Running as unit: dedup1.service; invocation ID: e34952cf7a394efa8589a37a8b442fb64747client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.05 seconds)4748client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4749client # [ 46.761240] nix[896]: don't know how to build these paths:4750client # [ 46.763274] nix[896]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4751worker1 # [ 46.779416] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:547764752worker2 # [ 46.792677] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04753worker2 # [ 46.794817] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14754worker1 # [ 46.789680] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547764755worker1 # [ 46.803934] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:547764756client # [ 46.793545] nix[896]: copying 1 paths to 'lb:50051'...4757client # [ 46.794930] nix[896]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4758worker1 # [ 46.813716] nix-grpc-daemon[784]: ts=2026-09-20T09:33:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:547764759worker1 # [ 46.826838] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4760worker1 # [ 46.850785] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=niks3_push_started pid=8684761worker1 # [ 47.358674] nix-daemon[843]: accepted connection from pid 881, user nix-grpc-daemon (trusted)4762worker1 # [ 47.378217] nix-daemon[843]: reaped child process 884, status = succeeded4763worker1 # [ 47.553244] nix-daemon[843]: accepted connection from pid 886, user nix-grpc-daemon (trusted)4764worker1 # [ 47.568444] nix-daemon[843]: reaped child process 889, status = succeeded4765lb # [ 47.594522] niks3-server[1067]: 2026/09/20 09:33:26 INFO Received uploads request method=POST path=/api/pending_closures4766worker1 # [ 47.593345] nix-grpc-daemon[868]: time=2026-09-20T09:33:27.750Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4767worker1 # [ 47.595966] nix-grpc-daemon[868]: time=2026-09-20T09:33:27.753Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4768lb # [ 47.634398] niks3-server[1067]: 2026/09/20 09:33:26 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4769lb # [ 47.641896] niks3-server[1067]: 2026/09/20 09:33:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4770lb # [ 47.649055] niks3-server[1067]: 2026/09/20 09:33:26 INFO Signed narinfos id=9 count=14771worker1 # [ 47.641522] nix-grpc-daemon[868]: time=2026-09-20T09:33:27.800Z level=INFO msg="Uploading 1 narinfos"4772lb # [ 47.652492] niks3-server[1067]: 2026/09/20 09:33:26 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4773lb # [ 47.662325] niks3-server[1067]: 2026/09/20 09:33:26 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4774worker1 # [ 47.658518] nix-grpc-daemon[868]: time=2026-09-20T09:33:27.817Z level=INFO msg="Upload complete. (498ms)"4775lb # [ 47.668646] niks3-server[1067]: 2026/09/20 09:33:26 INFO Completed upload id=94776worker1 # [ 47.660716] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=0 paths=1 nar_bytes_in=4324777lb # [ 47.671649] niks3-server[1067]: 2026/09/20 09:33:26 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4778client # [ 47.636173] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4779client # Running as unit: dedup2.service; invocation ID: e37531aeb27a4eb88c25d953829051d04780worker1 # [ 47.669850] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547764781worker1 # [ 47.671537] nix-daemon[843]: reaped child process 866, status = succeeded4782worker1 # [ 47.675106] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4783client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4784worker1 # [ 47.683517] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4785client # [ 47.664668] nix[896]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4786worker2 # [ 47.892217] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409724787client # [ 47.869423] nix[912]: don't know how to build these paths:4788client # [ 47.870336] nix[912]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4789worker2 # [ 47.901551] nix-grpc-daemon[784]: ts=2026-09-20T09:33:27Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04790client # [ 47.880422] nix[912]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4791worker1 # [ 47.898303] nix-grpc-daemon[784]: ts=2026-09-20T09:33:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547764792lb # [ 47.968109] envoy[908]: conn peer=[2001:db8:1::1]:60514 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9507 tx=3799 ms=58414793lb # [ 47.969672] envoy[908]: conn peer=[2001:db8:1::1]:35544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=478 ms=394794lb # [ 47.971077] envoy[908]: conn peer=[2001:db8:1::1]:35548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4126 tx=1688 ms=20924795client: must succeed: sleep 24796client: (finished: must succeed: sleep 2, in 2.05 seconds)4797??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4798 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394799client: waiting for success: ! systemctl is-active dedup1 dedup24800??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4801 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394802worker1 # [ 51.103638] nix-daemon[843]: reaped child process 895, status = succeeded4803worker1 # [ 51.255773] nix-daemon[843]: accepted connection from pid 922, user nix-grpc-daemon (trusted)4804worker1 # [ 51.269885] nix-daemon[843]: reaped child process 925, status = succeeded4805lb # [ 51.294527] niks3-server[1067]: 2026/09/20 09:33:30 INFO Received uploads request method=POST path=/api/pending_closures4806worker1 # [ 51.292380] nix-grpc-daemon[868]: time=2026-09-20T09:33:31.451Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4807worker1 # [ 51.293961] nix-grpc-daemon[868]: time=2026-09-20T09:33:31.453Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4808lb # [ 51.345076] niks3-server[1067]: 2026/09/20 09:33:30 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4809lb # [ 51.349718] niks3-server[1067]: 2026/09/20 09:33:30 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4810lb # [ 51.361114] niks3-server[1067]: 2026/09/20 09:33:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4811lb # [ 51.366326] niks3-server[1067]: 2026/09/20 09:33:30 INFO Signed narinfos id=10 count=14812worker1 # [ 51.359969] nix-grpc-daemon[868]: time=2026-09-20T09:33:31.518Z level=INFO msg="Uploading 1 narinfos"4813lb # [ 51.372042] niks3-server[1067]: 2026/09/20 09:33:30 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4814lb # [ 51.380699] niks3-server[1067]: 2026/09/20 09:33:30 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4815lb # [ 51.388622] niks3-server[1067]: 2026/09/20 09:33:30 INFO Completed upload id=104816worker1 # [ 51.378696] nix-grpc-daemon[868]: time=2026-09-20T09:33:31.537Z level=INFO msg="Upload complete. (280ms)"4817lb # [ 51.390235] niks3-server[1067]: 2026/09/20 09:33:30 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4818worker1 # [ 51.384984] nix-grpc-daemon[784]: ts=2026-09-20T09:33:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14819worker1 # [ 51.390933] nix-grpc-daemon[784]: ts=2026-09-20T09:33:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14820worker1 # [ 51.399784] nix-daemon[843]: reaped child process 891, status = succeeded4821client # [ 51.417311] systemd[1]: dedup1.service: Deactivated successfully.4822client # [ 51.420825] systemd[1]: dedup1.service: Consumed 104ms CPU time over 4.856s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.4823client # [ 51.427380] systemd[1]: dedup2.service: Deactivated successfully.4824client # [ 51.429544] systemd[1]: dedup2.service: Consumed 128ms CPU time over 3.791s wall clock time, 33.1M memory peak, 7.5K incoming IP traffic, 6.4K outgoing IP traffic.4825client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4826client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4827client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4828worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4829worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4830worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4831worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)4832worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4833worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.07 seconds)4834worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4835worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.04 seconds)4836(finished: subtest: two clients wanting the same drv build it once, in 5.87 seconds)4837subtest: build hook: nix-daemon with builders = grpc://lb4838client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4839client # [ 52.601311] systemd[1]: Started Nix Daemon.4840client # [ 52.769838] nix-daemon[954]: accepted connection from pid 951, user root (trusted)4841worker2 # [ 52.977363] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:409724842worker2 # [ 52.982561] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724843worker2 # [ 52.993159] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04844worker2 # [ 52.998206] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=14845worker2 # [ 53.037235] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724846worker2 # [ 53.050183] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724847worker2 # [ 53.061427] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409724848worker2 # [ 53.072153] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4849worker2 # [ 53.244327] nix-daemon[848]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)4850worker2 # [ 53.256766] nix-daemon[848]: reaped child process 1079, status = succeeded4851worker2 # [ 53.395511] nix-daemon[848]: accepted connection from pid 1081, user nix-grpc-daemon (trusted)4852worker2 # [ 53.406989] nix-daemon[848]: reaped child process 1084, status = succeeded4853lb # [ 53.421950] niks3-server[1067]: 2026/09/20 09:33:32 INFO Received uploads request method=POST path=/api/pending_closures4854worker2 # [ 53.428726] nix-grpc-daemon[860]: time=2026-09-20T09:33:33.522Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4855worker2 # [ 53.430208] nix-grpc-daemon[860]: time=2026-09-20T09:33:33.524Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4856lb # [ 53.453126] niks3-server[1067]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4857lb # [ 53.463215] niks3-server[1067]: 2026/09/20 09:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4858lb # [ 53.468251] niks3-server[1067]: 2026/09/20 09:33:32 INFO Signed narinfos id=11 count=14859worker2 # [ 53.468755] nix-grpc-daemon[860]: time=2026-09-20T09:33:33.563Z level=INFO msg="Uploading 1 narinfos"4860lb # [ 53.474564] niks3-server[1067]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4861lb # [ 53.487273] niks3-server[1067]: 2026/09/20 09:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4862lb # [ 53.494370] niks3-server[1067]: 2026/09/20 09:33:32 INFO Completed upload id=114863worker2 # [ 53.494059] nix-grpc-daemon[860]: time=2026-09-20T09:33:33.588Z level=INFO msg="Upload complete. (409ms)"4864lb # [ 53.497294] niks3-server[1067]: 2026/09/20 09:33:32 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4865worker2 # [ 53.498255] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_in=5124866worker2 # [ 53.502840] nix-grpc-daemon[784]: ts=2026-09-20T09:33:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724867worker2 # [ 53.513116] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4868worker2 # [ 53.519320] nix-daemon[848]: reaped child process 1074, status = succeeded4869worker2 # [ 53.524074] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4870worker2 # [ 54.343786] nix-daemon[848]: reaped child process 1088, status = succeeded4871worker2 # [ 54.495827] nix-daemon[848]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)4872worker2 # [ 54.507083] nix-daemon[848]: reaped child process 1106, status = succeeded4873lb # [ 54.525502] niks3-server[1067]: 2026/09/20 09:33:33 INFO Received uploads request method=POST path=/api/pending_closures4874worker2 # [ 54.531472] nix-grpc-daemon[860]: time=2026-09-20T09:33:34.625Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4875worker2 # [ 54.533000] nix-grpc-daemon[860]: time=2026-09-20T09:33:34.627Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4876lb # [ 54.570275] niks3-server[1067]: 2026/09/20 09:33:33 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4877lb # [ 54.577825] niks3-server[1067]: 2026/09/20 09:33:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4878worker2 # [ 54.581817] nix-grpc-daemon[860]: time=2026-09-20T09:33:34.676Z level=INFO msg="Uploading 1 narinfos"4879lb # [ 54.584582] niks3-server[1067]: 2026/09/20 09:33:33 INFO Signed narinfos id=12 count=14880lb # [ 54.589801] niks3-server[1067]: 2026/09/20 09:33:33 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4881lb # [ 54.594136] niks3-server[1067]: 2026/09/20 09:33:33 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4882lb # [ 54.604514] niks3-server[1067]: 2026/09/20 09:33:33 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4883lb # [ 54.614990] niks3-server[1067]: 2026/09/20 09:33:33 INFO Completed upload id=124884worker2 # [ 54.613951] nix-grpc-daemon[860]: time=2026-09-20T09:33:34.708Z level=INFO msg="Upload complete. (283ms)"4885worker2 # [ 54.617294] nix-grpc-daemon[784]: ts=2026-09-20T09:33:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14886lb # [ 54.619910] niks3-server[1067]: 2026/09/20 09:33:33 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4887worker2 # [ 54.624199] nix-daemon[848]: reaped child process 1086, status = succeeded4888worker1 # [ 54.617819] nix-grpc-daemon[784]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:547764889worker1 # [ 54.626566] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4890worker2 # [ 54.667936] nix-grpc-daemon[784]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409724891worker2 # [ 54.673149] nix-grpc-daemon[784]: ts=2026-09-20T09:33:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_out=1284892worker1 # [ 54.872812] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:547764893worker2 # [ 54.885279] nix-grpc-daemon[784]: ts=2026-09-20T09:33:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724894worker2 # [ 54.892993] nix-grpc-daemon[860]: time=2026-09-20T09:33:34.987Z level=INFO msg="All 1 paths already cached"4895worker2 # [ 54.906179] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04896worker2 # [ 54.911406] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=14897worker2 # [ 54.916906] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724898worker2 # [ 54.927704] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724899worker2 # [ 54.939667] nix-grpc-daemon[860]: time=2026-09-20T09:33:35.033Z level=INFO msg="All 1 paths already cached"4900worker2 # [ 54.946581] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409724901worker2 # [ 54.954524] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4902worker2 # [ 55.125146] nix-daemon[848]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)4903worker2 # [ 55.138946] nix-daemon[848]: reaped child process 1115, status = succeeded4904worker2 # [ 55.282530] nix-daemon[848]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)4905worker2 # [ 55.295398] nix-daemon[848]: reaped child process 1120, status = succeeded4906lb # [ 55.307725] niks3-server[1067]: 2026/09/20 09:33:34 INFO Received uploads request method=POST path=/api/pending_closures4907worker2 # [ 55.312555] nix-grpc-daemon[860]: time=2026-09-20T09:33:35.406Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4908worker2 # [ 55.314163] nix-grpc-daemon[860]: time=2026-09-20T09:33:35.408Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4909lb # [ 55.336724] niks3-server[1067]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4910lb # [ 55.348382] niks3-server[1067]: 2026/09/20 09:33:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4911lb # [ 55.354580] niks3-server[1067]: 2026/09/20 09:33:34 INFO Signed narinfos id=13 count=14912worker2 # [ 55.355040] nix-grpc-daemon[860]: time=2026-09-20T09:33:35.449Z level=INFO msg="Uploading 1 narinfos"4913lb # [ 55.359150] niks3-server[1067]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4914lb # [ 55.371766] niks3-server[1067]: 2026/09/20 09:33:34 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4915lb # [ 55.376367] niks3-server[1067]: 2026/09/20 09:33:34 INFO Completed upload id=134916worker2 # [ 55.373591] nix-grpc-daemon[860]: time=2026-09-20T09:33:35.467Z level=INFO msg="Upload complete. (406ms)"4917lb # [ 55.378944] niks3-server[1067]: 2026/09/20 09:33:34 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4918worker2 # [ 55.377129] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_in=6244919worker2 # [ 55.382791] nix-grpc-daemon[784]: ts=2026-09-20T09:33:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724920worker2 # [ 55.390559] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4921worker2 # [ 55.395529] nix-daemon[848]: reaped child process 1110, status = succeeded4922worker2 # [ 55.399544] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4923worker2 # [ 56.225884] nix-daemon[848]: reaped child process 1124, status = succeeded4924worker2 # [ 56.374219] nix-daemon[848]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)4925worker2 # [ 56.385229] nix-daemon[848]: reaped child process 1142, status = succeeded4926lb # [ 56.403576] niks3-server[1067]: 2026/09/20 09:33:35 INFO Received uploads request method=POST path=/api/pending_closures4927worker2 # [ 56.409412] nix-grpc-daemon[860]: time=2026-09-20T09:33:36.503Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4928worker2 # [ 56.411339] nix-grpc-daemon[860]: time=2026-09-20T09:33:36.505Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4929lb # [ 56.442340] niks3-server[1067]: 2026/09/20 09:33:35 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4930lb # [ 56.447748] niks3-server[1067]: 2026/09/20 09:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4931worker2 # [ 56.451234] nix-grpc-daemon[860]: time=2026-09-20T09:33:36.545Z level=INFO msg="Uploading 1 narinfos"4932lb # [ 56.453485] niks3-server[1067]: 2026/09/20 09:33:35 INFO Signed narinfos id=14 count=14933lb # [ 56.465364] niks3-server[1067]: 2026/09/20 09:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4934lb # [ 56.491772] niks3-server[1067]: 2026/09/20 09:33:35 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4935lb # [ 56.496809] niks3-server[1067]: 2026/09/20 09:33:35 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4936lb # [ 56.501624] niks3-server[1067]: 2026/09/20 09:33:35 INFO Completed upload id=144937worker2 # [ 56.501461] nix-grpc-daemon[860]: time=2026-09-20T09:33:36.595Z level=INFO msg="Upload complete. (294ms)"4938lb # [ 56.505385] niks3-server[1067]: 2026/09/20 09:33:35 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4939worker2 # [ 56.509740] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14940worker1 # [ 56.512964] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:547764941worker2 # [ 56.526225] nix-daemon[848]: reaped child process 1122, status = succeeded4942worker1 # [ 56.523635] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4943worker2 # [ 56.559179] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409724944worker2 # [ 56.564122] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_out=1764945worker2 # [ 56.807327] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:409724946worker1 # [ 56.802910] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:547764947worker1 # [ 56.834346] nix-grpc-daemon[868]: time=2026-09-20T09:33:36.992Z level=INFO msg="All 2 paths already cached"4948worker2 # [ 56.855230] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04949worker2 # [ 56.857449] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=14950worker2 # [ 56.860131] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724951worker2 # [ 56.867843] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409724952worker2 # [ 56.876988] nix-grpc-daemon[860]: time=2026-09-20T09:33:36.971Z level=INFO msg="All 2 paths already cached"4953worker2 # [ 56.881565] nix-grpc-daemon[784]: ts=2026-09-20T09:33:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409724954worker2 # [ 56.886439] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4955worker2 # [ 57.015841] nix-daemon[848]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)4956worker2 # [ 57.028355] nix-daemon[848]: reaped child process 1151, status = succeeded4957worker2 # [ 57.175093] nix-daemon[848]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)4958worker2 # [ 57.187111] nix-daemon[848]: reaped child process 1156, status = succeeded4959lb # [ 57.204944] niks3-server[1067]: 2026/09/20 09:33:36 INFO Received uploads request method=POST path=/api/pending_closures4960worker2 # [ 57.210714] nix-grpc-daemon[860]: time=2026-09-20T09:33:37.304Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4961worker2 # [ 57.212066] nix-grpc-daemon[860]: time=2026-09-20T09:33:37.304Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4962lb # [ 57.232129] niks3-server[1067]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4963lb # [ 57.241373] niks3-server[1067]: 2026/09/20 09:33:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4964lb # [ 57.247130] niks3-server[1067]: 2026/09/20 09:33:36 INFO Signed narinfos id=15 count=14965worker2 # [ 57.247708] nix-grpc-daemon[860]: time=2026-09-20T09:33:37.341Z level=INFO msg="Uploading 1 narinfos"4966lb # [ 57.252584] niks3-server[1067]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4967lb # [ 57.264267] niks3-server[1067]: 2026/09/20 09:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4968worker2 # [ 57.268079] nix-grpc-daemon[860]: time=2026-09-20T09:33:37.362Z level=INFO msg="Upload complete. (376ms)"4969lb # [ 57.270563] niks3-server[1067]: 2026/09/20 09:33:36 INFO Completed upload id=154970worker2 # [ 57.271084] nix-grpc-daemon[784]: ts=2026-09-20T09:33:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_in=7524971lb # [ 57.274425] niks3-server[1067]: 2026/09/20 09:33:36 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4972worker2 # [ 57.276378] nix-grpc-daemon[784]: ts=2026-09-20T09:33:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409724973worker2 # [ 57.278141] nix-daemon[848]: reaped child process 1146, status = succeeded4974worker2 # [ 57.281089] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4975worker2 # [ 57.287806] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4976lb # [ 57.969205] envoy[908]: conn peer=[2001:db8:1::1]:35552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1353 ms=46194977lb # [ 57.972914] envoy[908]: conn peer=[2001:db8:1::1]:35566 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=35224978lb # [ 57.976541] envoy[908]: conn peer=[2001:db8:1::1]:39218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=284979lb # [ 57.979909] envoy[908]: conn peer=[2001:db8:1::1]:39204 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3421 tx=1771 ms=17134980lb # [ 57.983666] envoy[908]: conn peer=[2001:db8:1::1]:39230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=574981lb # [ 57.987751] envoy[908]: conn peer=[2001:db8:1::1]:39228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3879 tx=1987 ms=17324982worker2 # [ 58.104938] nix-daemon[848]: reaped child process 1160, status = succeeded4983worker2 # [ 58.246739] nix-daemon[848]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)4984worker2 # [ 58.257203] nix-daemon[848]: reaped child process 1178, status = succeeded4985lb # [ 58.276072] niks3-server[1067]: 2026/09/20 09:33:37 INFO Received uploads request method=POST path=/api/pending_closures4986worker2 # [ 58.282423] nix-grpc-daemon[860]: time=2026-09-20T09:33:38.376Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4987worker2 # [ 58.283955] nix-grpc-daemon[860]: time=2026-09-20T09:33:38.378Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4988lb # [ 58.313891] niks3-server[1067]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4989lb # [ 58.324656] niks3-server[1067]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4990lb # [ 58.332583] niks3-server[1067]: 2026/09/20 09:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4991lb # [ 58.336829] niks3-server[1067]: 2026/09/20 09:33:37 INFO Signed narinfos id=16 count=14992worker2 # [ 58.337234] nix-grpc-daemon[860]: time=2026-09-20T09:33:38.431Z level=INFO msg="Uploading 1 narinfos"4993lb # [ 58.345073] niks3-server[1067]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4994lb # [ 58.354758] niks3-server[1067]: 2026/09/20 09:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4995lb # [ 58.357739] niks3-server[1067]: 2026/09/20 09:33:37 INFO Completed upload id=164996lb # [ 58.359510] niks3-server[1067]: 2026/09/20 09:33:37 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4997worker2 # [ 58.356557] nix-grpc-daemon[860]: time=2026-09-20T09:33:38.450Z level=INFO msg="Upload complete. (258ms)"4998worker2 # [ 58.363386] nix-grpc-daemon[784]: ts=2026-09-20T09:33:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14999worker2 # [ 58.373991] nix-daemon[848]: reaped child process 1158, status = succeeded5000worker1 # [ 58.399567] nix-grpc-daemon[784]: ts=2026-09-20T09:33:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:547765001worker1 # [ 58.411457] nix-grpc-daemon[784]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5002worker1 # [ 58.435778] nix-grpc-daemon[784]: ts=2026-09-20T09:33:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:547765003worker1 # [ 58.441513] nix-grpc-daemon[784]: ts=2026-09-20T09:33:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=0 paths=1 nar_bytes_out=2325004client # [ 58.516973] nix-daemon[954]: reaped child process 956, status = succeeded5005client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.13 seconds)5006client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5007client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5008client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5009client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)5010client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5011client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5012(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.22 seconds)5013subtest: an input only one worker has still reaches the builder5014client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5015client # this derivation will be built:5016client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5017client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5018client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.41 seconds)5019client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5020client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.17 seconds)5021worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5022worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.22 seconds)5023worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5024worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5025client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25026client # [ 59.631441] nix-daemon[954]: accepted connection from pid 1036, user root (trusted)5027client # this derivation will be built:5028client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5029worker2 # [ 59.830335] nix-grpc-daemon[784]: ts=2026-09-20T09:33:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:409725030client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5031worker1 # [ 59.825929] nix-grpc-daemon[784]: ts=2026-09-20T09:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:547765032worker1 # [ 60.003803] nix-daemon[843]: accepted connection from pid 946, user nix-grpc-daemon (trusted)5033worker1 # [ 60.015823] nix-daemon[843]: reaped child process 949, status = succeeded5034lb # [ 60.042976] niks3-server[1067]: 2026/09/20 09:33:39 INFO Received uploads request method=POST path=/api/pending_closures5035worker1 # [ 60.040369] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.199Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5036worker1 # [ 60.042322] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.201Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5037lb # [ 60.068586] niks3-server[1067]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5038lb # [ 60.081584] niks3-server[1067]: 2026/09/20 09:33:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5039worker1 # [ 60.077641] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.236Z level=INFO msg="Uploading 1 narinfos"5040lb # [ 60.087626] niks3-server[1067]: 2026/09/20 09:33:39 INFO Signed narinfos id=17 count=15041lb # [ 60.092276] niks3-server[1067]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5042lb # [ 60.105346] niks3-server[1067]: 2026/09/20 09:33:39 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5043worker1 # [ 60.101188] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.260Z level=INFO msg="Upload complete. (270ms)"5044lb # [ 60.111345] niks3-server[1067]: 2026/09/20 09:33:39 INFO Completed upload id=175045lb # [ 60.114641] niks3-server[1067]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5046worker2 # [ 60.123236] nix-grpc-daemon[784]: ts=2026-09-20T09:33:40Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05047worker2 # [ 60.125302] nix-grpc-daemon[784]: ts=2026-09-20T09:33:40Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15048worker1 # [ 60.159559] nix-grpc-daemon[784]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547765049worker1 # [ 60.171753] nix-grpc-daemon[784]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:547765050client # copying 2 paths to 'lb:50051'...5051client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5052worker1 # [ 60.186389] nix-grpc-daemon[784]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:547765053client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5054worker1 # [ 60.197155] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5055worker1 # [ 60.370487] nix-daemon[843]: accepted connection from pid 953, user nix-grpc-daemon (trusted)5056worker1 # [ 60.383158] nix-daemon[843]: reaped child process 956, status = succeeded5057worker1 # [ 60.553038] nix-daemon[843]: accepted connection from pid 958, user nix-grpc-daemon (trusted)5058worker1 # [ 60.564714] nix-daemon[843]: reaped child process 961, status = succeeded5059lb # [ 60.591884] niks3-server[1067]: 2026/09/20 09:33:39 INFO Received uploads request method=POST path=/api/pending_closures5060lb # [ 60.600666] niks3-server[1067]: 2026/09/20 09:33:39 INFO Received uploads request method=POST path=/api/pending_closures5061worker1 # [ 60.599186] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.757Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5062worker1 # [ 60.600583] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.757Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5063worker1 # [ 60.602134] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.757Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5064lb # [ 60.636178] niks3-server[1067]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5065lb # [ 60.644069] niks3-server[1067]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5066lb # [ 60.660397] niks3-server[1067]: 2026/09/20 09:33:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5067lb # [ 60.663280] niks3-server[1067]: 2026/09/20 09:33:39 INFO Signed narinfos id=19 count=25068lb # [ 60.664764] niks3-server[1067]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5069lb # [ 60.667197] niks3-server[1067]: 2026/09/20 09:33:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5070lb # [ 60.669230] niks3-server[1067]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5071lb # [ 60.671260] niks3-server[1067]: 2026/09/20 09:33:39 INFO Signed narinfos id=18 count=15072worker1 # [ 60.661693] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.820Z level=INFO msg="Uploading 3 narinfos"5073lb # [ 60.680639] niks3-server[1067]: 2026/09/20 09:33:39 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5074lb # [ 60.687940] niks3-server[1067]: 2026/09/20 09:33:40 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5075lb # [ 60.690289] niks3-server[1067]: 2026/09/20 09:33:40 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5076lb # [ 60.692791] niks3-server[1067]: 2026/09/20 09:33:40 INFO Completed upload id=185077lb # [ 60.694583] niks3-server[1067]: 2026/09/20 09:33:40 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5078lb # [ 60.696784] niks3-server[1067]: 2026/09/20 09:33:40 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5079lb # [ 60.699153] niks3-server[1067]: 2026/09/20 09:33:40 INFO Completed upload id=195080worker1 # [ 60.689021] nix-grpc-daemon[868]: time=2026-09-20T09:33:40.848Z level=INFO msg="Upload complete. (477ms)"5081worker1 # [ 60.690979] nix-grpc-daemon[784]: ts=2026-09-20T09:33:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=0 paths=2 nar_bytes_in=9765082worker1 # [ 60.693413] nix-grpc-daemon[784]: ts=2026-09-20T09:33:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547765083worker1 # [ 60.699754] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5084worker1 # [ 60.702879] nix-daemon[843]: reaped child process 951, status = succeeded5085client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5086worker1 # [ 60.707770] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5087client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5088worker1 # [ 61.119640] nix-daemon[843]: reaped child process 965, status = succeeded5089worker1 # [ 61.274710] nix-daemon[843]: accepted connection from pid 980, user nix-grpc-daemon (trusted)5090worker1 # [ 61.285377] nix-daemon[843]: reaped child process 983, status = succeeded5091lb # [ 61.312501] niks3-server[1067]: 2026/09/20 09:33:40 INFO Received uploads request method=POST path=/api/pending_closures5092worker1 # [ 61.311266] nix-grpc-daemon[868]: time=2026-09-20T09:33:41.470Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5093lb # [ 61.343077] niks3-server[1067]: 2026/09/20 09:33:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5094worker1 # [ 61.340785] nix-grpc-daemon[868]: time=2026-09-20T09:33:41.499Z level=INFO msg="Uploading 1 narinfos"5095lb # [ 61.351240] niks3-server[1067]: 2026/09/20 09:33:40 INFO Signed narinfos id=20 count=15096lb # [ 61.356628] niks3-server[1067]: 2026/09/20 09:33:40 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5097lb # [ 61.362584] niks3-server[1067]: 2026/09/20 09:33:40 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5098lb # [ 61.374670] niks3-server[1067]: 2026/09/20 09:33:40 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5099lb # [ 61.378964] niks3-server[1067]: 2026/09/20 09:33:40 INFO Completed upload id=205100worker1 # [ 61.371287] nix-grpc-daemon[868]: time=2026-09-20T09:33:41.530Z level=INFO msg="Upload complete. (258ms)"5101lb # [ 61.382700] niks3-server[1067]: 2026/09/20 09:33:40 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5102worker1 # [ 61.375458] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=0 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15103client # copying 1 paths...5104worker2 # [ 61.393327] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:409725105worker1 # [ 61.386120] nix-daemon[843]: reaped child process 963, status = succeeded5106worker2 # [ 61.401695] nix-grpc-daemon[784]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5107client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5108worker1 # [ 61.421187] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:547765109worker1 # [ 61.425752] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=0 paths=1 nar_bytes_out=1205110client # [ 61.473935] nix-daemon[954]: reaped child process 1039, status = succeeded5111client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.05 seconds)5112client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25113client # [ 61.664953] nix-daemon[954]: accepted connection from pid 1058, user root (trusted)5114client # this derivation will be built:5115client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5116worker1 # [ 61.867919] nix-grpc-daemon[784]: ts=2026-09-20T09:33:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:547765117client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5118worker2 # [ 61.878880] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409725119worker2 # [ 61.900254] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05120worker2 # [ 61.904510] nix-grpc-daemon[784]: ts=2026-09-20T09:33:41Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15121worker2 # [ 61.952731] nix-grpc-daemon[784]: ts=2026-09-20T09:33:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409725122worker2 # [ 61.964534] nix-grpc-daemon[784]: ts=2026-09-20T09:33:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409725123client # copying 1 paths to 'lb:50051'...5124client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5125worker2 # [ 61.981977] nix-grpc-daemon[784]: ts=2026-09-20T09:33:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409725126worker2 # [ 61.993104] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5127worker2 # [ 62.008145] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)5128worker2 # [ 62.057908] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 2/5)5129worker2 # [ 62.229410] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 151 ms (attempt 3/5)5130worker2 # [ 62.380861] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 4/5)5131worker2 # [ 62.775647] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5132worker2 # [ 62.781670] nix-grpc-daemon[784]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5133worker2 # [ 62.971374] nix-daemon[848]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)5134worker2 # [ 62.985086] nix-daemon[848]: reaped child process 1197, status = succeeded5135worker2 # [ 63.132403] nix-daemon[848]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)5136worker2 # [ 63.145339] nix-daemon[848]: reaped child process 1202, status = succeeded5137lb # [ 63.162669] niks3-server[1067]: 2026/09/20 09:33:42 INFO Received uploads request method=POST path=/api/pending_closures5138worker2 # [ 63.169463] nix-grpc-daemon[860]: time=2026-09-20T09:33:43.263Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5139worker2 # [ 63.171123] nix-grpc-daemon[860]: time=2026-09-20T09:33:43.263Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5140lb # [ 63.187566] niks3-server[1067]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5141lb # [ 63.197178] niks3-server[1067]: 2026/09/20 09:33:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5142lb # [ 63.201968] niks3-server[1067]: 2026/09/20 09:33:42 INFO Signed narinfos id=21 count=15143worker2 # [ 63.202420] nix-grpc-daemon[860]: time=2026-09-20T09:33:43.296Z level=INFO msg="Uploading 1 narinfos"5144lb # [ 63.213123] niks3-server[1067]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5145lb # [ 63.218997] niks3-server[1067]: 2026/09/20 09:33:42 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5146worker2 # [ 63.222136] nix-grpc-daemon[860]: time=2026-09-20T09:33:43.316Z level=INFO msg="Upload complete. (422ms)"5147lb # [ 63.224700] niks3-server[1067]: 2026/09/20 09:33:42 INFO Completed upload id=215148worker2 # [ 63.224194] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 paths=1 nar_bytes_in=5605149lb # [ 63.229762] niks3-server[1067]: 2026/09/20 09:33:42 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5150worker2 # [ 63.230722] nix-daemon[848]: reaped child process 1188, status = succeeded5151worker2 # [ 63.232751] nix-grpc-daemon[784]: ts=2026-09-20T09:33:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409725152worker2 # [ 63.237153] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5153worker2 # [ 63.241452] nix-grpc-daemon[784]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5154worker2 # [ 63.250348] nix-daemon[848]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5155client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5156client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5157worker2 # [ 64.474544] nix-daemon[848]: reaped child process 1207, status = succeeded5158worker2 # [ 64.615946] nix-daemon[848]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5159worker2 # [ 64.628117] nix-daemon[848]: reaped child process 1225, status = succeeded5160lb # [ 64.646167] niks3-server[1067]: 2026/09/20 09:33:43 INFO Received uploads request method=POST path=/api/pending_closures5161worker2 # [ 64.651223] nix-grpc-daemon[860]: time=2026-09-20T09:33:44.745Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5162lb # [ 64.675215] niks3-server[1067]: 2026/09/20 09:33:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5163lb # [ 64.680266] niks3-server[1067]: 2026/09/20 09:33:43 INFO Signed narinfos id=22 count=15164worker2 # [ 64.680750] nix-grpc-daemon[860]: time=2026-09-20T09:33:44.774Z level=INFO msg="Uploading 1 narinfos"5165lb # [ 64.689149] niks3-server[1067]: 2026/09/20 09:33:44 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5166lb # [ 64.700577] niks3-server[1067]: 2026/09/20 09:33:44 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5167lb # [ 64.705849] niks3-server[1067]: 2026/09/20 09:33:44 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5168lb # [ 64.710443] niks3-server[1067]: 2026/09/20 09:33:44 INFO Completed upload id=225169worker2 # [ 64.707578] nix-grpc-daemon[860]: time=2026-09-20T09:33:44.801Z level=WARN msg="Failed to register uploaded object" key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo error="server returned 404: object is not pending upload\n"5170worker2 # [ 64.713680] nix-grpc-daemon[860]: time=2026-09-20T09:33:44.805Z level=INFO msg="Upload complete. (253ms)"5171client # copying 1 paths...5172worker2 # [ 64.717412] nix-grpc-daemon[784]: ts=2026-09-20T09:33:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15173worker1 # [ 64.714105] nix-grpc-daemon[784]: ts=2026-09-20T09:33:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:547765174worker2 # [ 64.727430] nix-daemon[848]: reaped child process 1204, status = succeeded5175worker1 # [ 64.723208] nix-grpc-daemon[784]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5176client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5177worker2 # [ 64.754545] nix-grpc-daemon[784]: ts=2026-09-20T09:33:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409725178worker2 # [ 64.759624] nix-grpc-daemon[784]: ts=2026-09-20T09:33:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_out=1205179client # [ 64.787450] nix-daemon[954]: reaped child process 1061, status = succeeded5180client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.31 seconds)5181client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25182client # [ 64.964512] nix-daemon[954]: accepted connection from pid 1080, user root (trusted)5183client # this derivation will be built:5184client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5185worker1 # [ 65.160063] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:547765186client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5187worker2 # [ 65.175943] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409725188worker2 # [ 65.185330] nix-grpc-daemon[860]: time=2026-09-20T09:33:45.279Z level=INFO msg="All 1 paths already cached"5189worker2 # [ 65.199170] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05190worker2 # [ 65.203827] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15191worker1 # [ 65.198452] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547765192worker1 # [ 65.210292] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:547765193client # copying 1 paths to 'lb:50051'...5194worker1 # [ 65.223756] nix-grpc-daemon[868]: time=2026-09-20T09:33:45.381Z level=INFO msg="All 1 paths already cached"5195client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5196worker1 # [ 65.227846] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:547765197worker1 # [ 65.238762] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5198worker1 # [ 65.414152] nix-daemon[843]: accepted connection from pid 988, user nix-grpc-daemon (trusted)5199worker1 # [ 65.426191] nix-daemon[843]: reaped child process 991, status = succeeded5200worker1 # [ 65.612107] nix-daemon[843]: accepted connection from pid 993, user nix-grpc-daemon (trusted)5201worker1 # [ 65.625985] nix-daemon[843]: reaped child process 996, status = succeeded5202lb # [ 65.652881] niks3-server[1067]: 2026/09/20 09:33:44 INFO Received uploads request method=POST path=/api/pending_closures5203worker1 # [ 65.650222] nix-grpc-daemon[868]: time=2026-09-20T09:33:45.809Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5204worker1 # [ 65.651735] nix-grpc-daemon[868]: time=2026-09-20T09:33:45.810Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5205lb # [ 65.680153] niks3-server[1067]: 2026/09/20 09:33:44 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5206lb # [ 65.691230] niks3-server[1067]: 2026/09/20 09:33:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5207lb # [ 65.695634] niks3-server[1067]: 2026/09/20 09:33:45 INFO Signed narinfos id=23 count=15208worker1 # [ 65.688585] nix-grpc-daemon[868]: time=2026-09-20T09:33:45.847Z level=INFO msg="Uploading 1 narinfos"5209lb # [ 65.705609] niks3-server[1067]: 2026/09/20 09:33:45 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5210lb # [ 65.715051] niks3-server[1067]: 2026/09/20 09:33:45 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5211lb # [ 65.721532] niks3-server[1067]: 2026/09/20 09:33:45 INFO Completed upload id=235212worker1 # [ 65.711252] nix-grpc-daemon[868]: time=2026-09-20T09:33:45.870Z level=INFO msg="Upload complete. (461ms)"5213lb # [ 65.724340] niks3-server[1067]: 2026/09/20 09:33:45 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5214worker1 # [ 65.716375] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=0 paths=1 nar_bytes_in=5605215worker1 # [ 65.721258] nix-grpc-daemon[784]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:547765216worker1 # [ 65.730451] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5217worker1 # [ 65.733981] nix-daemon[843]: reaped child process 986, status = succeeded5218worker1 # [ 65.739434] nix-daemon[843]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5219client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5220client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5221worker1 # [ 66.956492] nix-daemon[843]: reaped child process 1000, status = succeeded5222worker1 # [ 67.082089] nix-daemon[843]: accepted connection from pid 1015, user nix-grpc-daemon (trusted)5223worker1 # [ 67.092893] nix-daemon[843]: reaped child process 1018, status = succeeded5224lb # [ 67.119693] niks3-server[1067]: 2026/09/20 09:33:46 INFO Received uploads request method=POST path=/api/pending_closures5225worker1 # [ 67.118954] nix-grpc-daemon[868]: time=2026-09-20T09:33:47.277Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5226lb # [ 67.142561] niks3-server[1067]: 2026/09/20 09:33:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5227lb # [ 67.146180] niks3-server[1067]: 2026/09/20 09:33:46 INFO Signed narinfos id=24 count=15228worker1 # [ 67.136200] nix-grpc-daemon[868]: time=2026-09-20T09:33:47.295Z level=INFO msg="Uploading 1 narinfos"5229lb # [ 67.149560] niks3-server[1067]: 2026/09/20 09:33:46 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5230lb # [ 67.152886] niks3-server[1067]: 2026/09/20 09:33:46 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5231lb # [ 67.157353] niks3-server[1067]: 2026/09/20 09:33:46 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5232lb # [ 67.160126] niks3-server[1067]: 2026/09/20 09:33:46 INFO Completed upload id=245233worker1 # [ 67.151310] nix-grpc-daemon[868]: time=2026-09-20T09:33:47.310Z level=INFO msg="Upload complete. (203ms)"5234lb # [ 67.163142] niks3-server[1067]: 2026/09/20 09:33:46 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5235worker1 # [ 67.153903] nix-grpc-daemon[784]: ts=2026-09-20T09:33:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54776 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15236worker1 # [ 67.160693] nix-daemon[843]: reaped child process 998, status = succeeded5237client # copying 1 paths...5238worker1 # [ 67.199670] nix-grpc-daemon[784]: ts=2026-09-20T09:33:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:547765239client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5240worker2 # [ 67.229519] nix-grpc-daemon[784]: ts=2026-09-20T09:33:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409725241worker2 # [ 67.240440] nix-grpc-daemon[784]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5242worker2 # [ 67.253623] nix-grpc-daemon[784]: ts=2026-09-20T09:33:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40972 duration_s=0 paths=1 nar_bytes_out=1205243client # [ 67.334961] nix-daemon[954]: reaped child process 1083, status = succeeded5244client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.55 seconds)5245(finished: subtest: an input only one worker has still reaches the builder, in 8.73 seconds)5246subtest: upload whose reference the worker lacks is completed from the cache5247client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5248client # this derivation will be built:5249client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5250client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5251client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.38 seconds)5252client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5253client # this derivation will be built:5254client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5255lb # [ 67.970378] envoy[908]: conn peer=[2001:db8:1::1]:39236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4314 tx=2220 ms=16925256lb # [ 67.974384] envoy[908]: conn peer=[2001:db8:1::1]:39244 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=555257lb # [ 67.977926] envoy[908]: conn peer=[2001:db8:1::1]:39246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3832 tx=1859 ms=16205258lb # [ 67.981718] envoy[908]: conn peer=[2001:db8:1::1]:39254 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=215259lb # [ 67.985462] envoy[908]: conn peer=[2001:db8:1::1]:39264 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3684 tx=1916 ms=28935260lb # [ 67.989706] envoy[908]: conn peer=[2001:db8:1::1]:38656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=135261lb # [ 67.993323] envoy[908]: conn peer=[2001:db8:1::1]:38682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=775262lb # [ 67.996892] envoy[908]: conn peer=[2001:db8:1::1]:38672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3712 tx=1777 ms=21395263client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5264client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5265client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5266client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.17 seconds)5267worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5268worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.21 seconds)5269worker2 # [ 68.566257] systemd[1]: Stopping Nix worker-protocol over gRPC...5270worker2 # [ 69.235883] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=signal_exit5271worker2 # [ 69.490642] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=scheduler_yield5272worker1 # [ 69.486159] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=scheduler_restarting addr=lb:500515273lb # [ 69.498649] niks3-server[1067]: 2026/09/20 09:33:48 INFO lead: released remote=192.168.1.4:378465274worker2 # [ 69.496953] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:390685275worker2 # [ 69.502690] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=worker_gone addr=192.168.1.3:500515276worker2 # [ 69.505673] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=scheduler_restarting addr=lb:500515277worker2 # [ 69.508673] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:390685278worker2 # [ 69.512442] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=worker_gone addr=192.168.1.4:500515279worker1 # [ 69.685934] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=scheduler_connected addr=lb:500515280worker1 # [ 69.689974] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5281worker2 # [ 69.702425] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=scheduler_connected addr=lb:500515282worker2 # [ 69.706169] nix-grpc-daemon[784]: ts=2026-09-20T09:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker1 # [ 69.890746] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_connected addr=lb:500515284worker1 # [ 69.894731] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5285worker2 # [ 69.907169] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_connected addr=lb:500515286worker2 # [ 69.911145] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5287worker1 # [ 70.095100] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_connected addr=lb:500515288worker1 # [ 70.097068] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5289worker2 # [ 70.111884] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_connected addr=lb:500515290worker2 # [ 70.116134] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5291worker1 # [ 70.297836] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_connected addr=lb:500515292worker1 # [ 70.301764] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5293worker2 # [ 70.318183] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_connected addr=lb:500515294worker2 # [ 70.321311] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5295worker2 # [ 70.494499] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_restarting5296worker1 # [ 70.503489] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_connected addr=lb:500515297worker1 # [ 70.507065] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5298worker2 # [ 70.517375] nix-daemon[848]: reaped child process 850, status = succeeded5299worker2 # [ 70.534671] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5300worker1 # [ 70.708074] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_connected addr=lb:500515301worker1 # [ 70.712268] nix-grpc-daemon[784]: ts=2026-09-20T09:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5302worker1 # [ 70.912812] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_connected addr=lb:500515303worker1 # [ 70.916617] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker1 # [ 71.117450] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_connected addr=lb:500515305worker1 # [ 71.121334] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5306worker1 # [ 71.322120] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_connected addr=lb:500515307worker1 # [ 71.326445] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5308worker2 # [ 71.377455] nix-grpc-daemon[784]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5309worker2 # [ 71.380885] nix-grpc-daemon[784]: E0000 00:00:1789896831.470554 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5310worker2 # [ 71.385441] nix-grpc-daemon[784]: E0000 00:00:1789896831.471008 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5311worker1 # [ 71.527861] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_connected addr=lb:500515312worker1 # [ 71.531540] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker1 # [ 71.732421] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_connected addr=lb:500515314worker1 # [ 71.736239] nix-grpc-daemon[784]: ts=2026-09-20T09:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5315worker1 # [ 71.937126] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=scheduler_connected addr=lb:500515316worker1 # [ 71.941134] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker1 # [ 72.026929] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=scheduler_take_over5318lb # [ 72.037502] niks3-server[1067]: 2026/09/20 09:33:51 INFO lead: acquired remote=192.168.1.3:445945319worker1 # [ 72.141789] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=scheduler_connected addr=lb:500515320worker1 # [ 72.145646] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5321worker1 # [ 72.346444] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=scheduler_connected addr=lb:500515322worker1 # [ 72.350301] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker2 # [ 72.471481] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5324worker2 # [ 72.475952] systemd[1]: Stopped Nix worker-protocol over gRPC.5325worker2 # [ 72.479607] systemd[1]: nix-grpc-daemon.service: Consumed 2.592s CPU time over 54.507s wall clock time, 64M memory peak, 90.1K incoming IP traffic, 178.4K outgoing IP traffic.5326worker2 # [ 72.489737] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5327worker2 # [ 72.493865] systemd[1]: Closed Nix worker-protocol over gRPC.5328worker2 # [ 72.497396] systemd[1]: nix-grpc-daemon.socket: Consumed 280us CPU time over 1min 3.157s wall clock time, 256K memory peak, 88.1K incoming IP traffic, 67.4K outgoing IP traffic.5329lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5330lb: must succeed: curl -sf localhost:9901/clusters5331worker1 # [ 72.550889] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=scheduler_connected addr=lb:500515332worker1 # [ 72.558122] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5334lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5335client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25336worker1 # [ 72.761411] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:547765337worker1 # [ 72.764929] nix-grpc-daemon[784]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:547765338client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5339client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.23 seconds)5340worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5341worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5342worker2 # [ 72.879643] systemd[1]: Listening on Nix worker-protocol over gRPC.5343worker1 # [ 72.930958] systemd[1]: Stopping Nix worker-protocol over gRPC...5344worker1 # [ 73.226478] nix-grpc-daemon[784]: ts=2026-09-20T09:33:53Z level=info event=signal_exit5345worker2 # [ 73.463887] systemd[1]: Starting Nix worker-protocol over gRPC...5346worker1 # [ 73.488288] nix-grpc-daemon[784]: ts=2026-09-20T09:33:53Z level=info event=scheduler_yield5347lb # [ 73.500859] niks3-server[1067]: 2026/09/20 09:33:52 INFO lead: released remote=192.168.1.3:445945348worker1 # [ 73.494891] nix-grpc-daemon[784]: ts=2026-09-20T09:33:53Z level=info event=scheduler_restarting addr=lb:500515349worker1 # [ 73.499126] nix-grpc-daemon[784]: ts=2026-09-20T09:33:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:584905350worker1 # [ 73.502369] nix-grpc-daemon[784]: ts=2026-09-20T09:33:53Z level=info event=worker_gone addr=192.168.1.3:500515351worker2 # [ 73.644355] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5352worker2 # [ 73.647966] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15353worker2 # [ 73.655454] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5354worker2 # [ 73.659300] systemd[1]: Started Nix worker-protocol over gRPC.5355lb # [ 73.667916] niks3-server[1067]: 2026/09/20 09:33:52 INFO lead: acquired remote=192.168.1.4:529865356worker2 # [ 73.665402] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:53Z level=info event=scheduler_take_over5357worker2 # [ 73.687776] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:53Z level=info event=scheduler_connected addr=lb:500515358worker2 # [ 73.689416] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5359worker1 # [ 73.700476] nix-grpc-daemon[784]: ts=2026-09-20T09:33:53Z level=info event=scheduler_connected addr=lb:500515360worker1 # [ 73.702248] nix-grpc-daemon[784]: ts=2026-09-20T09:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5361worker2 # [ 73.890580] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:53Z level=info event=scheduler_connected addr=lb:500515362worker2 # [ 73.894456] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5363worker1 # [ 73.900944] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=scheduler_connected addr=lb:500515364worker1 # [ 73.904982] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5365worker2 # [ 74.095422] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:54Z level=info event=scheduler_connected addr=lb:500515366worker2 # [ 74.099623] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367worker1 # [ 74.105356] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=scheduler_connected addr=lb:500515368worker1 # [ 74.109284] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5369worker2 # [ 74.299912] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:54Z level=info event=scheduler_connected addr=lb:500515370worker2 # [ 74.301683] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5371worker1 # [ 74.310088] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=scheduler_connected addr=lb:500515372worker1 # [ 74.313961] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5373worker1 # [ 74.492322] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=scheduler_restarting5374worker2 # [ 74.502715] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:54Z level=info event=scheduler_connected addr=lb:500515375worker2 # [ 74.507492] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker1 # [ 74.526905] nix-grpc-daemon[784]: ts=2026-09-20T09:33:54Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5377worker1 # [ 74.531428] nix-daemon[843]: reaped child process 845, status = succeeded5378worker2 # [ 74.707829] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:54Z level=info event=scheduler_connected addr=lb:500515379worker2 # [ 74.711515] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05380worker1 # [ 75.377201] nix-grpc-daemon[784]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5381worker1 # [ 75.380521] nix-grpc-daemon[784]: E0000 00:00:1789896835.534930 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5382worker1 # [ 75.385200] nix-grpc-daemon[784]: E0000 00:00:1789896835.535503 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5383worker1 # [ 76.421594] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5384worker1 # [ 76.423300] systemd[1]: Stopped Nix worker-protocol over gRPC.5385worker1 # [ 76.424995] systemd[1]: nix-grpc-daemon.service: Consumed 1.458s CPU time over 58.473s wall clock time, 50.4M memory peak, 57.3K incoming IP traffic, 106.6K outgoing IP traffic.5386worker1 # [ 76.429467] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5387worker1 # [ 76.431358] systemd[1]: Closed Nix worker-protocol over gRPC.5388worker1 # [ 76.432776] systemd[1]: nix-grpc-daemon.socket: Consumed 285us CPU time over 1min 7.114s wall clock time, 256K memory peak, 63.9K incoming IP traffic, 58.9K outgoing IP traffic.5389lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5390lb: must succeed: curl -sf localhost:9901/clusters5391lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5392lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5393client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25394worker2 # [ 76.731578] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604285395worker2 # [ 76.742943] nix-daemon[848]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5396worker2 # [ 76.759897] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 1/5)5397worker2 # [ 76.804693] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 2/5)5398worker2 # [ 76.825424] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 272 ms (attempt 3/5)5399worker2 # [ 77.103392] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 596 ms (attempt 4/5)5400worker2 # [ 77.705866] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5401client # copying 2 paths...5402client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5403worker2 # [ 77.724668] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604285404client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5405worker2 # [ 77.737200] nix-daemon[848]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5406worker2 # [ 77.760494] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:57Z level=info event=niks3_push_started pid=13055407lb # [ 77.971728] envoy[908]: conn peer=192.168.1.4:59662 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=14915 tx=3104 ms=512975408lb # [ 77.976176] envoy[908]: conn peer=[2001:db8:1::1]:38690 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=155409lb # [ 77.979644] envoy[908]: conn peer=[2001:db8:1::3]:39248 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=17355 tx=3237 ms=553745410worker2 # [ 78.100197] nix-daemon[848]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5411worker2 # [ 78.110766] nix-daemon[848]: reaped child process 1321, status = succeeded5412lb # [ 78.131591] niks3-server[1067]: 2026/09/20 09:33:57 INFO Received uploads request method=POST path=/api/pending_closures5413lb # [ 78.140677] niks3-server[1067]: 2026/09/20 09:33:57 INFO Received uploads request method=POST path=/api/pending_closures5414worker2 # [ 78.146062] nix-grpc-daemon[1305]: time=2026-09-20T09:33:58.240Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5415worker2 # [ 78.149146] nix-grpc-daemon[1305]: time=2026-09-20T09:33:58.242Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5416worker2 # [ 78.150824] nix-grpc-daemon[1305]: time=2026-09-20T09:33:58.242Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5417lb # [ 78.181341] niks3-server[1067]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5418lb # [ 78.200348] niks3-server[1067]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5419lb # [ 78.207547] niks3-server[1067]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5420lb # [ 78.215163] niks3-server[1067]: 2026/09/20 09:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5421lb # [ 78.223327] niks3-server[1067]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5422lb # [ 78.227083] niks3-server[1067]: 2026/09/20 09:33:57 INFO Signed narinfos id=25 count=15423lb # [ 78.228567] niks3-server[1067]: 2026/09/20 09:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5424lb # [ 78.230417] niks3-server[1067]: 2026/09/20 09:33:57 INFO Signed narinfos id=26 count=25425worker2 # [ 78.230214] nix-grpc-daemon[1305]: time=2026-09-20T09:33:58.323Z level=INFO msg="Uploading 3 narinfos"5426lb # [ 78.251948] niks3-server[1067]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5427lb # [ 78.253882] niks3-server[1067]: 2026/09/20 09:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5428lb # [ 78.255940] niks3-server[1067]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5429lb # [ 78.261854] niks3-server[1067]: 2026/09/20 09:33:57 INFO Completed upload id=255430lb # [ 78.263146] niks3-server[1067]: 2026/09/20 09:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5431lb # [ 78.265408] niks3-server[1067]: 2026/09/20 09:33:57 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5432lb # [ 78.268089] niks3-server[1067]: 2026/09/20 09:33:57 INFO Completed upload id=265433worker2 # [ 78.265889] nix-grpc-daemon[1305]: time=2026-09-20T09:33:58.360Z level=INFO msg="Upload complete. (321ms)"5434worker2 # [ 78.270499] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60428 duration_s=0 paths=2 nar_bytes_in=3125435worker2 # [ 78.283147] nix-daemon[848]: reaped child process 1303, status = succeeded5436client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.80 seconds)5437worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5438worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5439worker1 # [ 78.368108] systemd[1]: Listening on Nix worker-protocol over gRPC.5440worker1 # [ 78.372585] systemd[1]: Starting Nix worker-protocol over gRPC...5441worker1 # [ 78.515462] nix-grpc-daemon[1041]: ts=2026-09-20T09:33:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5442worker1 # [ 78.518676] nix-grpc-daemon[1041]: ts=2026-09-20T09:33:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15443worker1 # [ 78.525776] nix-grpc-daemon[1041]: ts=2026-09-20T09:33:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5444worker1 # [ 78.527379] systemd[1]: Started Nix worker-protocol over gRPC.5445lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5446lb: must succeed: curl -sf localhost:9901/clusters5447worker1 # [ 78.557764] nix-grpc-daemon[1041]: ts=2026-09-20T09:33:58Z level=info event=scheduler_connected addr=lb:500515448worker2 # [ 78.567895] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05449lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5450lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5451(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.23 seconds)5452subtest: scheduler down: the other node takes the lock and keeps it5453lb: must succeed: curl -sf localhost:9901/clusters5454lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5455lb: must succeed: curl -sf localhost:9901/clusters5456lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5457lb: must succeed: curl -sf localhost:9901/clusters5458lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5459lb: must succeed: curl -sf localhost:9901/clusters5460lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5461worker2 # [ 78.805927] systemd[1]: Stopping Nix worker-protocol over gRPC...5462worker2 # [ 79.663781] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=signal_exit5463worker2 # [ 79.673165] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=scheduler_yield5464lb # [ 79.678475] niks3-server[1067]: 2026/09/20 09:33:58 INFO lead: released remote=192.168.1.4:529865465worker1 # [ 79.668758] nix-grpc-daemon[1041]: ts=2026-09-20T09:33:59Z level=info event=scheduler_restarting addr=lb:500515466worker2 # [ 79.677800] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:604225467worker2 # [ 79.685749] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=worker_gone addr=192.168.1.3:500515468worker2 # [ 79.688971] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=scheduler_restarting addr=lb:500515469worker2 # [ 79.691883] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:604225470worker2 # [ 79.695065] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=worker_gone addr=192.168.1.4:500515471worker1 # [ 79.869094] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_connected addr=lb:500515472worker1 # [ 79.873418] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5473worker2 # [ 79.886133] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=scheduler_connected addr=lb:500515474worker2 # [ 79.889832] nix-grpc-daemon[1249]: ts=2026-09-20T09:33:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5475worker1 # [ 80.073880] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_connected addr=lb:500515476worker1 # [ 80.077751] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5477worker2 # [ 80.090652] nix-grpc-daemon[1249]: ts=2026-09-20T09:34:00Z level=info event=scheduler_connected addr=lb:500515478worker2 # [ 80.094419] nix-grpc-daemon[1249]: ts=2026-09-20T09:34:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5479worker1 # [ 80.278526] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_connected addr=lb:500515480worker1 # [ 80.282307] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5481worker2 # [ 80.294739] nix-grpc-daemon[1249]: ts=2026-09-20T09:34:00Z level=info event=scheduler_connected addr=lb:500515482worker2 # [ 80.296635] nix-grpc-daemon[1249]: ts=2026-09-20T09:34:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5483worker1 # [ 80.483205] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_connected addr=lb:500515484worker1 # [ 80.487275] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5485worker2 # [ 80.497653] nix-grpc-daemon[1249]: ts=2026-09-20T09:34:00Z level=info event=scheduler_connected addr=lb:500515486worker2 # [ 80.501572] nix-grpc-daemon[1249]: ts=2026-09-20T09:34:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5487worker2 # [ 80.680777] nix-grpc-daemon[1249]: ts=2026-09-20T09:34:00Z level=info event=scheduler_restarting5488worker2 # [ 80.685345] nix-grpc-daemon[1249]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5489worker2 # [ 80.688687] nix-grpc-daemon[1249]: E0000 00:00:1789896840.778224 1292 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5490worker2 # [ 80.693451] nix-grpc-daemon[1249]: E0000 00:00:1789896840.778328 1292 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5491worker1 # [ 80.689574] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_connected addr=lb:500515492worker1 # [ 80.692814] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5493worker2 # [ 80.715203] nix-grpc-daemon[1249]: ts=2026-09-20T09:34:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5494worker2 # [ 80.726303] nix-daemon[848]: reaped child process 1298, status = succeeded5495worker1 # [ 80.890595] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_connected addr=lb:500515496worker1 # [ 80.894598] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker1 # [ 81.095373] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_connected addr=lb:500515498worker1 # [ 81.099279] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5499worker1 # [ 81.300071] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_connected addr=lb:500515500worker1 # [ 81.304159] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5501worker1 # [ 81.504867] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_connected addr=lb:500515502worker1 # [ 81.508889] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5503worker2 # [ 81.681761] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5504worker2 # [ 81.686691] systemd[1]: Stopped Nix worker-protocol over gRPC.5505worker2 # [ 81.689363] systemd[1]: nix-grpc-daemon.service: Consumed 360ms CPU time over 8.217s wall clock time, 32.6M memory peak, 18.1K incoming IP traffic, 24.2K outgoing IP traffic.5506worker2 # [ 81.700224] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5507worker2 # [ 81.704568] systemd[1]: Closed Nix worker-protocol over gRPC.5508worker2 # [ 81.708414] systemd[1]: nix-grpc-daemon.socket: Consumed 278us CPU time over 8.828s wall clock time, 256K memory peak, 19.1K incoming IP traffic, 14.7K outgoing IP traffic.5509worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5510worker1 # [ 81.710570] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_connected addr=lb:500515511worker1 # [ 81.714283] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5512worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5513worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5514worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5515worker1 # [ 81.912110] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515516worker1 # [ 81.916184] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5517worker1 # [ 82.116482] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515518worker1 # [ 82.118308] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5519worker1 # [ 82.319359] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515520worker1 # [ 82.323628] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5521worker1 # [ 82.524099] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515522worker1 # [ 82.526486] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5523worker1 # [ 82.728252] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_connected addr=lb:500515524worker1 # [ 82.731992] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5525worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5526worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5527worker1 # [ 82.932739] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515528worker1 # [ 82.934824] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5529worker1 # [ 83.135378] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515530worker1 # [ 83.137230] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5531worker1 # [ 83.337680] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515532worker1 # [ 83.339520] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5533worker1 # [ 83.539110] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_take_over5534worker1 # [ 83.540939] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515535lb # [ 83.549759] niks3-server[1067]: 2026/09/20 09:34:02 INFO lead: acquired remote=192.168.1.3:597485536worker1 # [ 83.543735] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5537worker1 # [ 83.744770] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=scheduler_connected addr=lb:500515538worker1 # [ 83.748280] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05539worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5540worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5541client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25542worker1 # [ 84.161071] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:574065543worker1 # [ 84.172159] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5544client # don't know how to build these paths:5545client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5546worker1 # [ 84.186202] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:574065547worker1 # [ 84.196795] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)5548worker1 # [ 84.209354] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 158 ms (attempt 2/5)5549worker1 # [ 84.373310] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 304 ms (attempt 3/5)5550worker1 # [ 84.677735] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 4/5)5551worker1 # [ 84.774695] nix-grpc-daemon[1041]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5552worker1 # [ 84.790121] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:04Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05553worker1 # [ 84.792276] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:04Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15554worker1 # [ 84.833748] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574065555worker1 # [ 84.840023] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:574065556client # copying 1 paths to 'lb:50051'...5557worker1 # [ 84.846606] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:574065558client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5559worker1 # [ 84.851870] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5560worker1 # [ 84.862777] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:05Z level=info event=niks3_push_started pid=11205561worker1 # [ 85.187947] nix-daemon[843]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5562worker1 # [ 85.200062] nix-daemon[843]: reaped child process 1136, status = succeeded5563worker1 # [ 85.360978] nix-daemon[843]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)5564worker1 # [ 85.372879] nix-daemon[843]: reaped child process 1141, status = succeeded5565lb # [ 85.398855] niks3-server[1067]: 2026/09/20 09:34:04 INFO Received uploads request method=POST path=/api/pending_closures5566worker1 # [ 85.412601] nix-grpc-daemon[1120]: time=2026-09-20T09:34:05.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5567worker1 # [ 85.415916] nix-grpc-daemon[1120]: time=2026-09-20T09:34:05.573Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5568lb # [ 85.450490] niks3-server[1067]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5569lb # [ 85.458725] niks3-server[1067]: 2026/09/20 09:34:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5570lb # [ 85.474549] niks3-server[1067]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5571lb # [ 85.481486] niks3-server[1067]: 2026/09/20 09:34:04 INFO Signed narinfos id=27 count=15572worker1 # [ 85.472048] nix-grpc-daemon[1120]: time=2026-09-20T09:34:05.630Z level=INFO msg="Uploading 1 narinfos"5573lb # [ 85.492070] niks3-server[1067]: 2026/09/20 09:34:04 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5574lb # [ 85.501312] niks3-server[1067]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5575lb # [ 85.504661] niks3-server[1067]: 2026/09/20 09:34:04 INFO Completed upload id=275576worker1 # [ 85.496054] nix-grpc-daemon[1120]: time=2026-09-20T09:34:05.654Z level=INFO msg="Upload complete. (458ms)"5577worker1 # [ 85.500494] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57406 duration_s=0 paths=1 nar_bytes_in=5205578worker1 # [ 85.507346] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574065579worker1 # [ 85.517567] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5580worker1 # [ 85.526501] nix-daemon[843]: reaped child process 1118, status = succeeded5581client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5582client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5583worker1 # [ 85.536463] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5584client # farm-a> LOG-farm-a-on-standby5585worker1 # [ 86.751621] nix-daemon[843]: reaped child process 1145, status = succeeded5586worker1 # [ 86.894749] nix-daemon[843]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5587worker1 # [ 86.906426] nix-daemon[843]: reaped child process 1163, status = succeeded5588lb # [ 86.929049] niks3-server[1067]: 2026/09/20 09:34:06 INFO Received uploads request method=POST path=/api/pending_closures5589worker1 # [ 86.926279] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.084Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5590worker1 # [ 86.930250] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.084Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5591lb # [ 86.960641] niks3-server[1067]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5592lb # [ 86.977142] niks3-server[1067]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5593lb # [ 86.983784] niks3-server[1067]: 2026/09/20 09:34:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5594lb # [ 86.988805] niks3-server[1067]: 2026/09/20 09:34:06 INFO Signed narinfos id=28 count=15595worker1 # [ 86.979072] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.137Z level=INFO msg="Uploading 1 narinfos"5596lb # [ 86.992622] niks3-server[1067]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5597lb # [ 87.000508] niks3-server[1067]: 2026/09/20 09:34:06 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5598lb # [ 87.003577] niks3-server[1067]: 2026/09/20 09:34:06 INFO Completed upload id=285599worker1 # [ 86.993586] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.152Z level=INFO msg="Upload complete. (247ms)"5600lb # [ 87.005894] niks3-server[1067]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5601worker1 # [ 86.998773] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57406 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15602worker1 # [ 87.014908] nix-daemon[843]: reaped child process 1143, status = succeeded5603worker1 # [ 87.016250] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:07Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05604worker1 # [ 87.018033] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:07Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15605worker1 # [ 87.060506] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574065606worker1 # [ 87.072377] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:574065607worker1 # [ 87.084234] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.243Z level=INFO msg="All 1 paths already cached"5608client # copying 1 paths to 'lb:50051'...5609client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5610worker1 # [ 87.092151] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:574065611worker1 # [ 87.102185] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5612worker1 # [ 87.271466] nix-daemon[843]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5613worker1 # [ 87.283805] nix-daemon[843]: reaped child process 1172, status = succeeded5614worker1 # [ 87.440219] nix-daemon[843]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5615worker1 # [ 87.451784] nix-daemon[843]: reaped child process 1177, status = succeeded5616lb # [ 87.478223] niks3-server[1067]: 2026/09/20 09:34:06 INFO Received uploads request method=POST path=/api/pending_closures5617worker1 # [ 87.472194] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.631Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5618worker1 # [ 87.473905] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.632Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5619lb # [ 87.495708] niks3-server[1067]: 2026/09/20 09:34:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5620lb # [ 87.498285] niks3-server[1067]: 2026/09/20 09:34:06 INFO Signed narinfos id=29 count=15621worker1 # [ 87.489568] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.648Z level=INFO msg="Uploading 1 narinfos"5622lb # [ 87.504658] niks3-server[1067]: 2026/09/20 09:34:06 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5623lb # [ 87.511180] niks3-server[1067]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5624lb # [ 87.515948] niks3-server[1067]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5625lb # [ 87.529290] niks3-server[1067]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5626lb # [ 87.535961] niks3-server[1067]: 2026/09/20 09:34:06 INFO Completed upload id=295627worker1 # [ 87.525893] nix-grpc-daemon[1120]: time=2026-09-20T09:34:07.684Z level=INFO msg="Upload complete. (413ms)"5628worker1 # [ 87.529073] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57406 duration_s=0 paths=1 nar_bytes_in=6405629worker1 # [ 87.531486] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574065630worker1 # [ 87.537714] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5631worker1 # [ 87.539963] nix-daemon[843]: reaped child process 1167, status = succeeded5632worker1 # [ 87.545143] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5633client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5634client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5635lb # [ 87.971836] envoy[908]: conn peer=[2001:db8:1::1]:55956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=305 ms=15525636lb # [ 87.977710] envoy[908]: conn peer=[2001:db8:1::4]:36576 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2293 tx=535 ms=70215637client # farm-b> LOG-farm-b-on-standby5638worker1 # [ 88.369738] nix-daemon[843]: reaped child process 1181, status = succeeded5639worker1 # [ 88.531153] nix-daemon[843]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5640worker1 # [ 88.543606] nix-daemon[843]: reaped child process 1199, status = succeeded5641lb # [ 88.571724] niks3-server[1067]: 2026/09/20 09:34:07 INFO Received uploads request method=POST path=/api/pending_closures5642worker1 # [ 88.586664] nix-grpc-daemon[1120]: time=2026-09-20T09:34:08.745Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5643worker1 # [ 88.588693] nix-grpc-daemon[1120]: time=2026-09-20T09:34:08.747Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5644lb # [ 88.620506] niks3-server[1067]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5645lb # [ 88.632989] niks3-server[1067]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5646lb # [ 88.643672] niks3-server[1067]: 2026/09/20 09:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5647lb # [ 88.649051] niks3-server[1067]: 2026/09/20 09:34:07 INFO Signed narinfos id=30 count=15648worker1 # [ 88.639519] nix-grpc-daemon[1120]: time=2026-09-20T09:34:08.798Z level=INFO msg="Uploading 1 narinfos"5649lb # [ 88.656836] niks3-server[1067]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5650lb # [ 88.663952] niks3-server[1067]: 2026/09/20 09:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5651lb # [ 88.668543] niks3-server[1067]: 2026/09/20 09:34:07 INFO Completed upload id=305652lb # [ 88.671380] niks3-server[1067]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5653worker1 # [ 88.659636] nix-grpc-daemon[1120]: time=2026-09-20T09:34:08.818Z level=INFO msg="Upload complete. (307ms)"5654worker1 # [ 88.668736] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57406 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15655worker1 # [ 88.678585] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:08Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05656worker1 # [ 88.682474] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:08Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15657worker1 # [ 88.692584] nix-daemon[843]: reaped child process 1179, status = succeeded5658worker1 # [ 88.722298] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574065659worker1 # [ 88.734853] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:574065660worker1 # [ 88.746751] nix-grpc-daemon[1120]: time=2026-09-20T09:34:08.905Z level=INFO msg="All 2 paths already cached"5661client # copying 1 paths to 'lb:50051'...5662client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5663worker1 # [ 88.755160] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:574065664worker1 # [ 88.765161] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5665worker1 # [ 88.924339] nix-daemon[843]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5666worker1 # [ 88.936785] nix-daemon[843]: reaped child process 1208, status = succeeded5667worker1 # [ 89.098295] nix-daemon[843]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)5668worker1 # [ 89.109837] nix-daemon[843]: reaped child process 1213, status = succeeded5669lb # [ 89.132711] niks3-server[1067]: 2026/09/20 09:34:08 INFO Received uploads request method=POST path=/api/pending_closures5670worker1 # [ 89.130562] nix-grpc-daemon[1120]: time=2026-09-20T09:34:09.289Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5671worker1 # [ 89.132073] nix-grpc-daemon[1120]: time=2026-09-20T09:34:09.291Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5672lb # [ 89.159548] niks3-server[1067]: 2026/09/20 09:34:08 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5673lb # [ 89.171415] niks3-server[1067]: 2026/09/20 09:34:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5674worker1 # [ 89.166511] nix-grpc-daemon[1120]: time=2026-09-20T09:34:09.325Z level=INFO msg="Uploading 1 narinfos"5675lb # [ 89.176607] niks3-server[1067]: 2026/09/20 09:34:08 INFO Signed narinfos id=31 count=15676lb # [ 89.182624] niks3-server[1067]: 2026/09/20 09:34:08 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5677lb # [ 89.192509] niks3-server[1067]: 2026/09/20 09:34:08 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5678lb # [ 89.197991] niks3-server[1067]: 2026/09/20 09:34:08 INFO Completed upload id=315679worker1 # [ 89.187581] nix-grpc-daemon[1120]: time=2026-09-20T09:34:09.346Z level=INFO msg="Upload complete. (411ms)"5680lb # [ 89.202587] niks3-server[1067]: 2026/09/20 09:34:08 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5681worker1 # [ 89.192093] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57406 duration_s=0 paths=1 nar_bytes_in=7685682worker1 # [ 89.196751] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574065683worker1 # [ 89.204960] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5684worker1 # [ 89.209351] nix-daemon[843]: reaped child process 1203, status = succeeded5685client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5686worker1 # [ 89.215091] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5687client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5688client # farm-top> LOG-farm-top-on-standby5689worker1 # [ 90.228380] nix-daemon[843]: reaped child process 1217, status = succeeded5690worker1 # [ 90.354764] nix-daemon[843]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5691worker1 # [ 90.365501] nix-daemon[843]: reaped child process 1235, status = succeeded5692lb # [ 90.392624] niks3-server[1067]: 2026/09/20 09:34:09 INFO Received uploads request method=POST path=/api/pending_closures5693worker1 # [ 90.390444] nix-grpc-daemon[1120]: time=2026-09-20T09:34:10.549Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5694worker1 # [ 90.392047] nix-grpc-daemon[1120]: time=2026-09-20T09:34:10.551Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5695lb # [ 90.430788] niks3-server[1067]: 2026/09/20 09:34:09 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5696lb # [ 90.442250] niks3-server[1067]: 2026/09/20 09:34:09 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5697lb # [ 90.449122] niks3-server[1067]: 2026/09/20 09:34:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5698worker1 # [ 90.443894] nix-grpc-daemon[1120]: time=2026-09-20T09:34:10.602Z level=INFO msg="Uploading 1 narinfos"5699lb # [ 90.454145] niks3-server[1067]: 2026/09/20 09:34:09 INFO Signed narinfos id=32 count=15700lb # [ 90.458808] niks3-server[1067]: 2026/09/20 09:34:09 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5701lb # [ 90.468693] niks3-server[1067]: 2026/09/20 09:34:09 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5702lb # [ 90.473994] niks3-server[1067]: 2026/09/20 09:34:09 INFO Completed upload id=325703worker1 # [ 90.463896] nix-grpc-daemon[1120]: time=2026-09-20T09:34:10.622Z level=INFO msg="Upload complete. (243ms)"5704lb # [ 90.476949] niks3-server[1067]: 2026/09/20 09:34:09 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5705worker1 # [ 90.472479] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57406 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15706worker1 # [ 90.483660] nix-daemon[843]: reaped child process 1215, status = succeeded5707client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.58 seconds)5708client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5709client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.19 seconds)5710worker2 # [ 90.777782] systemd[1]: Listening on Nix worker-protocol over gRPC.5711worker2 # [ 90.785153] systemd[1]: Starting Nix worker-protocol over gRPC...5712worker2 # [ 90.951515] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5713worker2 # [ 90.955658] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15714worker2 # [ 90.962068] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5715worker2 # [ 90.967109] systemd[1]: Started Nix worker-protocol over gRPC.5716lb: must succeed: curl -sf localhost:9901/clusters5717worker2 # [ 90.991035] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:11Z level=info event=scheduler_connected addr=lb:500515718worker1 # [ 90.984814] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05719lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5720worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5721worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5722lb: must succeed: curl -sf localhost:9901/clusters5723lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5724lb: must succeed: curl -sf localhost:9901/clusters5725lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5726worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5727worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5728(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.65 seconds)5729subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5730client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5731client # [ 91.261593] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5732client # Running as unit: rs.service; invocation ID: 8f05812ac0ce4fa8ac42c575a740d88a5733client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.05 seconds)5734client # [ 91.459793] nix[1184]: don't know how to build these paths:5735client # [ 91.460706] nix[1184]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5736worker1 # [ 91.474972] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:574065737worker1 # [ 91.487268] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:11Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05738worker1 # [ 91.491372] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:11Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15739worker1 # [ 91.540760] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574065740worker1 # [ 91.552807] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:574065741client # [ 91.547248] nix[1184]: copying 1 paths to 'lb:50051'...5742worker1 # [ 91.565908] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:574065743client # [ 91.550076] nix[1184]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5744worker1 # [ 91.577338] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5745worker1 # [ 91.763276] nix-daemon[843]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)5746worker1 # [ 91.775855] nix-daemon[843]: reaped child process 1253, status = succeeded5747worker1 # [ 91.941480] nix-daemon[843]: accepted connection from pid 1255, user nix-grpc-daemon (trusted)5748worker1 # [ 91.953453] nix-daemon[843]: reaped child process 1258, status = succeeded5749lb # [ 91.979145] niks3-server[1067]: 2026/09/20 09:34:11 INFO Received uploads request method=POST path=/api/pending_closures5750worker1 # [ 91.973515] nix-grpc-daemon[1120]: time=2026-09-20T09:34:12.132Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5751worker1 # [ 91.975055] nix-grpc-daemon[1120]: time=2026-09-20T09:34:12.134Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5752lb # [ 91.995715] niks3-server[1067]: 2026/09/20 09:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5753lb # [ 91.998533] niks3-server[1067]: 2026/09/20 09:34:11 INFO Signed narinfos id=33 count=15754worker1 # [ 91.989689] nix-grpc-daemon[1120]: time=2026-09-20T09:34:12.148Z level=INFO msg="Uploading 1 narinfos"5755lb # [ 92.002502] niks3-server[1067]: 2026/09/20 09:34:11 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5756lb # [ 92.005104] niks3-server[1067]: 2026/09/20 09:34:11 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5757lb # [ 92.009146] niks3-server[1067]: 2026/09/20 09:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5758lb # [ 92.012935] niks3-server[1067]: 2026/09/20 09:34:11 INFO Completed upload id=335759worker1 # [ 92.002667] nix-grpc-daemon[1120]: time=2026-09-20T09:34:12.161Z level=INFO msg="Upload complete. (413ms)"5760lb # [ 92.015277] niks3-server[1067]: 2026/09/20 09:34:11 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5761worker1 # [ 92.004591] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57406 duration_s=0 paths=1 nar_bytes_in=4165762worker1 # [ 92.007461] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:574065763worker1 # [ 92.013956] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5764worker1 # [ 92.016463] nix-daemon[843]: reaped child process 1248, status = succeeded5765client # [ 92.002852] nix[1184]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5766worker1 # [ 92.021636] nix-daemon[843]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)5767lb: must succeed: curl -sf localhost:9901/clusters5768lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5769lb: must succeed: curl -sf localhost:9901/clusters5770lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5771lb: must succeed: curl -sf localhost:9901/clusters5772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5773lb: must succeed: curl -sf localhost:9901/clusters5774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5775worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5776worker2 # [ 94.069293] systemd-journald[287]: Received client request to rotate journal, rotating.5777worker2 # [ 94.085482] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/20d610c1f3594f68878b70fda61d80e2.5778worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)5779worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5780worker1 # [ 93.992682] systemd[1]: Stopping Nix worker-protocol over gRPC...5781worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5782??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5783 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735784client: must succeed: sleep 5.05785worker1 # [ 94.535804] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:14Z level=info event=draining5786lb # [ 97.972684] envoy[908]: conn peer=[2001:db8:1::1]:52206 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9577 tx=3837 ms=63265787client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5788worker1 # [ 99.239759] nix-daemon[843]: reaped child process 1262, status = succeeded5789worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5790worker1 # [ 99.385362] nix-daemon[843]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5791worker1 # [ 99.397638] nix-daemon[843]: reaped child process 1305, status = succeeded5792lb # [ 99.423403] niks3-server[1067]: 2026/09/20 09:34:18 INFO Received uploads request method=POST path=/api/pending_closures5793worker1 # [ 99.421469] nix-grpc-daemon[1120]: time=2026-09-20T09:34:19.580Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5794worker1 # [ 99.425149] nix-grpc-daemon[1120]: time=2026-09-20T09:34:19.582Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5795lb # [ 99.464138] niks3-server[1067]: 2026/09/20 09:34:18 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5796lb # [ 99.479959] niks3-server[1067]: 2026/09/20 09:34:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5797lb # [ 99.484903] niks3-server[1067]: 2026/09/20 09:34:18 INFO Signed narinfos id=34 count=15798worker1 # [ 99.477356] nix-grpc-daemon[1120]: time=2026-09-20T09:34:19.636Z level=INFO msg="Uploading 1 narinfos"5799lb # [ 99.491562] niks3-server[1067]: 2026/09/20 09:34:18 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5800lb # [ 99.496204] niks3-server[1067]: 2026/09/20 09:34:18 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5801lb # [ 99.505143] niks3-server[1067]: 2026/09/20 09:34:18 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5802lb # [ 99.508069] niks3-server[1067]: 2026/09/20 09:34:18 INFO Completed upload id=345803lb # [ 99.509726] niks3-server[1067]: 2026/09/20 09:34:18 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5804worker1 # [ 99.499497] nix-grpc-daemon[1120]: time=2026-09-20T09:34:19.658Z level=INFO msg="Upload complete. (265ms)"5805worker1 # [ 99.507626] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57406 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15806worker1 # [ 99.520266] nix-daemon[843]: reaped child process 1260, status = succeeded5807client # [ 99.515905] systemd[1]: rs.service: Deactivated successfully.5808worker1 # [ 99.536632] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:19Z level=info event=signal_exit5809client # [ 99.518458] systemd[1]: rs.service: Consumed 104ms CPU time over 8.253s wall clock time, 37.1M memory peak, 8.6K incoming IP traffic, 8.9K outgoing IP traffic.5810worker1 # [ 99.757500] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:19Z level=info event=scheduler_yield5811worker2 # [ 99.767364] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:19Z level=info event=scheduler_restarting addr=lb:500515812lb # [ 99.769105] niks3-server[1067]: 2026/09/20 09:34:19 INFO lead: released remote=192.168.1.3:597485813worker1 # [ 99.761333] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:19Z level=info event=scheduler_restarting addr=lb:500515814worker1 # [ 99.765498] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573965815worker1 # [ 99.769942] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:19Z level=info event=worker_gone addr=192.168.1.4:500515816worker1 # [ 99.773221] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:573965817worker1 # [ 99.776715] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:19Z level=info event=worker_gone addr=192.168.1.3:500515818worker2 # [ 99.967396] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515819worker2 # [ 99.969486] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker1 # [ 99.962177] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515821worker1 # [ 99.965984] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822worker2 # [ 100.170037] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 100.171832] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5824worker1 # [ 100.166922] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515825worker1 # [ 100.171876] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5826worker2 # [ 100.372660] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515827worker2 # [ 100.376687] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5828worker1 # [ 100.372730] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515829worker1 # [ 100.376695] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5830worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.09 seconds)5831client: waiting for success: ! systemctl is-active rs5832client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)5833client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5834client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5835client: must fail: journalctl -u rs | grep -q 'reconnecting'5836client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5837lb: must succeed: curl -sf localhost:9901/clusters5838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5839worker2 # [ 100.577480] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515840worker2 # [ 100.581337] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5841worker1 # [ 100.579563] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515842worker1 # [ 100.582806] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5843worker1 # [ 100.758297] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=scheduler_restarting5844worker2 # [ 100.783494] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_connected addr=lb:500515845worker2 # [ 100.786665] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5846worker1 # [ 100.794721] nix-grpc-daemon[1041]: ts=2026-09-20T09:34:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5847worker1 # [ 100.799417] nix-daemon[843]: reaped child process 1111, status = succeeded5848worker2 # [ 100.974489] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_take_over5849lb # [ 100.977289] niks3-server[1067]: 2026/09/20 09:34:20 INFO lead: acquired remote=192.168.1.4:603005850worker2 # [ 100.984273] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500515851worker2 # [ 100.988093] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5852worker1 # [ 101.126804] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5853worker1 # [ 101.130474] systemd[1]: Stopped Nix worker-protocol over gRPC.5854worker1 # [ 101.132578] systemd[1]: nix-grpc-daemon.service: Consumed 1.247s CPU time over 22.753s wall clock time, 35.4M memory peak, 48.3K incoming IP traffic, 92.8K outgoing IP traffic.5855worker1 # [ 101.149588] systemd[1]: Starting Nix worker-protocol over gRPC...5856worker2 # [ 101.188895] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500515857worker2 # [ 101.192808] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5858worker1 # [ 101.298535] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5859worker1 # [ 101.301895] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15860worker1 # [ 101.308671] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5861worker1 # [ 101.310569] systemd[1]: Started Nix worker-protocol over gRPC.5862worker1 # [ 101.336168] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500515863worker1 # [ 101.337876] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5864worker2 # [ 101.393641] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500515865worker2 # [ 101.397431] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5866lb: must succeed: curl -sf localhost:9901/clusters5867worker1 # [ 101.538971] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500515868worker1 # [ 101.543069] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5869worker2 # [ 101.598346] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500515870worker2 # [ 101.602849] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5871lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5872worker1 # [ 101.744089] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500515873worker2 # [ 101.760801] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05874worker2 # [ 101.804690] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=scheduler_connected addr=lb:500515875worker2 # [ 101.807796] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05876lb: must succeed: curl -sf localhost:9901/clusters5877lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5878worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5879worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5880(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.48 seconds)5881subtest: scheduler restart mid-build: build finishes, no double build5882client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5883client # [ 102.762837] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5884client # Running as unit: mid.service; invocation ID: 7f85e91295ad4e24afa99152b3f8505a5885client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5886worker1 # [ 102.997607] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480965887worker1 # [ 103.008839] nix-daemon[843]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5888client # [ 103.002309] nix[1220]: don't know how to build these paths:5889client # [ 103.003241] nix[1220]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5890worker2 # [ 103.038544] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:23Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05891worker2 # [ 103.042474] nix-grpc-daemon[1342]: ts=2026-09-20T09:34:23Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15892worker1 # [ 103.073350] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480965893worker1 # [ 103.085100] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 1/5)5894worker1 # [ 103.147738] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 2/5)5895worker1 # [ 103.337458] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 3/5)5896worker1 # [ 103.483413] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 444 ms (attempt 4/5)5897worker1 # [ 103.934760] nix-grpc-daemon[1312]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5898worker1 # [ 103.955120] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480965899client # [ 103.948738] nix[1220]: copying 1 paths to 'lb:50051'...5900client # [ 103.952194] nix[1220]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5901worker1 # [ 103.970520] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480965902worker1 # [ 103.981107] nix-daemon[843]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5903worker1 # [ 103.999115] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:24Z level=info event=niks3_push_started pid=13765904worker1 # [ 104.279394] nix-daemon[843]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)5905worker1 # [ 104.289774] nix-daemon[843]: reaped child process 1392, status = succeeded5906worker1 # [ 104.442212] nix-daemon[843]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5907worker1 # [ 104.453902] nix-daemon[843]: reaped child process 1397, status = succeeded5908lb # [ 104.479174] niks3-server[1067]: 2026/09/20 09:34:23 INFO Received uploads request method=POST path=/api/pending_closures5909worker1 # [ 104.480307] nix-grpc-daemon[1376]: time=2026-09-20T09:34:24.638Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5910worker1 # [ 104.482192] nix-grpc-daemon[1376]: time=2026-09-20T09:34:24.638Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5911lb # [ 104.511631] niks3-server[1067]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5912lb # [ 104.521569] niks3-server[1067]: 2026/09/20 09:34:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5913lb # [ 104.526502] niks3-server[1067]: 2026/09/20 09:34:23 INFO Signed narinfos id=35 count=15914worker1 # [ 104.519506] nix-grpc-daemon[1376]: time=2026-09-20T09:34:24.678Z level=INFO msg="Uploading 1 narinfos"5915lb # [ 104.538107] niks3-server[1067]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5916lb # [ 104.546255] niks3-server[1067]: 2026/09/20 09:34:23 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5917lb # [ 104.551601] niks3-server[1067]: 2026/09/20 09:34:23 INFO Completed upload id=355918worker1 # [ 104.544075] nix-grpc-daemon[1376]: time=2026-09-20T09:34:24.703Z level=INFO msg="Upload complete. (405ms)"5919lb # [ 104.556367] niks3-server[1067]: 2026/09/20 09:34:23 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5920worker1 # [ 104.548123] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48096 duration_s=0 paths=1 nar_bytes_in=4245921worker1 # [ 104.553494] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480965922worker1 # [ 104.561308] nix-daemon[843]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5923worker1 # [ 104.565291] nix-daemon[843]: reaped child process 1374, status = succeeded5924worker1 # [ 104.569952] nix-daemon[843]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5925client # [ 104.552431] nix[1220]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5926lb: must succeed: curl -sf localhost:9901/clusters5927lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5928lb: must succeed: curl -sf localhost:9901/clusters5929lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5930worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5931worker2 # [ 106.589640] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1342 (nix-grpc-daemon) on client request.5932lb # [ 106.599080] niks3-server[1067]: 2026/09/20 09:34:25 INFO lead: released remote=192.168.1.4:603005933worker2 # [ 106.596487] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5934worker1 # [ 106.592173] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"5935client # [ 106.576383] nix[1220]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5936worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.07 seconds)5937client: must succeed: sleep 5.05938worker2 # [ 106.620241] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5939worker2 # [ 106.622160] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5940worker2 # [ 106.623921] systemd[1]: nix-grpc-daemon.service: Consumed 195ms CPU time over 15.834s wall clock time, 12.8M memory peak, 10.1K incoming IP traffic, 14.2K outgoing IP traffic.5941worker2 # [ 106.728284] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5942worker2 # [ 106.738179] systemd[1]: Starting Nix worker-protocol over gRPC...5943worker1 # [ 106.792676] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:26Z level=info event=scheduler_connected addr=lb:500515944worker2 # [ 106.903599] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5945worker2 # [ 106.905694] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15946worker2 # [ 106.910895] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5947worker2 # [ 106.914140] systemd[1]: Started Nix worker-protocol over gRPC.5948worker2 # [ 106.923160] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:27Z level=info event=scheduler_take_over5949lb # [ 106.924748] niks3-server[1067]: 2026/09/20 09:34:26 INFO lead: acquired remote=192.168.1.4:371285950worker2 # [ 106.929403] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=15951worker2 # [ 106.946308] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:27Z level=info event=scheduler_connected addr=lb:500515952worker2 # [ 106.947957] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05953lb # [ 107.972851] envoy[908]: conn peer=[2001:db8:1::1]:52220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2917 tx=1328 ms=80415954lb # [ 107.976565] envoy[908]: conn peer=[2001:db8:1::3]:52660 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=6195 tx=1480 ms=222275955lb # [ 107.980533] envoy[908]: conn peer=[2001:db8:1::4]:49904 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2487 tx=630 ms=156115956client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5957client: waiting for success: ! systemctl is-active mid5958worker1 # [ 112.001657] nix-daemon[843]: reaped child process 1401, status = succeeded5959worker1 # [ 112.163609] nix-daemon[843]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)5960worker1 # [ 112.174096] nix-daemon[843]: reaped child process 1440, status = succeeded5961lb # [ 112.198982] niks3-server[1067]: 2026/09/20 09:34:31 INFO Received uploads request method=POST path=/api/pending_closures5962worker1 # [ 112.198259] nix-grpc-daemon[1376]: time=2026-09-20T09:34:32.356Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5963worker1 # [ 112.201929] nix-grpc-daemon[1376]: time=2026-09-20T09:34:32.358Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5964lb # [ 112.238602] niks3-server[1067]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5965lb # [ 112.253756] niks3-server[1067]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5966lb # [ 112.262495] niks3-server[1067]: 2026/09/20 09:34:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5967lb # [ 112.272769] niks3-server[1067]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5968lb # [ 112.280269] niks3-server[1067]: 2026/09/20 09:34:31 INFO Signed narinfos id=36 count=15969worker1 # [ 112.271209] nix-grpc-daemon[1376]: time=2026-09-20T09:34:32.429Z level=INFO msg="Uploading 1 narinfos"5970lb # [ 112.289341] niks3-server[1067]: 2026/09/20 09:34:31 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5971lb # [ 112.294267] niks3-server[1067]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5972lb # [ 112.298218] niks3-server[1067]: 2026/09/20 09:34:31 INFO Completed upload id=365973worker1 # [ 112.288318] nix-grpc-daemon[1376]: time=2026-09-20T09:34:32.447Z level=INFO msg="Upload complete. (305ms)"5974worker1 # [ 112.293597] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48096 duration_s=7 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15975worker1 # [ 112.307172] nix-daemon[843]: reaped child process 1399, status = succeeded5976client # [ 112.315237] systemd[1]: mid.service: Deactivated successfully.5977client # [ 112.318899] systemd[1]: mid.service: Consumed 118ms CPU time over 9.556s wall clock time, 39.3M memory peak, 9.1K incoming IP traffic, 9.3K outgoing IP traffic.5978client: (finished: waiting for success: ! systemctl is-active mid, in 1.11 seconds)5979client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5980client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5981(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.27 seconds)5982subtest: interrupting the client stops the build on the worker5983client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5984client # [ 113.003214] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5985client # Running as unit: intr.service; invocation ID: 865fd9dca8b8416c911f60a35538c8ca5986client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5987client # [ 113.214515] nix[1256]: don't know how to build these paths:5988client # [ 113.215480] nix[1256]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5989worker1 # [ 113.231141] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480965990worker2 # [ 113.250558] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:33Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05991worker2 # [ 113.254179] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:33Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=15992worker2 # [ 113.293672] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398285993worker2 # [ 113.298825] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5994worker2 # [ 113.306488] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5)5995worker2 # [ 113.333864] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 2/5)5996worker2 # [ 113.400413] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 314 ms (attempt 3/5)5997worker2 # [ 113.717495] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 116 ms (attempt 4/5)5998worker2 # [ 113.839711] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5999worker2 # [ 113.856954] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398286000client # [ 113.842367] nix[1256]: copying 1 paths to 'lb:50051'...6001client # [ 113.846197] nix[1256]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6002worker2 # [ 113.872677] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:398286003worker2 # [ 113.883231] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6004worker2 # [ 113.903090] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:33Z level=info event=niks3_push_started pid=15416005worker2 # [ 114.248536] nix-daemon[848]: accepted connection from pid 1554, user nix-grpc-daemon (trusted)6006worker2 # [ 114.265530] nix-daemon[848]: reaped child process 1560, status = succeeded6007worker2 # [ 114.428236] nix-daemon[848]: accepted connection from pid 1563, user nix-grpc-daemon (trusted)6008worker2 # [ 114.439737] nix-daemon[848]: reaped child process 1566, status = succeeded6009lb # [ 114.459373] niks3-server[1067]: 2026/09/20 09:34:33 INFO Received uploads request method=POST path=/api/pending_closures6010worker2 # [ 114.468305] nix-grpc-daemon[1541]: time=2026-09-20T09:34:34.561Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6011worker2 # [ 114.469656] nix-grpc-daemon[1541]: time=2026-09-20T09:34:34.561Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6012lb # [ 114.494372] niks3-server[1067]: 2026/09/20 09:34:33 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6013lb # [ 114.503570] niks3-server[1067]: 2026/09/20 09:34:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6014lb # [ 114.508174] niks3-server[1067]: 2026/09/20 09:34:33 INFO Signed narinfos id=37 count=16015worker2 # [ 114.505573] nix-grpc-daemon[1541]: time=2026-09-20T09:34:34.599Z level=INFO msg="Uploading 1 narinfos"6016lb # [ 114.520714] niks3-server[1067]: 2026/09/20 09:34:33 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6017lb # [ 114.527715] niks3-server[1067]: 2026/09/20 09:34:33 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6018lb # [ 114.534957] niks3-server[1067]: 2026/09/20 09:34:33 INFO Completed upload id=376019worker2 # [ 114.532505] nix-grpc-daemon[1541]: time=2026-09-20T09:34:34.626Z level=INFO msg="Upload complete. (441ms)"6020lb # [ 114.537442] niks3-server[1067]: 2026/09/20 09:34:33 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6021worker2 # [ 114.536826] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=0 paths=1 nar_bytes_in=4246022worker2 # [ 114.542768] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286023worker2 # [ 114.552116] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6024worker2 # [ 114.556306] nix-daemon[848]: reaped child process 1539, status = succeeded6025worker2 # [ 114.561148] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6026client # [ 114.535904] nix[1256]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6027client: must succeed: systemctl kill -s INT intr6028client # [ 115.440075] systemd[1]: intr.service: Sent signal SIGINT to main process 1256 (nix) on client request.6029client # [ 115.446471] nix[1256]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6030client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6031client # [ 115.454391] nix[1256]: error: interrupted by the user6032client # [ 115.483566] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6033client # [ 115.485591] systemd[1]: intr.service: Failed with result 'exit-code'.6034client # [ 115.487534] systemd[1]: intr.service: Consumed 103ms CPU time over 2.483s wall clock time, 39.3M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.6035worker2 # [ 115.582958] nix-daemon[848]: reaped child process 1568, status = succeeded6036worker2 # [ 115.588274] nix-daemon[1570]: unexpected Nix daemon error: error: interrupted by the user6037worker2 # [ 115.594156] nix-daemon[848]: reaped child process 1570, status = failed with exit code 16038(finished: subtest: interrupting the client stops the build on the worker, in 2.60 seconds)6039subtest: a deploy mid-build drains: the build finishes, the new generation takes over6040client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6041client # [ 115.602499] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6042client # Running as unit: sw.service; invocation ID: 40a38ed6ff30485b9fdd72aa0bf747496043client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6044worker2 # [ 115.835734] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:398286045client # [ 115.816857] nix[1276]: don't know how to build these paths:6046client # [ 115.817860] nix[1276]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6047worker2 # [ 115.850555] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:35Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06048worker2 # [ 115.854985] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:35Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16049worker1 # [ 115.888552] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480966050worker1 # [ 115.902075] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480966051client # [ 115.895184] nix[1276]: copying 1 paths to 'lb:50051'...6052worker1 # [ 115.913955] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480966053client # [ 115.897096] nix[1276]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6054worker1 # [ 115.924071] nix-daemon[843]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)6055worker1 # [ 116.108976] nix-daemon[843]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6056worker1 # [ 116.120831] nix-daemon[843]: reaped child process 1467, status = succeeded6057worker1 # [ 116.294151] nix-daemon[843]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6058worker1 # [ 116.308120] nix-daemon[843]: reaped child process 1472, status = succeeded6059lb # [ 116.337235] niks3-server[1067]: 2026/09/20 09:34:35 INFO Received uploads request method=POST path=/api/pending_closures6060worker1 # [ 116.333779] nix-grpc-daemon[1376]: time=2026-09-20T09:34:36.492Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6061worker1 # [ 116.335302] nix-grpc-daemon[1376]: time=2026-09-20T09:34:36.494Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6062lb # [ 116.366302] niks3-server[1067]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6063lb # [ 116.373530] niks3-server[1067]: 2026/09/20 09:34:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6064lb # [ 116.378495] niks3-server[1067]: 2026/09/20 09:34:35 INFO Signed narinfos id=38 count=16065worker1 # [ 116.371329] nix-grpc-daemon[1376]: time=2026-09-20T09:34:36.530Z level=INFO msg="Uploading 1 narinfos"6066lb # [ 116.388972] niks3-server[1067]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6067lb # [ 116.398187] niks3-server[1067]: 2026/09/20 09:34:35 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6068lb # [ 116.405271] niks3-server[1067]: 2026/09/20 09:34:35 INFO Completed upload id=386069worker1 # [ 116.395115] nix-grpc-daemon[1376]: time=2026-09-20T09:34:36.554Z level=INFO msg="Upload complete. (457ms)"6070lb # [ 116.408762] niks3-server[1067]: 2026/09/20 09:34:35 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6071worker1 # [ 116.398966] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48096 duration_s=0 paths=1 nar_bytes_in=4166072worker1 # [ 116.405101] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480966073worker1 # [ 116.415948] nix-daemon[843]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)6074worker1 # [ 116.422197] nix-daemon[843]: reaped child process 1462, status = succeeded6075worker1 # [ 116.425308] nix-daemon[843]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)6076client # [ 116.406781] nix[1276]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6077lb # [ 117.973527] envoy[908]: conn peer=[2001:db8:1::1]:40468 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1423 ms=93116078lb # [ 117.977565] envoy[908]: conn peer=[2001:db8:1::1]:51460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=733 ms=22516079worker1: must succeed: systemctl reload nix-grpc-daemon.service6080worker1 # [ 118.169135] systemd[1]: Reloading Nix worker-protocol over gRPC...6081worker1 # [ 118.209804] systemd[1]: Reloaded Nix worker-protocol over gRPC.6082worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6083worker1: must succeed: pgrep -f 'read -t [9]0 x'6084worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6085client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26086worker1 # [ 118.319171] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:38Z level=info event=draining6087worker2 # [ 118.489242] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:398286088client # don't know how to build these paths:6089client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6090worker1 # [ 118.487891] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480966091worker2 # [ 118.516037] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:38Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06092worker2 # [ 118.518194] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:38Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16093worker2 # [ 118.561491] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286094worker2 # [ 118.569140] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398286095client # copying 1 paths to 'lb:50051'...6096worker2 # [ 118.576832] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:398286097client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6098worker2 # [ 118.585537] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6099worker2 # [ 118.745490] nix-daemon[848]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6100worker2 # [ 118.757383] nix-daemon[848]: reaped child process 1619, status = succeeded6101worker2 # [ 118.916642] nix-daemon[848]: accepted connection from pid 1621, user nix-grpc-daemon (trusted)6102worker2 # [ 118.929530] nix-daemon[848]: reaped child process 1624, status = succeeded6103lb # [ 118.947339] niks3-server[1067]: 2026/09/20 09:34:38 INFO Received uploads request method=POST path=/api/pending_closures6104worker2 # [ 118.952945] nix-grpc-daemon[1541]: time=2026-09-20T09:34:39.046Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6105worker2 # [ 118.954713] nix-grpc-daemon[1541]: time=2026-09-20T09:34:39.049Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6106lb # [ 118.976498] niks3-server[1067]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6107lb # [ 118.989228] niks3-server[1067]: 2026/09/20 09:34:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6108lb # [ 118.993976] niks3-server[1067]: 2026/09/20 09:34:38 INFO Signed narinfos id=39 count=16109worker2 # [ 118.994700] nix-grpc-daemon[1541]: time=2026-09-20T09:34:39.089Z level=INFO msg="Uploading 1 narinfos"6110lb # [ 119.008670] niks3-server[1067]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6111lb # [ 119.014695] niks3-server[1067]: 2026/09/20 09:34:38 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6112worker2 # [ 119.017978] nix-grpc-daemon[1541]: time=2026-09-20T09:34:39.112Z level=INFO msg="Upload complete. (426ms)"6113lb # [ 119.020688] niks3-server[1067]: 2026/09/20 09:34:38 INFO Completed upload id=396114lb # [ 119.024217] niks3-server[1067]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6115worker2 # [ 119.021233] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=0 paths=1 nar_bytes_in=5286116worker2 # [ 119.026659] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286117worker2 # [ 119.035879] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6118worker2 # [ 119.041231] nix-daemon[848]: reaped child process 1614, status = succeeded6119client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6120client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6121worker2 # [ 119.046137] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6122client # farm-a> LOG-farm-a-during-drain6123worker2 # [ 120.061470] nix-daemon[848]: reaped child process 1628, status = succeeded6124worker2 # [ 120.203092] nix-daemon[848]: accepted connection from pid 1643, user nix-grpc-daemon (trusted)6125worker2 # [ 120.213576] nix-daemon[848]: reaped child process 1646, status = succeeded6126lb # [ 120.232669] niks3-server[1067]: 2026/09/20 09:34:39 INFO Received uploads request method=POST path=/api/pending_closures6127worker2 # [ 120.238250] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.332Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6128worker2 # [ 120.239893] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.334Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6129lb # [ 120.278175] niks3-server[1067]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6130lb # [ 120.288911] niks3-server[1067]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6131lb # [ 120.295915] niks3-server[1067]: 2026/09/20 09:34:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6132lb # [ 120.302358] niks3-server[1067]: 2026/09/20 09:34:39 INFO Signed narinfos id=40 count=16133worker2 # [ 120.302478] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.396Z level=INFO msg="Uploading 1 narinfos"6134lb # [ 120.307048] niks3-server[1067]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6135lb # [ 120.317600] niks3-server[1067]: 2026/09/20 09:34:39 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6136lb # [ 120.323567] niks3-server[1067]: 2026/09/20 09:34:39 INFO Completed upload id=406137worker2 # [ 120.321211] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.415Z level=INFO msg="Upload complete. (268ms)"6138lb # [ 120.325269] niks3-server[1067]: 2026/09/20 09:34:39 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6139worker2 # [ 120.327970] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=16140worker2 # [ 120.338691] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:40Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06141worker2 # [ 120.342208] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:40Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16142worker2 # [ 120.350761] nix-daemon[848]: reaped child process 1626, status = succeeded6143worker2 # [ 120.387919] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286144worker2 # [ 120.400453] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398286145worker2 # [ 120.412445] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.506Z level=INFO msg="All 1 paths already cached"6146client # copying 1 paths to 'lb:50051'...6147worker2 # [ 120.421279] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:398286148client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6149worker2 # [ 120.440146] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6150worker2 # [ 120.599856] nix-daemon[848]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6151worker2 # [ 120.611847] nix-daemon[848]: reaped child process 1655, status = succeeded6152worker2 # [ 120.761068] nix-daemon[848]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6153worker2 # [ 120.772744] nix-daemon[848]: reaped child process 1660, status = succeeded6154lb # [ 120.789593] niks3-server[1067]: 2026/09/20 09:34:40 INFO Received uploads request method=POST path=/api/pending_closures6155worker2 # [ 120.794258] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.888Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6156worker2 # [ 120.795786] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.890Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6157lb # [ 120.819404] niks3-server[1067]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6158lb # [ 120.828319] niks3-server[1067]: 2026/09/20 09:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6159worker2 # [ 120.830295] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.924Z level=INFO msg="Uploading 1 narinfos"6160lb # [ 120.833114] niks3-server[1067]: 2026/09/20 09:34:40 INFO Signed narinfos id=41 count=16161lb # [ 120.839288] niks3-server[1067]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6162lb # [ 120.850795] niks3-server[1067]: 2026/09/20 09:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6163worker2 # [ 120.854188] nix-grpc-daemon[1541]: time=2026-09-20T09:34:40.948Z level=INFO msg="Upload complete. (402ms)"6164lb # [ 120.856975] niks3-server[1067]: 2026/09/20 09:34:40 INFO Completed upload id=416165worker2 # [ 120.855584] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=0 paths=1 nar_bytes_in=6406166lb # [ 120.859624] niks3-server[1067]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6167worker2 # [ 120.863618] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286168worker2 # [ 120.869737] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6169worker2 # [ 120.871982] nix-daemon[848]: reaped child process 1650, status = succeeded6170worker2 # [ 120.877094] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6171client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6172client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6173client # farm-b> LOG-farm-b-during-drain6174worker2 # [ 121.890280] nix-daemon[848]: reaped child process 1664, status = succeeded6175worker2 # [ 122.029286] nix-daemon[848]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6176worker2 # [ 122.041300] nix-daemon[848]: reaped child process 1682, status = succeeded6177lb # [ 122.059692] niks3-server[1067]: 2026/09/20 09:34:41 INFO Received uploads request method=POST path=/api/pending_closures6178worker2 # [ 122.065237] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.159Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6179worker2 # [ 122.066709] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.161Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6180lb # [ 122.090150] niks3-server[1067]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6181lb # [ 122.104168] niks3-server[1067]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6182lb # [ 122.114865] niks3-server[1067]: 2026/09/20 09:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6183worker2 # [ 122.116370] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.210Z level=INFO msg="Uploading 1 narinfos"6184lb # [ 122.119114] niks3-server[1067]: 2026/09/20 09:34:41 INFO Signed narinfos id=42 count=16185lb # [ 122.124347] niks3-server[1067]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6186lb # [ 122.133166] niks3-server[1067]: 2026/09/20 09:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6187lb # [ 122.136357] niks3-server[1067]: 2026/09/20 09:34:41 INFO Completed upload id=426188lb # [ 122.138034] niks3-server[1067]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6189worker2 # [ 122.135169] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.229Z level=INFO msg="Upload complete. (251ms)"6190worker2 # [ 122.146489] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=16191worker2 # [ 122.151293] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:42Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06192worker2 # [ 122.155176] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:42Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16193worker2 # [ 122.163557] nix-daemon[848]: reaped child process 1662, status = succeeded6194worker2 # [ 122.189532] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286195worker2 # [ 122.201182] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398286196worker2 # [ 122.213561] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.307Z level=INFO msg="All 2 paths already cached"6197client # copying 1 paths to 'lb:50051'...6198client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6199worker2 # [ 122.220284] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:398286200worker2 # [ 122.228466] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6201worker2 # [ 122.386071] nix-daemon[848]: accepted connection from pid 1688, user nix-grpc-daemon (trusted)6202worker2 # [ 122.397721] nix-daemon[848]: reaped child process 1691, status = succeeded6203worker2 # [ 122.554300] nix-daemon[848]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6204worker2 # [ 122.565663] nix-daemon[848]: reaped child process 1696, status = succeeded6205lb # [ 122.583915] niks3-server[1067]: 2026/09/20 09:34:41 INFO Received uploads request method=POST path=/api/pending_closures6206worker2 # [ 122.589991] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.684Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6207worker2 # [ 122.591793] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.685Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6208lb # [ 122.611629] niks3-server[1067]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6209lb # [ 122.621776] niks3-server[1067]: 2026/09/20 09:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6210worker2 # [ 122.624135] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.718Z level=INFO msg="Uploading 1 narinfos"6211lb # [ 122.627254] niks3-server[1067]: 2026/09/20 09:34:41 INFO Signed narinfos id=43 count=16212lb # [ 122.631813] niks3-server[1067]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6213lb # [ 122.642799] niks3-server[1067]: 2026/09/20 09:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6214lb # [ 122.648632] niks3-server[1067]: 2026/09/20 09:34:41 INFO Completed upload id=436215worker2 # [ 122.648326] nix-grpc-daemon[1541]: time=2026-09-20T09:34:42.742Z level=INFO msg="Upload complete. (408ms)"6216worker2 # [ 122.650283] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=0 paths=1 nar_bytes_in=7686217lb # [ 122.653554] niks3-server[1067]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6218worker2 # [ 122.654694] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286219worker2 # [ 122.660268] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6220worker2 # [ 122.662491] nix-daemon[848]: reaped child process 1686, status = succeeded6221client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6222client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6223worker2 # [ 122.667766] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6224client # farm-top> LOG-farm-top-during-drain6225worker2 # [ 123.692908] nix-daemon[848]: reaped child process 1700, status = succeeded6226worker2 # [ 123.847685] nix-daemon[848]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6227worker2 # [ 123.859549] nix-daemon[848]: reaped child process 1718, status = succeeded6228lb # [ 123.878633] niks3-server[1067]: 2026/09/20 09:34:43 INFO Received uploads request method=POST path=/api/pending_closures6229worker2 # [ 123.883303] nix-grpc-daemon[1541]: time=2026-09-20T09:34:43.977Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6230worker2 # [ 123.884820] nix-grpc-daemon[1541]: time=2026-09-20T09:34:43.979Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6231lb # [ 123.916236] niks3-server[1067]: 2026/09/20 09:34:43 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6232lb # [ 123.921758] niks3-server[1067]: 2026/09/20 09:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6233lb # [ 123.930539] niks3-server[1067]: 2026/09/20 09:34:43 INFO Signed narinfos id=44 count=16234worker2 # [ 123.933302] nix-grpc-daemon[1541]: time=2026-09-20T09:34:44.027Z level=INFO msg="Uploading 1 narinfos"6235lb # [ 123.944162] niks3-server[1067]: 2026/09/20 09:34:43 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6236lb # [ 123.956391] niks3-server[1067]: 2026/09/20 09:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6237lb # [ 123.962710] niks3-server[1067]: 2026/09/20 09:34:43 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6238lb # [ 123.965623] niks3-server[1067]: 2026/09/20 09:34:43 INFO Completed upload id=446239worker2 # [ 123.963163] nix-grpc-daemon[1541]: time=2026-09-20T09:34:44.057Z level=INFO msg="Upload complete. (288ms)"6240lb # [ 123.967245] niks3-server[1067]: 2026/09/20 09:34:43 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6241worker2 # [ 123.966364] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=16242worker2 # [ 123.973105] nix-daemon[848]: reaped child process 1698, status = succeeded6243client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.73 seconds)6244client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6245client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6246worker1: must succeed: pkill -f 'read -t [9]0 x'6247worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6248client: waiting for success: ! systemctl is-active sw6249worker1 # [ 124.459421] nix-daemon[843]: reaped child process 1476, status = succeeded6250worker1 # [ 124.619416] nix-daemon[843]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6251worker1 # [ 124.630259] nix-daemon[843]: reaped child process 1520, status = succeeded6252lb # [ 124.655653] niks3-server[1067]: 2026/09/20 09:34:43 INFO Received uploads request method=POST path=/api/pending_closures6253worker1 # [ 124.653032] nix-grpc-daemon[1376]: time=2026-09-20T09:34:44.811Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6254worker1 # [ 124.654588] nix-grpc-daemon[1376]: time=2026-09-20T09:34:44.813Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6255lb # [ 124.685589] niks3-server[1067]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6256lb # [ 124.698204] niks3-server[1067]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6257lb # [ 124.709964] niks3-server[1067]: 2026/09/20 09:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6258worker1 # [ 124.704888] nix-grpc-daemon[1376]: time=2026-09-20T09:34:44.863Z level=INFO msg="Uploading 1 narinfos"6259lb # [ 124.715130] niks3-server[1067]: 2026/09/20 09:34:44 INFO Signed narinfos id=45 count=16260lb # [ 124.720879] niks3-server[1067]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6261lb # [ 124.729817] niks3-server[1067]: 2026/09/20 09:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6262lb # [ 124.734819] niks3-server[1067]: 2026/09/20 09:34:44 INFO Completed upload id=456263worker1 # [ 124.726468] nix-grpc-daemon[1376]: time=2026-09-20T09:34:44.885Z level=INFO msg="Upload complete. (286ms)"6264lb # [ 124.737682] niks3-server[1067]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6265worker1 # [ 124.734301] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48096 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=16266worker1 # [ 124.746317] nix-daemon[843]: reaped child process 1474, status = succeeded6267client # [ 124.742152] systemd[1]: sw.service: Deactivated successfully.6268client # [ 124.744026] systemd[1]: sw.service: Consumed 100ms CPU time over 9.140s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.6269worker1 # [ 125.320593] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:45Z level=info event=signal_exit6270client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6271client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6272client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6273worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6274worker2 # [ 126.551682] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:506646275worker1 # [ 126.547241] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:46Z level=info event=scheduler_restarting6276worker2 # [ 126.555546] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:46Z level=info event=worker_gone addr=192.168.1.3:500516277worker1 # [ 126.549951] nix-grpc-daemon[1312]: ts=2026-09-20T09:34:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6278worker1 # [ 126.561995] nix-daemon[843]: reaped child process 1365, status = succeeded6279worker1 # [ 126.642438] nix-grpc-daemon[1312]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6280worker1 # [ 126.643887] nix-grpc-daemon[1312]: E0000 00:00:1789896886.801276 1355 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6281worker1 # [ 126.645814] nix-grpc-daemon[1312]: E0000 00:00:1789896886.801488 1355 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6282worker1 # [ 127.344456] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6283worker1 # [ 127.346239] systemd[1]: nix-grpc-daemon.service: Consumed 791ms CPU time over 26.195s wall clock time, 32.8M memory peak, 23.8K incoming IP traffic, 33.4K outgoing IP traffic.6284worker1 # [ 127.352571] systemd[1]: Starting Nix worker-protocol over gRPC...6285worker1 # [ 127.498356] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6286worker1 # [ 127.501826] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16287worker1 # [ 127.508734] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6288worker1 # [ 127.512407] systemd[1]: Started Nix worker-protocol over gRPC.6289worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.16 seconds)6290lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6291lb: must succeed: curl -sf localhost:9901/clusters6292worker1 # [ 127.540356] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:47Z level=info event=scheduler_connected addr=lb:500516293worker2 # [ 127.550388] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06294lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6295lb # [ 127.973625] envoy[908]: conn peer=[2001:db8:1::1]:51484 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9598 tx=3835 ms=54966296lb # [ 127.977791] envoy[908]: conn peer=[2001:db8:1::1]:51472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1349 ms=89146297lb # [ 127.981769] envoy[908]: conn peer=[2001:db8:1::3]:55056 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1563 tx=475 ms=252116298lb: must succeed: curl -sf localhost:9901/clusters6299lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6300lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)6301lb: must succeed: curl -sf localhost:9901/clusters6302lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6303worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6304worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)6305(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.19 seconds)6306subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6307client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16308worker1 # [ 129.012294] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:399466309worker1 # [ 129.023071] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6310worker2 # [ 129.043806] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398286311worker2 # [ 129.085478] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:49Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06312worker2 # [ 129.087168] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:49Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16313worker2 # [ 129.088854] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:49Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06314worker2 # [ 129.090423] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:49Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06315worker1 # [ 129.123228] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466316worker1 # [ 129.136321] nix-grpc-daemon[1534]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 1/5)6317worker1 # [ 129.181870] nix-grpc-daemon[1534]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 179 ms (attempt 2/5)6318worker1 # [ 129.367337] nix-grpc-daemon[1534]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 269 ms (attempt 3/5)6319worker1 # [ 129.642282] nix-grpc-daemon[1534]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 641 ms (attempt 4/5)6320worker1 # [ 130.289780] nix-grpc-daemon[1534]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6321worker1 # [ 130.305585] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466322worker1 # [ 130.317462] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466323worker1 # [ 130.325862] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6324worker1 # [ 130.347345] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:50Z level=info event=niks3_push_started pid=15906325worker1 # [ 130.689507] nix-daemon[843]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)6326worker1 # [ 130.700470] nix-daemon[843]: reaped child process 1606, status = succeeded6327worker1 # [ 130.859122] nix-daemon[843]: accepted connection from pid 1608, user nix-grpc-daemon (trusted)6328worker1 # [ 130.870945] nix-daemon[843]: reaped child process 1611, status = succeeded6329lb # [ 130.897595] niks3-server[1067]: 2026/09/20 09:34:50 INFO Received uploads request method=POST path=/api/pending_closures6330worker1 # [ 130.896320] nix-grpc-daemon[1590]: time=2026-09-20T09:34:51.054Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6331worker1 # [ 130.897759] nix-grpc-daemon[1590]: time=2026-09-20T09:34:51.054Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6332lb # [ 130.931608] niks3-server[1067]: 2026/09/20 09:34:50 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6333lb # [ 130.940677] niks3-server[1067]: 2026/09/20 09:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6334worker1 # [ 130.935373] nix-grpc-daemon[1590]: time=2026-09-20T09:34:51.094Z level=INFO msg="Uploading 1 narinfos"6335lb # [ 130.945353] niks3-server[1067]: 2026/09/20 09:34:50 INFO Signed narinfos id=46 count=16336lb # [ 130.958600] niks3-server[1067]: 2026/09/20 09:34:50 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6337lb # [ 130.966556] niks3-server[1067]: 2026/09/20 09:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6338worker1 # [ 130.962021] nix-grpc-daemon[1590]: time=2026-09-20T09:34:51.120Z level=INFO msg="Upload complete. (425ms)"6339lb # [ 130.972266] niks3-server[1067]: 2026/09/20 09:34:50 INFO Completed upload id=466340worker1 # [ 130.964449] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=5046341lb # [ 130.975702] niks3-server[1067]: 2026/09/20 09:34:50 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6342worker1 # [ 130.971213] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466343worker1 # [ 130.980931] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6344worker1 # [ 130.984371] nix-daemon[843]: reaped child process 1588, status = succeeded6345worker1 # [ 130.989445] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6346worker1 # [ 132.203102] nix-daemon[843]: reaped child process 1615, status = succeeded6347worker1 # [ 132.361113] nix-daemon[843]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6348worker1 # [ 132.373510] nix-daemon[843]: reaped child process 1633, status = succeeded6349lb # [ 132.401642] niks3-server[1067]: 2026/09/20 09:34:51 INFO Received uploads request method=POST path=/api/pending_closures6350worker1 # [ 132.400306] nix-grpc-daemon[1590]: time=2026-09-20T09:34:52.558Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6351worker1 # [ 132.401786] nix-grpc-daemon[1590]: time=2026-09-20T09:34:52.558Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6352lb # [ 132.436625] niks3-server[1067]: 2026/09/20 09:34:51 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6353lb # [ 132.450187] niks3-server[1067]: 2026/09/20 09:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6354worker1 # [ 132.448918] nix-grpc-daemon[1590]: time=2026-09-20T09:34:52.607Z level=INFO msg="Uploading 1 narinfos"6355lb # [ 132.458507] niks3-server[1067]: 2026/09/20 09:34:51 INFO Signed narinfos id=47 count=16356lb # [ 132.465613] niks3-server[1067]: 2026/09/20 09:34:51 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6357lb # [ 132.471185] niks3-server[1067]: 2026/09/20 09:34:51 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6358lb # [ 132.479515] niks3-server[1067]: 2026/09/20 09:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6359lb # [ 132.484331] niks3-server[1067]: 2026/09/20 09:34:51 INFO Completed upload id=476360worker1 # [ 132.474351] nix-grpc-daemon[1590]: time=2026-09-20T09:34:52.633Z level=INFO msg="Upload complete. (278ms)"6361lb # [ 132.486535] niks3-server[1067]: 2026/09/20 09:34:51 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6362worker1 # [ 132.479929] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=6 outputs=16363worker2 # [ 132.488828] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:52Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16364worker1 # [ 132.487455] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466365worker2 # [ 132.498328] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:52Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06366worker1 # [ 132.497934] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466367worker1 # [ 132.500108] nix-daemon[843]: reaped child process 1613, status = succeeded6368worker1 # [ 132.503530] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466369worker1 # [ 132.508591] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6370worker1 # [ 132.660278] nix-daemon[843]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)6371worker1 # [ 132.673048] nix-daemon[843]: reaped child process 1640, status = succeeded6372worker1 # [ 132.841804] nix-daemon[843]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6373worker1 # [ 132.852714] nix-daemon[843]: reaped child process 1645, status = succeeded6374lb # [ 132.875100] niks3-server[1067]: 2026/09/20 09:34:52 INFO Received uploads request method=POST path=/api/pending_closures6375worker1 # [ 132.872389] nix-grpc-daemon[1590]: time=2026-09-20T09:34:53.031Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6376worker1 # [ 132.874049] nix-grpc-daemon[1590]: time=2026-09-20T09:34:53.033Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6377lb # [ 132.906302] niks3-server[1067]: 2026/09/20 09:34:52 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6378lb # [ 132.915873] niks3-server[1067]: 2026/09/20 09:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6379worker1 # [ 132.910688] nix-grpc-daemon[1590]: time=2026-09-20T09:34:53.069Z level=INFO msg="Uploading 1 narinfos"6380lb # [ 132.920817] niks3-server[1067]: 2026/09/20 09:34:52 INFO Signed narinfos id=48 count=16381lb # [ 132.927514] niks3-server[1067]: 2026/09/20 09:34:52 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6382lb # [ 132.937103] niks3-server[1067]: 2026/09/20 09:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6383lb # [ 132.941616] niks3-server[1067]: 2026/09/20 09:34:52 INFO Completed upload id=486384worker1 # [ 132.931353] nix-grpc-daemon[1590]: time=2026-09-20T09:34:53.090Z level=WARN msg="Failed to register uploaded object" key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo error="server returned 404: object is not pending upload\n"6385worker1 # [ 132.934737] nix-grpc-daemon[1590]: time=2026-09-20T09:34:53.092Z level=INFO msg="Upload complete. (420ms)"6386worker1 # [ 132.938472] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=5046387worker1 # [ 132.942180] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466388worker1 # [ 132.949061] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6389worker1 # [ 132.952456] nix-daemon[843]: reaped child process 1635, status = succeeded6390worker1 # [ 132.957648] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6391worker1 # [ 134.183949] nix-daemon[843]: reaped child process 1649, status = succeeded6392worker1 # [ 134.351517] nix-daemon[843]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6393worker1 # [ 134.364803] nix-daemon[843]: reaped child process 1667, status = succeeded6394lb # [ 134.392713] niks3-server[1067]: 2026/09/20 09:34:53 INFO Received uploads request method=POST path=/api/pending_closures6395worker1 # [ 134.390318] nix-grpc-daemon[1590]: time=2026-09-20T09:34:54.549Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6396worker1 # [ 134.392440] nix-grpc-daemon[1590]: time=2026-09-20T09:34:54.550Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6397lb # [ 134.422961] niks3-server[1067]: 2026/09/20 09:34:53 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6398lb # [ 134.437626] niks3-server[1067]: 2026/09/20 09:34:53 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6399lb # [ 134.446215] niks3-server[1067]: 2026/09/20 09:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6400lb # [ 134.452620] niks3-server[1067]: 2026/09/20 09:34:53 INFO Signed narinfos id=49 count=16401worker1 # [ 134.442788] nix-grpc-daemon[1590]: time=2026-09-20T09:34:54.601Z level=INFO msg="Uploading 1 narinfos"6402lb # [ 134.456285] niks3-server[1067]: 2026/09/20 09:34:53 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6403lb # [ 134.464823] niks3-server[1067]: 2026/09/20 09:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6404lb # [ 134.473435] niks3-server[1067]: 2026/09/20 09:34:53 INFO Completed upload id=496405lb # [ 134.474627] niks3-server[1067]: 2026/09/20 09:34:53 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6406worker1 # [ 134.463389] nix-grpc-daemon[1590]: time=2026-09-20T09:34:54.622Z level=INFO msg="Upload complete. (298ms)"6407worker2 # [ 134.476064] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:54Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16408worker2 # [ 134.481297] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:54Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06409worker1 # [ 134.471676] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=7 outputs=16410worker1 # [ 134.477340] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466411worker1 # [ 134.486213] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466412worker1 # [ 134.493515] nix-daemon[843]: reaped child process 1647, status = succeeded6413worker1 # [ 134.498182] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466414worker1 # [ 134.502162] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6415worker1 # [ 134.660812] nix-daemon[843]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)6416worker1 # [ 134.672620] nix-daemon[843]: reaped child process 1674, status = succeeded6417worker1 # [ 134.835422] nix-daemon[843]: accepted connection from pid 1676, user nix-grpc-daemon (trusted)6418worker1 # [ 134.847034] nix-daemon[843]: reaped child process 1679, status = succeeded6419lb # [ 134.873416] niks3-server[1067]: 2026/09/20 09:34:54 INFO Received uploads request method=POST path=/api/pending_closures6420worker1 # [ 134.867223] nix-grpc-daemon[1590]: time=2026-09-20T09:34:55.026Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6421worker1 # [ 134.868787] nix-grpc-daemon[1590]: time=2026-09-20T09:34:55.027Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6422lb # [ 134.889239] niks3-server[1067]: 2026/09/20 09:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6423lb # [ 134.891076] niks3-server[1067]: 2026/09/20 09:34:54 INFO Signed narinfos id=50 count=16424worker1 # [ 134.882147] nix-grpc-daemon[1590]: time=2026-09-20T09:34:55.041Z level=INFO msg="Uploading 1 narinfos"6425lb # [ 134.894627] niks3-server[1067]: 2026/09/20 09:34:54 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6426lb # [ 134.896817] niks3-server[1067]: 2026/09/20 09:34:54 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6427lb # [ 134.901900] niks3-server[1067]: 2026/09/20 09:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6428lb # [ 134.904358] niks3-server[1067]: 2026/09/20 09:34:54 INFO Completed upload id=506429worker1 # [ 134.894287] nix-grpc-daemon[1590]: time=2026-09-20T09:34:55.053Z level=INFO msg="Upload complete. (386ms)"6430lb # [ 134.907142] niks3-server[1067]: 2026/09/20 09:34:54 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6431worker1 # [ 134.896314] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=5046432worker1 # [ 134.899185] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466433worker1 # [ 134.904956] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6434worker1 # [ 134.909299] nix-daemon[843]: reaped child process 1669, status = succeeded6435worker1 # [ 134.913377] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6436worker1 # [ 135.933365] nix-daemon[843]: reaped child process 1683, status = succeeded6437worker1 # [ 136.087933] nix-daemon[843]: accepted connection from pid 1698, user nix-grpc-daemon (trusted)6438worker1 # [ 136.099649] nix-daemon[843]: reaped child process 1701, status = succeeded6439lb # [ 136.125258] niks3-server[1067]: 2026/09/20 09:34:55 INFO Received uploads request method=POST path=/api/pending_closures6440worker1 # [ 136.122449] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.281Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6441worker1 # [ 136.125066] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.283Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6442lb # [ 136.142879] niks3-server[1067]: 2026/09/20 09:34:55 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6443lb # [ 136.148468] niks3-server[1067]: 2026/09/20 09:34:55 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6444lb # [ 136.152721] niks3-server[1067]: 2026/09/20 09:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6445lb # [ 136.154412] niks3-server[1067]: 2026/09/20 09:34:55 INFO Signed narinfos id=51 count=16446worker1 # [ 136.144591] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.303Z level=INFO msg="Uploading 1 narinfos"6447lb # [ 136.160053] niks3-server[1067]: 2026/09/20 09:34:55 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6448lb # [ 136.161900] niks3-server[1067]: 2026/09/20 09:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6449lb # [ 136.165166] niks3-server[1067]: 2026/09/20 09:34:55 INFO Completed upload id=516450worker1 # [ 136.156272] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.315Z level=INFO msg="Upload complete. (236ms)"6451lb # [ 136.166895] niks3-server[1067]: 2026/09/20 09:34:55 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6452worker1 # [ 136.158899] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=8 outputs=16453worker1 # [ 136.162589] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466454worker2 # [ 136.168317] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:56Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16455worker1 # [ 136.169301] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466456worker1 # [ 136.171810] nix-daemon[843]: reaped child process 1681, status = succeeded6457worker2 # [ 136.177800] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:56Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06458worker1 # [ 136.175421] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.334Z level=INFO msg="All 1 paths already cached"6459worker1 # [ 136.179079] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466460worker1 # [ 136.183176] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6461worker1 # [ 136.339220] nix-daemon[843]: accepted connection from pid 1705, user nix-grpc-daemon (trusted)6462worker1 # [ 136.352902] nix-daemon[843]: reaped child process 1708, status = succeeded6463worker1 # [ 136.520024] nix-daemon[843]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6464worker1 # [ 136.531777] nix-daemon[843]: reaped child process 1713, status = succeeded6465lb # [ 136.557460] niks3-server[1067]: 2026/09/20 09:34:55 INFO Received uploads request method=POST path=/api/pending_closures6466worker1 # [ 136.553039] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.712Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6467worker1 # [ 136.554553] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.713Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6468lb # [ 136.582433] niks3-server[1067]: 2026/09/20 09:34:55 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6469lb # [ 136.594461] niks3-server[1067]: 2026/09/20 09:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6470lb # [ 136.599525] niks3-server[1067]: 2026/09/20 09:34:55 INFO Signed narinfos id=52 count=16471worker1 # [ 136.592357] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.751Z level=INFO msg="Uploading 1 narinfos"6472lb # [ 136.605054] niks3-server[1067]: 2026/09/20 09:34:55 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6473lb # [ 136.618290] niks3-server[1067]: 2026/09/20 09:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6474worker1 # [ 136.612821] nix-grpc-daemon[1590]: time=2026-09-20T09:34:56.771Z level=INFO msg="Upload complete. (423ms)"6475lb # [ 136.622962] niks3-server[1067]: 2026/09/20 09:34:55 INFO Completed upload id=526476lb # [ 136.625875] niks3-server[1067]: 2026/09/20 09:34:55 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6477worker1 # [ 136.616133] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=6246478worker1 # [ 136.620796] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466479worker1 # [ 136.629280] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6480worker1 # [ 136.632149] nix-daemon[843]: reaped child process 1703, status = succeeded6481worker1 # [ 136.637241] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6482worker1 # [ 137.862758] nix-daemon[843]: reaped child process 1717, status = succeeded6483worker1 # [ 138.023767] nix-daemon[843]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)6484worker1 # [ 138.034165] nix-daemon[843]: reaped child process 1735, status = succeeded6485lb # [ 138.061844] niks3-server[1067]: 2026/09/20 09:34:57 INFO Received uploads request method=POST path=/api/pending_closures6486worker1 # [ 138.059700] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.218Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6487worker1 # [ 138.061190] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.220Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6488lb # [ 138.102879] niks3-server[1067]: 2026/09/20 09:34:57 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6489lb # [ 138.109470] niks3-server[1067]: 2026/09/20 09:34:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6490worker1 # [ 138.103619] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.262Z level=INFO msg="Uploading 1 narinfos"6491lb # [ 138.113272] niks3-server[1067]: 2026/09/20 09:34:57 INFO Signed narinfos id=53 count=16492lb # [ 138.119617] niks3-server[1067]: 2026/09/20 09:34:57 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6493lb # [ 138.130378] niks3-server[1067]: 2026/09/20 09:34:57 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6494lb # [ 138.135108] niks3-server[1067]: 2026/09/20 09:34:57 INFO Completed upload id=536495worker1 # [ 138.124924] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.283Z level=INFO msg="Upload complete. (280ms)"6496lb # [ 138.136377] niks3-server[1067]: 2026/09/20 09:34:57 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6497lb # [ 138.139149] niks3-server[1067]: 2026/09/20 09:34:57 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6498worker1 # [ 138.130262] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=9 outputs=16499worker2 # [ 138.137828] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:58Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16500worker1 # [ 138.137431] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466501worker2 # [ 138.148372] nix-grpc-daemon[1471]: ts=2026-09-20T09:34:58Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06502worker1 # [ 138.147841] nix-daemon[843]: reaped child process 1715, status = succeeded6503worker1 # [ 138.149305] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466504worker1 # [ 138.154703] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.313Z level=INFO msg="All 1 paths already cached"6505worker1 # [ 138.158424] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466506worker1 # [ 138.162212] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6507worker1 # [ 138.326514] nix-daemon[843]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6508worker1 # [ 138.338633] nix-daemon[843]: reaped child process 1742, status = succeeded6509worker1 # [ 138.504940] nix-daemon[843]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6510worker1 # [ 138.517258] nix-daemon[843]: reaped child process 1747, status = succeeded6511lb # [ 138.539461] niks3-server[1067]: 2026/09/20 09:34:57 INFO Received uploads request method=POST path=/api/pending_closures6512worker1 # [ 138.538623] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.697Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6513worker1 # [ 138.540144] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.699Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6514lb # [ 138.573376] niks3-server[1067]: 2026/09/20 09:34:57 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6515lb # [ 138.582984] niks3-server[1067]: 2026/09/20 09:34:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6516lb # [ 138.589678] niks3-server[1067]: 2026/09/20 09:34:57 INFO Signed narinfos id=54 count=16517worker1 # [ 138.582414] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.741Z level=INFO msg="Uploading 1 narinfos"6518lb # [ 138.593105] niks3-server[1067]: 2026/09/20 09:34:57 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6519lb # [ 138.605657] niks3-server[1067]: 2026/09/20 09:34:57 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6520worker1 # [ 138.601656] nix-grpc-daemon[1590]: time=2026-09-20T09:34:58.760Z level=INFO msg="Upload complete. (433ms)"6521lb # [ 138.611904] niks3-server[1067]: 2026/09/20 09:34:57 INFO Completed upload id=546522lb # [ 138.614150] niks3-server[1067]: 2026/09/20 09:34:57 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6523worker1 # [ 138.604760] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=6246524worker1 # [ 138.610070] nix-grpc-daemon[1534]: ts=2026-09-20T09:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466525worker1 # [ 138.621035] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6526worker1 # [ 138.626148] nix-daemon[843]: reaped child process 1737, status = succeeded6527worker1 # [ 138.630615] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6528worker1 # [ 139.857296] nix-daemon[843]: reaped child process 1751, status = succeeded6529worker1 # [ 139.994244] nix-daemon[843]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6530worker1 # [ 140.006706] nix-daemon[843]: reaped child process 1769, status = succeeded6531lb # [ 140.034235] niks3-server[1067]: 2026/09/20 09:34:59 INFO Received uploads request method=POST path=/api/pending_closures6532worker1 # [ 140.032716] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.191Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6533worker1 # [ 140.034570] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.193Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6534lb # [ 140.075112] niks3-server[1067]: 2026/09/20 09:34:59 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6535lb # [ 140.083856] niks3-server[1067]: 2026/09/20 09:34:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6536lb # [ 140.089760] niks3-server[1067]: 2026/09/20 09:34:59 INFO Signed narinfos id=55 count=16537worker1 # [ 140.083229] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.242Z level=INFO msg="Uploading 1 narinfos"6538lb # [ 140.093892] niks3-server[1067]: 2026/09/20 09:34:59 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6539lb # [ 140.099866] niks3-server[1067]: 2026/09/20 09:34:59 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6540lb # [ 140.107888] niks3-server[1067]: 2026/09/20 09:34:59 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6541lb # [ 140.110714] niks3-server[1067]: 2026/09/20 09:34:59 INFO Completed upload id=556542worker1 # [ 140.101830] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.260Z level=INFO msg="Upload complete. (263ms)"6543lb # [ 140.113556] niks3-server[1067]: 2026/09/20 09:34:59 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6544worker1 # [ 140.106855] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=10 outputs=16545worker2 # [ 140.115486] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:00Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16546worker2 # [ 140.125308] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:00Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06547worker1 # [ 140.119831] nix-daemon[843]: reaped child process 1749, status = succeeded6548worker1 # [ 140.158762] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466549worker1 # [ 140.170737] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466550worker1 # [ 140.183461] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.342Z level=INFO msg="All 1 paths already cached"6551worker1 # [ 140.190916] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466552worker1 # [ 140.201158] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6553worker1 # [ 140.375872] nix-daemon[843]: accepted connection from pid 1773, user nix-grpc-daemon (trusted)6554worker1 # [ 140.389128] nix-daemon[843]: reaped child process 1776, status = succeeded6555worker1 # [ 140.558227] nix-daemon[843]: accepted connection from pid 1778, user nix-grpc-daemon (trusted)6556worker1 # [ 140.570237] nix-daemon[843]: reaped child process 1781, status = succeeded6557lb # [ 140.597456] niks3-server[1067]: 2026/09/20 09:34:59 INFO Received uploads request method=POST path=/api/pending_closures6558worker1 # [ 140.594703] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.753Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6559worker1 # [ 140.596100] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.753Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6560lb # [ 140.619712] niks3-server[1067]: 2026/09/20 09:34:59 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6561lb # [ 140.632122] niks3-server[1067]: 2026/09/20 09:34:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6562worker1 # [ 140.627370] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.786Z level=INFO msg="Uploading 1 narinfos"6563lb # [ 140.637418] niks3-server[1067]: 2026/09/20 09:34:59 INFO Signed narinfos id=56 count=16564lb # [ 140.643640] niks3-server[1067]: 2026/09/20 09:34:59 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6565lb # [ 140.654874] niks3-server[1067]: 2026/09/20 09:34:59 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6566worker1 # [ 140.650877] nix-grpc-daemon[1590]: time=2026-09-20T09:35:00.809Z level=INFO msg="Upload complete. (438ms)"6567lb # [ 140.661094] niks3-server[1067]: 2026/09/20 09:34:59 INFO Completed upload id=566568lb # [ 140.664162] niks3-server[1067]: 2026/09/20 09:34:59 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6569worker1 # [ 140.652972] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=6246570worker1 # [ 140.658252] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466571worker1 # [ 140.667698] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6572worker1 # [ 140.670976] nix-daemon[843]: reaped child process 1771, status = succeeded6573worker1 # [ 140.676046] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6574worker1 # [ 142.101609] nix-daemon[843]: reaped child process 1785, status = succeeded6575worker1 # [ 142.266264] nix-daemon[843]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6576worker1 # [ 142.278111] nix-daemon[843]: reaped child process 1803, status = succeeded6577lb # [ 142.305639] niks3-server[1067]: 2026/09/20 09:35:01 INFO Received uploads request method=POST path=/api/pending_closures6578worker1 # [ 142.303181] nix-grpc-daemon[1590]: time=2026-09-20T09:35:02.462Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6579worker1 # [ 142.304698] nix-grpc-daemon[1590]: time=2026-09-20T09:35:02.463Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6580lb # [ 142.345145] niks3-server[1067]: 2026/09/20 09:35:01 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6581lb # [ 142.354774] niks3-server[1067]: 2026/09/20 09:35:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6582worker1 # [ 142.350947] nix-grpc-daemon[1590]: time=2026-09-20T09:35:02.509Z level=INFO msg="Uploading 1 narinfos"6583lb # [ 142.361659] niks3-server[1067]: 2026/09/20 09:35:01 INFO Signed narinfos id=57 count=16584lb # [ 142.364158] niks3-server[1067]: 2026/09/20 09:35:01 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6585lb # [ 142.371466] niks3-server[1067]: 2026/09/20 09:35:01 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6586lb # [ 142.381151] niks3-server[1067]: 2026/09/20 09:35:01 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6587lb # [ 142.387108] niks3-server[1067]: 2026/09/20 09:35:01 INFO Completed upload id=576588lb # [ 142.388179] niks3-server[1067]: 2026/09/20 09:35:01 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6589worker1 # [ 142.376996] nix-grpc-daemon[1590]: time=2026-09-20T09:35:02.535Z level=INFO msg="Upload complete. (291ms)"6590worker2 # [ 142.393647] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:02Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16591worker1 # [ 142.382545] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=11 outputs=16592worker1 # [ 142.391543] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466593worker2 # [ 142.400843] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:02Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06594worker1 # [ 142.403523] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466595worker1 # [ 142.408924] nix-daemon[843]: reaped child process 1783, status = succeeded6596worker1 # [ 142.414498] nix-grpc-daemon[1590]: time=2026-09-20T09:35:02.573Z level=INFO msg="All 2 paths already cached"6597worker1 # [ 142.420649] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466598worker1 # [ 142.427157] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6599worker1 # [ 142.584973] nix-daemon[843]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)6600worker1 # [ 142.596979] nix-daemon[843]: reaped child process 1810, status = succeeded6601worker1 # [ 142.771301] nix-daemon[843]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6602worker1 # [ 142.783583] nix-daemon[843]: reaped child process 1815, status = succeeded6603lb # [ 142.810380] niks3-server[1067]: 2026/09/20 09:35:02 INFO Received uploads request method=POST path=/api/pending_closures6604worker1 # [ 142.807340] nix-grpc-daemon[1590]: time=2026-09-20T09:35:02.966Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6605worker1 # [ 142.808907] nix-grpc-daemon[1590]: time=2026-09-20T09:35:02.967Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6606lb # [ 142.837514] niks3-server[1067]: 2026/09/20 09:35:02 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6607lb # [ 142.848590] niks3-server[1067]: 2026/09/20 09:35:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6608lb # [ 142.854097] niks3-server[1067]: 2026/09/20 09:35:02 INFO Signed narinfos id=58 count=16609worker1 # [ 142.846494] nix-grpc-daemon[1590]: time=2026-09-20T09:35:03.005Z level=INFO msg="Uploading 1 narinfos"6610lb # [ 142.858607] niks3-server[1067]: 2026/09/20 09:35:02 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6611lb # [ 142.870089] niks3-server[1067]: 2026/09/20 09:35:02 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6612lb # [ 142.875702] niks3-server[1067]: 2026/09/20 09:35:02 INFO Completed upload id=586613worker1 # [ 142.865802] nix-grpc-daemon[1590]: time=2026-09-20T09:35:03.024Z level=INFO msg="Upload complete. (432ms)"6614lb # [ 142.880375] niks3-server[1067]: 2026/09/20 09:35:02 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6615worker1 # [ 142.869667] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=7526616worker1 # [ 142.875683] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466617worker1 # [ 142.884617] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6618worker1 # [ 142.890886] nix-daemon[843]: reaped child process 1805, status = succeeded6619worker1 # [ 142.897359] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6620worker1 # [ 143.908660] nix-daemon[843]: reaped child process 1819, status = succeeded6621worker1 # [ 144.052200] nix-daemon[843]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6622worker1 # [ 144.063767] nix-daemon[843]: reaped child process 1837, status = succeeded6623lb # [ 144.091948] niks3-server[1067]: 2026/09/20 09:35:03 INFO Received uploads request method=POST path=/api/pending_closures6624worker1 # [ 144.090691] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.249Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6625worker1 # [ 144.092260] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.251Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6626lb # [ 144.135373] niks3-server[1067]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6627lb # [ 144.142979] niks3-server[1067]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6628lb # [ 144.152252] niks3-server[1067]: 2026/09/20 09:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6629lb # [ 144.157079] niks3-server[1067]: 2026/09/20 09:35:03 INFO Signed narinfos id=59 count=16630worker1 # [ 144.149648] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.308Z level=INFO msg="Uploading 1 narinfos"6631lb # [ 144.164377] niks3-server[1067]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6632lb # [ 144.174924] niks3-server[1067]: 2026/09/20 09:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6633lb # [ 144.180515] niks3-server[1067]: 2026/09/20 09:35:03 INFO Completed upload id=596634worker1 # [ 144.170584] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.329Z level=INFO msg="Upload complete. (267ms)"6635lb # [ 144.182526] niks3-server[1067]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6636worker2 # [ 144.184635] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:04Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16637worker1 # [ 144.176888] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=12 outputs=16638worker1 # [ 144.184146] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466639worker1 # [ 144.195544] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466640worker1 # [ 144.200910] nix-daemon[843]: reaped child process 1817, status = succeeded6641worker1 # [ 144.206238] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.365Z level=INFO msg="All 2 paths already cached"6642worker1 # [ 144.212061] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466643worker1 # [ 144.219892] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6644worker1 # [ 144.376802] nix-daemon[843]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)6645worker1 # [ 144.390876] nix-daemon[843]: reaped child process 1844, status = succeeded6646worker1 # [ 144.555084] nix-daemon[843]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6647worker1 # [ 144.565578] nix-daemon[843]: reaped child process 1849, status = succeeded6648lb # [ 144.592288] niks3-server[1067]: 2026/09/20 09:35:03 INFO Received uploads request method=POST path=/api/pending_closures6649worker1 # [ 144.590366] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.749Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6650worker1 # [ 144.591957] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.751Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6651lb # [ 144.620463] niks3-server[1067]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6652lb # [ 144.633172] niks3-server[1067]: 2026/09/20 09:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6653worker1 # [ 144.630130] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.789Z level=INFO msg="Uploading 1 narinfos"6654lb # [ 144.640333] niks3-server[1067]: 2026/09/20 09:35:03 INFO Signed narinfos id=60 count=16655lb # [ 144.644551] niks3-server[1067]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6656lb # [ 144.654722] niks3-server[1067]: 2026/09/20 09:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6657worker1 # [ 144.651591] nix-grpc-daemon[1590]: time=2026-09-20T09:35:04.810Z level=INFO msg="Upload complete. (427ms)"6658lb # [ 144.661768] niks3-server[1067]: 2026/09/20 09:35:03 INFO Completed upload id=606659worker1 # [ 144.653610] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=7526660lb # [ 144.665234] niks3-server[1067]: 2026/09/20 09:35:03 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6661worker1 # [ 144.658976] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466662worker1 # [ 144.664129] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6663worker1 # [ 144.671064] nix-daemon[843]: reaped child process 1839, status = succeeded6664worker1 # [ 144.674414] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6665worker1 # [ 145.499595] nix-daemon[843]: reaped child process 1853, status = succeeded6666worker1 # [ 145.648282] nix-daemon[843]: accepted connection from pid 1868, user nix-grpc-daemon (trusted)6667worker1 # [ 145.660099] nix-daemon[843]: reaped child process 1871, status = succeeded6668lb # [ 145.686485] niks3-server[1067]: 2026/09/20 09:35:05 INFO Received uploads request method=POST path=/api/pending_closures6669worker1 # [ 145.685170] nix-grpc-daemon[1590]: time=2026-09-20T09:35:05.844Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6670worker1 # [ 145.686669] nix-grpc-daemon[1590]: time=2026-09-20T09:35:05.845Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6671lb # [ 145.720461] niks3-server[1067]: 2026/09/20 09:35:05 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6672lb # [ 145.731819] niks3-server[1067]: 2026/09/20 09:35:05 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6673lb # [ 145.742851] niks3-server[1067]: 2026/09/20 09:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6674lb # [ 145.746569] niks3-server[1067]: 2026/09/20 09:35:05 INFO Signed narinfos id=61 count=16675worker1 # [ 145.740827] nix-grpc-daemon[1590]: time=2026-09-20T09:35:05.899Z level=INFO msg="Uploading 1 narinfos"6676lb # [ 145.754045] niks3-server[1067]: 2026/09/20 09:35:05 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6677lb # [ 145.763516] niks3-server[1067]: 2026/09/20 09:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6678lb # [ 145.769199] niks3-server[1067]: 2026/09/20 09:35:05 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6679lb # [ 145.771603] niks3-server[1067]: 2026/09/20 09:35:05 INFO Completed upload id=616680worker1 # [ 145.761624] nix-grpc-daemon[1590]: time=2026-09-20T09:35:05.920Z level=INFO msg="Upload complete. (280ms)"6681worker2 # [ 145.775499] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:05Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16682worker1 # [ 145.766801] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=13 outputs=16683worker1 # [ 145.781100] nix-daemon[843]: reaped child process 1851, status = succeeded6684worker1 # [ 145.812226] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466685worker1 # [ 145.824694] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466686worker1 # [ 145.836927] nix-grpc-daemon[1590]: time=2026-09-20T09:35:05.995Z level=INFO msg="All 2 paths already cached"6687worker1 # [ 145.844170] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399466688worker1 # [ 145.854130] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6689worker1 # [ 146.020240] nix-daemon[843]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6690worker1 # [ 146.032429] nix-daemon[843]: reaped child process 1878, status = succeeded6691worker1 # [ 146.192147] nix-daemon[843]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6692worker1 # [ 146.203682] nix-daemon[843]: reaped child process 1883, status = succeeded6693lb # [ 146.230482] niks3-server[1067]: 2026/09/20 09:35:05 INFO Received uploads request method=POST path=/api/pending_closures6694worker1 # [ 146.228423] nix-grpc-daemon[1590]: time=2026-09-20T09:35:06.387Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6695worker1 # [ 146.229917] nix-grpc-daemon[1590]: time=2026-09-20T09:35:06.389Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6696lb # [ 146.259443] niks3-server[1067]: 2026/09/20 09:35:05 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6697lb # [ 146.271679] niks3-server[1067]: 2026/09/20 09:35:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6698worker1 # [ 146.268083] nix-grpc-daemon[1590]: time=2026-09-20T09:35:06.426Z level=INFO msg="Uploading 1 narinfos"6699lb # [ 146.278179] niks3-server[1067]: 2026/09/20 09:35:05 INFO Signed narinfos id=62 count=16700lb # [ 146.282478] niks3-server[1067]: 2026/09/20 09:35:05 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6701lb # [ 146.294441] niks3-server[1067]: 2026/09/20 09:35:05 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6702worker1 # [ 146.290385] nix-grpc-daemon[1590]: time=2026-09-20T09:35:06.449Z level=INFO msg="Upload complete. (424ms)"6703lb # [ 146.300707] niks3-server[1067]: 2026/09/20 09:35:05 INFO Completed upload id=626704worker1 # [ 146.292631] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=0 paths=1 nar_bytes_in=7526705lb # [ 146.304106] niks3-server[1067]: 2026/09/20 09:35:05 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6706worker1 # [ 146.297949] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399466707worker1 # [ 146.308105] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6708worker1 # [ 146.315135] nix-daemon[843]: reaped child process 1873, status = succeeded6709worker1 # [ 146.318688] nix-daemon[843]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6710worker1 # [ 147.330504] nix-daemon[843]: reaped child process 1887, status = succeeded6711worker1 # [ 147.480748] nix-daemon[843]: accepted connection from pid 1902, user nix-grpc-daemon (trusted)6712worker1 # [ 147.492645] nix-daemon[843]: reaped child process 1905, status = succeeded6713lb # [ 147.521644] niks3-server[1067]: 2026/09/20 09:35:06 INFO Received uploads request method=POST path=/api/pending_closures6714worker1 # [ 147.519297] nix-grpc-daemon[1590]: time=2026-09-20T09:35:07.678Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6715worker1 # [ 147.522059] nix-grpc-daemon[1590]: time=2026-09-20T09:35:07.680Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6716lb # [ 147.561628] niks3-server[1067]: 2026/09/20 09:35:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6717lb # [ 147.566958] niks3-server[1067]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6718lb # [ 147.574379] niks3-server[1067]: 2026/09/20 09:35:06 INFO Signed narinfos id=63 count=16719worker1 # [ 147.567513] nix-grpc-daemon[1590]: time=2026-09-20T09:35:07.726Z level=INFO msg="Uploading 1 narinfos"6720lb # [ 147.582673] niks3-server[1067]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6721lb # [ 147.587355] niks3-server[1067]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6722lb # [ 147.596250] niks3-server[1067]: 2026/09/20 09:35:06 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6723lb # [ 147.601321] niks3-server[1067]: 2026/09/20 09:35:06 INFO Completed upload id=636724worker1 # [ 147.591182] nix-grpc-daemon[1590]: time=2026-09-20T09:35:07.750Z level=INFO msg="Upload complete. (266ms)"6725lb # [ 147.603709] niks3-server[1067]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6726worker1 # [ 147.595710] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39946 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=14 outputs=16727worker1 # [ 147.606426] nix-daemon[843]: reaped child process 1885, status = succeeded6728client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.92 seconds)6729worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6730worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.07 seconds)6731client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16732lb # [ 147.973847] envoy[908]: conn peer=[2001:db8:1::1]:51608 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27098 tx=10617 ms=186476733worker2 # [ 147.996131] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:398286734worker1 # [ 147.991366] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466735worker2 # [ 148.021284] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:08Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06736client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.29 seconds)6737(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.27 seconds)6738subtest: low disk drains a worker and builds go to the other6739worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6740worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.99 seconds)6741worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6742worker1 # [ 179.530554] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:39Z level=info event=unhealthy reason=min_free6743worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.08 seconds)6744client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26745worker2 # [ 180.325743] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:398286746client # don't know how to build these paths:6747client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6748worker1 # [ 180.324550] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399466749worker2 # [ 180.350172] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:40Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06750worker2 # [ 180.355394] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:40Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16751worker2 # [ 180.394292] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286752worker2 # [ 180.407411] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398286753client # copying 1 paths to 'lb:50051'...6754client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6755worker2 # [ 180.420418] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:398286756worker2 # [ 180.432381] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6757worker2 # [ 180.591326] nix-daemon[848]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6758worker2 # [ 180.603232] nix-daemon[848]: reaped child process 1748, status = succeeded6759worker2 # [ 180.737746] nix-daemon[848]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6760worker2 # [ 180.749112] nix-daemon[848]: reaped child process 1753, status = succeeded6761lb # [ 180.765458] niks3-server[1067]: 2026/09/20 09:35:40 INFO Received uploads request method=POST path=/api/pending_closures6762worker2 # [ 180.769320] nix-grpc-daemon[1541]: time=2026-09-20T09:35:40.863Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6763worker2 # [ 180.770677] nix-grpc-daemon[1541]: time=2026-09-20T09:35:40.863Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6764lb # [ 180.782159] niks3-server[1067]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6765lb # [ 180.786856] niks3-server[1067]: 2026/09/20 09:35:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6766lb # [ 180.789214] niks3-server[1067]: 2026/09/20 09:35:40 INFO Signed narinfos id=64 count=16767worker2 # [ 180.786766] nix-grpc-daemon[1541]: time=2026-09-20T09:35:40.880Z level=INFO msg="Uploading 1 narinfos"6768lb # [ 180.792190] niks3-server[1067]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6769lb # [ 180.797072] niks3-server[1067]: 2026/09/20 09:35:40 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6770lb # [ 180.801393] niks3-server[1067]: 2026/09/20 09:35:40 INFO Completed upload id=646771worker2 # [ 180.798825] nix-grpc-daemon[1541]: time=2026-09-20T09:35:40.893Z level=INFO msg="Upload complete. (354ms)"6772worker2 # [ 180.800846] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=0 paths=1 nar_bytes_in=5126773lb # [ 180.804053] niks3-server[1067]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6774worker2 # [ 180.803856] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286775worker2 # [ 180.809972] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6776worker2 # [ 180.812141] nix-daemon[848]: reaped child process 1743, status = succeeded6777worker2 # [ 180.817342] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6778client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6779client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6780client # farm-a> LOG-farm-a-drain6781worker2 # [ 181.630530] nix-daemon[848]: reaped child process 1757, status = succeeded6782worker2 # [ 181.768824] nix-daemon[848]: accepted connection from pid 1772, user nix-grpc-daemon (trusted)6783worker2 # [ 181.780504] nix-daemon[848]: reaped child process 1775, status = succeeded6784lb # [ 181.797631] niks3-server[1067]: 2026/09/20 09:35:41 INFO Received uploads request method=POST path=/api/pending_closures6785worker2 # [ 181.806708] nix-grpc-daemon[1541]: time=2026-09-20T09:35:41.900Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6786worker2 # [ 181.808668] nix-grpc-daemon[1541]: time=2026-09-20T09:35:41.902Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6787lb # [ 181.839679] niks3-server[1067]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6788lb # [ 181.848451] niks3-server[1067]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6789lb # [ 181.855358] niks3-server[1067]: 2026/09/20 09:35:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6790worker2 # [ 181.858513] nix-grpc-daemon[1541]: time=2026-09-20T09:35:41.952Z level=INFO msg="Uploading 1 narinfos"6791lb # [ 181.862289] niks3-server[1067]: 2026/09/20 09:35:41 INFO Signed narinfos id=65 count=16792lb # [ 181.864655] niks3-server[1067]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6793lb # [ 181.869559] niks3-server[1067]: 2026/09/20 09:35:41 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6794lb # [ 181.873356] niks3-server[1067]: 2026/09/20 09:35:41 INFO Completed upload id=656795worker2 # [ 181.870934] nix-grpc-daemon[1541]: time=2026-09-20T09:35:41.965Z level=INFO msg="Upload complete. (247ms)"6796lb # [ 181.875326] niks3-server[1067]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6797worker2 # [ 181.876504] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=15 outputs=16798worker2 # [ 181.886642] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:41Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06799worker2 # [ 181.890950] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:41Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16800worker2 # [ 181.894501] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286801worker2 # [ 181.899929] nix-daemon[848]: reaped child process 1755, status = succeeded6802worker2 # [ 181.903607] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398286803worker2 # [ 181.909763] nix-grpc-daemon[1541]: time=2026-09-20T09:35:42.004Z level=INFO msg="All 1 paths already cached"6804client # copying 1 paths to 'lb:50051'...6805worker2 # [ 181.914860] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:398286806client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6807worker2 # [ 181.920738] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6808worker2 # [ 182.066223] nix-daemon[848]: accepted connection from pid 1781, user nix-grpc-daemon (trusted)6809worker2 # [ 182.078235] nix-daemon[848]: reaped child process 1784, status = succeeded6810worker2 # [ 182.228333] nix-daemon[848]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6811worker2 # [ 182.239960] nix-daemon[848]: reaped child process 1789, status = succeeded6812lb # [ 182.254941] niks3-server[1067]: 2026/09/20 09:35:41 INFO Received uploads request method=POST path=/api/pending_closures6813worker2 # [ 182.261126] nix-grpc-daemon[1541]: time=2026-09-20T09:35:42.355Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6814worker2 # [ 182.262597] nix-grpc-daemon[1541]: time=2026-09-20T09:35:42.356Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6815lb # [ 182.284856] niks3-server[1067]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6816lb # [ 182.295510] niks3-server[1067]: 2026/09/20 09:35:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6817lb # [ 182.300908] niks3-server[1067]: 2026/09/20 09:35:41 INFO Signed narinfos id=66 count=16818worker2 # [ 182.300618] nix-grpc-daemon[1541]: time=2026-09-20T09:35:42.394Z level=INFO msg="Uploading 1 narinfos"6819lb # [ 182.306097] niks3-server[1067]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6820lb # [ 182.316279] niks3-server[1067]: 2026/09/20 09:35:41 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6821lb # [ 182.322672] niks3-server[1067]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6822lb # [ 182.325260] niks3-server[1067]: 2026/09/20 09:35:41 INFO Completed upload id=666823worker2 # [ 182.323905] nix-grpc-daemon[1541]: time=2026-09-20T09:35:42.418Z level=INFO msg="Upload complete. (398ms)"6824worker2 # [ 182.327451] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=0 paths=1 nar_bytes_in=6246825worker2 # [ 182.332277] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286826worker2 # [ 182.342113] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6827worker2 # [ 182.346593] nix-daemon[848]: reaped child process 1779, status = succeeded6828worker2 # [ 182.351230] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6829client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6830client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6831client # farm-b> LOG-farm-b-drain6832worker2 # [ 183.374538] nix-daemon[848]: reaped child process 1793, status = succeeded6833worker2 # [ 183.528353] nix-daemon[848]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6834worker2 # [ 183.538758] nix-daemon[848]: reaped child process 1811, status = succeeded6835lb # [ 183.559280] niks3-server[1067]: 2026/09/20 09:35:42 INFO Received uploads request method=POST path=/api/pending_closures6836worker2 # [ 183.565052] nix-grpc-daemon[1541]: time=2026-09-20T09:35:43.659Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6837worker2 # [ 183.566675] nix-grpc-daemon[1541]: time=2026-09-20T09:35:43.660Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6838lb # [ 183.595494] niks3-server[1067]: 2026/09/20 09:35:42 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6839lb # [ 183.609142] niks3-server[1067]: 2026/09/20 09:35:42 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6840lb # [ 183.615981] niks3-server[1067]: 2026/09/20 09:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6841lb # [ 183.620493] niks3-server[1067]: 2026/09/20 09:35:42 INFO Signed narinfos id=67 count=16842worker2 # [ 183.620623] nix-grpc-daemon[1541]: time=2026-09-20T09:35:43.714Z level=INFO msg="Uploading 1 narinfos"6843lb # [ 183.627835] niks3-server[1067]: 2026/09/20 09:35:42 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6844lb # [ 183.636766] niks3-server[1067]: 2026/09/20 09:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6845lb # [ 183.641857] niks3-server[1067]: 2026/09/20 09:35:42 INFO Completed upload id=676846worker2 # [ 183.639391] nix-grpc-daemon[1541]: time=2026-09-20T09:35:43.733Z level=INFO msg="Upload complete. (279ms)"6847lb # [ 183.644326] niks3-server[1067]: 2026/09/20 09:35:42 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6848worker2 # [ 183.644746] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=16 outputs=16849worker2 # [ 183.655136] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:43Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06850worker2 # [ 183.658887] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:43Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16851worker2 # [ 183.666524] nix-daemon[848]: reaped child process 1791, status = succeeded6852worker2 # [ 183.698838] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286853worker2 # [ 183.711289] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:398286854worker2 # [ 183.723457] nix-grpc-daemon[1541]: time=2026-09-20T09:35:43.817Z level=INFO msg="All 2 paths already cached"6855client # copying 1 paths to 'lb:50051'...6856client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6857worker2 # [ 183.730547] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:398286858worker2 # [ 183.740169] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6859worker2 # [ 183.903712] nix-daemon[848]: accepted connection from pid 1817, user nix-grpc-daemon (trusted)6860worker2 # [ 183.915721] nix-daemon[848]: reaped child process 1820, status = succeeded6861worker2 # [ 184.054669] nix-daemon[848]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6862worker2 # [ 184.066620] nix-daemon[848]: reaped child process 1825, status = succeeded6863lb # [ 184.080075] niks3-server[1067]: 2026/09/20 09:35:43 INFO Received uploads request method=POST path=/api/pending_closures6864worker2 # [ 184.086598] nix-grpc-daemon[1541]: time=2026-09-20T09:35:44.180Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6865worker2 # [ 184.088429] nix-grpc-daemon[1541]: time=2026-09-20T09:35:44.182Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6866lb # [ 184.116116] niks3-server[1067]: 2026/09/20 09:35:43 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6867lb # [ 184.125134] niks3-server[1067]: 2026/09/20 09:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6868worker2 # [ 184.130208] nix-grpc-daemon[1541]: time=2026-09-20T09:35:44.224Z level=INFO msg="Uploading 1 narinfos"6869lb # [ 184.132618] niks3-server[1067]: 2026/09/20 09:35:43 INFO Signed narinfos id=68 count=16870lb # [ 184.136136] niks3-server[1067]: 2026/09/20 09:35:43 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6871lb # [ 184.148904] niks3-server[1067]: 2026/09/20 09:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6872lb # [ 184.154355] niks3-server[1067]: 2026/09/20 09:35:43 INFO Completed upload id=686873worker2 # [ 184.151831] nix-grpc-daemon[1541]: time=2026-09-20T09:35:44.246Z level=INFO msg="Upload complete. (400ms)"6874lb # [ 184.157160] niks3-server[1067]: 2026/09/20 09:35:43 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6875worker2 # [ 184.154121] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=0 paths=1 nar_bytes_in=7526876worker2 # [ 184.156937] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:398286877worker2 # [ 184.163612] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6878worker2 # [ 184.165853] nix-daemon[848]: reaped child process 1815, status = succeeded6879client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6880worker2 # [ 184.171212] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6881client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6882client # farm-top> LOG-farm-top-drain6883worker2 # [ 185.398670] nix-daemon[848]: reaped child process 1829, status = succeeded6884worker2 # [ 185.526091] nix-daemon[848]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6885worker2 # [ 185.536888] nix-daemon[848]: reaped child process 1847, status = succeeded6886lb # [ 185.556799] niks3-server[1067]: 2026/09/20 09:35:44 INFO Received uploads request method=POST path=/api/pending_closures6887worker2 # [ 185.558973] nix-grpc-daemon[1541]: time=2026-09-20T09:35:45.653Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6888worker2 # [ 185.560570] nix-grpc-daemon[1541]: time=2026-09-20T09:35:45.654Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6889lb # [ 185.580631] niks3-server[1067]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6890lb # [ 185.595573] niks3-server[1067]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6891lb # [ 185.600313] niks3-server[1067]: 2026/09/20 09:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6892lb # [ 185.604194] niks3-server[1067]: 2026/09/20 09:35:44 INFO Signed narinfos id=69 count=16893worker2 # [ 185.604247] nix-grpc-daemon[1541]: time=2026-09-20T09:35:45.698Z level=INFO msg="Uploading 1 narinfos"6894lb # [ 185.612948] niks3-server[1067]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6895lb # [ 185.615137] niks3-server[1067]: 2026/09/20 09:35:44 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6896lb # [ 185.618702] niks3-server[1067]: 2026/09/20 09:35:44 INFO Completed upload id=696897worker2 # [ 185.616137] nix-grpc-daemon[1541]: time=2026-09-20T09:35:45.710Z level=INFO msg="Upload complete. (237ms)"6898lb # [ 185.621146] niks3-server[1067]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6899worker2 # [ 185.619285] nix-grpc-daemon[1471]: ts=2026-09-20T09:35:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39828 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=17 outputs=16900worker2 # [ 185.625078] nix-daemon[848]: reaped child process 1827, status = succeeded6901client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.57 seconds)6902client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6903client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.21 seconds)6904worker1: must succeed: rm /nix/.rw-store/fill6905worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)6906worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6907worker1 # [ 186.537133] nix-grpc-daemon[1534]: ts=2026-09-20T09:35:46Z level=info event=healthy reason=min_free6908worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)6909(finished: subtest: low disk drains a worker and builds go to the other, in 9.16 seconds)6910(finished: run the VM test script, in 188.11 seconds)6911test script finished in 188.33s6912cleanup6913kill QemuMachine (pid 49)6914client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6915kill QemuMachine (pid 50)6916lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6917kill QemuMachine (pid 51)6918worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6919kill QemuMachine (pid 52)6920worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6921(finished: cleanup, in 0.90 seconds)6922additionally exposed symbols:6923 client, lb, worker1, worker2,6924 vlan1,6925 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh