vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #307
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.01 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.zryndv3aBu', fmt=raw size=107374182414client # mke2fs 1.47.4 (6-Mar-2025)15client # Discarding device blocks: 0/262144 done16client: QEMU running (pid 50)17client # Creating filesystem with 262144 4k blocks and 65536 inodes18client # Filesystem UUID: bf11b64a-78b5-473a-8081-ea0ec9959e9c19client # Superblock backups stored on blocks:20client # 32768, 98304, 163840, 22937621client # 22client # Allocating group tables: 0/8 done23client # Writing inode tables: 0/8 done24client # Creating journal (8192 blocks): done25client # Writing superblocks and filesystem accounting information: 0/8 done26worker1 # Disk image does not exist, creating the virtualisation disk image...27client # 28worker1 # Formatting '/build/vm-state-worker1/tmp.dA7wqeXcQL', fmt=raw size=107374182429client # Virtualisation disk image created.30worker1 # mke2fs 1.47.4 (6-Mar-2025)31worker1: QEMU running (pid 51)32worker1 # Discarding device blocks: 0/262144 done33worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes34worker1 # Filesystem UUID: 0f3477fd-08be-44a0-b4f7-fb16ff94f01635worker1 # Superblock backups stored on blocks:36worker1 # 32768, 98304, 163840, 22937637worker1 # 38worker1 # Allocating group tables: 0/8 done39worker1 # Writing inode tables: 0/8 done40worker1 # Creating journal (8192 blocks): done41worker1 # Writing superblocks and filesystem accounting information: 0/8 done42worker1 # 43worker1 # Virtualisation disk image created.44worker2: QEMU running (pid 52)45worker2 # Disk image does not exist, creating the virtualisation disk image...46worker2 # Formatting '/build/vm-state-worker2/tmp.0BE3cHsOL1', fmt=raw size=107374182447worker2 # mke2fs 1.47.4 (6-Mar-2025)48worker2 # Discarding device blocks: 0/262144 done49worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes50worker2 # Filesystem UUID: 964685d3-00fd-40cf-948a-cf19bfa8a55e51worker2 # Superblock backups stored on blocks:52worker2 # 32768, 98304, 163840, 22937653worker2 # 54worker2 # Allocating group tables: 0/8 done55worker2 # Writing inode tables: 0/8 done56worker2 # Creating journal (8192 blocks): done57worker2 # Writing superblocks and filesystem accounting information: 0/8 done58worker2 # 59worker2 # Virtualisation disk image created.60lb # Disk image does not exist, creating the virtualisation disk image...61lb: QEMU running (pid 49)62lb # Formatting '/build/vm-state-lb/tmp.HbvIuJ8PyC', fmt=raw size=107374182463lb # mke2fs 1.47.4 (6-Mar-2025)64lb # Discarding device blocks: 0/262144 done65lb # Creating filesystem with 262144 4k blocks and 65536 inodes66(finished: start all VMs, in 0.24 seconds)67lb # Filesystem UUID: 4d4f5cd4-eb71-4688-9606-9693b30d296868client: waiting for unit network-addresses-eth1.service69lb # Superblock backups stored on blocks:70client: waiting for the VM to finish booting71lb # 32768, 98304, 163840, 22937672lb # 73lb # Allocating group tables: 0/8 done74lb # Writing inode tables: 0/8 done75lb # Creating journal (8192 blocks): done76lb # Writing superblocks and filesystem accounting information: 0/8 done77lb # 78lb # Virtualisation disk image created.79client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83client # 84client # 85client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # 87worker1 # 88client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...89client # 90client # 91client # 92client # 93worker2 # 94worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0095worker2 # 96client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0097worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0098worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...99worker1 # 100worker1 # 101worker1 # 102worker1 # 103worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...104worker2 # 105worker2 # 106worker2 # 107worker2 # 108worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00109client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...110client # 111client # 112worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...114worker1 # 115worker1 # 116worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117worker2 # 118worker2 # 119lb # 120lb # 121lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123lb # 124lb # 125lb # 126lb # 127lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129lb # 130lb # 131lb # Booting from ROM...132client # Booting from ROM...133worker1 # Booting from ROM...134worker2 # Booting from ROM...135client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test/init regInfo=/nix/store/9qrjs3171pxvd0p653zlbv22i1fimhl2-closure-info/registration console=ttyS0,115200n8 console=tty0137client # [ 0.000000] BIOS-provided physical RAM map:138client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026147client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved148worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026149client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved150client # [ 0.000000] NX (Execute Disable) protection: active151client # [ 0.000000] APIC: Static calls initialized152client # [ 0.000000] SMBIOS 2.8 present.153worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g27xn1m1f16f9vvzlv5lxp45ik86lzk5-nixos-system-worker1-test/init regInfo=/nix/store/nz37j0fi8ibw0hm3wbnnfs2268ccybhn-closure-info/registration console=ttyS0,115200n8 console=tty0154client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014155worker1 # [ 0.000000] BIOS-provided physical RAM map:156client # [ 0.000000] DMI: Memory slots populated: 1/1157worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/57k37pgcxfmcvcaikbpq6wxxg9c64rf7-nixos-system-worker2-test/init regInfo=/nix/store/8iq91pzaxdh07c3b8j2h7ipsghrhxcbk-closure-info/registration console=ttyS0,115200n8 console=tty0158client # [ 0.000000] Hypervisor detected: KVM159worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable160worker2 # [ 0.000000] BIOS-provided physical RAM map:161client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000162worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved163worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable164client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00165worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved166client # [ 0.000000] kvm-clock: using sched offset of 459136215 cycles167worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved168worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable169worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved170client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns171worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved172worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable173client # [ 0.000005] tsc: Detected 2400.012 MHz processor174worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved175client # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000176worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved177worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved178client # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs179worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved180worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved181client # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT182worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved183worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved184client # [ 0.002744] found SMP MP-table at [mem 0x000f5450-0x000f545f]185worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved186client # [ 0.002756] Using GB pages for direct mapping187worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved188worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved189client # [ 0.002918] RAMDISK: [mem 0x5e366000-0x5ffcffff]190worker1 # [ 0.000000] NX (Execute Disable) protection: active191client # [ 0.002927] ACPI: Early table checksum verification disabled192worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved193worker1 # [ 0.000000] APIC: Static calls initialized194worker1 # [ 0.000000] SMBIOS 2.8 present.195worker2 # [ 0.000000] NX (Execute Disable) protection: active196client # [ 0.002929] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )197worker2 # [ 0.000000] APIC: Static calls initialized198worker2 # [ 0.000000] SMBIOS 2.8 present.199client # [ 0.002933] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)200worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014201worker1 # [ 0.000000] DMI: Memory slots populated: 1/1202worker1 # [ 0.000000] Hypervisor detected: KVM203client # [ 0.002937] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)204worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014205worker2 # [ 0.000000] DMI: Memory slots populated: 1/1206worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000207worker2 # [ 0.000000] Hypervisor detected: KVM208client # [ 0.002943] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)209worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00210client # [ 0.002945] ACPI: FACS 0x000000005FFE0000 000040211worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000212worker1 # [ 0.000000] kvm-clock: using sched offset of 465619201 cycles213worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00214client # [ 0.002947] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)215worker2 # [ 0.000000] kvm-clock: using sched offset of 462821945 cycles216worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns217client # [ 0.002948] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)218worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor219lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026220worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns221worker1 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000222client # [ 0.002950] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)223worker2 # [ 0.000006] tsc: Detected 2400.012 MHz processor224worker1 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs225worker2 # [ 0.000807] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000226client # [ 0.002951] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)227worker1 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT228worker2 # [ 0.000834] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs229client # [ 0.002953] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]230worker1 # [ 0.002739] found SMP MP-table at [mem 0x000f5450-0x000f545f]231worker2 # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT232client # [ 0.002954] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]233worker1 # [ 0.002750] Using GB pages for direct mapping234lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0235worker1 # [ 0.002831] RAMDISK: [mem 0x5e367000-0x5ffcffff]236worker2 # [ 0.002742] found SMP MP-table at [mem 0x000f5450-0x000f545f]237client # [ 0.002954] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]238lb # [ 0.000000] BIOS-provided physical RAM map:239worker2 # [ 0.002754] Using GB pages for direct mapping240worker1 # [ 0.002839] ACPI: Early table checksum verification disabled241client # [ 0.002955] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]242lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable243worker2 # [ 0.002861] RAMDISK: [mem 0x5e367000-0x5ffcffff]244worker1 # [ 0.002843] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )245worker2 # [ 0.002869] ACPI: Early table checksum verification disabled246client # [ 0.002955] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]247lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved248worker1 # [ 0.002847] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)249worker2 # [ 0.002873] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )250client # [ 0.002956] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]251lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved252worker1 # [ 0.002851] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)253worker2 # [ 0.002877] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)254client # [ 0.002956] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]255lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable256client # [ 0.003187] No NUMA configuration found257worker1 # [ 0.002858] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)258worker2 # [ 0.002882] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)259lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved260client # [ 0.003188] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]261worker1 # [ 0.002860] ACPI: FACS 0x000000005FFE0000 000040262lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved263worker2 # [ 0.002889] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)264client # [ 0.003190] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]265client # [ 0.005749] Zone ranges:266worker1 # [ 0.002862] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)267worker2 # [ 0.002891] ACPI: FACS 0x000000005FFE0000 000040268lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved269client # [ 0.005750] DMA [mem 0x0000000000001000-0x0000000000ffffff]270lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved271worker1 # [ 0.002863] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)272worker2 # [ 0.002893] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)273client # [ 0.005752] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]274client # [ 0.005753] Normal empty275lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved276worker1 # [ 0.002865] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)277client # [ 0.005754] Device empty278worker2 # [ 0.002894] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)279client # [ 0.005754] Movable zone start for each node280lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved281client # [ 0.005755] Early memory node ranges282worker1 # [ 0.002866] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)283worker2 # [ 0.002896] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)284lb # [ 0.000000] NX (Execute Disable) protection: active285client # [ 0.005755] node 0: [mem 0x0000000000001000-0x000000000009efff]286lb # [ 0.000000] APIC: Static calls initialized287worker1 # [ 0.002867] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]288lb # [ 0.000000] SMBIOS 2.8 present.289worker2 # [ 0.002898] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)290client # [ 0.005756] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]291worker1 # [ 0.002869] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]292worker2 # [ 0.002899] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]293lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014294client # [ 0.005757] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]295worker1 # [ 0.002869] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]296worker2 # [ 0.002900] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]297lb # [ 0.000000] DMI: Memory slots populated: 1/1298client # [ 0.005778] On node 0, zone DMA: 1 pages in unavailable ranges299lb # [ 0.000000] Hypervisor detected: KVM300worker1 # [ 0.002870] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]301worker2 # [ 0.002900] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]302client # [ 0.006044] On node 0, zone DMA: 97 pages in unavailable ranges303lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000304worker1 # [ 0.002870] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]305worker2 # [ 0.002908] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]306client # [ 0.033265] On node 0, zone DMA32: 40 pages in unavailable ranges307lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00308client # [ 0.033726] ACPI: PM-Timer IO Port: 0x608309worker1 # [ 0.002871] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]310lb # [ 0.000001] kvm-clock: using sched offset of 475056395 cycles311worker2 # [ 0.002909] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]312client # [ 0.033736] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])313worker1 # [ 0.002871] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]314worker2 # [ 0.002909] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]315worker1 # [ 0.003100] No NUMA configuration found316client # [ 0.033763] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23317lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns318worker2 # [ 0.002909] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]319lb # [ 0.000005] tsc: Detected 2400.012 MHz processor320worker1 # [ 0.003101] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]321client # [ 0.033766] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)322worker2 # [ 0.003142] No NUMA configuration found323lb # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000324worker1 # [ 0.003104] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]325client # [ 0.033768] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)326worker1 # [ 0.005543] Zone ranges:327worker2 # [ 0.003143] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]328lb # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs329client # [ 0.033769] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)330worker1 # [ 0.005544] DMA [mem 0x0000000000001000-0x0000000000ffffff]331worker2 # [ 0.003145] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]332worker2 # [ 0.005794] Zone ranges:333worker1 # [ 0.005546] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]334client # [ 0.033770] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)335lb # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT336worker1 # [ 0.005547] Normal empty337worker2 # [ 0.005796] DMA [mem 0x0000000000001000-0x0000000000ffffff]338worker1 # [ 0.005548] Device empty339lb # [ 0.002746] found SMP MP-table at [mem 0x000f5450-0x000f545f]340client # [ 0.033771] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)341worker2 # [ 0.005797] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]342lb # [ 0.002757] Using GB pages for direct mapping343worker1 # [ 0.005548] Movable zone start for each node344worker2 # [ 0.005799] Normal empty345client # [ 0.033773] ACPI: Using ACPI (MADT) for SMP configuration information346worker1 # [ 0.005549] Early memory node ranges347lb # [ 0.002887] RAMDISK: [mem 0x5e367000-0x5ffcffff]348worker2 # [ 0.005800] Device empty349client # [ 0.033774] ACPI: HPET id: 0x8086a201 base: 0xfed00000350worker2 # [ 0.005800] Movable zone start for each node351worker1 # [ 0.005549] node 0: [mem 0x0000000000001000-0x000000000009efff]352lb # [ 0.002895] ACPI: Early table checksum verification disabled353client # [ 0.033777] TSC deadline timer available354worker2 # [ 0.005801] Early memory node ranges355lb # [ 0.002898] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )356worker1 # [ 0.005550] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]357client # [ 0.033781] CPU topo: Max. logical packages: 1358worker2 # [ 0.005801] node 0: [mem 0x0000000000001000-0x000000000009efff]359client # [ 0.033782] CPU topo: Max. logical dies: 1360worker1 # [ 0.005551] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]361lb # [ 0.002903] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)362worker2 # [ 0.005802] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]363client # [ 0.033782] CPU topo: Max. dies per package: 1364worker1 # [ 0.005573] On node 0, zone DMA: 1 pages in unavailable ranges365client # [ 0.033786] CPU topo: Max. threads per core: 1366lb # [ 0.002908] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)367worker2 # [ 0.005803] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]368worker1 # [ 0.005855] On node 0, zone DMA: 97 pages in unavailable ranges369client # [ 0.033786] CPU topo: Num. cores per package: 1370worker2 # [ 0.005827] On node 0, zone DMA: 1 pages in unavailable ranges371client # [ 0.033787] CPU topo: Num. threads per package: 1372worker1 # [ 0.034502] On node 0, zone DMA32: 40 pages in unavailable ranges373lb # [ 0.002914] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)374worker2 # [ 0.006106] On node 0, zone DMA: 97 pages in unavailable ranges375worker1 # [ 0.034959] ACPI: PM-Timer IO Port: 0x608376lb # [ 0.002916] ACPI: FACS 0x000000005FFE0000 000040377client # [ 0.033787] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs378worker2 # [ 0.034362] On node 0, zone DMA32: 40 pages in unavailable ranges379worker1 # [ 0.034971] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])380client # [ 0.033806] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()381worker2 # [ 0.034822] ACPI: PM-Timer IO Port: 0x608382lb # [ 0.002918] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)383worker1 # [ 0.034997] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23384worker2 # [ 0.034835] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])385client # [ 0.033837] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]386lb # [ 0.002919] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)387worker1 # [ 0.035000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)388worker2 # [ 0.034866] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23389client # [ 0.033838] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]390worker1 # [ 0.035002] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)391lb # [ 0.002921] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)392worker2 # [ 0.034869] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)393client # [ 0.033840] [mem 0x60000000-0xafffffff] available for PCI devices394worker1 # [ 0.035003] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)395client # [ 0.033841] Booting paravirtualized kernel on KVM396lb # [ 0.002922] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)397worker2 # [ 0.034870] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)398worker1 # [ 0.035004] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)399worker2 # [ 0.034871] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)400lb # [ 0.002924] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]401client # [ 0.033844] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns402worker1 # [ 0.035005] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)403worker2 # [ 0.034873] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)404lb # [ 0.002925] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]405worker1 # [ 0.035007] ACPI: Using ACPI (MADT) for SMP configuration information406client # [ 0.038313] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1407worker2 # [ 0.034873] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)408lb # [ 0.002925] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]409worker1 # [ 0.035008] ACPI: HPET id: 0x8086a201 base: 0xfed00000410client # [ 0.040497] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152411worker1 # [ 0.035012] TSC deadline timer available412worker2 # [ 0.034876] ACPI: Using ACPI (MADT) for SMP configuration information413lb # [ 0.002926] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]414client # [ 0.040540] kvm-guest: PV spinlocks disabled, single CPU415worker1 # [ 0.035016] CPU topo: Max. logical packages: 1416worker2 # [ 0.034877] ACPI: HPET id: 0x8086a201 base: 0xfed00000417lb # [ 0.002926] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]418worker1 # [ 0.035017] CPU topo: Max. logical dies: 1419worker2 # [ 0.034881] TSC deadline timer available420worker1 # [ 0.035017] CPU topo: Max. dies per package: 1421worker2 # [ 0.034885] CPU topo: Max. logical packages: 1422lb # [ 0.002927] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]423worker1 # [ 0.035021] CPU topo: Max. threads per core: 1424worker2 # [ 0.034886] CPU topo: Max. logical dies: 1425worker1 # [ 0.035021] CPU topo: Num. cores per package: 1426worker2 # [ 0.034886] CPU topo: Max. dies per package: 1427lb # [ 0.002927] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]428lb # [ 0.003153] No NUMA configuration found429worker1 # [ 0.035022] CPU topo: Num. threads per package: 1430worker2 # [ 0.034890] CPU topo: Max. threads per core: 1431worker2 # [ 0.034890] CPU topo: Num. cores per package: 1432lb # [ 0.003153] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]433client # [ 0.040542] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test/init regInfo=/nix/store/9qrjs3171pxvd0p653zlbv22i1fimhl2-closure-info/registration console=ttyS0,115200n8 console=tty0434worker1 # [ 0.035022] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs435worker2 # [ 0.034891] CPU topo: Num. threads per package: 1436lb # [ 0.003156] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]437worker1 # [ 0.035041] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()438lb # [ 0.005539] Zone ranges:439worker2 # [ 0.034891] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs440lb # [ 0.005540] DMA [mem 0x0000000000001000-0x0000000000ffffff]441worker1 # [ 0.035074] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]442client # [ 0.040634] Unknown kernel command line parameters "regInfo=/nix/store/9qrjs3171pxvd0p653zlbv22i1fimhl2-closure-info/registration", will be passed to user space.443worker2 # [ 0.034922] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()444client # [ 0.040647] random: crng init done445lb # [ 0.005543] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]446worker1 # [ 0.035075] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]447lb # [ 0.005544] Normal empty448worker2 # [ 0.034956] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]449client # [ 0.040648] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes450lb # [ 0.005545] Device empty451worker1 # [ 0.035077] [mem 0x60000000-0xafffffff] available for PCI devices452lb # [ 0.005545] Movable zone start for each node453worker2 # [ 0.034958] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]454worker1 # [ 0.035079] Booting paravirtualized kernel on KVM455client # [ 0.042790] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)456lb # [ 0.005546] Early memory node ranges457worker2 # [ 0.034959] [mem 0x60000000-0xafffffff] available for PCI devices458lb # [ 0.005546] node 0: [mem 0x0000000000001000-0x000000000009efff]459client # [ 0.043875] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)460worker1 # [ 0.035082] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns461worker2 # [ 0.034961] Booting paravirtualized kernel on KVM462client # [ 0.043907] Fallback order for Node 0: 0463lb # [ 0.005548] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]464worker1 # [ 0.039563] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1465client # [ 0.043910] Built 1 zonelists, mobility grouping on. Total pages: 393078466worker2 # [ 0.034964] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns467lb # [ 0.005549] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]468client # [ 0.043911] Policy zone: DMA32469worker1 # [ 0.041890] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152470lb # [ 0.005572] On node 0, zone DMA: 1 pages in unavailable ranges471worker2 # [ 0.039438] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1472client # [ 0.046965] mem auto-init: stack:all(zero), heap alloc:on, heap free:off473worker1 # [ 0.041932] kvm-guest: PV spinlocks disabled, single CPU474lb # [ 0.005852] On node 0, zone DMA: 97 pages in unavailable ranges475worker2 # [ 0.041728] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152476client # [ 0.050208] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1477lb # [ 0.034239] On node 0, zone DMA32: 40 pages in unavailable ranges478worker2 # [ 0.041776] kvm-guest: PV spinlocks disabled, single CPU479client # [ 0.053587] allocated 3145728 bytes of page_ext480lb # [ 0.034699] ACPI: PM-Timer IO Port: 0x608481client # [ 0.063618] ftrace: allocating 48731 entries in 192 pages482lb # [ 0.034712] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])483client # [ 0.063620] ftrace: allocated 192 pages with 2 groups484lb # [ 0.034742] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23485client # [ 0.064471] Dynamic Preempt: lazy486client # [ 0.064652] rcu: Preemptible hierarchical RCU implementation.487worker1 # [ 0.041934] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g27xn1m1f16f9vvzlv5lxp45ik86lzk5-nixos-system-worker1-test/init regInfo=/nix/store/nz37j0fi8ibw0hm3wbnnfs2268ccybhn-closure-info/registration console=ttyS0,115200n8 console=tty0488lb # [ 0.034744] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)489client # [ 0.064653] rcu: RCU event tracing is enabled.490lb # [ 0.034746] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)491worker2 # [ 0.041777] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/57k37pgcxfmcvcaikbpq6wxxg9c64rf7-nixos-system-worker2-test/init regInfo=/nix/store/8iq91pzaxdh07c3b8j2h7ipsghrhxcbk-closure-info/registration console=ttyS0,115200n8 console=tty0492lb # [ 0.034747] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)493client # [ 0.064653] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.494worker1 # [ 0.042032] Unknown kernel command line parameters "regInfo=/nix/store/nz37j0fi8ibw0hm3wbnnfs2268ccybhn-closure-info/registration", will be passed to user space.495worker1 # [ 0.042045] random: crng init done496client # [ 0.064655] Trampoline variant of Tasks RCU enabled.497lb # [ 0.034748] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)498client # [ 0.064656] Rude variant of Tasks RCU enabled.499worker2 # [ 0.041877] Unknown kernel command line parameters "regInfo=/nix/store/8iq91pzaxdh07c3b8j2h7ipsghrhxcbk-closure-info/registration", will be passed to user space.500worker1 # [ 0.042046] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes501lb # [ 0.034749] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)502client # [ 0.064656] Tracing variant of Tasks RCU enabled.503worker2 # [ 0.041891] random: crng init done504worker1 # [ 0.044603] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)505lb # [ 0.034751] ACPI: Using ACPI (MADT) for SMP configuration information506client # [ 0.064657] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.507worker2 # [ 0.041892] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes508lb # [ 0.034752] ACPI: HPET id: 0x8086a201 base: 0xfed00000509worker1 # [ 0.045728] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)510client # [ 0.064657] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1511lb # [ 0.034756] TSC deadline timer available512worker2 # [ 0.044155] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)513worker1 # [ 0.045761] Fallback order for Node 0: 0514lb # [ 0.034761] CPU topo: Max. logical packages: 1515worker2 # [ 0.045276] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)516lb # [ 0.034761] CPU topo: Max. logical dies: 1517worker1 # [ 0.045764] Built 1 zonelists, mobility grouping on. Total pages: 393078518client # [ 0.064678] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.519worker1 # [ 0.045765] Policy zone: DMA32520worker2 # [ 0.045311] Fallback order for Node 0: 0521lb # [ 0.034762] CPU topo: Max. dies per package: 1522client # [ 0.064680] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.523lb # [ 0.034765] CPU topo: Max. threads per core: 1524worker1 # [ 0.048431] mem auto-init: stack:all(zero), heap alloc:on, heap free:off525worker2 # [ 0.045315] Built 1 zonelists, mobility grouping on. Total pages: 393078526worker2 # [ 0.045316] Policy zone: DMA32527lb # [ 0.034766] CPU topo: Num. cores per package: 1528worker1 # [ 0.051916] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1529client # [ 0.064681] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.530lb # [ 0.034766] CPU topo: Num. threads per package: 1531worker2 # [ 0.048239] mem auto-init: stack:all(zero), heap alloc:on, heap free:off532worker1 # [ 0.055432] allocated 3145728 bytes of page_ext533client # [ 0.069063] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16534lb # [ 0.034766] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs535worker1 # [ 0.066132] ftrace: allocating 48731 entries in 192 pages536worker2 # [ 0.051753] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1537client # [ 0.069342] rcu: srcu_init: Setting srcu_struct sizes based on contention.538worker1 # [ 0.066134] ftrace: allocated 192 pages with 2 groups539worker2 # [ 0.055284] allocated 3145728 bytes of page_ext540lb # [ 0.034787] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()541worker1 # [ 0.066989] Dynamic Preempt: lazy542worker2 # [ 0.065600] ftrace: allocating 48731 entries in 192 pages543lb # [ 0.034819] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]544worker1 # [ 0.067166] rcu: Preemptible hierarchical RCU implementation.545client # [ 0.069348] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns546worker2 # [ 0.065602] ftrace: allocated 192 pages with 2 groups547worker1 # [ 0.067167] rcu: RCU event tracing is enabled.548worker2 # [ 0.066488] Dynamic Preempt: lazy549lb # [ 0.034821] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]550client # [ 0.069454] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)551worker2 # [ 0.066629] rcu: Preemptible hierarchical RCU implementation.552worker1 # [ 0.067167] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.553lb # [ 0.034822] [mem 0x60000000-0xafffffff] available for PCI devices554client # [ 0.072999] Console: colour VGA+ 80x25555worker2 # [ 0.066630] rcu: RCU event tracing is enabled.556worker1 # [ 0.067169] Trampoline variant of Tasks RCU enabled.557lb # [ 0.034824] Booting paravirtualized kernel on KVM558client # [ 0.073002] printk: legacy console [tty0] enabled559worker1 # [ 0.067170] Rude variant of Tasks RCU enabled.560worker2 # [ 0.066631] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.561client # [ 0.102589] printk: legacy console [ttyS0] enabled562worker1 # [ 0.067170] Tracing variant of Tasks RCU enabled.563worker2 # [ 0.066632] Trampoline variant of Tasks RCU enabled.564lb # [ 0.034828] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns565client # [ 0.200230] ACPI: Core revision 20250807566worker2 # [ 0.066633] Rude variant of Tasks RCU enabled.567worker1 # [ 0.067171] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.568lb # [ 0.039302] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1569worker2 # [ 0.066633] Tracing variant of Tasks RCU enabled.570worker1 # [ 0.067171] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1571lb # [ 0.041592] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152572worker2 # [ 0.066634] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.573client # [ 0.201125] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns574lb # [ 0.041637] kvm-guest: PV spinlocks disabled, single CPU575worker1 # [ 0.067193] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.576worker2 # [ 0.066634] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1577client # [ 0.202662] APIC: Switch to symmetric I/O mode setup578worker1 # [ 0.067195] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.579client # [ 0.203596] x2apic enabled580worker2 # [ 0.066696] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.581worker1 # [ 0.067196] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.582client # [ 0.204331] APIC: Switched APIC routing to: physical x2apic583worker2 # [ 0.066698] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.584worker1 # [ 0.071624] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16585worker2 # [ 0.066699] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.586lb # [ 0.041638] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0qln5mjy767qjp5wbmxjg8jay8k05dsm-nixos-system-lb-test/init regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration console=ttyS0,115200n8 console=tty0587worker1 # [ 0.071906] rcu: srcu_init: Setting srcu_struct sizes based on contention.588client # [ 0.206089] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1589worker2 # [ 0.071165] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16590worker1 # [ 0.071912] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns591worker2 # [ 0.071443] rcu: srcu_init: Setting srcu_struct sizes based on contention.592lb # [ 0.041736] Unknown kernel command line parameters "regInfo=/nix/store/nwvvzfrgncwhnws6wdjigc6jviavrdxi-closure-info/registration", will be passed to user space.593lb # [ 0.041749] random: crng init done594client # [ 0.207026] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns595worker1 # [ 0.072023] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)596worker2 # [ 0.071452] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns597worker1 # [ 0.075564] Console: colour VGA+ 80x25598lb # [ 0.041750] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes599worker1 # [ 0.075567] printk: legacy console [tty0] enabled600client # [ 0.208578] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)601worker2 # [ 0.071563] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)602lb # [ 0.043999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)603worker1 # [ 0.105276] printk: legacy console [ttyS0] enabled604worker2 # [ 0.075108] Console: colour VGA+ 80x25605client # [ 0.210661] x86/cpu: User Mode Instruction Prevention (UMIP) activated606worker2 # [ 0.075111] printk: legacy console [tty0] enabled607worker1 # [ 0.200579] ACPI: Core revision 20250807608lb # [ 0.045121] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)609worker2 # [ 0.104964] printk: legacy console [ttyS0] enabled610lb # [ 0.045155] Fallback order for Node 0: 0611client # [ 0.211703] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127612worker2 # [ 0.200589] ACPI: Core revision 20250807613lb # [ 0.045158] Built 1 zonelists, mobility grouping on. Total pages: 393078614worker1 # [ 0.201440] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns615lb # [ 0.045159] Policy zone: DMA32616client # [ 0.212576] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0617worker1 # [ 0.202930] APIC: Switch to symmetric I/O mode setup618lb # [ 0.048179] mem auto-init: stack:all(zero), heap alloc:on, heap free:off619worker2 # [ 0.201447] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns620worker1 # [ 0.203859] x2apic enabled621lb # [ 0.051671] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1622client # [ 0.213579] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto623worker2 # [ 0.202959] APIC: Switch to symmetric I/O mode setup624lb # [ 0.055211] allocated 3145728 bytes of page_ext625worker1 # [ 0.204591] APIC: Switched APIC routing to: physical x2apic626worker2 # [ 0.203894] x2apic enabled627lb # [ 0.065623] ftrace: allocating 48731 entries in 192 pages628lb # [ 0.065626] ftrace: allocated 192 pages with 2 groups629client # [ 0.215575] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl630lb # [ 0.066587] Dynamic Preempt: lazy631worker2 # [ 0.204634] APIC: Switched APIC routing to: physical x2apic632client # [ 0.216575] Transient Scheduler Attacks: Vulnerable: No microcode633worker1 # [ 0.206401] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1634lb # [ 0.066718] rcu: Preemptible hierarchical RCU implementation.635lb # [ 0.066719] rcu: RCU event tracing is enabled.636client # [ 0.217574] Spectre V2 : Mitigation: Enhanced / Automatic IBRS637worker2 # [ 0.206437] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1638lb # [ 0.066720] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.639worker1 # [ 0.207368] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns640client # [ 0.218574] Speculative Return Stack Overflow: Mitigation: Safe RET641lb # [ 0.066721] Trampoline variant of Tasks RCU enabled.642lb # [ 0.066722] Rude variant of Tasks RCU enabled.643worker1 # [ 0.208974] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)644worker2 # [ 0.207401] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns645lb # [ 0.066722] Tracing variant of Tasks RCU enabled.646lb # [ 0.066723] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.647worker1 # [ 0.210060] x86/cpu: User Mode Instruction Prevention (UMIP) activated648worker2 # [ 0.209074] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)649lb # [ 0.066723] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1650worker1 # [ 0.211143] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127651worker2 # [ 0.210160] x86/cpu: User Mode Instruction Prevention (UMIP) activated652client # [ 0.219574] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization653lb # [ 0.066740] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.654worker1 # [ 0.211972] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0655client # [ 0.221582] Spectre V2 : Enabling IBPB for BPF656worker2 # [ 0.211216] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127657lb # [ 0.066742] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.658worker2 # [ 0.212071] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0659client # [ 0.222575] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier660worker1 # [ 0.212975] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto661lb # [ 0.066743] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.662client # [ 0.223575] active return thunk: srso_alias_return_thunk663lb # [ 0.071350] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16664worker1 # [ 0.213971] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl665worker2 # [ 0.213075] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto666lb # [ 0.071643] rcu: srcu_init: Setting srcu_struct sizes based on contention.667client # [ 0.224596] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'668worker1 # [ 0.214971] Transient Scheduler Attacks: Vulnerable: No microcode669lb # [ 0.071651] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns670worker2 # [ 0.215071] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl671client # [ 0.225574] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'672worker1 # [ 0.215971] Spectre V2 : Mitigation: Enhanced / Automatic IBRS673worker2 # [ 0.216072] Transient Scheduler Attacks: Vulnerable: No microcode674lb # [ 0.071762] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)675client # [ 0.226574] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'676worker1 # [ 0.216971] Speculative Return Stack Overflow: Mitigation: Safe RET677lb # [ 0.075426] Console: colour VGA+ 80x25678worker2 # [ 0.217071] Spectre V2 : Mitigation: Enhanced / Automatic IBRS679client # [ 0.227574] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'680lb # [ 0.075429] printk: legacy console [tty0] enabled681lb # [ 0.105013] printk: legacy console [ttyS0] enabled682worker2 # [ 0.218071] Speculative Return Stack Overflow: Mitigation: Safe RET683client # [ 0.228574] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'684lb # [ 0.200640] ACPI: Core revision 20250807685client # [ 0.229574] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'686lb # [ 0.201530] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns687worker1 # [ 0.217971] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization688client # [ 0.230574] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'689lb # [ 0.203086] APIC: Switch to symmetric I/O mode setup690worker1 # [ 0.219979] Spectre V2 : Enabling IBPB for BPF691client # [ 0.231575] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256692lb # [ 0.204021] x2apic enabled693worker2 # [ 0.219070] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization694client # [ 0.232574] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64695worker1 # [ 0.220682] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier696worker2 # [ 0.221080] Spectre V2 : Enabling IBPB for BPF697lb # [ 0.204757] APIC: Switched APIC routing to: physical x2apic698client # [ 0.233574] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512699worker1 # [ 0.221972] active return thunk: srso_alias_return_thunk700worker2 # [ 0.222071] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier701client # [ 0.234574] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024702worker1 # [ 0.222992] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'703worker2 # [ 0.223071] active return thunk: srso_alias_return_thunk704lb # [ 0.206892] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1705client # [ 0.235574] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8706worker1 # [ 0.223971] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'707worker2 # [ 0.224092] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'708worker1 # [ 0.224971] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'709client # [ 0.236574] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.710lb # [ 0.207844] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns711worker2 # [ 0.225071] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'712worker1 # [ 0.225971] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'713lb # [ 0.209427] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)714worker2 # [ 0.226071] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'715worker1 # [ 0.226971] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'716worker2 # [ 0.227070] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'717lb # [ 0.210513] x86/cpu: User Mode Instruction Prevention (UMIP) activated718worker1 # [ 0.227971] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'719worker2 # [ 0.228070] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'720lb # [ 0.212321] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127721worker1 # [ 0.228971] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'722worker2 # [ 0.229070] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'723lb # [ 0.213096] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0724worker1 # [ 0.229971] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256725worker2 # [ 0.230071] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'726worker1 # [ 0.230971] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64727lb # [ 0.214164] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto728worker2 # [ 0.231071] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256729worker1 # [ 0.231971] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512730lb # [ 0.215424] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl731worker2 # [ 0.232768] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64732worker1 # [ 0.232971] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024733lb # [ 0.216424] Transient Scheduler Attacks: Vulnerable: No microcode734worker2 # [ 0.233770] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512735worker1 # [ 0.233971] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8736lb # [ 0.218126] Spectre V2 : Mitigation: Enhanced / Automatic IBRS737worker2 # [ 0.234071] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024738worker1 # [ 0.234971] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.739lb # [ 0.219107] Speculative Return Stack Overflow: Mitigation: Safe RET740worker2 # [ 0.235071] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8741worker2 # [ 0.236071] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.742lb # [ 0.219424] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization743lb # [ 0.221432] Spectre V2 : Enabling IBPB for BPF744lb # [ 0.222424] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier745lb # [ 0.223424] active return thunk: srso_alias_return_thunk746lb # [ 0.224444] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'747lb # [ 0.225424] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748lb # [ 0.226423] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749lb # [ 0.227423] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750lb # [ 0.228423] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751lb # [ 0.229423] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752lb # [ 0.230423] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753lb # [ 0.232334] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754lb # [ 0.233132] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755lb # [ 0.234140] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756lb # [ 0.235149] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757lb # [ 0.235424] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758lb # [ 0.236423] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759client # [ 0.272434] Freeing SMP alternatives memory: 44K760client # [ 0.273183] pid_max: default: 32768 minimum: 301761client # [ 0.273659] LSM: initializing lsm=capability,landlock,yama,bpf,ima762client # [ 0.274672] landlock: Up and running.763client # [ 0.275575] Yama: becoming mindful.764client # [ 0.276776] LSM support for eBPF active765client # [ 0.277549] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766client # [ 0.278612] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)767worker1 # [ 0.269342] Freeing SMP alternatives memory: 44K768worker1 # [ 0.269974] pid_max: default: 32768 minimum: 301769client # [ 0.281370] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)770worker1 # [ 0.270792] LSM: initializing lsm=capability,landlock,yama,bpf,ima771worker1 # [ 0.272044] landlock: Up and running.772worker2 # [ 0.271860] Freeing SMP alternatives memory: 44K773client # [ 0.282579] Performance Events: Fam17h+ core perfctr, AMD PMU driver.774worker1 # [ 0.272670] Yama: becoming mindful.775worker2 # [ 0.272652] pid_max: default: 32768 minimum: 301776client # [ 0.283579] ... version: 2777worker1 # [ 0.273187] LSM support for eBPF active778worker2 # [ 0.273172] LSM: initializing lsm=capability,landlock,yama,bpf,ima779client # [ 0.284318] ... bit width: 48780worker2 # [ 0.274178] landlock: Up and running.781client # [ 0.284591] ... generic counters: 6782worker1 # [ 0.274087] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)783worker2 # [ 0.275071] Yama: becoming mindful.784client # [ 0.285282] ... generic bitmap: 000000000000003f785worker1 # [ 0.275013] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)786worker2 # [ 0.276182] LSM support for eBPF active787client # [ 0.285577] ... fixed-purpose counters: 0788client # [ 0.286391] ... fixed-purpose bitmap: 0000000000000000789worker2 # [ 0.276929] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)790client # [ 0.287350] ... value mask: 0000ffffffffffff791worker1 # [ 0.278021] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)792worker2 # [ 0.278114] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)793client # [ 0.287576] ... max period: 00007fffffffffff794client # [ 0.288496] ... global_ctrl mask: 000000000000003f795worker1 # [ 0.279570] Performance Events: Fam17h+ core perfctr, AMD PMU driver.796client # [ 0.289408] signal: max sigframe size: 3376797worker1 # [ 0.279979] ... version: 2798worker2 # [ 0.280558] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)799worker1 # [ 0.280649] ... bit width: 48800client # [ 0.290220] rcu: Hierarchical SRCU implementation.801worker1 # [ 0.280974] ... generic counters: 6802client # [ 0.290580] rcu: Max phase no-delay instances is 400.803worker2 # [ 0.281659] Performance Events: Fam17h+ core perfctr, AMD PMU driver.804worker1 # [ 0.281690] ... generic bitmap: 000000000000003f805worker2 # [ 0.282076] ... version: 2806worker1 # [ 0.281973] ... fixed-purpose counters: 0807worker2 # [ 0.282793] ... bit width: 48808worker1 # [ 0.282722] ... fixed-purpose bitmap: 0000000000000000809worker2 # [ 0.283073] ... generic counters: 6810worker1 # [ 0.282973] ... value mask: 0000ffffffffffff811worker2 # [ 0.283756] ... generic bitmap: 000000000000003f812worker2 # [ 0.284080] ... fixed-purpose counters: 0813worker1 # [ 0.283861] ... max period: 00007fffffffffff814worker2 # [ 0.284801] ... fixed-purpose bitmap: 0000000000000000815worker1 # [ 0.284667] ... global_ctrl mask: 000000000000003f816client # [ 0.295942] smp: Bringing up secondary CPUs ...817worker1 # [ 0.285069] signal: max sigframe size: 3376818worker2 # [ 0.285073] ... value mask: 0000ffffffffffff819client # [ 0.296592] smp: Brought up 1 node, 1 CPU820worker2 # [ 0.285957] ... max period: 00007fffffffffff821worker1 # [ 0.285907] rcu: Hierarchical SRCU implementation.822client # [ 0.297246] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)823worker2 # [ 0.286747] ... global_ctrl mask: 000000000000003f824worker1 # [ 0.286571] rcu: Max phase no-delay instances is 400.825lb # [ 0.271970] Freeing SMP alternatives memory: 44K826worker2 # [ 0.287180] signal: max sigframe size: 3376827lb # [ 0.272427] pid_max: default: 32768 minimum: 301828worker2 # [ 0.287969] rcu: Hierarchical SRCU implementation.829client # [ 0.297694] Memory: 1454548K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)830lb # [ 0.273521] LSM: initializing lsm=capability,landlock,yama,bpf,ima831worker2 # [ 0.288673] rcu: Max phase no-delay instances is 400.832client # [ 0.298816] devtmpfs: initialized833lb # [ 0.274522] landlock: Up and running.834lb # [ 0.275127] Yama: becoming mindful.835client # [ 0.299624] x86/mm: Memory block size: 128MB836lb # [ 0.275640] LSM support for eBPF active837worker1 # [ 0.291551] smp: Bringing up secondary CPUs ...838lb # [ 0.276557] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)839client # [ 0.301425] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)840worker1 # [ 0.291986] smp: Brought up 1 node, 1 CPU841lb # [ 0.277465] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)842worker1 # [ 0.292659] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)843client # [ 0.302477] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).844worker2 # [ 0.293642] smp: Bringing up secondary CPUs ...845client # [ 0.303662] pinctrl core: initialized pinctrl subsystem846worker2 # [ 0.294087] smp: Brought up 1 node, 1 CPU847worker1 # [ 0.293154] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)848client # [ 0.304779] PM: RTC time: 12:19:30, date: 2026-09-20849lb # [ 0.280394] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)850worker2 # [ 0.294770] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)851worker1 # [ 0.294248] devtmpfs: initialized852lb # [ 0.280970] Performance Events: Fam17h+ core perfctr, AMD PMU driver.853worker1 # [ 0.295197] x86/mm: Memory block size: 128MB854lb # [ 0.281429] ... version: 2855worker2 # [ 0.295293] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)856lb # [ 0.282117] ... bit width: 48857worker2 # [ 0.296421] devtmpfs: initialized858lb # [ 0.282426] ... generic counters: 6859client # [ 0.308368] NET: Registered PF_NETLINK/PF_ROUTE protocol family860worker1 # [ 0.297053] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)861worker2 # [ 0.297201] x86/mm: Memory block size: 128MB862lb # [ 0.283101] ... generic bitmap: 000000000000003f863lb # [ 0.283476] ... fixed-purpose counters: 0864client # [ 0.309968] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations865worker1 # [ 0.297999] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).866lb # [ 0.284146] ... fixed-purpose bitmap: 0000000000000000867worker1 # [ 0.299054] pinctrl core: initialized pinctrl subsystem868client # [ 0.310604] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations869lb # [ 0.284427] ... value mask: 0000ffffffffffff870worker2 # [ 0.299421] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)871worker1 # [ 0.300150] PM: RTC time: 12:19:30, date: 2026-09-20872lb # [ 0.285270] ... max period: 00007fffffffffff873client # [ 0.311855] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations874worker2 # [ 0.300103] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).875lb # [ 0.286075] ... global_ctrl mask: 000000000000003f876client # [ 0.312588] audit: initializing netlink subsys (disabled)877worker2 # [ 0.301165] pinctrl core: initialized pinctrl subsystem878lb # [ 0.286556] signal: max sigframe size: 3376879lb # [ 0.287298] rcu: Hierarchical SRCU implementation.880worker2 # [ 0.302199] PM: RTC time: 12:19:30, date: 2026-09-20881client # [ 0.313721] thermal_sys: Registered thermal governor 'fair_share'882lb # [ 0.287971] rcu: Max phase no-delay instances is 400.883worker1 # [ 0.303799] NET: Registered PF_NETLINK/PF_ROUTE protocol family884client # [ 0.313724] thermal_sys: Registered thermal governor 'bang_bang'885client # [ 0.314578] thermal_sys: Registered thermal governor 'step_wise'886worker1 # [ 0.305181] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations887client # [ 0.315511] thermal_sys: Registered thermal governor 'user_space'888worker2 # [ 0.305819] NET: Registered PF_NETLINK/PF_ROUTE protocol family889worker1 # [ 0.305997] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations890client # [ 0.316363] audit: type=2000 audit(1789906770.888:1): state=initialized audit_enabled=0 res=1891worker2 # [ 0.306608] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations892client # [ 0.318580] thermal_sys: Registered thermal governor 'power_allocator'893worker1 # [ 0.307269] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations894lb # [ 0.292992] smp: Bringing up secondary CPUs ...895client # [ 0.318595] cpuidle: using governor menu896worker1 # [ 0.307983] audit: initializing netlink subsys (disabled)897worker2 # [ 0.307099] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations898lb # [ 0.293440] smp: Brought up 1 node, 1 CPU899worker1 # [ 0.309095] thermal_sys: Registered thermal governor 'fair_share'900client # [ 0.320776] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5901lb # [ 0.294090] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)902worker2 # [ 0.308358] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations903worker1 # [ 0.309098] thermal_sys: Registered thermal governor 'bang_bang'904worker2 # [ 0.309083] audit: initializing netlink subsys (disabled)905client # [ 0.321817] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]906worker1 # [ 0.309974] thermal_sys: Registered thermal governor 'step_wise'907lb # [ 0.294639] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)908client # [ 0.322580] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry909worker1 # [ 0.310934] thermal_sys: Registered thermal governor 'user_space'910worker2 # [ 0.310075] audit: type=2000 audit(1789906770.898:1): state=initialized audit_enabled=0 res=1911lb # [ 0.295678] devtmpfs: initialized912client # [ 0.323589] PCI: Using configuration type 1 for base access913worker2 # [ 0.311195] thermal_sys: Registered thermal governor 'fair_share'914worker1 # [ 0.311680] audit: type=2000 audit(1789906770.896:1): state=initialized audit_enabled=0 res=1915lb # [ 0.296546] x86/mm: Memory block size: 128MB916worker2 # [ 0.311197] thermal_sys: Registered thermal governor 'bang_bang'917worker1 # [ 0.312976] thermal_sys: Registered thermal governor 'power_allocator'918client # [ 0.324692] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.919worker1 # [ 0.312989] cpuidle: using governor menu920worker2 # [ 0.312074] thermal_sys: Registered thermal governor 'step_wise'921lb # [ 0.298329] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)922worker2 # [ 0.312984] thermal_sys: Registered thermal governor 'user_space'923worker1 # [ 0.315189] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5924worker2 # [ 0.313785] thermal_sys: Registered thermal governor 'power_allocator'925lb # [ 0.299349] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).926worker2 # [ 0.314086] cpuidle: using governor menu927lb # [ 0.300502] pinctrl core: initialized pinctrl subsystem928worker1 # [ 0.316212] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]929lb # [ 0.301628] PM: RTC time: 12:19:30, date: 2026-09-20930worker1 # [ 0.316976] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry931worker2 # [ 0.316312] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5932client # [ 0.329811] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages933worker1 # [ 0.317986] PCI: Using configuration type 1 for base access934client # [ 0.330578] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page935worker2 # [ 0.317294] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]936worker1 # [ 0.319116] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.937worker2 # [ 0.318078] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry938worker2 # [ 0.319084] PCI: Using configuration type 1 for base access939lb # [ 0.305256] NET: Registered PF_NETLINK/PF_ROUTE protocol family940worker2 # [ 0.320119] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.941lb # [ 0.306732] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations942client # [ 0.335578] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages943lb # [ 0.307451] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations944client # [ 0.336577] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page945worker1 # [ 0.324230] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages946lb # [ 0.308722] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations947worker1 # [ 0.324975] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page948lb # [ 0.309435] audit: initializing netlink subsys (disabled)949lb # [ 0.310659] thermal_sys: Registered thermal governor 'fair_share'950lb # [ 0.310662] thermal_sys: Registered thermal governor 'bang_bang'951worker2 # [ 0.327152] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages952lb # [ 0.311431] audit: type=2000 audit(1789906770.919:1): state=initialized audit_enabled=0 res=1953worker2 # [ 0.328075] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page954worker1 # [ 0.329975] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages955lb # [ 0.313428] thermal_sys: Registered thermal governor 'step_wise'956worker1 # [ 0.330973] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page957lb # [ 0.313430] thermal_sys: Registered thermal governor 'user_space'958lb # [ 0.314412] thermal_sys: Registered thermal governor 'power_allocator'959lb # [ 0.315246] cpuidle: using governor menu960client # [ 0.347042] ACPI: Added _OSI(Module Device)961worker2 # [ 0.333075] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages962lb # [ 0.316972] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5963client # [ 0.347579] ACPI: Added _OSI(Processor Device)964worker2 # [ 0.334074] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page965client # [ 0.348318] ACPI: Added _OSI(Processor Aggregator Device)966lb # [ 0.317682] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]967lb # [ 0.318429] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry968lb # [ 0.319439] PCI: Using configuration type 1 for base access969lb # [ 0.320584] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.970worker1 # [ 0.341375] ACPI: Added _OSI(Module Device)971worker1 # [ 0.341975] ACPI: Added _OSI(Processor Device)972worker1 # [ 0.342748] ACPI: Added _OSI(Processor Aggregator Device)973client # [ 0.357096] ACPI: 1 ACPI AML tables successfully acquired and loaded974worker2 # [ 0.344507] ACPI: Added _OSI(Module Device)975worker2 # [ 0.345075] ACPI: Added _OSI(Processor Device)976lb # [ 0.327443] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages977worker2 # [ 0.345822] ACPI: Added _OSI(Processor Aggregator Device)978lb # [ 0.328427] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page979client # [ 0.360998] ACPI: Interpreter enabled980client # [ 0.361592] ACPI: PM: (supports S0 S3 S4 S5)981client # [ 0.362280] ACPI: Using IOAPIC for interrupt routing982worker1 # [ 0.351501] ACPI: 1 ACPI AML tables successfully acquired and loaded983lb # [ 0.333427] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages984lb # [ 0.334427] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page985client # [ 0.364633] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug986client # [ 0.365578] PCI: Using E820 reservations for host bridge windows987worker2 # [ 0.353791] ACPI: 1 ACPI AML tables successfully acquired and loaded988worker1 # [ 0.355364] ACPI: Interpreter enabled989worker1 # [ 0.355989] ACPI: PM: (supports S0 S3 S4 S5)990client # [ 0.368656] ACPI: Enabled 2 GPEs in block 00 to 3F991worker1 # [ 0.356785] ACPI: Using IOAPIC for interrupt routing992worker2 # [ 0.357482] ACPI: Interpreter enabled993worker2 # [ 0.358090] ACPI: PM: (supports S0 S3 S4 S5)994worker2 # [ 0.358857] ACPI: Using IOAPIC for interrupt routing995worker1 # [ 0.360052] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug996worker2 # [ 0.359133] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug997worker1 # [ 0.360974] PCI: Using E820 reservations for host bridge windows998worker2 # [ 0.360074] PCI: Using E820 reservations for host bridge windows999worker1 # [ 0.362091] ACPI: Enabled 2 GPEs in block 00 to 3F1000lb # [ 0.345660] ACPI: Added _OSI(Module Device)1001lb # [ 0.346381] ACPI: Added _OSI(Processor Device)1002client # [ 0.376294] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1003lb # [ 0.346963] ACPI: Added _OSI(Processor Aggregator Device)1004worker2 # [ 0.363209] ACPI: Enabled 2 GPEs in block 00 to 3F1005client # [ 0.376584] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1006client # [ 0.377656] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1007client # [ 0.378699] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1008client # [ 0.379939] PCI host bridge to bus 0000:001009client # [ 0.380581] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1010client # [ 0.381577] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1011worker1 # [ 0.370703] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1012lb # [ 0.353173] ACPI: 1 ACPI AML tables successfully acquired and loaded1013client # [ 0.382577] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1014worker1 # [ 0.370981] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1015client # [ 0.383577] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1016worker2 # [ 0.371351] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1017worker1 # [ 0.372196] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1018client # [ 0.384576] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1019worker2 # [ 0.372080] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1020worker1 # [ 0.373096] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1021client # [ 0.385577] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1022lb # [ 0.356896] ACPI: Interpreter enabled1023worker2 # [ 0.373149] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1024lb # [ 0.357442] ACPI: PM: (supports S0 S3 S4 S5)1025worker1 # [ 0.374326] PCI host bridge to bus 0000:001026client # [ 0.386577] pci_bus 0000:00: root bus resource [bus 00-ff]1027lb # [ 0.358131] ACPI: Using IOAPIC for interrupt routing1028worker1 # [ 0.374978] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1029worker2 # [ 0.374196] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1030client # [ 0.387539] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1031worker2 # [ 0.375420] PCI host bridge to bus 0000:001032worker1 # [ 0.375974] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1033worker2 # [ 0.376078] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1034worker1 # [ 0.376974] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1035client # [ 0.388980] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1036worker2 # [ 0.377074] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1037lb # [ 0.360492] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1038worker1 # [ 0.377974] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1039lb # [ 0.361427] PCI: Using E820 reservations for host bridge windows1040worker2 # [ 0.378073] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1041worker1 # [ 0.378973] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1042worker2 # [ 0.379073] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1043worker1 # [ 0.379973] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1044worker2 # [ 0.380073] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1045worker1 # [ 0.380978] pci_bus 0000:00: root bus resource [bus 00-ff]1046lb # [ 0.364541] ACPI: Enabled 2 GPEs in block 00 to 3F1047worker2 # [ 0.381074] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1048worker1 # [ 0.381910] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1049worker2 # [ 0.382074] pci_bus 0000:00: root bus resource [bus 00-ff]1050worker1 # [ 0.383478] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1051worker2 # [ 0.383086] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1052client # [ 0.391634] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1053worker2 # [ 0.384510] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1054client # [ 0.392589] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1055client # [ 0.393553] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1056lb # [ 0.371997] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1057client # [ 0.394766] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1058lb # [ 0.372434] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1059client # [ 0.396285] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1060lb # [ 0.373512] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1061lb # [ 0.374548] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1062worker1 # [ 0.386069] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1063client # [ 0.398605] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1064lb # [ 0.375797] PCI host bridge to bus 0000:001065worker1 # [ 0.386986] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1066client # [ 0.399428] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1067lb # [ 0.376431] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1068worker2 # [ 0.387139] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1069worker1 # [ 0.387933] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1070client # [ 0.400284] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1071lb # [ 0.377426] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1072worker2 # [ 0.388085] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1073client # [ 0.400583] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1074lb # [ 0.378426] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1075worker1 # [ 0.389112] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1076worker2 # [ 0.388999] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1077lb # [ 0.379427] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1078client # [ 0.402293] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1079worker1 # [ 0.390701] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1080worker2 # [ 0.390435] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1081lb # [ 0.380426] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1082lb # [ 0.381426] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1083worker2 # [ 0.391792] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1084client # [ 0.404601] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1085lb # [ 0.382426] pci_bus 0000:00: root bus resource [bus 00-ff]1086worker1 # [ 0.392999] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1087client # [ 0.405418] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1088lb # [ 0.383386] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1089worker1 # [ 0.393853] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1090client # [ 0.406289] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1091worker1 # [ 0.394692] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1092worker2 # [ 0.394083] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1093lb # [ 0.384847] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1094worker1 # [ 0.394979] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1095client # [ 0.407135] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1096worker2 # [ 0.394919] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1097worker2 # [ 0.395786] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1098worker1 # [ 0.396864] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1099worker2 # [ 0.396079] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1100client # [ 0.409685] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1101worker2 # [ 0.397627] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1102worker1 # [ 0.399005] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1103client # [ 0.410583] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1104worker1 # [ 0.399980] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1105client # [ 0.411515] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1106worker1 # [ 0.400941] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1107worker2 # [ 0.399109] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1108client # [ 0.412926] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1109worker2 # [ 0.400079] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1110lb # [ 0.387491] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1111worker1 # [ 0.402535] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1112worker2 # [ 0.401045] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1113client # [ 0.414600] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1114lb # [ 0.388571] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1115client # [ 0.415418] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1116lb # [ 0.389448] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1117worker2 # [ 0.402522] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118client # [ 0.416286] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1119lb # [ 0.390790] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1120worker1 # [ 0.404981] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1121client # [ 0.417129] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1122worker1 # [ 0.405980] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1123lb # [ 0.392136] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1124worker1 # [ 0.406957] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1125worker2 # [ 0.405296] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1126client # [ 0.418584] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1127worker2 # [ 0.406080] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1128client # [ 0.419412] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1129worker1 # [ 0.408387] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1130worker2 # [ 0.407028] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1131lb # [ 0.394457] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1132client # [ 0.420288] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1133lb # [ 0.395292] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1134worker2 # [ 0.408444] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1135worker1 # [ 0.409981] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1136lb # [ 0.396174] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1137client # [ 0.421147] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1138worker1 # [ 0.410827] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1139lb # [ 0.396432] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1140worker2 # [ 0.410101] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1141worker1 # [ 0.411730] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1142worker2 # [ 0.410953] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1143lb # [ 0.397969] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1144client # [ 0.423593] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1145worker2 # [ 0.411827] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1146worker1 # [ 0.412547] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1147client # [ 0.424411] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1148lb # [ 0.399449] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1149client # [ 0.425290] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1150worker2 # [ 0.412612] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1151worker1 # [ 0.414003] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1152lb # [ 0.400306] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1153worker1 # [ 0.414840] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1154client # [ 0.426182] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1155lb # [ 0.401191] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1156worker2 # [ 0.414103] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1157worker1 # [ 0.415715] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1158worker2 # [ 0.414916] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1159lb # [ 0.402012] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1160client # [ 0.427592] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1161worker1 # [ 0.416553] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1162worker2 # [ 0.415791] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1163client # [ 0.428407] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1164client # [ 0.429288] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1165worker2 # [ 0.416736] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1166client # [ 0.429583] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1167lb # [ 0.404448] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1168worker1 # [ 0.419002] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1169lb # [ 0.405302] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1170worker1 # [ 0.419873] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1171worker2 # [ 0.419018] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1172client # [ 0.431064] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1173lb # [ 0.406141] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1174worker1 # [ 0.420676] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1175worker2 # [ 0.419676] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1176client # [ 0.432444] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1177lb # [ 0.407019] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1178worker2 # [ 0.420095] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1179client # [ 0.433293] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1180worker1 # [ 0.421572] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1181worker2 # [ 0.421650] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1182lb # [ 0.408440] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1183client # [ 0.434118] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1184worker1 # [ 0.423006] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1185lb # [ 0.409272] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1186worker1 # [ 0.423924] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1187worker2 # [ 0.423110] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1188client # [ 0.435596] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1189worker1 # [ 0.424727] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1190lb # [ 0.410196] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1191worker2 # [ 0.423967] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1192client # [ 0.436421] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1193worker1 # [ 0.424979] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1194worker2 # [ 0.424802] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1195client # [ 0.437331] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1196lb # [ 0.411967] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1197worker2 # [ 0.425079] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1198worker1 # [ 0.426540] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1199lb # [ 0.413434] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1200client # [ 0.438123] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1201worker2 # [ 0.426610] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1202lb # [ 0.414262] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1203worker1 # [ 0.428810] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1204client # [ 0.440584] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1205lb # [ 0.415177] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1206worker1 # [ 0.429718] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1207worker2 # [ 0.427918] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1208client # [ 0.441455] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1209worker2 # [ 0.428813] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1210lb # [ 0.415984] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1211client # [ 0.442331] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1212worker1 # [ 0.430524] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1213worker2 # [ 0.429612] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1214client # [ 0.443177] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1215worker1 # [ 0.432007] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1216client # [ 0.444197] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1217worker1 # [ 0.432875] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1218lb # [ 0.417689] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1219worker2 # [ 0.431112] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1220lb # [ 0.418433] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1221worker2 # [ 0.431965] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1222worker1 # [ 0.433717] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1223client # [ 0.444775] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1224lb # [ 0.419346] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1225worker2 # [ 0.432753] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1226client # [ 0.446116] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1227worker1 # [ 0.434515] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1228lb # [ 0.420856] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1229worker2 # [ 0.434125] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1230worker1 # [ 0.435981] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1231lb # [ 0.422435] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1232worker1 # [ 0.436872] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1233worker2 # [ 0.436081] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1234lb # [ 0.423319] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1235worker1 # [ 0.437771] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1236worker2 # [ 0.436946] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1237lb # [ 0.424139] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1238worker2 # [ 0.437813] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1239lb # [ 0.424432] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1240worker1 # [ 0.439391] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1241worker2 # [ 0.438659] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1242worker1 # [ 0.440554] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1243lb # [ 0.426399] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1244worker2 # [ 0.439728] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1245worker1 # [ 0.441202] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1246lb # [ 0.428260] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1247worker2 # [ 0.440301] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1248lb # [ 0.429193] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1249worker1 # [ 0.442516] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1250client # [ 0.446784] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1251worker2 # [ 0.441598] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1252lb # [ 0.429962] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1253client # [ 0.449109] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1254lb # [ 0.431449] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1255client # [ 0.449819] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1256lb # [ 0.432313] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1257client # [ 0.451097] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1258lb # [ 0.433213] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1259client # [ 0.451845] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1260lb # [ 0.434975] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1261client # [ 0.453057] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1262lb # [ 0.436433] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1263client # [ 0.453849] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1264lb # [ 0.437263] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1265client # [ 0.455375] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1266worker1 # [ 0.443165] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1267lb # [ 0.438202] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1268client # [ 0.456186] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1269worker2 # [ 0.442263] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1270worker1 # [ 0.445514] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1271lb # [ 0.439802] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1272client # [ 0.456902] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1273worker2 # [ 0.444608] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1274worker1 # [ 0.446202] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1275lb # [ 0.441125] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1276client # [ 0.458136] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1277worker2 # [ 0.445327] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1278worker1 # [ 0.447500] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1279lb # [ 0.442159] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1280worker2 # [ 0.446639] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1281worker1 # [ 0.448245] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1282lb # [ 0.444020] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1283worker2 # [ 0.447337] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1284client # [ 0.461739] ACPI: PCI: Interrupt link LNKA configured for IRQ 101285worker1 # [ 0.449267] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1286client # [ 0.462683] ACPI: PCI: Interrupt link LNKB configured for IRQ 101287worker2 # [ 0.448355] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1288worker1 # [ 0.450295] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1289client # [ 0.463678] ACPI: PCI: Interrupt link LNKC configured for IRQ 111290worker2 # [ 0.449330] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1291client # [ 0.464675] ACPI: PCI: Interrupt link LNKD configured for IRQ 111292worker1 # [ 0.451802] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1293client # [ 0.465674] ACPI: PCI: Interrupt link LNKE configured for IRQ 101294worker1 # [ 0.452631] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1295worker2 # [ 0.450911] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1296client # [ 0.466683] ACPI: PCI: Interrupt link LNKF configured for IRQ 101297worker2 # [ 0.451711] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1298worker1 # [ 0.453290] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1299client # [ 0.467683] ACPI: PCI: Interrupt link LNKG configured for IRQ 111300worker2 # [ 0.452388] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1301client # [ 0.468675] ACPI: PCI: Interrupt link LNKH configured for IRQ 111302worker1 # [ 0.454589] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1303client # [ 0.469628] ACPI: PCI: Interrupt link GSIA configured for IRQ 161304worker2 # [ 0.453669] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1305client # [ 0.470600] ACPI: PCI: Interrupt link GSIB configured for IRQ 171306client # [ 0.471599] ACPI: PCI: Interrupt link GSIC configured for IRQ 181307lb # [ 0.444632] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1308worker1 # [ 0.458301] ACPI: PCI: Interrupt link LNKA configured for IRQ 101309client # [ 0.472592] ACPI: PCI: Interrupt link GSID configured for IRQ 191310lb # [ 0.446922] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1311client # [ 0.473519] ACPI: PCI: Interrupt link GSIE configured for IRQ 201312worker1 # [ 0.459078] ACPI: PCI: Interrupt link LNKB configured for IRQ 101313worker2 # [ 0.457633] ACPI: PCI: Interrupt link LNKA configured for IRQ 101314client # [ 0.474299] ACPI: PCI: Interrupt link GSIF configured for IRQ 211315worker1 # [ 0.460073] ACPI: PCI: Interrupt link LNKC configured for IRQ 111316lb # [ 0.447653] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1317worker2 # [ 0.458184] ACPI: PCI: Interrupt link LNKB configured for IRQ 101318client # [ 0.474588] ACPI: PCI: Interrupt link GSIG configured for IRQ 221319worker1 # [ 0.461068] ACPI: PCI: Interrupt link LNKD configured for IRQ 111320worker2 # [ 0.459172] ACPI: PCI: Interrupt link LNKC configured for IRQ 111321client # [ 0.475506] ACPI: PCI: Interrupt link GSIH configured for IRQ 231322lb # [ 0.448963] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1323worker1 # [ 0.462072] ACPI: PCI: Interrupt link LNKE configured for IRQ 101324worker2 # [ 0.460175] ACPI: PCI: Interrupt link LNKD configured for IRQ 111325worker1 # [ 0.463092] ACPI: PCI: Interrupt link LNKF configured for IRQ 101326client # [ 0.477323] iommu: Default domain type: Translated1327lb # [ 0.449701] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1328worker2 # [ 0.461172] ACPI: PCI: Interrupt link LNKE configured for IRQ 101329worker1 # [ 0.464075] ACPI: PCI: Interrupt link LNKG configured for IRQ 111330client # [ 0.477586] iommu: DMA domain TLB invalidation policy: lazy mode1331worker2 # [ 0.462170] ACPI: PCI: Interrupt link LNKF configured for IRQ 101332lb # [ 0.450715] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1333worker1 # [ 0.465082] ACPI: PCI: Interrupt link LNKH configured for IRQ 111334client # [ 0.478767] ACPI: bus type USB registered1335worker2 # [ 0.463176] ACPI: PCI: Interrupt link LNKG configured for IRQ 111336client # [ 0.479486] usbcore: registered new interface driver usbfs1337worker1 # [ 0.466013] ACPI: PCI: Interrupt link GSIA configured for IRQ 161338lb # [ 0.451693] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1339worker2 # [ 0.464172] ACPI: PCI: Interrupt link LNKH configured for IRQ 111340client # [ 0.480240] usbcore: registered new interface driver hub1341worker1 # [ 0.466967] ACPI: PCI: Interrupt link GSIB configured for IRQ 171342client # [ 0.480597] usbcore: registered new device driver usb1343worker2 # [ 0.465112] ACPI: PCI: Interrupt link GSIA configured for IRQ 161344worker1 # [ 0.467942] ACPI: PCI: Interrupt link GSIC configured for IRQ 181345lb # [ 0.453235] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1346worker2 # [ 0.466087] ACPI: PCI: Interrupt link GSIB configured for IRQ 171347lb # [ 0.454028] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1348worker1 # [ 0.468696] ACPI: PCI: Interrupt link GSID configured for IRQ 191349client # [ 0.482272] NetLabel: Initializing1350worker2 # [ 0.467062] ACPI: PCI: Interrupt link GSIC configured for IRQ 181351worker1 # [ 0.468989] ACPI: PCI: Interrupt link GSIE configured for IRQ 201352client # [ 0.482582] NetLabel: domain hash size = 1281353lb # [ 0.454764] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1354client # [ 0.483283] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1355worker2 # [ 0.467823] ACPI: PCI: Interrupt link GSID configured for IRQ 191356worker1 # [ 0.469928] ACPI: PCI: Interrupt link GSIF configured for IRQ 211357client # [ 0.483612] NetLabel: unlabeled traffic allowed by default1358worker2 # [ 0.468086] ACPI: PCI: Interrupt link GSIE configured for IRQ 201359worker1 # [ 0.470682] ACPI: PCI: Interrupt link GSIG configured for IRQ 221360lb # [ 0.456006] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1361client # [ 0.484456] PCI: Using ACPI for IRQ routing1362worker2 # [ 0.469021] ACPI: PCI: Interrupt link GSIF configured for IRQ 211363worker1 # [ 0.470990] ACPI: PCI: Interrupt link GSIH configured for IRQ 231364worker2 # [ 0.469834] ACPI: PCI: Interrupt link GSIG configured for IRQ 221365worker2 # [ 0.470087] ACPI: PCI: Interrupt link GSIH configured for IRQ 231366worker1 # [ 0.473106] iommu: Default domain type: Translated1367worker1 # [ 0.473988] iommu: DMA domain TLB invalidation policy: lazy mode1368lb # [ 0.459598] ACPI: PCI: Interrupt link LNKA configured for IRQ 101369worker2 # [ 0.472114] iommu: Default domain type: Translated1370worker1 # [ 0.475244] ACPI: bus type USB registered1371lb # [ 0.460538] ACPI: PCI: Interrupt link LNKB configured for IRQ 101372worker2 # [ 0.472916] iommu: DMA domain TLB invalidation policy: lazy mode1373worker1 # [ 0.476010] usbcore: registered new interface driver usbfs1374worker2 # [ 0.474057] ACPI: bus type USB registered1375lb # [ 0.461523] ACPI: PCI: Interrupt link LNKC configured for IRQ 111376worker1 # [ 0.476926] usbcore: registered new interface driver hub1377worker2 # [ 0.474659] usbcore: registered new interface driver usbfs1378lb # [ 0.462523] ACPI: PCI: Interrupt link LNKD configured for IRQ 111379worker1 # [ 0.477613] usbcore: registered new device driver usb1380worker2 # [ 0.475090] usbcore: registered new interface driver hub1381lb # [ 0.463526] ACPI: PCI: Interrupt link LNKE configured for IRQ 101382worker1 # [ 0.478821] NetLabel: Initializing1383worker2 # [ 0.476005] usbcore: registered new device driver usb1384lb # [ 0.464522] ACPI: PCI: Interrupt link LNKF configured for IRQ 101385worker1 # [ 0.479427] NetLabel: domain hash size = 1281386worker1 # [ 0.479973] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1387worker2 # [ 0.477679] NetLabel: Initializing1388lb # [ 0.465529] ACPI: PCI: Interrupt link LNKG configured for IRQ 111389worker2 # [ 0.478074] NetLabel: domain hash size = 1281390worker1 # [ 0.480890] NetLabel: unlabeled traffic allowed by default1391lb # [ 0.466522] ACPI: PCI: Interrupt link LNKH configured for IRQ 111392worker1 # [ 0.481666] PCI: Using ACPI for IRQ routing1393worker2 # [ 0.478824] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1394lb # [ 0.467464] ACPI: PCI: Interrupt link GSIA configured for IRQ 161395worker2 # [ 0.479111] NetLabel: unlabeled traffic allowed by default1396lb # [ 0.468436] ACPI: PCI: Interrupt link GSIB configured for IRQ 171397worker2 # [ 0.480050] PCI: Using ACPI for IRQ routing1398lb # [ 0.469408] ACPI: PCI: Interrupt link GSIC configured for IRQ 181399lb # [ 0.470234] ACPI: PCI: Interrupt link GSID configured for IRQ 191400lb # [ 0.471207] ACPI: PCI: Interrupt link GSIE configured for IRQ 201401lb # [ 0.472202] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402lb # [ 0.473246] ACPI: PCI: Interrupt link GSIG configured for IRQ 221403lb # [ 0.474226] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404lb # [ 0.476250] iommu: Default domain type: Translated1405lb # [ 0.477058] iommu: DMA domain TLB invalidation policy: lazy mode1406lb # [ 0.477690] ACPI: bus type USB registered1407lb # [ 0.478455] usbcore: registered new interface driver usbfs1408lb # [ 0.479379] usbcore: registered new interface driver hub1409lb # [ 0.480090] usbcore: registered new device driver usb1410lb # [ 0.481300] NetLabel: Initializing1411lb # [ 0.481912] NetLabel: domain hash size = 1281412lb # [ 0.482426] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413lb # [ 0.483408] NetLabel: unlabeled traffic allowed by default1414lb # [ 0.484171] PCI: Using ACPI for IRQ routing1415client # [ 0.528680] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416client # [ 0.529573] pci 0000:00:01.0: vgaarb: bridge control possible1417client # [ 0.529573] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418client # [ 0.529581] vgaarb: loaded1419client # [ 0.530294] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420client # [ 0.530578] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421client # [ 0.533666] clocksource: Switched to clocksource kvm-clock1422client # [ 0.537196] VFS: Disk quotas dquot_6.6.01423worker1 # [ 0.525498] pci 0000:00:01.0: vgaarb: setting as boot VGA device1424client # [ 0.537912] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1425worker1 # [ 0.525970] pci 0000:00:01.0: vgaarb: bridge control possible1426client # [ 0.539205] pnp: PnP ACPI init1427worker2 # [ 0.523963] pci 0000:00:01.0: vgaarb: setting as boot VGA device1428worker2 # [ 0.524069] pci 0000:00:01.0: vgaarb: bridge control possible1429client # [ 0.539989] ACPI: IRQ 4 override to edge(!), high(!)1430worker1 # [ 0.525970] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1431worker1 # [ 0.525979] vgaarb: loaded1432worker2 # [ 0.524069] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1433client # [ 0.540921] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1434worker2 # [ 0.524078] vgaarb: loaded1435worker1 # [ 0.526725] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01436client # [ 0.542406] pnp: PnP ACPI: found 6 devices1437worker2 # [ 0.524791] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01438worker1 # [ 0.526976] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1439worker2 # [ 0.525074] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1440worker1 # [ 0.530062] clocksource: Switched to clocksource kvm-clock1441worker1 # [ 0.531833] VFS: Disk quotas dquot_6.6.01442worker2 # [ 0.529160] clocksource: Switched to clocksource kvm-clock1443client # [ 0.550057] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1444client # [ 0.551471] clocksource: Switched to clocksource acpi_pm1445worker1 # [ 0.534409] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1446worker2 # [ 0.532648] VFS: Disk quotas dquot_6.6.01447worker1 # [ 0.535853] pnp: PnP ACPI init1448client # [ 0.552428] NET: Registered PF_INET protocol family1449worker2 # [ 0.533360] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1450worker1 # [ 0.536659] ACPI: IRQ 4 override to edge(!), high(!)1451client # [ 0.553585] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1452worker2 # [ 0.534696] pnp: PnP ACPI init1453worker1 # [ 0.537689] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1454worker2 # [ 0.535487] ACPI: IRQ 4 override to edge(!), high(!)1455worker1 # [ 0.539230] pnp: PnP ACPI: found 6 devices1456lb # [ 0.527700] pci 0000:00:01.0: vgaarb: setting as boot VGA device1457worker2 # [ 0.536472] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1458lb # [ 0.528422] pci 0000:00:01.0: vgaarb: bridge control possible1459worker2 # [ 0.537988] pnp: PnP ACPI: found 6 devices1460lb # [ 0.528422] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1461lb # [ 0.528431] vgaarb: loaded1462lb # [ 0.529158] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01463lb # [ 0.529426] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1464worker1 # [ 0.546870] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1465worker1 # [ 0.548337] clocksource: Switched to clocksource acpi_pm1466worker2 # [ 0.545609] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1467worker1 # [ 0.549323] NET: Registered PF_INET protocol family1468lb # [ 0.533518] clocksource: Switched to clocksource kvm-clock1469worker2 # [ 0.547077] clocksource: Switched to clocksource acpi_pm1470worker2 # [ 0.548004] NET: Registered PF_INET protocol family1471worker1 # [ 0.550477] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1472worker2 # [ 0.549133] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1473lb # [ 0.536995] VFS: Disk quotas dquot_6.6.01474lb # [ 0.537672] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1475client # [ 0.570061] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1476lb # [ 0.538988] pnp: PnP ACPI init1477client # [ 0.571582] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1478lb # [ 0.539766] ACPI: IRQ 4 override to edge(!), high(!)1479client # [ 0.572891] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1480lb # [ 0.540749] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1481client # [ 0.574172] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1482lb # [ 0.542269] pnp: PnP ACPI: found 6 devices1483client # [ 0.575390] TCP: Hash tables configured (established 16384 bind 16384)1484client # [ 0.576464] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1485client # [ 0.577691] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1486client # [ 0.578740] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1487client # [ 0.579916] NET: Registered PF_UNIX/PF_LOCAL protocol family1488client # [ 0.580842] NET: Registered PF_XDP protocol family1489client # [ 0.581700] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1490client # [ 0.582678] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1491lb # [ 0.549892] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1492lb # [ 0.551265] clocksource: Switched to clocksource acpi_pm1493client # [ 0.583648] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1494lb # [ 0.552190] NET: Registered PF_INET protocol family1495client # [ 0.584713] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1496worker1 # [ 0.567659] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1497client # [ 0.585769] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1498worker2 # [ 0.566225] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1499worker1 # [ 0.569226] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1500lb # [ 0.553348] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1501client # [ 0.586817] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1502worker1 # [ 0.570451] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1503worker2 # [ 0.567750] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1504client # [ 0.588488] ACPI: \_SB_.GSIA: Enabled at IRQ 161505worker1 # [ 0.571803] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1506worker2 # [ 0.568997] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1507worker1 # [ 0.573086] TCP: Hash tables configured (established 16384 bind 16384)1508worker2 # [ 0.570310] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1509client # [ 0.591006] ACPI: \_SB_.GSIB: Enabled at IRQ 171510worker2 # [ 0.571538] TCP: Hash tables configured (established 16384 bind 16384)1511worker1 # [ 0.574221] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1512worker1 # [ 0.575450] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1513worker2 # [ 0.572668] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1514client # [ 0.592886] ACPI: \_SB_.GSIC: Enabled at IRQ 181515worker1 # [ 0.576546] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1516worker2 # [ 0.573901] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1517worker1 # [ 0.577764] NET: Registered PF_UNIX/PF_LOCAL protocol family1518worker2 # [ 0.574925] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1519client # [ 0.594810] ACPI: \_SB_.GSID: Enabled at IRQ 191520worker1 # [ 0.578704] NET: Registered PF_XDP protocol family1521worker2 # [ 0.576100] NET: Registered PF_UNIX/PF_LOCAL protocol family1522client # [ 0.596456] PCI: CLS 0 bytes, default 641523worker1 # [ 0.579563] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1524worker2 # [ 0.577008] NET: Registered PF_XDP protocol family1525client # [ 0.597366] Trying to unpack rootfs image as initramfs...1526worker1 # [ 0.580544] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1527worker2 # [ 0.577792] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1528worker2 # [ 0.578758] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1529worker1 # [ 0.581514] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1530worker2 # [ 0.579743] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1531worker1 # [ 0.582622] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1532worker2 # [ 0.580817] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1533worker1 # [ 0.583761] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1534worker2 # [ 0.581859] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1535worker1 # [ 0.584870] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1536worker2 # [ 0.582895] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1537client # [ 0.601921] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1538worker1 # [ 0.586603] ACPI: \_SB_.GSIA: Enabled at IRQ 161539lb # [ 0.570581] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1540worker2 # [ 0.584529] ACPI: \_SB_.GSIA: Enabled at IRQ 161541lb # [ 0.572078] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1542worker1 # [ 0.588968] ACPI: \_SB_.GSIB: Enabled at IRQ 171543lb # [ 0.573324] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1544worker2 # [ 0.586861] ACPI: \_SB_.GSIB: Enabled at IRQ 171545lb # [ 0.574605] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1546worker1 # [ 0.591113] ACPI: \_SB_.GSIC: Enabled at IRQ 181547lb # [ 0.575809] TCP: Hash tables configured (established 16384 bind 16384)1548worker2 # [ 0.589244] ACPI: \_SB_.GSIC: Enabled at IRQ 181549worker1 # [ 0.593033] ACPI: \_SB_.GSID: Enabled at IRQ 191550lb # [ 0.576887] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1551worker2 # [ 0.591394] ACPI: \_SB_.GSID: Enabled at IRQ 191552lb # [ 0.578099] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1553worker1 # [ 0.594720] PCI: CLS 0 bytes, default 641554lb # [ 0.579209] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1555worker2 # [ 0.593105] PCI: CLS 0 bytes, default 641556lb # [ 0.580380] NET: Registered PF_UNIX/PF_LOCAL protocol family1557worker1 # [ 0.595650] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1558lb # [ 0.581449] NET: Registered PF_XDP protocol family1559worker1 # [ 0.597466] Trying to unpack rootfs image as initramfs...1560worker2 # [ 0.594032] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1561lb # [ 0.582285] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1562worker2 # [ 0.595792] Trying to unpack rootfs image as initramfs...1563lb # [ 0.583267] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1564lb # [ 0.584233] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1565lb # [ 0.585292] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1566lb # [ 0.586353] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1567lb # [ 0.587417] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1568lb # [ 0.589085] ACPI: \_SB_.GSIA: Enabled at IRQ 161569lb # [ 0.591318] ACPI: \_SB_.GSIB: Enabled at IRQ 171570lb # [ 0.593270] ACPI: \_SB_.GSIC: Enabled at IRQ 181571lb # [ 0.595433] ACPI: \_SB_.GSID: Enabled at IRQ 191572lb # [ 0.597179] PCI: CLS 0 bytes, default 641573lb # [ 0.598044] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574lb # [ 0.599730] Trying to unpack rootfs image as initramfs...1575client # [ 0.642702] Initialise system trusted keyrings1576client # [ 0.646301] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.640078] Initialise system trusted keyrings1578worker1 # [ 0.643381] Initialise system trusted keyrings1579worker2 # [ 0.641008] workingset: timestamp_bits=40 max_order=19 bucket_order=01580worker1 # [ 0.646128] workingset: timestamp_bits=40 max_order=19 bucket_order=01581client # [ 0.667763] Key type asymmetric registered1582client # [ 0.668517] Asymmetric key parser 'x509' registered1583client # [ 0.671311] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1584client # [ 0.674314] io scheduler mq-deadline registered1585client # [ 0.675099] io scheduler kyber registered1586lb # [ 0.643993] Initialise system trusted keyrings1587client # [ 0.677873] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1588client # [ 0.679163] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1589lb # [ 0.646849] workingset: timestamp_bits=40 max_order=19 bucket_order=01590worker1 # [ 0.667352] Key type asymmetric registered1591worker2 # [ 0.664755] Key type asymmetric registered1592client # [ 0.684408] Linux agpgart interface v0.1031593worker1 # [ 0.668041] Asymmetric key parser 'x509' registered1594worker2 # [ 0.665478] Asymmetric key parser 'x509' registered1595client # [ 0.685162] ACPI: bus type drm_connector registered1596worker1 # [ 0.668891] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1597worker2 # [ 0.666338] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1598client # [ 0.688538] usbcore: registered new interface driver usbserial_generic1599client # [ 0.689604] usbserial: USB Serial support registered for generic1600worker1 # [ 0.674244] io scheduler mq-deadline registered1601worker2 # [ 0.671769] io scheduler mq-deadline registered1602worker1 # [ 0.675039] io scheduler kyber registered1603worker2 # [ 0.672538] io scheduler kyber registered1604client # [ 0.693263] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1605client # [ 0.693263] Please enable it if your BIOS has the CPPC option.1606worker2 # [ 0.675244] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1607worker1 # [ 0.678696] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1608client # [ 0.695416] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1609worker2 # [ 0.676486] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1610client # [ 0.696791] drop_monitor: Initializing network drop monitor service1611worker1 # [ 0.680022] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1612client # [ 0.697955] NET: Registered PF_INET6 protocol family1613worker1 # [ 0.683972] Linux agpgart interface v0.1031614worker2 # [ 0.682267] Linux agpgart interface v0.1031615client # [ 0.701889] Segment Routing with IPv61616lb # [ 0.669873] Key type asymmetric registered1617worker2 # [ 0.683040] ACPI: bus type drm_connector registered1618lb # [ 0.670571] Asymmetric key parser 'x509' registered1619worker1 # [ 0.687133] ACPI: bus type drm_connector registered1620lb # [ 0.671461] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1621client # [ 0.705272] In-situ OAM (IOAM) with IPv61622worker2 # [ 0.686078] usbcore: registered new interface driver usbserial_generic1623client # [ 0.706338] IPI shorthand broadcast: enabled1624worker2 # [ 0.687128] usbserial: USB Serial support registered for generic1625worker1 # [ 0.690211] usbcore: registered new interface driver usbserial_generic1626worker1 # [ 0.691325] usbserial: USB Serial support registered for generic1627lb # [ 0.675947] io scheduler mq-deadline registered1628lb # [ 0.676949] io scheduler kyber registered1629worker2 # [ 0.690654] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1630worker2 # [ 0.690654] Please enable it if your BIOS has the CPPC option.1631worker1 # [ 0.694095] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1632worker1 # [ 0.694095] Please enable it if your BIOS has the CPPC option.1633worker2 # [ 0.692800] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1634lb # [ 0.680450] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1635worker1 # [ 0.696326] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1636lb # [ 0.681751] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1637client # [ 0.714880] sched_clock: Marking stable (579024518, 135225834)->(784833406, -70583054)1638worker2 # [ 0.696764] drop_monitor: Initializing network drop monitor service1639worker2 # [ 0.697902] NET: Registered PF_INET6 protocol family1640lb # [ 0.685572] Linux agpgart interface v0.1031641worker1 # [ 0.701312] drop_monitor: Initializing network drop monitor service1642worker2 # [ 0.699163] Segment Routing with IPv61643lb # [ 0.686399] ACPI: bus type drm_connector registered1644worker1 # [ 0.702713] NET: Registered PF_INET6 protocol family1645client # [ 0.720395] registered taskstats version 11646worker2 # [ 0.701695] In-situ OAM (IOAM) with IPv61647client # [ 0.721358] Loading compiled-in X.509 certificates1648lb # [ 0.689260] usbcore: registered new interface driver usbserial_generic1649worker1 # [ 0.705541] Segment Routing with IPv61650worker2 # [ 0.702737] IPI shorthand broadcast: enabled1651lb # [ 0.690344] usbserial: USB Serial support registered for generic1652worker1 # [ 0.709102] In-situ OAM (IOAM) with IPv61653worker1 # [ 0.710196] IPI shorthand broadcast: enabled1654lb # [ 0.694099] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1655lb # [ 0.694099] Please enable it if your BIOS has the CPPC option.1656lb # [ 0.696293] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1657worker2 # [ 0.711088] sched_clock: Marking stable (577014273, 133618622)->(789588252, -78955357)1658lb # [ 0.699950] drop_monitor: Initializing network drop monitor service1659lb # [ 0.701310] NET: Registered PF_INET6 protocol family1660worker2 # [ 0.715910] registered taskstats version 11661worker1 # [ 0.718568] sched_clock: Marking stable (585013497, 133057994)->(795520643, -77449152)1662worker2 # [ 0.716836] Loading compiled-in X.509 certificates1663lb # [ 0.704293] Segment Routing with IPv61664client # [ 0.738057] Demotion targets for Node 0: null1665worker1 # [ 0.722269] registered taskstats version 11666lb # [ 0.706832] In-situ OAM (IOAM) with IPv61667worker1 # [ 0.723278] Loading compiled-in X.509 certificates1668lb # [ 0.707919] IPI shorthand broadcast: enabled1669client # [ 0.741313] Key type .fscrypt registered1670client # [ 0.741996] Key type fscrypt-provisioning registered1671client # [ 0.742902] ima: No TPM chip found, activating TPM-bypass!1672client # [ 0.745256] ima: Allocated hash algorithm: sha11673client # [ 0.746049] ima: No architecture policies found1674client # [ 0.748264] PM: Magic number: 10:66:3301675client # [ 0.748950] tty tty51: hash matches1676lb # [ 0.716293] sched_clock: Marking stable (583014129, 132772392)->(789025904, -73239383)1677client # [ 0.752595] RAS: Correctable Errors collector initialized.1678worker2 # [ 0.734646] Demotion targets for Node 0: null1679lb # [ 0.722058] registered taskstats version 11680lb # [ 0.723002] Loading compiled-in X.509 certificates1681worker2 # [ 0.737683] Key type .fscrypt registered1682worker2 # [ 0.738358] Key type fscrypt-provisioning registered1683worker1 # [ 0.742090] Demotion targets for Node 0: null1684worker2 # [ 0.739294] ima: No TPM chip found, activating TPM-bypass!1685worker1 # [ 0.744130] Key type .fscrypt registered1686worker2 # [ 0.741650] ima: Allocated hash algorithm: sha11687client # [ 0.761414] clk: Disabling unused clocks1688worker1 # [ 0.744836] Key type fscrypt-provisioning registered1689worker2 # [ 0.742441] ima: No architecture policies found1690client # [ 0.762107] PM: genpd: Disabling unused power domains1691worker1 # [ 0.745783] ima: No TPM chip found, activating TPM-bypass!1692worker2 # [ 0.744645] PM: Magic number: 10:66:3301693worker2 # [ 0.745401] tty tty51: hash matches1694worker1 # [ 0.750095] ima: Allocated hash algorithm: sha11695worker1 # [ 0.750912] ima: No architecture policies found1696worker2 # [ 0.749012] RAS: Correctable Errors collector initialized.1697worker1 # [ 0.753083] PM: Magic number: 10:66:3301698worker1 # [ 0.753828] tty tty51: hash matches1699worker1 # [ 0.757445] RAS: Correctable Errors collector initialized.1700lb # [ 0.741803] Demotion targets for Node 0: null1701lb # [ 0.742686] Key type .fscrypt registered1702lb # [ 0.743429] Key type fscrypt-provisioning registered1703worker2 # [ 0.757603] clk: Disabling unused clocks1704lb # [ 0.746931] ima: No TPM chip found, activating TPM-bypass!1705lb # [ 0.747954] ima: Allocated hash algorithm: sha11706worker2 # [ 0.761651] PM: genpd: Disabling unused power domains1707lb # [ 0.748748] ima: No architecture policies found1708worker1 # [ 0.766106] clk: Disabling unused clocks1709lb # [ 0.751004] PM: Magic number: 10:66:3301710worker1 # [ 0.766821] PM: genpd: Disabling unused power domains1711lb # [ 0.751706] tty tty51: hash matches1712lb # [ 0.755268] RAS: Correctable Errors collector initialized.1713lb # [ 0.764074] clk: Disabling unused clocks1714lb # [ 0.764757] PM: genpd: Disabling unused power domains1715client # [ 0.948118] Freeing initrd memory: 29096K1716client # [ 0.951371] Freeing unused decrypted memory: 2028K1717worker2 # [ 0.932038] Freeing initrd memory: 29092K1718worker1 # [ 0.936934] Freeing initrd memory: 29092K1719client # [ 0.954011] Freeing unused kernel image (initmem) memory: 3644K1720worker2 # [ 0.935314] Freeing unused decrypted memory: 2028K1721client # [ 0.955068] Write protecting the kernel read-only data: 32768k1722worker1 # [ 0.940186] Freeing unused decrypted memory: 2028K1723client # [ 0.957007] Freeing unused kernel image (text/rodata gap) memory: 1208K1724worker2 # [ 0.938046] Freeing unused kernel image (initmem) memory: 3644K1725worker2 # [ 0.939105] Write protecting the kernel read-only data: 32768k1726client # [ 0.958652] Freeing unused kernel image (rodata/data gap) memory: 752K1727worker1 # [ 0.942862] Freeing unused kernel image (initmem) memory: 3644K1728worker2 # [ 0.941003] Freeing unused kernel image (text/rodata gap) memory: 1208K1729worker1 # [ 0.943922] Write protecting the kernel read-only data: 32768k1730worker2 # [ 0.942492] Freeing unused kernel image (rodata/data gap) memory: 752K1731worker1 # [ 0.945795] Freeing unused kernel image (text/rodata gap) memory: 1208K1732worker1 # [ 0.947316] Freeing unused kernel image (rodata/data gap) memory: 752K1733lb # [ 0.937758] Freeing initrd memory: 29092K1734lb # [ 0.941031] Freeing unused decrypted memory: 2028K1735lb # [ 0.943708] Freeing unused kernel image (initmem) memory: 3644K1736lb # [ 0.944801] Write protecting the kernel read-only data: 32768k1737lb # [ 0.946730] Freeing unused kernel image (text/rodata gap) memory: 1208K1738lb # [ 0.948388] Freeing unused kernel image (rodata/data gap) memory: 752K1739client # [ 1.010898] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740client # [ 1.011987] Run /init as init process1741worker2 # [ 0.993546] x86/mm: Checked W+X mappings: passed, no W+X pages found.1742worker2 # [ 0.994611] Run /init as init process1743worker1 # [ 0.998357] x86/mm: Checked W+X mappings: passed, no W+X pages found.1744worker1 # [ 0.999371] Run /init as init process1745client # [ 1.023705] systemd[1]: Inserted module 'autofs4'1746worker2 # [ 1.005065] systemd[1]: Inserted module 'autofs4'1747worker1 # [ 1.009886] systemd[1]: Inserted module 'autofs4'1748lb # [ 0.999411] x86/mm: Checked W+X mappings: passed, no W+X pages found.1749lb # [ 1.000538] Run /init as init process1750worker2 # [ 1.021373] fuse: init (API version 7.45)1751client # [ 1.041055] fuse: init (API version 7.45)1752worker1 # [ 1.026192] fuse: init (API version 7.45)1753lb # [ 1.011250] systemd[1]: Inserted module 'autofs4'1754worker2 # [ 1.027282] ACPI: \_SB_.GSIG: Enabled at IRQ 221755client # [ 1.047157] ACPI: \_SB_.GSIG: Enabled at IRQ 221756worker1 # [ 1.032042] ACPI: \_SB_.GSIG: Enabled at IRQ 221757worker2 # [ 1.030055] ACPI: \_SB_.GSIH: Enabled at IRQ 231758client # [ 1.049650] ACPI: \_SB_.GSIH: Enabled at IRQ 231759worker1 # [ 1.034569] ACPI: \_SB_.GSIH: Enabled at IRQ 231760client # [ 1.052612] ACPI: \_SB_.GSIE: Enabled at IRQ 201761worker2 # [ 1.033509] ACPI: \_SB_.GSIE: Enabled at IRQ 201762worker1 # [ 1.037874] ACPI: \_SB_.GSIE: Enabled at IRQ 201763client # [ 1.054823] ACPI: \_SB_.GSIF: Enabled at IRQ 211764worker2 # [ 1.036373] ACPI: \_SB_.GSIF: Enabled at IRQ 211765worker1 # [ 1.040452] ACPI: \_SB_.GSIF: Enabled at IRQ 211766lb # [ 1.028163] fuse: init (API version 7.45)1767lb # [ 1.034212] ACPI: \_SB_.GSIG: Enabled at IRQ 221768lb # [ 1.036990] ACPI: \_SB_.GSIH: Enabled at IRQ 231769lb # [ 1.040021] ACPI: \_SB_.GSIE: Enabled at IRQ 201770lb # [ 1.042295] ACPI: \_SB_.GSIF: Enabled at IRQ 211771client # [ 1.082146] systemd[1]: Successfully made /usr/ read-only.1772worker2 # [ 1.064471] systemd[1]: Successfully made /usr/ read-only.1773worker1 # [ 1.070219] systemd[1]: Successfully made /usr/ read-only.1774lb # [ 1.070029] systemd[1]: Successfully made /usr/ read-only.1775client # [ 1.418057] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1776client # [ 1.423313] systemd[1]: Detected virtualization kvm.1777worker2 # [ 1.400032] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778client # [ 1.424218] systemd[1]: Detected architecture x86-64.1779worker2 # [ 1.404831] systemd[1]: Detected virtualization kvm.1780client # [ 1.425115] systemd[1]: Running in initrd.1781worker2 # [ 1.405670] systemd[1]: Detected architecture x86-64.1782worker2 # [ 1.406507] systemd[1]: Running in initrd.1783client # [ 1.426367] systemd[1]: Initializing machine ID from random generator.1784worker2 # [ 1.407604] systemd[1]: Initializing machine ID from random generator.1785worker1 # [ 1.405867] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1786client # [ 1.427663] systemd[1]: Hostname set to <client>.1787worker2 # [ 1.408801] systemd[1]: Hostname set to <worker2>.1788worker1 # [ 1.411807] systemd[1]: Detected virtualization kvm.1789worker1 # [ 1.412690] systemd[1]: Detected architecture x86-64.1790worker1 # [ 1.413630] systemd[1]: Running in initrd.1791worker1 # [ 1.414831] systemd[1]: Initializing machine ID from random generator.1792worker1 # [ 1.416159] systemd[1]: Hostname set to <worker1>.1793lb # [ 1.405665] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1794lb # [ 1.410571] systemd[1]: Detected virtualization kvm.1795lb # [ 1.411442] systemd[1]: Detected architecture x86-64.1796lb # [ 1.412355] systemd[1]: Running in initrd.1797lb # [ 1.413557] systemd[1]: Initializing machine ID from random generator.1798lb # [ 1.415431] systemd[1]: Hostname set to <lb>.1799client # [ 1.624151] systemd[1]: bpf-restrict-fs: LSM BPF program attached1800worker1 # [ 1.623591] systemd[1]: bpf-restrict-fs: LSM BPF program attached1801worker2 # [ 1.630737] systemd[1]: bpf-restrict-fs: LSM BPF program attached1802lb # [ 1.620543] systemd[1]: bpf-restrict-fs: LSM BPF program attached1803client # [ 1.666820] systemd[1]: Queued start job for default target Initrd Default Target.1804client # [ 1.670690] systemd[1]: Created slice Slice /system/modprobe.1805client # [ 1.671864] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1806client # [ 1.673266] systemd[1]: Expecting device /dev/disk/by-label/nixos...1807client # [ 1.674455] systemd[1]: Reached target Path Units.1808client # [ 1.675333] systemd[1]: Reached target Slice Units.1809client # [ 1.676179] systemd[1]: Reached target Swaps.1810client # [ 1.676971] systemd[1]: Reached target Timer Units.1811client # [ 1.677972] systemd[1]: Listening on D-Bus System Message Bus Socket.1812client # [ 1.679182] systemd[1]: Listening on Journal Socket (/dev/log).1813client # [ 1.680340] systemd[1]: Listening on Journal Sockets.1814worker1 # [ 1.663276] systemd[1]: Queued start job for default target Initrd Default Target.1815client # [ 1.681320] systemd[1]: Listening on udev Control Socket.1816client # [ 1.682391] systemd[1]: Listening on udev Kernel Socket.1817client # [ 1.683340] systemd[1]: Reached target Socket Units.1818worker1 # [ 1.667394] systemd[1]: Created slice Slice /system/modprobe.1819client # [ 1.685215] systemd[1]: Starting Create List of Static Device Nodes...1820worker1 # [ 1.668536] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1821worker1 # [ 1.669912] systemd[1]: Expecting device /dev/disk/by-label/nixos...1822worker1 # [ 1.671007] systemd[1]: Reached target Path Units.1823worker1 # [ 1.671962] systemd[1]: Reached target Slice Units.1824client # [ 1.689114] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1825worker1 # [ 1.672848] systemd[1]: Reached target Swaps.1826worker1 # [ 1.673645] systemd[1]: Reached target Timer Units.1827worker1 # [ 1.674602] systemd[1]: Listening on D-Bus System Message Bus Socket.1828worker1 # [ 1.675879] systemd[1]: Listening on Journal Socket (/dev/log).1829worker1 # [ 1.677032] systemd[1]: Listening on Journal Sockets.1830worker1 # [ 1.678086] systemd[1]: Listening on udev Control Socket.1831worker1 # [ 1.679095] systemd[1]: Listening on udev Kernel Socket.1832client # [ 1.695993] systemd[1]: Starting Load Kernel Module configfs...1833worker1 # [ 1.680034] systemd[1]: Reached target Socket Units.1834worker1 # [ 1.681883] systemd[1]: Starting Create List of Static Device Nodes...1835worker1 # [ 1.686147] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1836worker2 # [ 1.686122] systemd[1]: Queued start job for default target Initrd Default Target.1837lb # [ 1.674886] systemd[1]: Queued start job for default target Initrd Default Target.1838worker2 # [ 1.689995] systemd[1]: Created slice Slice /system/modprobe.1839worker1 # [ 1.692805] systemd[1]: Starting Load Kernel Module configfs...1840lb # [ 1.678160] systemd[1]: Created slice Slice /system/modprobe.1841worker2 # [ 1.691207] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1842client # [ 1.711321] systemd[1]: Starting Journal Service...1843lb # [ 1.679285] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1844worker2 # [ 1.692680] systemd[1]: Expecting device /dev/disk/by-label/nixos...1845lb # [ 1.680653] systemd[1]: Expecting device /dev/disk/by-label/nixos...1846worker2 # [ 1.693816] systemd[1]: Reached target Path Units.1847lb # [ 1.681690] systemd[1]: Reached target Path Units.1848worker2 # [ 1.694757] systemd[1]: Reached target Slice Units.1849lb # [ 1.682520] systemd[1]: Reached target Slice Units.1850worker2 # [ 1.695658] systemd[1]: Reached target Swaps.1851lb # [ 1.683363] systemd[1]: Reached target Swaps.1852worker2 # [ 1.696486] systemd[1]: Reached target Timer Units.1853lb # [ 1.684173] systemd[1]: Reached target Timer Units.1854worker2 # [ 1.697490] systemd[1]: Listening on D-Bus System Message Bus Socket.1855lb # [ 1.685116] systemd[1]: Listening on D-Bus System Message Bus Socket.1856worker2 # [ 1.698754] systemd[1]: Listening on Journal Socket (/dev/log).1857lb # [ 1.686340] systemd[1]: Listening on Journal Socket (/dev/log).1858worker2 # [ 1.699985] systemd[1]: Listening on Journal Sockets.1859lb # [ 1.687461] systemd[1]: Listening on Journal Sockets.1860worker2 # [ 1.701000] systemd[1]: Listening on udev Control Socket.1861lb # [ 1.688425] systemd[1]: Listening on udev Control Socket.1862worker2 # [ 1.702112] systemd[1]: Listening on udev Kernel Socket.1863lb # [ 1.689463] systemd[1]: Listening on udev Kernel Socket.1864worker2 # [ 1.703092] systemd[1]: Reached target Socket Units.1865lb # [ 1.690416] systemd[1]: Reached target Socket Units.1866worker2 # [ 1.704949] systemd[1]: Starting Create List of Static Device Nodes...1867worker1 # [ 1.708152] systemd[1]: Starting Journal Service...1868lb # [ 1.692331] systemd[1]: Starting Create List of Static Device Nodes...1869worker2 # [ 1.708712] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1870lb # [ 1.696906] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1871client # [ 1.722977] systemd[1]: Starting Load Kernel Modules...1872lb # [ 1.703891] systemd[1]: Starting Load Kernel Module configfs...1873worker2 # [ 1.717845] systemd[1]: Starting Load Kernel Module configfs...1874client # [ 1.745796] netfs: FS-Cache loaded1875worker1 # [ 1.721166] systemd[1]: Starting Load Kernel Modules...1876worker2 # [ 1.731712] systemd[1]: Starting Journal Service...1877client # [ 1.751874] 9pnet: Installing 9P2000 support1878lb # [ 1.720493] systemd[1]: Starting Journal Service...1879client # [ 1.756955] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1880worker1 # [ 1.741614] netfs: FS-Cache loaded1881worker1 # [ 1.748543] 9pnet: Installing 9P2000 support1882client # [ 1.769124] systemd[1]: Starting Coldplug All udev Devices...1883worker1 # [ 1.752290] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1884lb # [ 1.733892] systemd[1]: Starting Load Kernel Modules...1885worker2 # [ 1.745786] systemd[1]: Starting Load Kernel Modules...1886worker1 # [ 1.763408] systemd[1]: Starting Coldplug All udev Devices...1887lb # [ 1.753035] netfs: FS-Cache loaded1888worker2 # [ 1.770295] netfs: FS-Cache loaded1889lb # [ 1.758022] 9pnet: Installing 9P2000 support1890client # [ 1.791389] systemd[1]: Finished Create List of Static Device Nodes.1891worker2 # [ 1.776357] 9pnet: Installing 9P2000 support1892worker1 # [ 1.782170] systemd[1]: Finished Create List of Static Device Nodes.1893lb # [ 1.766224] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1894client # [ 1.803411] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1895worker2 # [ 1.785091] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1896worker1 # [ 1.790196] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897client # [ 1.808104] systemd-journald[67]: Collecting audit messages is disabled.1898lb # [ 1.776236] systemd[1]: Starting Coldplug All udev Devices...1899client # [ 1.810702] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1900worker1 # [ 1.795562] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1901client # [ 1.815724] systemd[1]: modprobe@configfs.service: Deactivated successfully.1902worker2 # [ 1.796912] systemd[1]: Starting Coldplug All udev Devices...1903worker1 # [ 1.800725] systemd-journald[67]: Collecting audit messages is disabled.1904worker1 # [ 1.802491] systemd[1]: modprobe@configfs.service: Deactivated successfully.1905lb # [ 1.791857] systemd[1]: Finished Create List of Static Device Nodes.1906client # [ 1.825708] systemd[1]: Finished Load Kernel Module configfs.1907worker1 # [ 1.810457] systemd[1]: Finished Load Kernel Module configfs.1908client # [ 1.827356] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1909lb # [ 1.796482] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1910worker1 # [ 1.812694] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1911worker2 # [ 1.811744] systemd[1]: Finished Create List of Static Device Nodes.1912client # [ 1.831819] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1913worker1 # [ 1.815514] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1914lb # [ 1.801476] systemd-journald[67]: Collecting audit messages is disabled.1915client # [ 1.836282] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1916worker2 # [ 1.817280] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1917lb # [ 1.807248] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1918worker1 # [ 1.822937] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1919worker2 # [ 1.824392] systemd-journald[67]: Collecting audit messages is disabled.1920worker1 # [ 1.827296] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1921lb # [ 1.812288] systemd[1]: modprobe@configfs.service: Deactivated successfully.1922worker2 # [ 1.825997] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1923client # [ 1.845574] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1924lb # [ 1.817196] systemd[1]: Finished Load Kernel Module configfs.1925lb # [ 1.820053] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1926worker2 # [ 1.834216] systemd[1]: modprobe@configfs.service: Deactivated successfully.1927lb # [ 1.824281] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1928worker2 # [ 1.839026] systemd[1]: Finished Load Kernel Module configfs.1929lb # [ 1.828810] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1930worker2 # [ 1.843099] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1931lb # [ 1.833696] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1932worker2 # [ 1.846382] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1933worker1 # [ 1.854651] systemd[1]: Finished Load Kernel Modules.1934client # [ 1.871824] systemd[1]: Finished Load Kernel Modules.1935worker2 # [ 1.856007] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1936worker2 # [ 1.857421] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1937worker1 # [ 1.862301] systemd[1]: Starting Apply Kernel Variables...1938client # [ 1.881511] systemd[1]: Starting Apply Kernel Variables...1939lb # [ 1.861335] systemd[1]: Finished Load Kernel Modules.1940worker1 # [ 1.878213] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1941client # [ 1.897631] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1942lb # [ 1.867856] systemd[1]: Starting Apply Kernel Variables...1943worker1 # [ 1.885285] systemd[1]: Starting Create Static Device Nodes in /dev...1944worker2 # [ 1.889241] systemd[1]: Finished Load Kernel Modules.1945client # [ 1.908938] systemd[1]: Starting Create Static Device Nodes in /dev...1946lb # [ 1.880962] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1947worker2 # [ 1.897854] systemd[1]: Starting Apply Kernel Variables...1948worker1 # [ 1.904401] systemd[1]: Finished Apply Kernel Variables.1949client # [ 1.921358] systemd[1]: Finished Apply Kernel Variables.1950lb # [ 1.891004] systemd[1]: Starting Create Static Device Nodes in /dev...1951worker2 # [ 1.910712] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1952worker1 # [ 1.921631] systemd[1]: Finished Create Static Device Nodes in /dev.1953worker2 # [ 1.918961] systemd[1]: Starting Create Static Device Nodes in /dev...1954worker1 # [ 1.925322] systemd[1]: Started Journal Service.1955worker1 # [ 1.794124] systemd-modules-load[68]: Inserted module 'dm_mod'1956lb # [ 1.911866] systemd[1]: Finished Apply Kernel Variables.1957client # [ 1.808456] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.944932] systemd[1]: Started Journal Service.1958worker1 # [ 1.795591] systemd-modules-load[68]: Inserted module 'virtio_balloon'1959client # 1960client # [ 1.812365] systemd-modules-load[68]: Inserted module 'virtio_balloon'1961worker1 # [ 1.799180] systemd-modules-load[68]: Inserted module 'virtio_gpu'1962client # [ 1.814426] systemd-modules-load[68]: Inserted module 'virtio_gpu'1963worker1 # [ 1.800252] systemd[1]: Reached target Preparation for Local File Systems.1964worker1 # [ 1.801412] systemd[1]: Reached target Local File Systems.1965worker1 # [ 1.806083] systemd[1]: Starting Create System Files and Directories...1966lb # [ 1.791656] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.925533] systemd[1]: Started Journal Service.1967client # [ 1.822961] systemd[1]: Finished Create Static Device Nodes in /dev.1968lb # 1969worker2 # [ 1.939818] systemd[1]: Finished Apply Kernel Variables.1970lb # [ 1.794349] systemd-modules-load[68]: Inserted module 'virtio_balloon'1971lb # [ 1.796295] systemd-modules-load[68]: Inserted module 'virtio_gpu'1972worker1 # [ 1.812075] systemd[1]: Starting Rule-based Manager for Device Events and Files...1973client # [ 1.828110] systemd[1]: Reached target Preparation for Local File Systems.1974client # [ 1.829190] systemd[1]: Reached target Local File Systems.1975lb # [ 1.799275] systemd[1]: Finished Create Static Device Nodes in /dev.1976worker2 # [ 1.813658] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.949817] systemd[1]: Started Journal Service.1977client # [ 1.834070] systemd[1]: Starting Create System Files and Directories...1978worker2 # 1979worker2 # [ 1.818332] systemd-modules-load[68]: Inserted module 'virtio_balloon'1980worker2 # [ 1.819362] systemd-modules-load[68]: Inserted module 'virtio_gpu'1981lb # [ 1.808108] systemd[1]: Reached target Preparation for Local File Systems.1982lb # [ 1.809139] systemd[1]: Reached target Local File Systems.1983client # [ 1.839398] systemd[1]: Starting Rule-based Manager for Device Events and Files...1984lb # [ 1.812072] systemd[1]: Starting Create System Files and Directories...1985worker2 # [ 1.828130] systemd[1]: Finished Create Static Device Nodes in /dev.1986lb # [ 1.817072] systemd[1]: Starting Rule-based Manager for Device Events and Files...1987worker2 # [ 1.831946] systemd[1]: Reached target Preparation for Local File Systems.1988worker2 # [ 1.835094] systemd[1]: Reached target Local File Systems.1989worker2 # [ 1.838098] systemd[1]: Starting Create System Files and Directories...1990worker2 # [ 1.845079] systemd[1]: Starting Rule-based Manager for Device Events and Files...1991worker1 # [ 1.851222] systemd[1]: Finished Create System Files and Directories.1992lb # [ 1.842992] systemd[1]: Finished Create System Files and Directories.1993client # [ 1.874235] systemd[1]: Finished Create System Files and Directories.1994worker2 # [ 1.870181] systemd[1]: Finished Create System Files and Directories.1995worker1 # [ 1.874795] systemd-udevd[76]: Using default interface naming scheme 'v261'.1996client # [ 1.897905] systemd-udevd[82]: Using default interface naming scheme 'v261'.1997lb # [ 1.868277] systemd-udevd[83]: Using default interface naming scheme 'v261'.1998worker2 # [ 1.896286] systemd-udevd[83]: Using default interface naming scheme 'v261'.1999worker1 # [ 1.900354] systemd[1]: Started Rule-based Manager for Device Events and Files.2000lb # [ 1.892313] systemd[1]: Started Rule-based Manager for Device Events and Files.
loading 2,014 hidden lines…
4015lb # [ 17.633901] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4016lb # [ 17.638499] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4017lb # [ 17.640398] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4018lb # [ 17.643697] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4019lb # [ 17.646106] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4020lb # [ 17.649204] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4021lb # [ 17.650837] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4022lb # [ 17.652817] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4023lb # [ 17.658433] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4024lb # [ 17.660704] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4025lb # [ 17.666521] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4026lb # [ 17.668507] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4027lb # [ 17.670613] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4028lb # [ 17.673455] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4029lb # [ 17.681177] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4030lb # [ 17.682936] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4031lb # [ 17.686162] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4032lb # [ 17.689043] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4033lb # [ 17.691706] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4034lb # [ 17.719621] envoy[919]: [2026-09-20 12:19:48.451][919][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4035lb # [ 17.721349] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4036lb # [ 17.723214] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4037lb # [ 17.725578] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4038lb # [ 17.736926] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4039lb # [ 17.739230] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4040lb # [ 17.742297] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4041lb # [ 17.747083] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4042lb # [ 17.749114] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24043lb # [ 17.751462] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4044lb # [ 17.753498] envoy[919]: [2026-09-20 12:19:48.452][919][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4045lb # [ 17.755423] envoy[919]: [2026-09-20 12:19:48.456][919][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4046lb # [ 17.758440] envoy[919]: [2026-09-20 12:19:48.456][919][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4047lb # [ 17.760718] envoy[919]: [2026-09-20 12:19:48.456][919][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4048lb # [ 17.763354] envoy[919]: [2026-09-20 12:19:48.456][919][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4049lb # [ 17.765034] envoy[919]: [2026-09-20 12:19:48.456][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4050lb # [ 17.767973] envoy[919]: [2026-09-20 12:19:48.456][919][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4051lb # [ 17.770079] envoy[919]: [2026-09-20 12:19:48.456][919][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24052lb # [ 17.771751] envoy[919]: [2026-09-20 12:19:48.456][919][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4053lb # [ 17.773525] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4054lb # [ 17.775676] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4055lb # [ 17.778331] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24056lb # [ 17.780122] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4057lb # [ 17.782621] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84058lb # [ 17.784520] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4059lb # [ 17.787682] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4060lb # [ 17.790389] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4061lb # [ 17.797137] envoy[919]: [2026-09-20 12:19:48.462][919][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4062lb # [ 17.798952] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4063lb # [ 17.800506] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4064lb # [ 17.803202] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4065lb # [ 17.809689] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4066lb # [ 17.812559] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4067lb # [ 17.814511] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4068lb # [ 17.817092] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4069lb # [ 17.819082] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4070lb # [ 17.824855] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4071lb # [ 17.831491] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4072lb # [ 17.833282] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4073lb # [ 17.835393] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4074lb # [ 17.837699] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4075lb # [ 17.839770] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4076lb # [ 17.843060] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4077lb # [ 17.852692] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4078lb # [ 17.855213] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4079lb # [ 17.857174] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4080lb # [ 17.858945] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4081lb # [ 17.861334] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4082lb # [ 17.863364] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4083lb # [ 17.866116] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4084lb # [ 17.870892] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4085lb # [ 17.872563] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4086lb # [ 17.874677] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4087lb # [ 17.877582] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4088lb # [ 17.885129] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4089lb # [ 17.888849] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4090lb # [ 17.890510] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4091lb # [ 17.892904] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4092lb # [ 17.896019] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4093lb # [ 17.897813] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4094lb # [ 17.900496] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4095lb # [ 17.902661] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4096lb # [ 17.904518] envoy[919]: [2026-09-20 12:19:48.463][919][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4097lb # [ 17.906344] envoy[919]: [2026-09-20 12:19:48.642][919][info][main] [source/server/server.cc:531] HTTP header map info:4098lb # [ 17.907673] envoy[919]: [2026-09-20 12:19:48.676][919][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4099lb # [ 17.921337] envoy[919]: [2026-09-20 12:19:48.676][919][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4100lb # [ 17.922763] envoy[919]: [2026-09-20 12:19:48.676][919][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4101lb # [ 17.931305] envoy[919]: [2026-09-20 12:19:48.676][919][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4102lb # [ 17.962531] envoy[919]: [2026-09-20 12:19:48.856][919][info][main] [source/server/server.cc:968] runtime: {}4103lb # [ 17.968641] envoy[919]: [2026-09-20 12:19:48.862][919][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4104lb # [ 17.970726] envoy[919]: [2026-09-20 12:19:48.864][919][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014105lb # [ 17.973389] envoy[919]: [2026-09-20 12:19:48.867][919][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014106lb # [ 17.976124] envoy[919]: [2026-09-20 12:19:48.869][919][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4107lb # [ 17.977738] envoy[919]: [2026-09-20 12:19:48.871][919][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4108lb # [ 17.979251] envoy[919]: [2026-09-20 12:19:48.871][919][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4109worker1 # [ 18.041541] systemd[1]: Starting Nix worker-protocol over gRPC...4110lb # [ 18.025746] envoy[919]: [2026-09-20 12:19:48.919][919][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4111worker2 # [ 18.039443] systemd[1]: Reached target Network is Online.4112worker2 # [ 18.042731] systemd[1]: Starting scheduler-token.service...4113lb # [ 18.049106] envoy[919]: [2026-09-20 12:19:48.942][919][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4114lb # [ 18.051991] envoy[919]: [2026-09-20 12:19:48.945][919][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4115lb # [ 18.055109] envoy[919]: [2026-09-20 12:19:48.945][919][info][main] [source/server/server.cc:1077] starting main dispatch loop4116worker2 # [ 18.147916] systemd[1]: scheduler-token.service: Deactivated successfully.4117worker2 # [ 18.149954] systemd[1]: Finished scheduler-token.service.4118worker2 # [ 18.151248] systemd[1]: scheduler-token.service: Consumed 27ms CPU time over 107ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4119worker2 # [ 18.158077] systemd[1]: Starting Nix worker-protocol over gRPC...4120worker1 # [ 18.235118] nix-grpc-daemon[853]: ts=2026-09-20T12:19:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4121worker1 # [ 18.245139] nix-grpc-daemon[853]: ts=2026-09-20T12:19:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4122worker1 # [ 18.248925] nix-grpc-daemon[853]: ts=2026-09-20T12:19:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14123worker1 # [ 18.287872] nix-grpc-daemon[853]: ts=2026-09-20T12:19:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4124worker1 # [ 18.292359] systemd[1]: Started Nix worker-protocol over gRPC.4125worker1 # [ 18.342515] nix-grpc-daemon[853]: ts=2026-09-20T12:19:49Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4126lb # [ 18.327442] envoy[919]: [2026-09-20 12:19:49.221][919][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4127lb # [ 18.331193] envoy[919]: [2026-09-20 12:19:49.224][919][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4128lb # [ 18.334786] envoy[919]: [2026-09-20 12:19:49.224][919][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4129lb # [ 18.338174] envoy[919]: [2026-09-20 12:19:49.224][919][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4130worker2 # [ 18.356411] nix-grpc-daemon[856]: ts=2026-09-20T12:19:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4131worker2 # [ 18.366945] nix-grpc-daemon[856]: ts=2026-09-20T12:19:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4132worker2 # [ 18.374776] nix-grpc-daemon[856]: ts=2026-09-20T12:19:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14133worker2 # [ 18.418406] nix-grpc-daemon[856]: ts=2026-09-20T12:19:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4134worker2 # [ 18.421529] systemd[1]: Started Nix worker-protocol over gRPC.4135worker2 # [ 18.466963] nix-grpc-daemon[856]: ts=2026-09-20T12:19:48Z level=info event=scheduler_connected addr=lb:500514136worker2 # [ 18.475101] nix-grpc-daemon[856]: ts=2026-09-20T12:19:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker1 # [ 18.543434] nix-grpc-daemon[853]: ts=2026-09-20T12:19:49Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4138worker2 # [ 18.677149] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 18.678852] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker2 # [ 18.879657] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_connected addr=lb:500514141worker2 # [ 18.881565] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker1 # [ 18.944098] nix-grpc-daemon[853]: ts=2026-09-20T12:19:49Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"4143worker2 # [ 19.082358] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_connected addr=lb:500514144worker2 # [ 19.084122] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 19.285411] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 19.289308] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147worker2 # [ 19.491292] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_connected addr=lb:500514148worker2 # [ 19.495143] nix-grpc-daemon[856]: ts=2026-09-20T12:19:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 19.696170] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_connected addr=lb:500514150worker2 # [ 19.700226] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4151worker1 # [ 19.748497] nix-grpc-daemon[853]: ts=2026-09-20T12:19:50Z level=info event=scheduler_connected addr=lb:500514152worker1 # [ 19.753229] nix-grpc-daemon[853]: ts=2026-09-20T12:19:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker2 # [ 19.900858] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_connected addr=lb:500514154worker2 # [ 19.902569] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4155worker1 # [ 19.954310] nix-grpc-daemon[853]: ts=2026-09-20T12:19:50Z level=info event=scheduler_connected addr=lb:500514156worker1 # [ 19.958134] nix-grpc-daemon[853]: ts=2026-09-20T12:19:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker2 # [ 20.103960] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_connected addr=lb:500514158worker2 # [ 20.107837] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker1 # [ 20.159120] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514160worker1 # [ 20.161927] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker2 # [ 20.308568] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 20.310320] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker1 # [ 20.362639] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 20.365178] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker2 # [ 20.511436] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 20.513531] nix-grpc-daemon[856]: ts=2026-09-20T12:19:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4167worker1 # [ 20.565960] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514168worker1 # [ 20.568387] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker2 # [ 20.714376] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 20.716259] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4171worker1 # [ 20.769681] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514172worker1 # [ 20.771900] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker2 # [ 20.916995] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 20.918800] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker1 # [ 20.972825] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514176worker1 # [ 20.975245] nix-grpc-daemon[853]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 21.119767] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514178worker2 # [ 21.122194] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker1 # [ 21.175979] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514180worker1 # [ 21.178471] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker2 # [ 21.322944] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 21.326965] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker1 # [ 21.379263] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 21.381623] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker2 # [ 21.528338] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 21.530320] nix-grpc-daemon[856]: ts=2026-09-20T12:19:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker1 # [ 21.582333] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 21.584664] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker2 # [ 21.731388] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514190worker2 # [ 21.733316] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker1 # [ 21.786930] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 21.790202] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 21.934087] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 21.935874] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker1 # [ 21.987712] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514196worker1 # [ 21.990363] nix-grpc-daemon[853]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker2 # [ 22.136878] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514198worker2 # [ 22.138908] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4199worker1 # [ 22.191161] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 22.193125] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker2 # [ 22.340292] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 22.344132] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4203worker1 # [ 22.393786] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 22.396093] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker2 # [ 22.544894] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 22.546704] nix-grpc-daemon[856]: ts=2026-09-20T12:19:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker1 # [ 22.596874] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514208worker1 # [ 22.599301] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker2 # [ 22.747903] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 22.749980] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker1 # [ 22.800564] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 22.802526] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 22.950867] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 22.952980] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker1 # [ 23.003495] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 23.006035] nix-grpc-daemon[853]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 23.153855] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514218worker2 # [ 23.155766] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker1 # [ 23.206393] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 23.208316] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 23.356747] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 23.358765] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker1 # [ 23.408889] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514224worker1 # [ 23.410736] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 23.560678] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_connected addr=lb:500514226worker2 # [ 23.562229] nix-grpc-daemon[856]: ts=2026-09-20T12:19:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker1 # [ 23.611411] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 23.613895] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker2 # [ 23.761764] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 23.764129] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker1 # [ 23.815618] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 23.817068] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker2 # [ 23.965448] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 23.967876] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker1 # [ 24.016418] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 24.018475] nix-grpc-daemon[853]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker2 # [ 24.168719] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 24.170800] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker1 # [ 24.219577] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514240worker1 # [ 24.222041] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker2 # [ 24.371729] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 24.373807] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker1 # [ 24.422764] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514244worker1 # [ 24.425247] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker2 # [ 24.575709] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_connected addr=lb:500514246worker2 # [ 24.577467] nix-grpc-daemon[856]: ts=2026-09-20T12:19:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker1 # [ 24.626208] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 24.628660] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker2 # [ 24.776879] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514250worker2 # [ 24.780513] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker1 # [ 24.830337] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514252worker1 # [ 24.832641] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker2 # [ 24.980517] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 24.982783] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker1 # [ 25.031773] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514256worker1 # [ 25.034074] nix-grpc-daemon[853]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker2 # [ 25.184191] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514258worker2 # [ 25.186895] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker1 # [ 25.234597] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514260worker1 # [ 25.236617] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker2 # [ 25.387831] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514262worker2 # [ 25.390248] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker1 # [ 25.437638] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514264worker1 # [ 25.442148] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker2 # [ 25.591445] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_connected addr=lb:500514266worker2 # [ 25.593377] nix-grpc-daemon[856]: ts=2026-09-20T12:19:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker1 # [ 25.642574] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514268worker1 # [ 25.645175] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker2 # [ 25.794952] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514270worker2 # [ 25.799320] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker1 # [ 25.846848] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514272worker1 # [ 25.848317] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker2 # [ 26.000422] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514274worker2 # [ 26.004340] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker1 # [ 26.047727] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514276worker1 # [ 26.049906] nix-grpc-daemon[853]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker2 # [ 26.204800] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514278worker2 # [ 26.206535] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker1 # [ 26.250747] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514280worker1 # [ 26.253216] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker2 # [ 26.408271] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514282worker2 # [ 26.412273] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker1 # [ 26.453607] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 26.457191] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker2 # [ 26.613639] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_connected addr=lb:500514286worker2 # [ 26.615667] nix-grpc-daemon[856]: ts=2026-09-20T12:19:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker1 # [ 26.657621] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514288worker1 # [ 26.659527] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker2 # [ 26.817649] nix-grpc-daemon[856]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514290worker2 # [ 26.821907] nix-grpc-daemon[856]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker1 # [ 26.860610] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514292worker1 # [ 26.862496] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker2 # [ 27.023395] nix-grpc-daemon[856]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 27.027489] nix-grpc-daemon[856]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker1 # [ 27.063565] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514296worker1 # [ 27.066818] nix-grpc-daemon[853]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker2 # [ 27.228375] nix-grpc-daemon[856]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514298worker2 # [ 27.230351] nix-grpc-daemon[856]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker1 # [ 27.267485] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514300worker1 # [ 27.269626] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker2 # [ 27.431300] nix-grpc-daemon[856]: ts=2026-09-20T12:19:57Z level=info event=scheduler_connected addr=lb:500514302worker2 # [ 27.434310] nix-grpc-daemon[856]: ts=2026-09-20T12:19:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker1 # [ 27.470207] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514304worker1 # [ 27.472483] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker2 # [ 27.635791] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514306worker2 # [ 27.638359] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker1 # [ 27.673203] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514308worker1 # [ 27.675156] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker2 # [ 27.839502] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514310worker2 # [ 27.842212] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker1 # [ 27.877582] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514312lb # [ 27.865310] rustfs-bucket-start[1030]: mb s3://farm4313worker1 # [ 27.881262] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4314lb # [ 27.872934] systemd[1]: Finished rustfs-bucket.service.4315lb # [ 27.881618] systemd[1]: Starting niks3 server...4316worker2 # [ 28.043407] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514317worker2 # [ 28.045815] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4318worker1 # [ 28.078223] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514319worker1 # [ 28.080627] nix-grpc-daemon[853]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4320lb # [ 28.185991] postgres[1050]: [1050] ERROR: relation "goose_db_version" does not exist at character 364321lb # [ 28.187936] postgres[1050]: [1050] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4322worker2 # [ 28.247169] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514323worker2 # [ 28.249736] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4324lb # [ 28.245693] niks3-server[1039]: 2026/09/20 12:19:59 OK 20241026095416_initial_model.sql (31.73ms)4325lb # [ 28.251948] niks3-server[1039]: 2026/09/20 12:19:59 OK 20251210153512_drop_unused_gin_index.sql (4.31ms)4326lb # [ 28.260313] niks3-server[1039]: 2026/09/20 12:19:59 OK 20251218171726_add_pins.sql (8.48ms)4327worker1 # [ 28.281297] nix-grpc-daemon[853]: ts=2026-09-20T12:19:59Z level=info event=scheduler_connected addr=lb:500514328worker1 # [ 28.283418] nix-grpc-daemon[853]: ts=2026-09-20T12:19:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329lb # [ 28.269488] niks3-server[1039]: 2026/09/20 12:19:59 OK 20260628120000_add_object_size_and_stats.sql (7.43ms)4330lb # [ 28.276906] niks3-server[1039]: 2026/09/20 12:19:59 OK 20260905000000_add_claims.sql (5.29ms)4331lb # [ 28.283341] niks3-server[1039]: 2026/09/20 12:19:59 OK 20260920000000_drop_claims.sql (6.52ms)4332lb # [ 28.285364] niks3-server[1039]: 2026/09/20 12:19:59 goose: successfully migrated database to version: 202609200000004333lb # [ 28.292756] niks3-server[1039]: 2026/09/20 12:19:59 OK 1_commit_pending_closure.sql (7.41ms)4334lb # [ 28.296407] niks3-server[1039]: 2026/09/20 12:19:59 OK 2_object_stats_trigger.sql (1.96ms)4335lb # [ 28.297974] niks3-server[1039]: 2026/09/20 12:19:59 goose: up to current file version: 24336lb # [ 28.304437] niks3-server[1039]: 2026/09/20 12:19:59 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4337lb # [ 28.400429] niks3-server[1039]: 2026/09/20 12:19:59 INFO Read proxy enabled — serving cache objects from S34338lb # [ 28.402803] niks3-server[1039]: 2026/09/20 12:19:59 INFO Using socket-activated listener address=0.0.0.0:57514339lb # [ 28.410123] systemd[1]: Started niks3 server.4340lb # [ 28.410973] niks3-server[1039]: 2026/09/20 12:19:59 INFO systemd watchdog enabled interval=15s4341lb # [ 28.414421] systemd[1]: Reached target Multi-User System.4342lb # [ 28.415774] niks3-server[1039]: 2026/09/20 12:19:59 INFO Starting HTTP server address=0.0.0.0:57514343lb # [ 28.418569] systemd[1]: Startup finished in 871ms (kernel) + 5.131s (initrd) + 22.414s (userspace) = 28.417s.4344lb # [ 28.427950] niks3-server[1039]: 2026/09/20 12:19:59 INFO lead: acquired remote=192.168.1.4:500384345worker2 # [ 28.448390] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_take_over4346worker2 # [ 28.450985] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_connected addr=lb:500514347worker2 # [ 28.453493] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4348worker1 # [ 28.484122] nix-grpc-daemon[853]: ts=2026-09-20T12:19:59Z level=info event=scheduler_connected addr=lb:500514349worker2 # [ 28.493942] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:479484350worker2 # [ 28.498092] nix-grpc-daemon[856]: ts=2026-09-20T12:19:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04351worker2 # [ 28.655269] nix-grpc-daemon[856]: ts=2026-09-20T12:19:59Z level=info event=scheduler_connected addr=lb:500514352worker2 # [ 28.676846] nix-grpc-daemon[856]: ts=2026-09-20T12:19:59Z level=info event=oidc_jwks provider=mock keys=14353worker2 # [ 28.680119] nix-grpc-daemon[856]: ts=2026-09-20T12:19:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:479484354worker2 # [ 28.682448] nix-grpc-daemon[856]: ts=2026-09-20T12:19:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04355lb: (finished: waiting for unit niks3.service, in 17.42 seconds)4356lb: waiting for TCP port 5751 on localhost4357lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4358lb: (finished: waiting for TCP port 5751 on localhost, in 0.09 seconds)4359lb: waiting for unit envoy.service4360lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4361lb: waiting for TCP port 50051 on localhost4362lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4363lb # [ 29.321580] envoy[919]: conn peer=[::1]:58950 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34364lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4365worker2 # [ 29.434843] systemd[1]: Starting scheduler-token.service...4366worker2 # [ 29.511821] systemd[1]: scheduler-token.service: Deactivated successfully.4367worker2 # [ 29.514413] systemd[1]: Finished scheduler-token.service.4368worker2 # [ 29.516760] systemd[1]: scheduler-token.service: Consumed 30ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4369lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4370??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4371 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284372lb: must succeed: curl -sf localhost:9901/clusters4373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4374lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4375subtest: one node schedules and both builders hold a WorkerSession on it4376lb: must succeed: curl -sf localhost:9901/clusters4377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4378worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4379worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4380lb: must succeed: curl -sf localhost:9901/clusters4381lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4382lb: must succeed: curl -sf localhost:9901/clusters4383lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4384worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4385worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)4386(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.25 seconds)4387subtest: balancer auth: client cert, bearer token, nothing, unknown CN4388worker1 # [ 30.244644] nix-grpc-daemon[853]: ts=2026-09-20T12:20:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372164389worker1 # [ 30.255615] systemd[1]: Started Nix Daemon.4390worker1 # [ 30.474172] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4391worker1 # [ 30.521759] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 1/5)4392worker1 # [ 30.536084] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 119 ms (attempt 2/5)4393worker1 # [ 30.658249] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 3/5)4394worker1 # [ 30.718488] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 182 ms (attempt 4/5)4395worker1 # [ 30.909567] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4396worker2 # [ 30.948821] nix-grpc-daemon[856]: ts=2026-09-20T12:20:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479544397worker2 # [ 30.955553] systemd[1]: Started Nix Daemon.4398worker2 # [ 31.208632] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)4399worker2 # [ 31.241925] nix-grpc-daemon[856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)4400worker2 # [ 31.281174] nix-grpc-daemon[856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 196 ms (attempt 2/5)4401worker2 # [ 31.483874] nix-grpc-daemon[856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 3/5)4402worker2 # [ 31.584512] nix-grpc-daemon[856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 4/5)4403worker2 # [ 31.651589] nix-grpc-daemon[856]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4404worker2 # [ 31.672387] nix-grpc-daemon[856]: ts=2026-09-20T12:20:02Z level=debug event=path_info_miss path=6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test error="error: path '/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test' is required, but there is no substituter that can build it"4405worker1 # [ 31.684641] nix-grpc-daemon[853]: ts=2026-09-20T12:20:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:372164406worker1 # [ 31.706189] nix-grpc-daemon[853]: ts=2026-09-20T12:20:02Z level=debug event=path_info_miss path=6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test error="error: path '/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test' is required, but there is no substituter that can build it"4407client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4408client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)4409client: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4410client: (finished: must succeed: install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.06 seconds)4411worker2 # [ 32.123991] nix-grpc-daemon[856]: ts=2026-09-20T12:20:02Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:479544412worker2 # [ 32.144572] nix-grpc-daemon[856]: ts=2026-09-20T12:20:02Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:479544413worker2 # [ 32.155182] nix-grpc-daemon[856]: ts=2026-09-20T12:20:02Z level=debug event=path_info_miss path=6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test error="error: path '/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test' is required, but there is no substituter that can build it"4414worker2 # [ 32.165155] nix-grpc-daemon[856]: ts=2026-09-20T12:20:02Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:479544415worker2 # [ 32.175739] nix-grpc-daemon[856]: ts=2026-09-20T12:20:02Z level=debug event=path_info_miss path=6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test error="error: path '/nix/store/6wlj2ay8s7brjjrdfwgnvhsxskfdjz0s-nixos-system-client-test' is required, but there is no substituter that can build it"4416client: must succeed: rm -r /var/lib/nix-grpc-store4417client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4418worker1 # [ 32.450187] nix-grpc-daemon[853]: ts=2026-09-20T12:20:03Z level=info event=denied method=QueryMissing cn=- role=none4419worker2 # [ 32.662116] nix-grpc-daemon[856]: ts=2026-09-20T12:20:03Z level=info event=denied method=QueryMissing cn=stranger role=none4420(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.83 seconds)4421subtest: DAG build is scheduled across workers and lands in the cache4422client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24423worker2 # [ 32.899121] nix-grpc-daemon[856]: ts=2026-09-20T12:20:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479544424client # don't know how to build these paths:4425client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4426worker1 # [ 32.917276] nix-grpc-daemon[853]: ts=2026-09-20T12:20:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164427worker2 # [ 32.951755] nix-grpc-daemon[856]: ts=2026-09-20T12:20:03Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04428worker2 # [ 32.954345] nix-grpc-daemon[856]: ts=2026-09-20T12:20:03Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14429worker1 # [ 33.010491] nix-grpc-daemon[853]: ts=2026-09-20T12:20:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164430worker1 # [ 33.030681] nix-grpc-daemon[853]: ts=2026-09-20T12:20:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164431client # copying 1 paths to 'lb:50051'...4432worker1 # [ 33.041440] nix-grpc-daemon[853]: ts=2026-09-20T12:20:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372164433client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4434worker1 # [ 33.058480] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4435worker1 # [ 33.076917] nix-grpc-daemon[853]: ts=2026-09-20T12:20:03Z level=info event=niks3_push_started pid=9214436worker1 # [ 33.388738] nix-grpc-daemon[921]: time=2026-09-20T12:20:04.267Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""4437worker1 # [ 33.588488] nix-daemon[911]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4438worker1 # [ 33.608960] nix-daemon[911]: reaped child process 937, status = succeeded4439worker1 # [ 33.749415] nix-daemon[911]: accepted connection from pid 939, user nix-grpc-daemon (trusted)4440worker1 # [ 33.764656] nix-daemon[911]: reaped child process 942, status = succeeded4441lb # [ 33.760500] niks3-server[1039]: 2026/09/20 12:20:04 INFO Received uploads request method=POST path=/api/pending_closures4442worker1 # [ 33.797931] nix-grpc-daemon[921]: time=2026-09-20T12:20:04.679Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4443worker1 # [ 33.800266] nix-grpc-daemon[921]: time=2026-09-20T12:20:04.682Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4444lb # [ 33.823236] niks3-server[1039]: 2026/09/20 12:20:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4445lb # [ 33.844863] niks3-server[1039]: 2026/09/20 12:20:04 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4446lb # [ 33.860214] niks3-server[1039]: 2026/09/20 12:20:04 INFO Signed narinfos id=1 count=14447worker1 # [ 33.878548] nix-grpc-daemon[921]: time=2026-09-20T12:20:04.760Z level=INFO msg="Uploading 1 narinfos"4448lb # [ 33.876224] niks3-server[1039]: 2026/09/20 12:20:04 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4449lb # [ 33.884097] niks3-server[1039]: 2026/09/20 12:20:04 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4450lb # [ 33.900495] niks3-server[1039]: 2026/09/20 12:20:04 INFO Completed upload id=14451worker1 # [ 33.916597] nix-grpc-daemon[921]: time=2026-09-20T12:20:04.798Z level=INFO msg="Upload complete. (501ms)"4452worker1 # [ 33.922645] nix-grpc-daemon[853]: ts=2026-09-20T12:20:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=5044453lb # [ 33.909835] niks3-server[1039]: 2026/09/20 12:20:04 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4454worker1 # [ 33.929348] nix-grpc-daemon[853]: ts=2026-09-20T12:20:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164455worker1 # [ 33.936300] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4456worker1 # [ 33.939141] nix-daemon[911]: reaped child process 919, status = succeeded4457worker1 # [ 33.945878] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4458client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4459client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4460client # farm-a> LOG-farm-a-t14461worker1 # [ 34.966157] nix-daemon[911]: reaped child process 946, status = succeeded4462worker1 # [ 35.117238] nix-daemon[911]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4463worker1 # [ 35.133409] nix-daemon[911]: reaped child process 964, status = succeeded4464lb # [ 35.126869] niks3-server[1039]: 2026/09/20 12:20:06 INFO Received uploads request method=POST path=/api/pending_closures4465worker1 # [ 35.153047] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.034Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4466worker1 # [ 35.155058] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.036Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4467lb # [ 35.175783] niks3-server[1039]: 2026/09/20 12:20:06 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4468lb # [ 35.183625] niks3-server[1039]: 2026/09/20 12:20:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4469lb # [ 35.189103] niks3-server[1039]: 2026/09/20 12:20:06 INFO Signed narinfos id=2 count=14470worker1 # [ 35.205533] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.086Z level=INFO msg="Uploading 1 narinfos"4471lb # [ 35.198609] niks3-server[1039]: 2026/09/20 12:20:06 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4472lb # [ 35.203164] niks3-server[1039]: 2026/09/20 12:20:06 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4473lb # [ 35.214675] niks3-server[1039]: 2026/09/20 12:20:06 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4474lb # [ 35.222491] niks3-server[1039]: 2026/09/20 12:20:06 INFO Completed upload id=24475worker1 # [ 35.238877] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.120Z level=INFO msg="Upload complete. (281ms)"4476lb # [ 35.225812] niks3-server[1039]: 2026/09/20 12:20:06 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4477worker1 # [ 35.245569] nix-grpc-daemon[853]: ts=2026-09-20T12:20:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14478worker1 # [ 35.257511] nix-daemon[911]: reaped child process 944, status = succeeded4479worker2 # [ 35.263898] nix-grpc-daemon[856]: ts=2026-09-20T12:20:05Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04480worker2 # [ 35.268522] nix-grpc-daemon[856]: ts=2026-09-20T12:20:05Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14481worker1 # [ 35.312070] nix-grpc-daemon[853]: ts=2026-09-20T12:20:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164482worker1 # [ 35.330339] nix-grpc-daemon[853]: ts=2026-09-20T12:20:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164483worker1 # [ 35.345819] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.227Z level=INFO msg="All 1 paths already cached"4484client # copying 1 paths to 'lb:50051'...4485client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4486worker1 # [ 35.354892] nix-grpc-daemon[853]: ts=2026-09-20T12:20:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372164487worker1 # [ 35.372197] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4488worker1 # [ 35.537295] nix-daemon[911]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4489worker1 # [ 35.554093] nix-daemon[911]: reaped child process 971, status = succeeded4490worker1 # [ 35.708502] nix-daemon[911]: accepted connection from pid 973, user nix-grpc-daemon (trusted)4491worker1 # [ 35.728057] nix-daemon[911]: reaped child process 976, status = succeeded4492lb # [ 35.721828] niks3-server[1039]: 2026/09/20 12:20:06 INFO Received uploads request method=POST path=/api/pending_closures4493worker1 # [ 35.746344] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.628Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4494worker1 # [ 35.749077] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.630Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4495lb # [ 35.749195] niks3-server[1039]: 2026/09/20 12:20:06 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4496lb # [ 35.757574] niks3-server[1039]: 2026/09/20 12:20:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4497lb # [ 35.761314] niks3-server[1039]: 2026/09/20 12:20:06 INFO Signed narinfos id=3 count=14498worker1 # [ 35.777467] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.659Z level=INFO msg="Uploading 1 narinfos"4499lb # [ 35.767887] niks3-server[1039]: 2026/09/20 12:20:06 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4500lb # [ 35.777036] niks3-server[1039]: 2026/09/20 12:20:06 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4501lb # [ 35.784303] niks3-server[1039]: 2026/09/20 12:20:06 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4502lb # [ 35.788971] niks3-server[1039]: 2026/09/20 12:20:06 INFO Completed upload id=34503worker1 # [ 35.804736] nix-grpc-daemon[921]: time=2026-09-20T12:20:06.686Z level=INFO msg="Upload complete. (416ms)"4504worker1 # [ 35.808113] nix-grpc-daemon[853]: ts=2026-09-20T12:20:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=6244505worker1 # [ 35.810817] nix-grpc-daemon[853]: ts=2026-09-20T12:20:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164506worker1 # [ 35.819856] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4507worker1 # [ 35.826846] nix-daemon[911]: reaped child process 966, status = succeeded4508client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4509client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4510worker1 # [ 35.834042] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4511client # farm-b> LOG-farm-b-t14512worker1 # [ 37.061370] nix-daemon[911]: reaped child process 980, status = succeeded4513worker1 # [ 37.243146] nix-daemon[911]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4514worker1 # [ 37.263631] nix-daemon[911]: reaped child process 999, status = succeeded4515lb # [ 37.261051] niks3-server[1039]: 2026/09/20 12:20:08 INFO Received uploads request method=POST path=/api/pending_closures4516worker1 # [ 37.282577] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.164Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4517worker1 # [ 37.285087] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.166Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4518lb # [ 37.284575] niks3-server[1039]: 2026/09/20 12:20:08 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4519lb # [ 37.291361] niks3-server[1039]: 2026/09/20 12:20:08 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4520lb # [ 37.294070] niks3-server[1039]: 2026/09/20 12:20:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4521lb # [ 37.296558] niks3-server[1039]: 2026/09/20 12:20:08 INFO Signed narinfos id=4 count=14522worker1 # [ 37.314482] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.196Z level=INFO msg="Uploading 1 narinfos"4523lb # [ 37.308258] niks3-server[1039]: 2026/09/20 12:20:08 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4524lb # [ 37.354905] niks3-server[1039]: 2026/09/20 12:20:08 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4525lb # [ 37.389461] niks3-server[1039]: 2026/09/20 12:20:08 INFO Completed upload id=44526worker1 # [ 37.405544] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.287Z level=INFO msg="Upload complete. (357ms)"4527lb # [ 37.395131] niks3-server[1039]: 2026/09/20 12:20:08 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4528worker1 # [ 37.409485] nix-grpc-daemon[853]: ts=2026-09-20T12:20:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14529worker1 # [ 37.419824] nix-daemon[911]: reaped child process 978, status = succeeded4530worker2 # [ 37.417193] nix-grpc-daemon[856]: ts=2026-09-20T12:20:07Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04531worker2 # [ 37.419761] nix-grpc-daemon[856]: ts=2026-09-20T12:20:07Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14532worker1 # [ 37.466392] nix-grpc-daemon[853]: ts=2026-09-20T12:20:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164533worker1 # [ 37.477624] nix-grpc-daemon[853]: ts=2026-09-20T12:20:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164534worker1 # [ 37.490086] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.371Z level=INFO msg="All 2 paths already cached"4535client # copying 1 paths to 'lb:50051'...4536worker1 # [ 37.498301] nix-grpc-daemon[853]: ts=2026-09-20T12:20:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372164537client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4538worker1 # [ 37.518139] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4539worker1 # [ 37.706913] nix-daemon[911]: accepted connection from pid 1003, user nix-grpc-daemon (trusted)4540worker1 # [ 37.727054] nix-daemon[911]: reaped child process 1006, status = succeeded4541worker1 # [ 37.867401] nix-daemon[911]: accepted connection from pid 1008, user nix-grpc-daemon (trusted)4542worker1 # [ 37.883634] nix-daemon[911]: reaped child process 1011, status = succeeded4543lb # [ 37.880274] niks3-server[1039]: 2026/09/20 12:20:08 INFO Received uploads request method=POST path=/api/pending_closures4544worker1 # [ 37.914879] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.796Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4545worker1 # [ 37.916472] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.796Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4546lb # [ 37.914712] niks3-server[1039]: 2026/09/20 12:20:08 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4547lb # [ 37.922281] niks3-server[1039]: 2026/09/20 12:20:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4548lb # [ 37.926593] niks3-server[1039]: 2026/09/20 12:20:08 INFO Signed narinfos id=5 count=14549worker1 # [ 37.942603] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.824Z level=INFO msg="Uploading 1 narinfos"4550lb # [ 37.929314] niks3-server[1039]: 2026/09/20 12:20:08 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4551lb # [ 37.938110] niks3-server[1039]: 2026/09/20 12:20:08 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4552lb # [ 37.948168] niks3-server[1039]: 2026/09/20 12:20:08 INFO Completed upload id=54553worker1 # [ 37.963973] nix-grpc-daemon[921]: time=2026-09-20T12:20:08.845Z level=INFO msg="Upload complete. (437ms)"4554lb # [ 37.950252] niks3-server[1039]: 2026/09/20 12:20:08 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4555worker1 # [ 37.968187] nix-grpc-daemon[853]: ts=2026-09-20T12:20:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=7524556worker1 # [ 37.972672] nix-grpc-daemon[853]: ts=2026-09-20T12:20:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164557worker1 # [ 37.980614] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4558worker1 # [ 37.988976] nix-daemon[911]: reaped child process 1001, status = succeeded4559client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4560client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4561worker1 # [ 37.997581] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4562lb # [ 38.033911] envoy[919]: conn peer=[2001:db8:1::1]:49784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=15454563lb # [ 38.035820] envoy[919]: conn peer=[2001:db8:1::1]:49790 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=654564lb # [ 38.037504] envoy[919]: conn peer=[2001:db8:1::1]:49794 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=144565lb # [ 38.039191] envoy[919]: conn peer=[2001:db8:1::1]:49802 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=114566client # farm-top> LOG-farm-top-t14567worker1 # [ 39.221664] nix-daemon[911]: reaped child process 1015, status = succeeded4568worker1 # [ 39.362675] nix-daemon[911]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4569worker1 # [ 39.378467] nix-daemon[911]: reaped child process 1034, status = succeeded4570lb # [ 39.370767] niks3-server[1039]: 2026/09/20 12:20:10 INFO Received uploads request method=POST path=/api/pending_closures4571worker1 # [ 39.393347] nix-grpc-daemon[921]: time=2026-09-20T12:20:10.274Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4572worker1 # [ 39.394721] nix-grpc-daemon[921]: time=2026-09-20T12:20:10.274Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4573lb # [ 39.387651] niks3-server[1039]: 2026/09/20 12:20:10 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4574lb # [ 39.396390] niks3-server[1039]: 2026/09/20 12:20:10 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4575lb # [ 39.401634] niks3-server[1039]: 2026/09/20 12:20:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4576lb # [ 39.404381] niks3-server[1039]: 2026/09/20 12:20:10 INFO Signed narinfos id=6 count=14577worker1 # [ 39.420237] nix-grpc-daemon[921]: time=2026-09-20T12:20:10.302Z level=INFO msg="Uploading 1 narinfos"4578lb # [ 39.408324] niks3-server[1039]: 2026/09/20 12:20:10 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4579lb # [ 39.412062] niks3-server[1039]: 2026/09/20 12:20:10 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4580lb # [ 39.417051] niks3-server[1039]: 2026/09/20 12:20:10 INFO Completed upload id=64581worker1 # [ 39.432559] nix-grpc-daemon[921]: time=2026-09-20T12:20:10.314Z level=INFO msg="Upload complete. (223ms)"4582lb # [ 39.418955] niks3-server[1039]: 2026/09/20 12:20:10 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4583worker1 # [ 39.436385] nix-grpc-daemon[853]: ts=2026-09-20T12:20:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14584worker1 # [ 39.442538] nix-daemon[911]: reaped child process 1013, status = succeeded4585client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.77 seconds)4586client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4587client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4588client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4589client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4590client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4591client # copying 3 paths...4592client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4593client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4594client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4595client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4596(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.32 seconds)4597subtest: repeat is answered Cached by the scheduler without building4598worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4599worker1 # finding garbage collector roots...4600worker1 # deleting specified paths...4601worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4602worker1 # deleting unused links...4603worker1 # note: hard linking is currently saving 0.0 KiB4604worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4605worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4606worker2 # finding garbage collector roots...4607worker2 # deleting specified paths...4608worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4609worker2 # deleting unused links...4610worker2 # note: hard linking is currently saving 0.0 KiB4611worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.30 seconds)4612client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24613worker1 # [ 40.762601] nix-grpc-daemon[853]: ts=2026-09-20T12:20:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372164614client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4615client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4616worker2 # [ 40.773910] nix-grpc-daemon[856]: ts=2026-09-20T12:20:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479544617worker2 # [ 40.805847] nix-grpc-daemon[856]: ts=2026-09-20T12:20:11Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14618client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.26 seconds)4619client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4620client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4621lb: must succeed: curl -sf localhost:9901/clusters4622lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4623lb: must succeed: curl -sf localhost:9901/clusters4624lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4625worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4626worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4627(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.24 seconds)4628subtest: inputs already in the cache are not sent to the scheduler4629lb: must succeed: curl -sf localhost:9901/clusters4630lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4631lb: must succeed: curl -sf localhost:9901/clusters4632lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4633worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4634worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4635client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24636client # don't know how to build these paths:4637client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4638worker2 # [ 41.600406] nix-grpc-daemon[856]: ts=2026-09-20T12:20:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479544639worker1 # [ 41.607642] nix-grpc-daemon[853]: ts=2026-09-20T12:20:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164640worker1 # [ 41.615684] nix-grpc-daemon[921]: time=2026-09-20T12:20:12.497Z level=INFO msg="All 2 paths already cached"4641worker2 # [ 41.622791] nix-grpc-daemon[856]: ts=2026-09-20T12:20:11Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04642worker2 # [ 41.627422] nix-grpc-daemon[856]: ts=2026-09-20T12:20:11Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14643worker1 # [ 41.670188] nix-grpc-daemon[853]: ts=2026-09-20T12:20:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164644worker1 # [ 41.677529] nix-grpc-daemon[853]: ts=2026-09-20T12:20:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164645worker1 # [ 41.685794] nix-grpc-daemon[921]: time=2026-09-20T12:20:12.567Z level=INFO msg="All 2 paths already cached"4646client # copying 1 paths to 'lb:50051'...4647client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4648worker1 # [ 41.690943] nix-grpc-daemon[853]: ts=2026-09-20T12:20:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372164649worker1 # [ 41.698185] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4650worker1 # [ 41.867665] nix-daemon[911]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4651worker1 # [ 41.884828] nix-daemon[911]: reaped child process 1054, status = succeeded4652worker1 # [ 42.033500] nix-daemon[911]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)4653worker1 # [ 42.046574] nix-daemon[911]: reaped child process 1059, status = succeeded4654lb # [ 42.043610] niks3-server[1039]: 2026/09/20 12:20:12 INFO Received uploads request method=POST path=/api/pending_closures4655worker1 # [ 42.065349] nix-grpc-daemon[921]: time=2026-09-20T12:20:12.947Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4656worker1 # [ 42.066983] nix-grpc-daemon[921]: time=2026-09-20T12:20:12.948Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4657lb # [ 42.064962] niks3-server[1039]: 2026/09/20 12:20:12 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4658lb # [ 42.071055] niks3-server[1039]: 2026/09/20 12:20:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4659lb # [ 42.074559] niks3-server[1039]: 2026/09/20 12:20:12 INFO Signed narinfos id=7 count=14660worker1 # [ 42.090310] nix-grpc-daemon[921]: time=2026-09-20T12:20:12.972Z level=INFO msg="Uploading 1 narinfos"4661lb # [ 42.078736] niks3-server[1039]: 2026/09/20 12:20:12 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4662lb # [ 42.086677] niks3-server[1039]: 2026/09/20 12:20:12 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4663lb # [ 42.090655] niks3-server[1039]: 2026/09/20 12:20:12 INFO Completed upload id=74664worker1 # [ 42.106461] nix-grpc-daemon[921]: time=2026-09-20T12:20:12.988Z level=INFO msg="Upload complete. (401ms)"4665worker1 # [ 42.109325] nix-grpc-daemon[853]: ts=2026-09-20T12:20:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=7524666lb # [ 42.094465] niks3-server[1039]: 2026/09/20 12:20:12 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4667worker1 # [ 42.112805] nix-grpc-daemon[853]: ts=2026-09-20T12:20:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164668worker1 # [ 42.118585] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4669worker1 # [ 42.123894] nix-daemon[911]: reaped child process 1049, status = succeeded4670client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4671client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4672worker1 # [ 42.130918] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4673client # farm-top> LOG-farm-top-t1b4674worker1 # [ 43.146746] nix-daemon[911]: reaped child process 1063, status = succeeded4675worker1 # [ 43.278532] nix-daemon[911]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4676worker1 # [ 43.291740] nix-daemon[911]: reaped child process 1082, status = succeeded4677lb # [ 43.290843] niks3-server[1039]: 2026/09/20 12:20:14 INFO Received uploads request method=POST path=/api/pending_closures4678worker1 # [ 43.311406] nix-grpc-daemon[921]: time=2026-09-20T12:20:14.193Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4679worker1 # [ 43.314370] nix-grpc-daemon[921]: time=2026-09-20T12:20:14.194Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4680lb # [ 43.318494] niks3-server[1039]: 2026/09/20 12:20:14 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4681lb # [ 43.321452] niks3-server[1039]: 2026/09/20 12:20:14 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4682lb # [ 43.325310] niks3-server[1039]: 2026/09/20 12:20:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4683lb # [ 43.328775] niks3-server[1039]: 2026/09/20 12:20:14 INFO Signed narinfos id=8 count=14684worker1 # [ 43.344796] nix-grpc-daemon[921]: time=2026-09-20T12:20:14.226Z level=INFO msg="Uploading 1 narinfos"4685lb # [ 43.332701] niks3-server[1039]: 2026/09/20 12:20:14 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4686lb # [ 43.338453] niks3-server[1039]: 2026/09/20 12:20:14 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4687lb # [ 43.341848] niks3-server[1039]: 2026/09/20 12:20:14 INFO Completed upload id=84688worker1 # [ 43.357545] nix-grpc-daemon[921]: time=2026-09-20T12:20:14.239Z level=INFO msg="Upload complete. (220ms)"4689lb # [ 43.343718] niks3-server[1039]: 2026/09/20 12:20:14 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4690worker1 # [ 43.361321] nix-grpc-daemon[853]: ts=2026-09-20T12:20:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14691worker1 # [ 43.367786] nix-daemon[911]: reaped child process 1061, status = succeeded4692client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.98 seconds)4693client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4694client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds)4695lb: must succeed: curl -sf localhost:9901/clusters4696lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4697lb: must succeed: curl -sf localhost:9901/clusters4698lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4699worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4700worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4701(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.41 seconds)4702subtest: two clients wanting the same drv build it once4703client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4704client # [ 43.718479] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4705client # Running as unit: dedup1.service; invocation ID: f1843f541da14702b21b519c2db621094706client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.05 seconds)4707client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4708worker2 # [ 43.911125] nix-grpc-daemon[856]: ts=2026-09-20T12:20:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479544709client # [ 43.930308] nix[887]: don't know how to build these paths:4710client # [ 43.931220] nix[887]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4711worker2 # [ 43.916604] nix-grpc-daemon[856]: ts=2026-09-20T12:20:14Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04712worker2 # [ 43.919235] nix-grpc-daemon[856]: ts=2026-09-20T12:20:14Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14713worker2 # [ 43.921082] nix-grpc-daemon[856]: ts=2026-09-20T12:20:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479544714worker2 # [ 43.928828] nix-grpc-daemon[856]: ts=2026-09-20T12:20:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479544715client # [ 43.951907] nix[887]: copying 1 paths to 'lb:50051'...4716worker2 # [ 43.934032] nix-grpc-daemon[856]: ts=2026-09-20T12:20:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479544717client # [ 43.952726] nix[887]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4718worker2 # [ 43.940635] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)4719worker2 # [ 43.952960] nix-grpc-daemon[856]: ts=2026-09-20T12:20:14Z level=info event=niks3_push_started pid=9764720worker2 # [ 44.198734] nix-grpc-daemon[976]: time=2026-09-20T12:20:14.563Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""4721worker2 # [ 44.403120] nix-daemon[925]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4722worker2 # [ 44.421272] nix-daemon[925]: reaped child process 991, status = succeeded4723worker2 # [ 44.568179] nix-daemon[925]: accepted connection from pid 994, user nix-grpc-daemon (trusted)4724worker2 # [ 44.581472] nix-daemon[925]: reaped child process 997, status = succeeded4725lb # [ 44.579646] niks3-server[1039]: 2026/09/20 12:20:15 INFO Received uploads request method=POST path=/api/pending_closures4726worker2 # [ 44.598494] nix-grpc-daemon[976]: time=2026-09-20T12:20:14.967Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4727worker2 # [ 44.600592] nix-grpc-daemon[976]: time=2026-09-20T12:20:14.969Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4728lb # [ 44.609917] niks3-server[1039]: 2026/09/20 12:20:15 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4729lb # [ 44.613700] niks3-server[1039]: 2026/09/20 12:20:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4730lb # [ 44.616339] niks3-server[1039]: 2026/09/20 12:20:15 INFO Signed narinfos id=9 count=14731worker2 # [ 44.629256] nix-grpc-daemon[976]: time=2026-09-20T12:20:14.998Z level=INFO msg="Uploading 1 narinfos"4732lb # [ 44.625705] niks3-server[1039]: 2026/09/20 12:20:15 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4733lb # [ 44.637831] niks3-server[1039]: 2026/09/20 12:20:15 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4734lb # [ 44.643118] niks3-server[1039]: 2026/09/20 12:20:15 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4735lb # [ 44.645788] niks3-server[1039]: 2026/09/20 12:20:15 INFO Completed upload id=94736worker2 # [ 44.658265] nix-grpc-daemon[976]: time=2026-09-20T12:20:15.027Z level=INFO msg="Upload complete. (435ms)"4737worker2 # [ 44.660714] nix-grpc-daemon[856]: ts=2026-09-20T12:20:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=0 paths=1 nar_bytes_in=4324738worker2 # [ 44.664422] nix-grpc-daemon[856]: ts=2026-09-20T12:20:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479544739worker2 # [ 44.671343] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)4740worker2 # [ 44.673758] nix-daemon[925]: reaped child process 974, status = succeeded4741client # [ 44.696267] nix[887]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4742worker2 # [ 44.679383] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)4743client # [ 44.775776] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4744client # Running as unit: dedup2.service; invocation ID: 835c4b552e9f43c48458786cb19bd70e4745client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.06 seconds)4746worker1 # [ 44.982596] nix-grpc-daemon[853]: ts=2026-09-20T12:20:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372164747client # [ 45.000958] nix[903]: don't know how to build these paths:4748client # [ 45.002227] nix[903]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4749worker2 # [ 44.987725] nix-grpc-daemon[856]: ts=2026-09-20T12:20:15Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04750worker2 # [ 44.993765] nix-grpc-daemon[856]: ts=2026-09-20T12:20:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479544751client # [ 45.014073] nix[903]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4752client: must succeed: sleep 24753client: (finished: must succeed: sleep 2, in 2.03 seconds)4754lb # [ 48.034637] envoy[919]: conn peer=[2001:db8:1::1]:38014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9524 tx=3858 ms=65744755lb # [ 48.038612] envoy[919]: conn peer=[2001:db8:1::1]:38024 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=479 ms=594756lb # [ 48.042186] envoy[919]: conn peer=[2001:db8:1::1]:38036 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4152 tx=1686 ms=17714757??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4758 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394759client: waiting for success: ! systemctl is-active dedup1 dedup24760??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4761 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394762worker2 # [ 48.305650] nix-daemon[925]: reaped child process 1001, status = succeeded4763worker2 # [ 48.458958] nix-daemon[925]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4764worker2 # [ 48.472630] nix-daemon[925]: reaped child process 1035, status = succeeded4765lb # [ 48.474604] niks3-server[1039]: 2026/09/20 12:20:19 INFO Received uploads request method=POST path=/api/pending_closures4766worker2 # [ 48.492052] nix-grpc-daemon[976]: time=2026-09-20T12:20:18.861Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4767worker2 # [ 48.493733] nix-grpc-daemon[976]: time=2026-09-20T12:20:18.862Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4768lb # [ 48.497972] niks3-server[1039]: 2026/09/20 12:20:19 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4769lb # [ 48.503311] niks3-server[1039]: 2026/09/20 12:20:19 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4770lb # [ 48.507510] niks3-server[1039]: 2026/09/20 12:20:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4771lb # [ 48.509168] niks3-server[1039]: 2026/09/20 12:20:19 INFO Signed narinfos id=10 count=14772worker2 # [ 48.522253] nix-grpc-daemon[976]: time=2026-09-20T12:20:18.890Z level=INFO msg="Uploading 1 narinfos"4773lb # [ 48.513660] niks3-server[1039]: 2026/09/20 12:20:19 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4774lb # [ 48.517783] niks3-server[1039]: 2026/09/20 12:20:19 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4775lb # [ 48.520478] niks3-server[1039]: 2026/09/20 12:20:19 INFO Completed upload id=104776worker2 # [ 48.532895] nix-grpc-daemon[976]: time=2026-09-20T12:20:18.902Z level=INFO msg="Upload complete. (240ms)"4777lb # [ 48.522858] niks3-server[1039]: 2026/09/20 12:20:19 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4778worker2 # [ 48.536359] nix-grpc-daemon[856]: ts=2026-09-20T12:20:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14779worker2 # [ 48.540209] nix-grpc-daemon[856]: ts=2026-09-20T12:20:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14780worker2 # [ 48.546563] nix-daemon[925]: reaped child process 999, status = succeeded4781client # [ 48.589636] systemd[1]: dedup2.service: Deactivated successfully.4782client # [ 48.592297] systemd[1]: dedup2.service: Consumed 117ms CPU time over 3.815s wall clock time, 31.1M memory peak, 7.4K incoming IP traffic, 6.4K outgoing IP traffic.4783client # [ 48.635124] systemd[1]: dedup1.service: Deactivated successfully.4784client # [ 48.637225] systemd[1]: dedup1.service: Consumed 137ms CPU time over 4.918s wall clock time, 37.3M memory peak, 8.6K incoming IP traffic, 9.1K outgoing IP traffic.4785client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4786client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4787client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4788worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4789worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4790worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4791worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4792worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4793worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4794worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4795worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4796(finished: subtest: two clients wanting the same drv build it once, in 5.84 seconds)4797subtest: build hook: nix-daemon with builders = grpc://lb4798client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4799client # [ 49.676757] systemd[1]: Started Nix Daemon.4800client # [ 49.809398] nix-daemon[945]: accepted connection from pid 942, user root (trusted)4801worker1 # [ 49.968909] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:372164802worker2 # [ 49.975391] nix-grpc-daemon[856]: ts=2026-09-20T12:20:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479544803worker2 # [ 49.986525] nix-grpc-daemon[856]: ts=2026-09-20T12:20:20Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04804worker2 # [ 49.988425] nix-grpc-daemon[856]: ts=2026-09-20T12:20:20Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14805worker1 # [ 50.034481] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164806worker1 # [ 50.043603] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164807worker1 # [ 50.051670] nix-grpc-daemon[853]: ts=2026-09-20T12:20:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372164808worker1 # [ 50.059215] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4809worker1 # [ 50.219946] nix-daemon[911]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4810worker1 # [ 50.235324] nix-daemon[911]: reaped child process 1110, status = succeeded4811worker1 # [ 50.375898] nix-daemon[911]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)4812worker1 # [ 50.388302] nix-daemon[911]: reaped child process 1115, status = succeeded4813lb # [ 50.386961] niks3-server[1039]: 2026/09/20 12:20:21 INFO Received uploads request method=POST path=/api/pending_closures4814worker1 # [ 50.408274] nix-grpc-daemon[921]: time=2026-09-20T12:20:21.290Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4815worker1 # [ 50.410246] nix-grpc-daemon[921]: time=2026-09-20T12:20:21.291Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4816lb # [ 50.412308] niks3-server[1039]: 2026/09/20 12:20:21 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4817lb # [ 50.417024] niks3-server[1039]: 2026/09/20 12:20:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4818lb # [ 50.419803] niks3-server[1039]: 2026/09/20 12:20:21 INFO Signed narinfos id=11 count=14819worker1 # [ 50.435796] nix-grpc-daemon[921]: time=2026-09-20T12:20:21.317Z level=INFO msg="Uploading 1 narinfos"4820lb # [ 50.424755] niks3-server[1039]: 2026/09/20 12:20:21 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4821lb # [ 50.432087] niks3-server[1039]: 2026/09/20 12:20:21 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4822worker1 # [ 50.455129] nix-grpc-daemon[921]: time=2026-09-20T12:20:21.337Z level=INFO msg="Upload complete. (388ms)"4823lb # [ 50.439796] niks3-server[1039]: 2026/09/20 12:20:21 INFO Completed upload id=114824worker1 # [ 50.457199] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=5124825lb # [ 50.442597] niks3-server[1039]: 2026/09/20 12:20:21 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4826worker1 # [ 50.461419] nix-grpc-daemon[853]: ts=2026-09-20T12:20:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164827worker1 # [ 50.468139] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4828worker1 # [ 50.470731] nix-daemon[911]: reaped child process 1105, status = succeeded4829worker1 # [ 50.476074] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4830worker1 # [ 51.692788] nix-daemon[911]: reaped child process 1119, status = succeeded4831worker1 # [ 51.836309] nix-daemon[911]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)4832worker1 # [ 51.851456] nix-daemon[911]: reaped child process 1137, status = succeeded4833lb # [ 51.848220] niks3-server[1039]: 2026/09/20 12:20:22 INFO Received uploads request method=POST path=/api/pending_closures4834worker1 # [ 51.872652] nix-grpc-daemon[921]: time=2026-09-20T12:20:22.754Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4835worker1 # [ 51.875065] nix-grpc-daemon[921]: time=2026-09-20T12:20:22.756Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4836lb # [ 51.884217] niks3-server[1039]: 2026/09/20 12:20:22 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4837lb # [ 51.900185] niks3-server[1039]: 2026/09/20 12:20:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4838lb # [ 51.906092] niks3-server[1039]: 2026/09/20 12:20:22 INFO Signed narinfos id=12 count=14839worker1 # [ 51.924666] nix-grpc-daemon[921]: time=2026-09-20T12:20:22.806Z level=INFO msg="Uploading 1 narinfos"4840lb # [ 51.920165] niks3-server[1039]: 2026/09/20 12:20:22 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4841lb # [ 51.925171] niks3-server[1039]: 2026/09/20 12:20:22 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4842lb # [ 51.931534] niks3-server[1039]: 2026/09/20 12:20:22 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4843lb # [ 51.934422] niks3-server[1039]: 2026/09/20 12:20:22 INFO Completed upload id=124844worker1 # [ 51.951676] nix-grpc-daemon[921]: time=2026-09-20T12:20:22.833Z level=INFO msg="Upload complete. (268ms)"4845lb # [ 51.936682] niks3-server[1039]: 2026/09/20 12:20:22 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4846worker1 # [ 51.955321] nix-grpc-daemon[853]: ts=2026-09-20T12:20:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14847worker2 # [ 51.955828] nix-grpc-daemon[856]: ts=2026-09-20T12:20:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479544848worker1 # [ 51.961977] nix-daemon[911]: reaped child process 1117, status = succeeded4849worker2 # [ 51.971372] nix-grpc-daemon[856]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4850worker2 # [ 52.001628] nix-grpc-daemon[856]: ts=2026-09-20T12:20:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:479544851worker2 # [ 52.004576] nix-grpc-daemon[856]: ts=2026-09-20T12:20:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=0 paths=1 nar_bytes_out=1284852worker2 # [ 52.248463] nix-grpc-daemon[856]: ts=2026-09-20T12:20:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:479544853worker2 # [ 52.251674] nix-grpc-daemon[856]: ts=2026-09-20T12:20:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479544854worker2 # [ 52.375498] nix-grpc-daemon[976]: time=2026-09-20T12:20:22.744Z level=INFO msg="All 1 paths already cached"4855worker2 # [ 52.386306] nix-grpc-daemon[856]: ts=2026-09-20T12:20:22Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04856worker2 # [ 52.388520] nix-grpc-daemon[856]: ts=2026-09-20T12:20:22Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14857worker1 # [ 52.393642] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164858worker1 # [ 52.401111] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164859worker1 # [ 52.538033] nix-grpc-daemon[921]: time=2026-09-20T12:20:23.419Z level=INFO msg="All 1 paths already cached"4860worker1 # [ 52.543835] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372164861worker1 # [ 52.550115] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4862worker1 # [ 52.700313] nix-daemon[911]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)4863worker1 # [ 52.715184] nix-daemon[911]: reaped child process 1144, status = succeeded4864worker1 # [ 52.852804] nix-daemon[911]: accepted connection from pid 1146, user nix-grpc-daemon (trusted)4865worker1 # [ 52.866193] nix-daemon[911]: reaped child process 1149, status = succeeded4866lb # [ 52.861825] niks3-server[1039]: 2026/09/20 12:20:23 INFO Received uploads request method=POST path=/api/pending_closures4867worker1 # [ 52.882689] nix-grpc-daemon[921]: time=2026-09-20T12:20:23.764Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4868worker1 # [ 52.884793] nix-grpc-daemon[921]: time=2026-09-20T12:20:23.766Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4869lb # [ 52.882868] niks3-server[1039]: 2026/09/20 12:20:23 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4870lb # [ 52.885893] niks3-server[1039]: 2026/09/20 12:20:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4871lb # [ 52.890210] niks3-server[1039]: 2026/09/20 12:20:23 INFO Signed narinfos id=13 count=14872worker1 # [ 52.906247] nix-grpc-daemon[921]: time=2026-09-20T12:20:23.788Z level=INFO msg="Uploading 1 narinfos"4873lb # [ 52.892597] niks3-server[1039]: 2026/09/20 12:20:23 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4874lb # [ 52.899384] niks3-server[1039]: 2026/09/20 12:20:23 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4875lb # [ 52.902236] niks3-server[1039]: 2026/09/20 12:20:23 INFO Completed upload id=134876worker1 # [ 52.918023] nix-grpc-daemon[921]: time=2026-09-20T12:20:23.800Z level=INFO msg="Upload complete. (360ms)"4877lb # [ 52.905182] niks3-server[1039]: 2026/09/20 12:20:23 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4878worker1 # [ 52.920240] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=6244879worker1 # [ 52.923338] nix-grpc-daemon[853]: ts=2026-09-20T12:20:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164880worker1 # [ 52.930057] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4881worker1 # [ 52.934442] nix-daemon[911]: reaped child process 1139, status = succeeded4882worker1 # [ 52.939772] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4883worker1 # [ 54.157687] nix-daemon[911]: reaped child process 1153, status = succeeded4884worker1 # [ 54.272521] nix-daemon[911]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)4885worker1 # [ 54.286809] nix-daemon[911]: reaped child process 1171, status = succeeded4886lb # [ 54.285842] niks3-server[1039]: 2026/09/20 12:20:25 INFO Received uploads request method=POST path=/api/pending_closures4887worker1 # [ 54.320394] nix-grpc-daemon[921]: time=2026-09-20T12:20:25.201Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4888worker1 # [ 54.321809] nix-grpc-daemon[921]: time=2026-09-20T12:20:25.201Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4889lb # [ 54.330270] niks3-server[1039]: 2026/09/20 12:20:25 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4890lb # [ 54.344470] niks3-server[1039]: 2026/09/20 12:20:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4891worker1 # [ 54.366963] nix-grpc-daemon[921]: time=2026-09-20T12:20:25.248Z level=INFO msg="Uploading 1 narinfos"4892lb # [ 54.351424] niks3-server[1039]: 2026/09/20 12:20:25 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4893lb # [ 54.354946] niks3-server[1039]: 2026/09/20 12:20:25 INFO Signed narinfos id=14 count=14894lb # [ 54.358411] niks3-server[1039]: 2026/09/20 12:20:25 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4895lb # [ 54.368051] niks3-server[1039]: 2026/09/20 12:20:25 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4896lb # [ 54.377385] niks3-server[1039]: 2026/09/20 12:20:25 INFO Completed upload id=144897worker1 # [ 54.394925] nix-grpc-daemon[921]: time=2026-09-20T12:20:25.276Z level=INFO msg="Upload complete. (247ms)"4898lb # [ 54.380911] niks3-server[1039]: 2026/09/20 12:20:25 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4899worker1 # [ 54.402793] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14900worker2 # [ 54.403174] nix-grpc-daemon[856]: ts=2026-09-20T12:20:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479544901worker2 # [ 54.410887] nix-grpc-daemon[856]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4902worker1 # [ 54.417170] nix-daemon[911]: reaped child process 1151, status = succeeded4903worker2 # [ 54.432509] nix-grpc-daemon[856]: ts=2026-09-20T12:20:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:479544904worker2 # [ 54.438475] nix-grpc-daemon[856]: ts=2026-09-20T12:20:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=0 paths=1 nar_bytes_out=1764905worker2 # [ 54.646068] nix-grpc-daemon[856]: ts=2026-09-20T12:20:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:479544906worker1 # [ 54.652252] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164907worker1 # [ 54.658031] nix-grpc-daemon[921]: time=2026-09-20T12:20:25.539Z level=INFO msg="All 2 paths already cached"4908worker2 # [ 54.668496] nix-grpc-daemon[856]: ts=2026-09-20T12:20:25Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04909worker2 # [ 54.674221] nix-grpc-daemon[856]: ts=2026-09-20T12:20:25Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14910worker1 # [ 54.721583] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164911worker1 # [ 54.729368] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164912worker1 # [ 54.737645] nix-grpc-daemon[921]: time=2026-09-20T12:20:25.619Z level=INFO msg="All 2 paths already cached"4913worker1 # [ 54.742789] nix-grpc-daemon[853]: ts=2026-09-20T12:20:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372164914worker1 # [ 54.749985] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4915worker1 # [ 54.901642] nix-daemon[911]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)4916worker1 # [ 54.919236] nix-daemon[911]: reaped child process 1178, status = succeeded4917worker1 # [ 55.079691] nix-daemon[911]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)4918worker1 # [ 55.095270] nix-daemon[911]: reaped child process 1183, status = succeeded4919lb # [ 55.088173] niks3-server[1039]: 2026/09/20 12:20:25 INFO Received uploads request method=POST path=/api/pending_closures4920worker1 # [ 55.112273] nix-grpc-daemon[921]: time=2026-09-20T12:20:25.993Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4921worker1 # [ 55.113642] nix-grpc-daemon[921]: time=2026-09-20T12:20:25.994Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4922lb # [ 55.107108] niks3-server[1039]: 2026/09/20 12:20:26 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4923lb # [ 55.108914] niks3-server[1039]: 2026/09/20 12:20:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4924lb # [ 55.111690] niks3-server[1039]: 2026/09/20 12:20:26 INFO Signed narinfos id=15 count=14925worker1 # [ 55.127760] nix-grpc-daemon[921]: time=2026-09-20T12:20:26.009Z level=INFO msg="Uploading 1 narinfos"4926lb # [ 55.115319] niks3-server[1039]: 2026/09/20 12:20:26 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4927lb # [ 55.120241] niks3-server[1039]: 2026/09/20 12:20:26 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4928lb # [ 55.122955] niks3-server[1039]: 2026/09/20 12:20:26 INFO Completed upload id=154929worker1 # [ 55.138771] nix-grpc-daemon[921]: time=2026-09-20T12:20:26.020Z level=INFO msg="Upload complete. (380ms)"4930worker1 # [ 55.140862] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=7524931lb # [ 55.125732] niks3-server[1039]: 2026/09/20 12:20:26 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4932worker1 # [ 55.144079] nix-grpc-daemon[853]: ts=2026-09-20T12:20:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372164933worker1 # [ 55.150728] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4934worker1 # [ 55.159261] nix-daemon[911]: reaped child process 1173, status = succeeded4935worker1 # [ 55.163481] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)4936worker1 # [ 56.579258] nix-daemon[911]: reaped child process 1187, status = succeeded4937worker1 # [ 56.727259] nix-daemon[911]: accepted connection from pid 1202, user nix-grpc-daemon (trusted)4938worker1 # [ 56.740766] nix-daemon[911]: reaped child process 1205, status = succeeded4939lb # [ 56.734798] niks3-server[1039]: 2026/09/20 12:20:27 INFO Received uploads request method=POST path=/api/pending_closures4940worker1 # [ 56.758319] nix-grpc-daemon[921]: time=2026-09-20T12:20:27.638Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4941worker1 # [ 56.759780] nix-grpc-daemon[921]: time=2026-09-20T12:20:27.638Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4942lb # [ 56.768198] niks3-server[1039]: 2026/09/20 12:20:27 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4943lb # [ 56.772543] niks3-server[1039]: 2026/09/20 12:20:27 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4944lb # [ 56.777898] niks3-server[1039]: 2026/09/20 12:20:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4945lb # [ 56.782904] niks3-server[1039]: 2026/09/20 12:20:27 INFO Signed narinfos id=16 count=14946worker1 # [ 56.798764] nix-grpc-daemon[921]: time=2026-09-20T12:20:27.680Z level=INFO msg="Uploading 1 narinfos"4947lb # [ 56.787642] niks3-server[1039]: 2026/09/20 12:20:27 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4948lb # [ 56.797627] niks3-server[1039]: 2026/09/20 12:20:27 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4949lb # [ 56.801132] niks3-server[1039]: 2026/09/20 12:20:27 INFO Completed upload id=164950worker1 # [ 56.816998] nix-grpc-daemon[921]: time=2026-09-20T12:20:27.698Z level=INFO msg="Upload complete. (242ms)"4951worker1 # [ 56.820331] nix-grpc-daemon[853]: ts=2026-09-20T12:20:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14952lb # [ 56.805436] niks3-server[1039]: 2026/09/20 12:20:27 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4953worker1 # [ 56.826644] nix-daemon[911]: reaped child process 1185, status = succeeded4954worker2 # [ 56.824290] nix-grpc-daemon[856]: ts=2026-09-20T12:20:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479544955worker2 # [ 56.834714] nix-grpc-daemon[856]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4956worker1 # [ 56.859691] nix-grpc-daemon[853]: ts=2026-09-20T12:20:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:372164957worker1 # [ 56.864668] nix-grpc-daemon[853]: ts=2026-09-20T12:20:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_out=2324958client # [ 56.933075] nix-daemon[945]: reaped child process 947, status = succeeded4959client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.42 seconds)4960client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4961client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4962client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4963client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)4964client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4965client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4966(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.50 seconds)4967subtest: an input only one worker has still reaches the builder4968client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4969client # this derivation will be built:4970client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4971client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4972client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.39 seconds)4973client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4974client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.16 seconds)4975worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4976worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)4977worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4978worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4979client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24980client # [ 57.937624] nix-daemon[945]: accepted connection from pid 1028, user root (trusted)4981client # this derivation will be built:4982client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4983lb # [ 58.035982] envoy[919]: conn peer=[2001:db8:1::1]:39472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=35744984lb # [ 58.039787] envoy[919]: conn peer=[2001:db8:1::1]:39464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2985 tx=1371 ms=46864985lb # [ 58.045106] envoy[919]: conn peer=[2001:db8:1::1]:39488 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3426 tx=1748 ms=20904986lb # [ 58.048695] envoy[919]: conn peer=[2001:db8:1::1]:39498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=554987lb # [ 58.052262] envoy[919]: conn peer=[2001:db8:1::1]:39508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3867 tx=1991 ms=22424988lb # [ 58.055801] envoy[919]: conn peer=[2001:db8:1::1]:34342 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=574989lb # [ 58.060195] envoy[919]: conn peer=[2001:db8:1::1]:34344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4298 tx=2179 ms=22234990lb # [ 58.063676] envoy[919]: conn peer=[2001:db8:1::1]:34358 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=411 tx=410 ms=124991worker2 # [ 58.090538] nix-grpc-daemon[856]: ts=2026-09-20T12:20:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:479544992client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...4993worker1 # [ 58.099410] nix-grpc-daemon[853]: ts=2026-09-20T12:20:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372164994worker1 # [ 58.276033] nix-daemon[911]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)4995worker1 # [ 58.288898] nix-daemon[911]: reaped child process 1215, status = succeeded4996lb # [ 58.287874] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received uploads request method=POST path=/api/pending_closures4997worker1 # [ 58.333941] nix-grpc-daemon[921]: time=2026-09-20T12:20:29.215Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4998worker1 # [ 58.335455] nix-grpc-daemon[921]: time=2026-09-20T12:20:29.215Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4999lb # [ 58.341340] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5000lb # [ 58.346134] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5001lb # [ 58.352120] niks3-server[1039]: 2026/09/20 12:20:29 INFO Signed narinfos id=17 count=15002worker1 # [ 58.371047] nix-grpc-daemon[921]: time=2026-09-20T12:20:29.252Z level=INFO msg="Uploading 1 narinfos"5003lb # [ 58.363965] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5004lb # [ 58.373842] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5005lb # [ 58.384117] niks3-server[1039]: 2026/09/20 12:20:29 INFO Completed upload id=175006worker1 # [ 58.400145] nix-grpc-daemon[921]: time=2026-09-20T12:20:29.282Z level=INFO msg="Upload complete. (301ms)"5007lb # [ 58.389426] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5008worker2 # [ 58.409792] nix-grpc-daemon[856]: ts=2026-09-20T12:20:28Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05009worker2 # [ 58.412643] nix-grpc-daemon[856]: ts=2026-09-20T12:20:28Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15010worker2 # [ 58.453680] nix-grpc-daemon[856]: ts=2026-09-20T12:20:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479545011worker2 # [ 58.464457] nix-grpc-daemon[856]: ts=2026-09-20T12:20:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479545012client # copying 2 paths to 'lb:50051'...5013client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5014worker2 # [ 58.483494] nix-grpc-daemon[856]: ts=2026-09-20T12:20:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479545015client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5016worker2 # [ 58.487920] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)5017worker2 # [ 58.667291] nix-daemon[925]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)5018worker2 # [ 58.683712] nix-daemon[925]: reaped child process 1065, status = succeeded5019worker2 # [ 58.853189] nix-daemon[925]: accepted connection from pid 1067, user nix-grpc-daemon (trusted)5020worker2 # [ 58.867819] nix-daemon[925]: reaped child process 1070, status = succeeded5021lb # [ 58.869218] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received uploads request method=POST path=/api/pending_closures5022lb # [ 58.874951] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received uploads request method=POST path=/api/pending_closures5023worker2 # [ 58.892715] nix-grpc-daemon[976]: time=2026-09-20T12:20:29.261Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5024worker2 # [ 58.894249] nix-grpc-daemon[976]: time=2026-09-20T12:20:29.261Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5025worker2 # [ 58.895988] nix-grpc-daemon[976]: time=2026-09-20T12:20:29.261Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5026lb # [ 58.896096] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5027lb # [ 58.902591] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5028lb # [ 58.907597] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5029lb # [ 58.910349] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5030lb # [ 58.913040] niks3-server[1039]: 2026/09/20 12:20:29 INFO Signed narinfos id=18 count=15031lb # [ 58.915237] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5032lb # [ 58.917858] niks3-server[1039]: 2026/09/20 12:20:29 INFO Signed narinfos id=19 count=25033worker2 # [ 58.930909] nix-grpc-daemon[976]: time=2026-09-20T12:20:29.299Z level=INFO msg="Uploading 3 narinfos"5034lb # [ 58.921300] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5035lb # [ 58.936181] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5036lb # [ 58.943908] niks3-server[1039]: 2026/09/20 12:20:29 INFO Completed upload id=185037lb # [ 58.945993] niks3-server[1039]: 2026/09/20 12:20:29 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5038lb # [ 58.950926] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5039lb # [ 58.954362] niks3-server[1039]: 2026/09/20 12:20:29 INFO Completed upload id=195040lb # [ 58.956027] niks3-server[1039]: 2026/09/20 12:20:29 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5041worker2 # [ 58.968468] nix-grpc-daemon[976]: time=2026-09-20T12:20:29.337Z level=INFO msg="Upload complete. (471ms)"5042worker2 # [ 58.971588] nix-grpc-daemon[856]: ts=2026-09-20T12:20:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=0 paths=2 nar_bytes_in=9765043worker2 # [ 58.977324] nix-grpc-daemon[856]: ts=2026-09-20T12:20:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479545044worker2 # [ 58.983556] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)5045worker2 # [ 58.990360] nix-daemon[925]: reaped child process 1060, status = succeeded5046worker2 # [ 58.997555] nix-grpc-daemon[856]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5047worker2 # [ 59.006077] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)5048client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5049client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5050lb # [ 59.186888] niks3-server[1039]: 2026/09/20 12:20:30 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5051worker2 # [ 60.223769] nix-daemon[925]: reaped child process 1075, status = succeeded5052worker2 # [ 60.347226] nix-daemon[925]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5053worker2 # [ 60.361195] nix-daemon[925]: reaped child process 1093, status = succeeded5054lb # [ 60.358582] niks3-server[1039]: 2026/09/20 12:20:31 INFO Received uploads request method=POST path=/api/pending_closures5055worker2 # [ 60.381502] nix-grpc-daemon[976]: time=2026-09-20T12:20:30.750Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5056lb # [ 60.395148] niks3-server[1039]: 2026/09/20 12:20:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5057lb # [ 60.410337] niks3-server[1039]: 2026/09/20 12:20:31 INFO Signed narinfos id=20 count=15058worker2 # [ 60.427421] nix-grpc-daemon[976]: time=2026-09-20T12:20:30.796Z level=INFO msg="Uploading 1 narinfos"5059lb # [ 60.415645] niks3-server[1039]: 2026/09/20 12:20:31 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5060lb # [ 60.423117] niks3-server[1039]: 2026/09/20 12:20:31 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5061lb # [ 60.433870] niks3-server[1039]: 2026/09/20 12:20:31 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5062lb # [ 60.442125] niks3-server[1039]: 2026/09/20 12:20:31 INFO Completed upload id=205063worker2 # [ 60.454357] nix-grpc-daemon[976]: time=2026-09-20T12:20:30.823Z level=INFO msg="Upload complete. (240ms)"5064lb # [ 60.445398] niks3-server[1039]: 2026/09/20 12:20:31 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5065worker2 # [ 60.459237] nix-grpc-daemon[856]: ts=2026-09-20T12:20:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15066worker2 # [ 60.466763] nix-daemon[925]: reaped child process 1072, status = succeeded5067client # copying 1 paths...5068worker1 # [ 60.507147] nix-grpc-daemon[853]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:372165069worker1 # [ 60.516539] nix-grpc-daemon[853]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5070client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5071worker1 # [ 60.542472] nix-grpc-daemon[853]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:372165072worker1 # [ 60.544304] nix-grpc-daemon[853]: ts=2026-09-20T12:20:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_out=1205073client # [ 60.649825] nix-daemon[945]: reaped child process 1031, status = succeeded5074client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.85 seconds)5075client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25076client # [ 60.832981] nix-daemon[945]: accepted connection from pid 1050, user root (trusted)5077client # this derivation will be built:5078client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5079worker2 # [ 60.982982] nix-grpc-daemon[856]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:479545080client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5081worker1 # [ 60.991141] nix-grpc-daemon[853]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372165082worker1 # [ 61.012167] nix-grpc-daemon[921]: time=2026-09-20T12:20:31.892Z level=INFO msg="All 1 paths already cached"5083worker2 # [ 61.021969] nix-grpc-daemon[856]: ts=2026-09-20T12:20:31Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05084worker2 # [ 61.027110] nix-grpc-daemon[856]: ts=2026-09-20T12:20:31Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15085worker1 # [ 61.069921] nix-grpc-daemon[853]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372165086worker1 # [ 61.081814] nix-grpc-daemon[853]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372165087client # copying 1 paths to 'lb:50051'...5088client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5089worker1 # [ 61.096944] nix-grpc-daemon[853]: ts=2026-09-20T12:20:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372165090worker1 # [ 61.108614] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5091worker1 # [ 61.125885] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)5092worker1 # [ 61.131596] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)5093worker1 # [ 61.334849] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 3/5)5094worker1 # [ 61.410971] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 606 ms (attempt 4/5)5095worker1 # [ 62.023969] nix-grpc-daemon[853]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5096worker1 # [ 62.027375] nix-grpc-daemon[853]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5097worker1 # [ 62.200484] nix-daemon[911]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5098worker1 # [ 62.218285] nix-daemon[911]: reaped child process 1227, status = succeeded5099worker1 # [ 62.361624] nix-daemon[911]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5100worker1 # [ 62.377329] nix-daemon[911]: reaped child process 1232, status = succeeded5101lb # [ 62.372686] niks3-server[1039]: 2026/09/20 12:20:33 INFO Received uploads request method=POST path=/api/pending_closures5102worker1 # [ 62.393935] nix-grpc-daemon[921]: time=2026-09-20T12:20:33.275Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5103worker1 # [ 62.397307] nix-grpc-daemon[921]: time=2026-09-20T12:20:33.278Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5104lb # [ 62.404816] niks3-server[1039]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5105lb # [ 62.407537] niks3-server[1039]: 2026/09/20 12:20:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5106worker1 # [ 62.429922] nix-grpc-daemon[921]: time=2026-09-20T12:20:33.311Z level=INFO msg="Uploading 1 narinfos"5107lb # [ 62.414629] niks3-server[1039]: 2026/09/20 12:20:33 INFO Signed narinfos id=21 count=15108lb # [ 62.419747] niks3-server[1039]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5109lb # [ 62.425340] niks3-server[1039]: 2026/09/20 12:20:33 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5110lb # [ 62.428798] niks3-server[1039]: 2026/09/20 12:20:33 INFO Completed upload id=215111worker1 # [ 62.445856] nix-grpc-daemon[921]: time=2026-09-20T12:20:33.327Z level=INFO msg="Upload complete. (405ms)"5112lb # [ 62.433268] niks3-server[1039]: 2026/09/20 12:20:33 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5113worker1 # [ 62.448084] nix-grpc-daemon[853]: ts=2026-09-20T12:20:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 paths=1 nar_bytes_in=5605114worker1 # [ 62.451891] nix-grpc-daemon[853]: ts=2026-09-20T12:20:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372165115worker1 # [ 62.458087] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5116worker1 # [ 62.465616] nix-daemon[911]: reaped child process 1218, status = succeeded5117client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5118worker1 # [ 62.469200] nix-daemon[911]: accepted connection from pid 853, user nix-grpc-daemon (trusted)5119client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5120worker1 # [ 63.289861] nix-daemon[911]: reaped child process 1236, status = succeeded5121worker1 # [ 63.456580] nix-daemon[911]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5122worker1 # [ 63.473435] nix-daemon[911]: reaped child process 1254, status = succeeded5123lb # [ 63.469347] niks3-server[1039]: 2026/09/20 12:20:34 INFO Received uploads request method=POST path=/api/pending_closures5124worker1 # [ 63.491103] nix-grpc-daemon[921]: time=2026-09-20T12:20:34.372Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5125lb # [ 63.491864] niks3-server[1039]: 2026/09/20 12:20:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5126lb # [ 63.494976] niks3-server[1039]: 2026/09/20 12:20:34 INFO Signed narinfos id=22 count=15127worker1 # [ 63.510901] nix-grpc-daemon[921]: time=2026-09-20T12:20:34.392Z level=INFO msg="Uploading 1 narinfos"5128lb # [ 63.500285] niks3-server[1039]: 2026/09/20 12:20:34 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5129lb # [ 63.505544] niks3-server[1039]: 2026/09/20 12:20:34 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5130lb # [ 63.512876] niks3-server[1039]: 2026/09/20 12:20:34 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5131lb # [ 63.516347] niks3-server[1039]: 2026/09/20 12:20:34 INFO Completed upload id=225132worker1 # [ 63.534393] nix-grpc-daemon[921]: time=2026-09-20T12:20:34.416Z level=INFO msg="Upload complete. (256ms)"5133lb # [ 63.520554] niks3-server[1039]: 2026/09/20 12:20:34 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5134worker1 # [ 63.538817] nix-grpc-daemon[853]: ts=2026-09-20T12:20:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15135client # copying 1 paths...5136worker2 # [ 63.540920] nix-grpc-daemon[856]: ts=2026-09-20T12:20:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479545137worker1 # [ 63.549501] nix-daemon[911]: reaped child process 1234, status = succeeded5138worker2 # [ 63.549129] nix-grpc-daemon[856]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5139client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5140worker1 # [ 63.574120] nix-grpc-daemon[853]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:372165141worker1 # [ 63.577266] nix-grpc-daemon[853]: ts=2026-09-20T12:20:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_out=1205142client # [ 63.687494] nix-daemon[945]: reaped child process 1053, status = succeeded5143client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.04 seconds)5144client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25145client # [ 63.871122] nix-daemon[945]: accepted connection from pid 1072, user root (trusted)5146client # this derivation will be built:5147client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5148worker1 # [ 64.072652] nix-grpc-daemon[853]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:372165149worker1 # [ 64.076378] nix-grpc-daemon[853]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372165150client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux'...5151worker1 # [ 64.088627] nix-grpc-daemon[921]: time=2026-09-20T12:20:34.970Z level=INFO msg="All 1 paths already cached"5152worker2 # [ 64.096797] nix-grpc-daemon[856]: ts=2026-09-20T12:20:34Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05153worker2 # [ 64.099427] nix-grpc-daemon[856]: ts=2026-09-20T12:20:34Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15154worker2 # [ 64.143866] nix-grpc-daemon[856]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479545155worker2 # [ 64.152213] nix-grpc-daemon[856]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479545156worker2 # [ 64.160983] nix-grpc-daemon[976]: time=2026-09-20T12:20:34.530Z level=INFO msg="All 1 paths already cached"5157client # copying 1 paths to 'lb:50051'...5158client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5159worker2 # [ 64.167853] nix-grpc-daemon[856]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479545160worker2 # [ 64.176623] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)5161worker2 # [ 64.348967] nix-daemon[925]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5162worker2 # [ 64.365103] nix-daemon[925]: reaped child process 1105, status = succeeded5163worker2 # [ 64.512167] nix-daemon[925]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5164worker2 # [ 64.528591] nix-daemon[925]: reaped child process 1110, status = succeeded5165lb # [ 64.528194] niks3-server[1039]: 2026/09/20 12:20:35 INFO Received uploads request method=POST path=/api/pending_closures5166worker2 # [ 64.545630] nix-grpc-daemon[976]: time=2026-09-20T12:20:34.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5167worker2 # [ 64.547442] nix-grpc-daemon[976]: time=2026-09-20T12:20:34.914Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5168lb # [ 64.544253] niks3-server[1039]: 2026/09/20 12:20:35 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5169lb # [ 64.550449] niks3-server[1039]: 2026/09/20 12:20:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5170lb # [ 64.553754] niks3-server[1039]: 2026/09/20 12:20:35 INFO Signed narinfos id=23 count=15171worker2 # [ 64.566132] nix-grpc-daemon[976]: time=2026-09-20T12:20:34.935Z level=INFO msg="Uploading 1 narinfos"5172lb # [ 64.556688] niks3-server[1039]: 2026/09/20 12:20:35 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5173lb # [ 64.562359] niks3-server[1039]: 2026/09/20 12:20:35 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5174lb # [ 64.568133] niks3-server[1039]: 2026/09/20 12:20:35 INFO Completed upload id=235175worker2 # [ 64.580621] nix-grpc-daemon[976]: time=2026-09-20T12:20:34.949Z level=INFO msg="Upload complete. (395ms)"5176lb # [ 64.570826] niks3-server[1039]: 2026/09/20 12:20:35 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5177worker2 # [ 64.584478] nix-grpc-daemon[856]: ts=2026-09-20T12:20:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=0 paths=1 nar_bytes_in=5605178worker2 # [ 64.588694] nix-grpc-daemon[856]: ts=2026-09-20T12:20:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479545179worker2 # [ 64.596546] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)5180worker2 # [ 64.599392] nix-daemon[925]: reaped child process 1100, status = succeeded5181worker2 # [ 64.606374] nix-daemon[925]: accepted connection from pid 856, user nix-grpc-daemon (trusted)5182client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5183client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5184worker2 # [ 65.624652] nix-daemon[925]: reaped child process 1114, status = succeeded5185worker2 # [ 65.746576] nix-daemon[925]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)5186worker2 # [ 65.759459] nix-daemon[925]: reaped child process 1132, status = succeeded5187lb # [ 65.757314] niks3-server[1039]: 2026/09/20 12:20:36 INFO Received uploads request method=POST path=/api/pending_closures5188worker2 # [ 65.775895] nix-grpc-daemon[976]: time=2026-09-20T12:20:36.144Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5189lb # [ 65.781323] niks3-server[1039]: 2026/09/20 12:20:36 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5190lb # [ 65.786297] niks3-server[1039]: 2026/09/20 12:20:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5191lb # [ 65.790409] niks3-server[1039]: 2026/09/20 12:20:36 INFO Signed narinfos id=24 count=15192worker2 # [ 65.803047] nix-grpc-daemon[976]: time=2026-09-20T12:20:36.172Z level=INFO msg="Uploading 1 narinfos"5193lb # [ 65.794996] niks3-server[1039]: 2026/09/20 12:20:36 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5194lb # [ 65.802174] niks3-server[1039]: 2026/09/20 12:20:36 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5195lb # [ 65.807754] niks3-server[1039]: 2026/09/20 12:20:36 INFO Completed upload id=245196worker2 # [ 65.820045] nix-grpc-daemon[976]: time=2026-09-20T12:20:36.189Z level=INFO msg="Upload complete. (205ms)"5197worker2 # [ 65.824313] nix-grpc-daemon[856]: ts=2026-09-20T12:20:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15198worker2 # [ 65.830410] nix-daemon[925]: reaped child process 1112, status = succeeded5199lb # [ 65.834563] niks3-server[1039]: 2026/09/20 12:20:36 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5200client # copying 1 paths...5201worker2 # [ 65.867302] nix-grpc-daemon[856]: ts=2026-09-20T12:20:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:479545202client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5203worker1 # [ 65.888801] nix-grpc-daemon[853]: ts=2026-09-20T12:20:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:372165204worker1 # [ 65.897477] nix-grpc-daemon[853]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5205worker1 # [ 65.906994] nix-grpc-daemon[853]: ts=2026-09-20T12:20:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_out=1205206client # [ 66.021803] nix-daemon[945]: reaped child process 1075, status = succeeded5207client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.33 seconds)5208(finished: subtest: an input only one worker has still reaches the builder, in 9.01 seconds)5209subtest: upload whose reference the worker lacks is completed from the cache5210client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5211client # this derivation will be built:5212client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5213client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5214client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.34 seconds)5215client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5216client # this derivation will be built:5217client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5218client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5219client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.35 seconds)5220client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5221client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.16 seconds)5222worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5223worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5224worker2 # [ 67.098218] systemd[1]: Stopping Nix worker-protocol over gRPC...5225worker2 # [ 67.442150] nix-grpc-daemon[856]: ts=2026-09-20T12:20:37Z level=info event=signal_exit5226worker2 # [ 67.679664] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_yield5227lb # [ 67.670562] niks3-server[1039]: 2026/09/20 12:20:38 INFO lead: released remote=192.168.1.4:500385228worker2 # [ 67.682655] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_restarting addr=lb:500515229worker1 # [ 67.691219] nix-grpc-daemon[853]: ts=2026-09-20T12:20:38Z level=info event=scheduler_restarting addr=lb:500515230worker2 # [ 67.687031] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:479485231worker2 # [ 67.689892] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=worker_gone addr=192.168.1.4:500515232worker2 # [ 67.691636] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:479485233worker2 # [ 67.693450] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=worker_gone addr=192.168.1.3:500515234worker2 # [ 67.883578] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500515235worker2 # [ 67.885997] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5236worker1 # [ 67.891585] nix-grpc-daemon[853]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500515237worker1 # [ 67.895230] nix-grpc-daemon[853]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5238lb # [ 68.036090] envoy[919]: conn peer=[2001:db8:1::1]:34386 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=505239lb # [ 68.038166] envoy[919]: conn peer=[2001:db8:1::1]:34370 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3850 tx=1878 ms=25275240lb # [ 68.040176] envoy[919]: conn peer=[2001:db8:1::1]:34392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3717 tx=1934 ms=26425241lb # [ 68.042075] envoy[919]: conn peer=[2001:db8:1::1]:42522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=535242lb # [ 68.043920] envoy[919]: conn peer=[2001:db8:1::1]:42524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3701 tx=1802 ms=18845243lb # [ 68.045831] envoy[919]: conn peer=[2001:db8:1::1]:42526 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=725244worker2 # [ 68.086919] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500515245worker2 # [ 68.089247] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5246worker1 # [ 68.095772] nix-grpc-daemon[853]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500515247worker1 # [ 68.097633] nix-grpc-daemon[853]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5248worker2 # [ 68.290313] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500515249worker2 # [ 68.292822] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5250worker1 # [ 68.298423] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500515251worker1 # [ 68.300532] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5252lb # [ 68.425900] niks3-server[1039]: 2026/09/20 12:20:39 INFO lead: acquired remote=192.168.1.3:381785253worker1 # [ 68.441785] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_take_over5254worker2 # [ 68.494958] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_connected addr=lb:500515255worker2 # [ 68.496740] nix-grpc-daemon[856]: ts=2026-09-20T12:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5256worker1 # [ 68.501247] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500515257worker1 # [ 68.503434] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258worker2 # [ 68.682820] nix-grpc-daemon[856]: ts=2026-09-20T12:20:39Z level=info event=scheduler_restarting5259worker1 # [ 68.704949] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500515260worker1 # [ 68.706554] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5261worker2 # [ 68.704449] nix-grpc-daemon[856]: ts=2026-09-20T12:20:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5262worker2 # [ 68.716090] nix-daemon[925]: reaped child process 927, status = succeeded5263worker1 # [ 68.905565] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500515264worker1 # [ 68.907641] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5265worker1 # [ 69.108543] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_connected addr=lb:500515266worker1 # [ 69.110928] nix-grpc-daemon[853]: ts=2026-09-20T12:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5267worker1 # [ 69.311375] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500515268worker1 # [ 69.313374] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5269worker1 # [ 69.513885] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=info event=scheduler_connected addr=lb:500515270worker1 # [ 69.519875] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:556505271worker1 # [ 69.521859] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05272worker2 # [ 69.543557] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5273worker2 # [ 69.545874] systemd[1]: Stopped Nix worker-protocol over gRPC.5274worker2 # [ 69.547258] systemd[1]: nix-grpc-daemon.service: Consumed 1.468s CPU time over 51.385s wall clock time, 49.9M memory peak, 54.5K incoming IP traffic, 137.5K outgoing IP traffic.5275worker2 # [ 69.555078] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5276worker2 # [ 69.557680] systemd[1]: Closed Nix worker-protocol over gRPC.5277worker2 # [ 69.560173] systemd[1]: nix-grpc-daemon.socket: Consumed 360us CPU time over 59.187s wall clock time, 256K memory peak, 73.7K incoming IP traffic, 62.1K outgoing IP traffic.5278lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5279lb: must succeed: curl -sf localhost:9901/clusters5280lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5281lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5282client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25283worker1 # [ 69.827054] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372165284worker1 # [ 69.831391] nix-grpc-daemon[853]: ts=2026-09-20T12:20:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:372165285client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5286client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.24 seconds)5287worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5288worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5289worker2 # [ 69.931104] systemd[1]: Listening on Nix worker-protocol over gRPC.5290worker1 # [ 69.991720] systemd[1]: Stopping Nix worker-protocol over gRPC...5291worker1 # [ 70.309295] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=signal_exit5292worker1 # [ 70.564258] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=scheduler_yield5293worker1 # [ 70.567433] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=scheduler_restarting addr=lb:500515294worker2 # [ 70.563151] systemd[1]: Starting scheduler-token.service...5295lb # [ 70.551829] niks3-server[1039]: 2026/09/20 12:20:41 INFO lead: released remote=192.168.1.3:381785296worker1 # [ 70.570270] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:556505297worker1 # [ 70.572433] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=worker_gone addr=192.168.1.3:500515298worker2 # [ 70.644641] systemd[1]: scheduler-token.service: Deactivated successfully.5299worker2 # [ 70.645816] systemd[1]: Finished scheduler-token.service.5300worker2 # [ 70.647357] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5301worker2 # [ 70.653252] systemd[1]: Starting Nix worker-protocol over gRPC...5302worker1 # [ 70.768651] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500515303worker1 # [ 70.770848] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker2 # [ 70.828979] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5305worker2 # [ 70.831056] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5306worker2 # [ 70.834905] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15307worker2 # [ 70.842075] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5308worker2 # [ 70.846176] systemd[1]: Started Nix worker-protocol over gRPC.5309worker2 # [ 70.855258] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_take_over5310lb # [ 70.843084] niks3-server[1039]: 2026/09/20 12:20:41 INFO lead: acquired remote=192.168.1.4:336225311worker2 # [ 70.861511] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500515312worker2 # [ 70.863245] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker1 # [ 70.971865] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500515314worker1 # [ 70.973823] nix-grpc-daemon[853]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5315worker2 # [ 71.064237] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500515316worker2 # [ 71.066509] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker1 # [ 71.174615] nix-grpc-daemon[853]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500515318worker1 # [ 71.177246] nix-grpc-daemon[853]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5319worker2 # [ 71.267923] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500515320worker2 # [ 71.271979] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5321worker1 # [ 71.378286] nix-grpc-daemon[853]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500515322worker1 # [ 71.382584] nix-grpc-daemon[853]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker2 # [ 71.473315] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_connected addr=lb:500515324worker2 # [ 71.477945] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5325worker1 # [ 71.566917] nix-grpc-daemon[853]: ts=2026-09-20T12:20:42Z level=info event=scheduler_restarting5326worker1 # [ 71.594403] nix-grpc-daemon[853]: ts=2026-09-20T12:20:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5327worker1 # [ 71.606492] nix-daemon[911]: reaped child process 913, status = succeeded5328worker2 # [ 71.678832] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500515329worker2 # [ 71.681121] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5330worker2 # [ 71.882863] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:42Z level=info event=scheduler_connected addr=lb:500515331worker2 # [ 71.896163] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:42Z level=info event=oidc_jwks provider=mock keys=15332worker2 # [ 71.900829] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:439925333worker2 # [ 71.904868] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05334worker1 # [ 72.448627] nix-grpc-daemon[853]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5335worker1 # [ 72.450196] nix-grpc-daemon[853]: E0000 00:00:1789906843.329865 896 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5336worker1 # [ 72.452371] nix-grpc-daemon[853]: E0000 00:00:1789906843.330234 896 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5337worker1 # [ 73.499860] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5338worker1 # [ 73.502344] systemd[1]: Stopped Nix worker-protocol over gRPC.5339worker1 # [ 73.504466] systemd[1]: nix-grpc-daemon.service: Consumed 2.911s CPU time over 55.459s wall clock time, 58.3M memory peak, 84.8K incoming IP traffic, 159.8K outgoing IP traffic.5340worker1 # [ 73.508992] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5341worker1 # [ 73.510898] systemd[1]: Closed Nix worker-protocol over gRPC.5342worker1 # [ 73.512482] systemd[1]: nix-grpc-daemon.socket: Consumed 461us CPU time over 1min 4.577s wall clock time, 256K memory peak, 77.7K incoming IP traffic, 64.5K outgoing IP traffic.5343lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5344lb: must succeed: curl -sf localhost:9901/clusters5345lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5346lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5347client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25348worker2 # [ 73.815091] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419225349worker2 # [ 73.821813] nix-daemon[925]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)5350worker2 # [ 73.831957] nix-grpc-daemon[1156]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)5351worker2 # [ 73.861906] nix-grpc-daemon[1156]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 2/5)5352worker2 # [ 73.946087] nix-grpc-daemon[1156]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 3/5)5353worker2 # [ 73.980955] nix-grpc-daemon[1156]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 384 ms (attempt 4/5)5354worker2 # [ 74.368138] nix-grpc-daemon[1156]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5355client # copying 2 paths...5356client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5357worker2 # [ 74.385621] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419225358client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5359worker2 # [ 74.392636] nix-daemon[925]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)5360worker2 # [ 74.415449] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:44Z level=info event=niks3_push_started pid=12125361worker2 # [ 74.575118] nix-grpc-daemon[1212]: time=2026-09-20T12:20:44.944Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5362worker2 # [ 74.741154] nix-daemon[925]: accepted connection from pid 1225, user nix-grpc-daemon (trusted)5363worker2 # [ 74.754099] nix-daemon[925]: reaped child process 1228, status = succeeded5364lb # [ 74.760721] niks3-server[1039]: 2026/09/20 12:20:45 INFO Received uploads request method=POST path=/api/pending_closures5365lb # [ 74.765425] niks3-server[1039]: 2026/09/20 12:20:45 INFO Received uploads request method=POST path=/api/pending_closures5366worker2 # [ 74.782834] nix-grpc-daemon[1212]: time=2026-09-20T12:20:45.152Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5367worker2 # [ 74.784562] nix-grpc-daemon[1212]: time=2026-09-20T12:20:45.153Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5368worker2 # [ 74.786797] nix-grpc-daemon[1212]: time=2026-09-20T12:20:45.154Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5369lb # [ 74.804578] niks3-server[1039]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5370lb # [ 74.829261] niks3-server[1039]: 2026/09/20 12:20:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5371lb # [ 74.833348] niks3-server[1039]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5372lb # [ 74.837097] niks3-server[1039]: 2026/09/20 12:20:45 INFO Signed narinfos id=26 count=25373lb # [ 74.839279] niks3-server[1039]: 2026/09/20 12:20:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5374lb # [ 74.841773] niks3-server[1039]: 2026/09/20 12:20:45 INFO Signed narinfos id=25 count=15375worker2 # [ 74.855106] nix-grpc-daemon[1212]: time=2026-09-20T12:20:45.223Z level=INFO msg="Uploading 3 narinfos"5376lb # [ 74.847878] niks3-server[1039]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5377lb # [ 74.871128] niks3-server[1039]: 2026/09/20 12:20:45 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5378lb # [ 74.882611] niks3-server[1039]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5379lb # [ 74.886777] niks3-server[1039]: 2026/09/20 12:20:45 INFO Completed upload id=255380lb # [ 74.888844] niks3-server[1039]: 2026/09/20 12:20:45 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5381lb # [ 74.892396] niks3-server[1039]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5382lb # [ 74.896297] niks3-server[1039]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5383lb # [ 74.924624] niks3-server[1039]: 2026/09/20 12:20:45 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5384lb # [ 74.936904] niks3-server[1039]: 2026/09/20 12:20:45 INFO Completed upload id=265385worker2 # [ 74.949587] nix-grpc-daemon[1212]: time=2026-09-20T12:20:45.318Z level=INFO msg="Upload complete. (369ms)"5386worker2 # [ 74.956232] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41922 duration_s=0 paths=2 nar_bytes_in=3125387worker2 # [ 74.967642] nix-daemon[925]: reaped child process 1210, status = succeeded5388client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.39 seconds)5389worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5390worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5391worker1 # [ 75.047281] systemd[1]: Listening on Nix worker-protocol over gRPC.5392worker1 # [ 75.051304] systemd[1]: Starting Nix worker-protocol over gRPC...5393worker1 # [ 75.206856] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5394worker1 # [ 75.209105] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5395worker1 # [ 75.212964] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15396worker1 # [ 75.219934] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5397worker1 # [ 75.224433] systemd[1]: Started Nix worker-protocol over gRPC.5398lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5399lb: must succeed: curl -sf localhost:9901/clusters5400worker1 # [ 75.235688] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:46Z level=info event=scheduler_connected addr=lb:500515401worker2 # [ 75.233955] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:439925402worker2 # [ 75.235819] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05403lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5404lb: must succeed: curl -sf localhost:9901/clusters5405lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5406lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)5407(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.32 seconds)5408subtest: scheduler down: the other node takes the lock and keeps it5409lb: must succeed: curl -sf localhost:9901/clusters5410lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5411lb: must succeed: curl -sf localhost:9901/clusters5412lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5413lb: must succeed: curl -sf localhost:9901/clusters5414lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5415lb: must succeed: curl -sf localhost:9901/clusters5416lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5417worker2 # [ 76.519739] systemd[1]: Stopping Nix worker-protocol over gRPC...5418worker2 # [ 76.847313] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=signal_exit5419worker2 # [ 76.858982] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=scheduler_yield5420lb # [ 76.848943] niks3-server[1039]: 2026/09/20 12:20:47 INFO lead: released remote=192.168.1.4:336225421worker1 # [ 76.864561] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:47Z level=info event=scheduler_restarting addr=lb:500515422worker2 # [ 76.863373] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:439925423worker2 # [ 76.865311] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=worker_gone addr=192.168.1.3:500515424worker2 # [ 76.868484] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=scheduler_restarting addr=lb:500515425worker2 # [ 76.869853] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:439925426worker2 # [ 76.871187] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=worker_gone addr=192.168.1.4:500515427worker1 # [ 77.065085] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:47Z level=info event=scheduler_connected addr=lb:500515428worker1 # [ 77.067494] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5429worker2 # [ 77.066294] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=scheduler_connected addr=lb:500515430worker2 # [ 77.068266] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker1 # [ 77.267947] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_connected addr=lb:500515432worker1 # [ 77.269730] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker2 # [ 77.269358] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=scheduler_connected addr=lb:500515434worker2 # [ 77.271233] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker1 # [ 77.470589] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_connected addr=lb:500515436worker1 # [ 77.474154] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker2 # [ 77.472598] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=scheduler_connected addr=lb:500515438worker2 # [ 77.476434] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5439worker1 # [ 77.674566] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_connected addr=lb:500515440worker1 # [ 77.676868] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5441worker2 # [ 77.677834] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:48Z level=info event=scheduler_connected addr=lb:500515442worker2 # [ 77.682110] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5443worker2 # [ 77.864267] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:48Z level=info event=scheduler_restarting5444worker2 # [ 77.870741] nix-grpc-daemon[1156]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5445worker1 # [ 77.877971] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_connected addr=lb:500515446worker1 # [ 77.879681] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5447worker2 # [ 77.875411] nix-grpc-daemon[1156]: E0000 00:00:1789906848.239436 1199 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5448worker2 # [ 77.881643] nix-grpc-daemon[1156]: E0000 00:00:1789906848.243667 1199 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5449worker2 # [ 77.899932] nix-grpc-daemon[1156]: ts=2026-09-20T12:20:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5450worker2 # [ 77.911984] nix-daemon[925]: reaped child process 1205, status = succeeded5451lb # [ 78.035819] envoy[919]: conn peer=[2001:db8:1::4]:48588 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=46356 tx=2276 ms=502455452lb # [ 78.038112] envoy[919]: conn peer=[2001:db8:1::1]:42536 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165453lb # [ 78.039965] envoy[919]: conn peer=192.168.1.3:37290 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13316 tx=2776 ms=523145454lb # [ 78.041950] envoy[919]: conn peer=[2001:db8:1::1]:45518 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=305 ms=11565455lb # [ 78.043936] envoy[919]: conn peer=[2001:db8:1::4]:57658 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8603 tx=535 ms=70325456worker1 # [ 78.080419] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_connected addr=lb:500515457worker1 # [ 78.082322] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5458worker1 # [ 78.282883] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_connected addr=lb:500515459worker1 # [ 78.284743] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5460worker1 # [ 78.485290] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_connected addr=lb:500515461worker1 # [ 78.487463] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5462worker1 # [ 78.688108] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_connected addr=lb:500515463worker1 # [ 78.689978] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5464worker2 # [ 78.856794] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5465worker2 # [ 78.858475] systemd[1]: Stopped Nix worker-protocol over gRPC.5466worker2 # [ 78.860120] systemd[1]: nix-grpc-daemon.service: Consumed 349ms CPU time over 8.203s wall clock time, 30.6M memory peak, 20.5K incoming IP traffic, 31.8K outgoing IP traffic.5467worker2 # [ 78.864596] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5468worker2 # [ 78.866421] systemd[1]: Closed Nix worker-protocol over gRPC.5469worker2 # [ 78.867672] systemd[1]: nix-grpc-daemon.socket: Consumed 469us CPU time over 8.942s wall clock time, 256K memory peak, 19.6K incoming IP traffic, 14.5K outgoing IP traffic.5470worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5471worker1 # [ 78.890961] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_connected addr=lb:500515472worker1 # [ 78.892815] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5473worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.04 seconds)5474worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5475worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5476worker1 # [ 79.093326] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_connected addr=lb:500515477worker1 # [ 79.095339] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5478worker1 # [ 79.295821] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_connected addr=lb:500515479worker1 # [ 79.298172] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5480worker1 # [ 79.498701] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_connected addr=lb:500515481worker1 # [ 79.500534] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker1 # [ 79.701376] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_connected addr=lb:500515483worker1 # [ 79.704840] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker1 # [ 79.906525] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_connected addr=lb:500515485worker1 # [ 79.910198] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5486worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5487worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5488worker1 # [ 80.110615] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_connected addr=lb:500515489worker1 # [ 80.112369] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5490lb # [ 80.213486] niks3-server[1039]: 2026/09/20 12:20:51 INFO lead: acquired remote=192.168.1.3:599585491worker1 # [ 80.229051] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:51Z level=info event=scheduler_take_over5492worker1 # [ 80.312843] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:51Z level=info event=scheduler_connected addr=lb:500515493worker1 # [ 80.314669] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5494worker1 # [ 80.515340] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:51Z level=info event=scheduler_connected addr=lb:500515495worker1 # [ 80.522121] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:374985496worker1 # [ 80.526246] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05497worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5498worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5499client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25500worker1 # [ 81.261788] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375065501worker1 # [ 81.273626] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5502client # don't know how to build these paths:5503client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5504worker1 # [ 81.288149] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375065505worker1 # [ 81.299236] nix-grpc-daemon[1278]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 1/5)5506worker1 # [ 81.367201] nix-grpc-daemon[1278]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5)5507worker1 # [ 81.525241] nix-grpc-daemon[1278]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 187 ms (attempt 3/5)5508worker1 # [ 81.718075] nix-grpc-daemon[1278]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 651 ms (attempt 4/5)5509worker1 # [ 82.375517] nix-grpc-daemon[1278]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5510worker1 # [ 82.393904] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:53Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05511worker1 # [ 82.396274] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:53Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15512worker1 # [ 82.437628] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375065513worker1 # [ 82.447944] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375065514client # copying 1 paths to 'lb:50051'...5515client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5516worker1 # [ 82.458786] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375065517worker1 # [ 82.470090] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5518worker1 # [ 82.525590] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:53Z level=info event=niks3_push_started pid=13515519worker1 # [ 82.708929] nix-grpc-daemon[1351]: time=2026-09-20T12:20:53.590Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""5520worker1 # [ 82.857766] nix-daemon[911]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5521worker1 # [ 82.868877] nix-daemon[911]: reaped child process 1367, status = succeeded5522worker1 # [ 83.028092] nix-daemon[911]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5523worker1 # [ 83.040863] nix-daemon[911]: reaped child process 1372, status = succeeded5524lb # [ 83.038762] niks3-server[1039]: 2026/09/20 12:20:53 INFO Received uploads request method=POST path=/api/pending_closures5525worker1 # [ 83.065398] nix-grpc-daemon[1351]: time=2026-09-20T12:20:53.947Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5526worker1 # [ 83.068507] nix-grpc-daemon[1351]: time=2026-09-20T12:20:53.948Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5527lb # [ 83.079461] niks3-server[1039]: 2026/09/20 12:20:53 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5528lb # [ 83.086784] niks3-server[1039]: 2026/09/20 12:20:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5529lb # [ 83.095726] niks3-server[1039]: 2026/09/20 12:20:53 INFO Signed narinfos id=27 count=15530worker1 # [ 83.114852] nix-grpc-daemon[1351]: time=2026-09-20T12:20:53.996Z level=INFO msg="Uploading 1 narinfos"5531lb # [ 83.107319] niks3-server[1039]: 2026/09/20 12:20:54 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5532lb # [ 83.114057] niks3-server[1039]: 2026/09/20 12:20:54 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5533lb # [ 83.123456] niks3-server[1039]: 2026/09/20 12:20:54 INFO Completed upload id=275534worker1 # [ 83.139314] nix-grpc-daemon[1351]: time=2026-09-20T12:20:54.021Z level=INFO msg="Upload complete. (427ms)"5535lb # [ 83.125123] niks3-server[1039]: 2026/09/20 12:20:54 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5536worker1 # [ 83.143654] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37506 duration_s=0 paths=1 nar_bytes_in=5205537worker1 # [ 83.151300] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375065538worker1 # [ 83.159496] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5539worker1 # [ 83.168831] nix-daemon[911]: reaped child process 1349, status = succeeded5540worker1 # [ 83.173401] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5541client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5542client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5543client # farm-a> LOG-farm-a-on-standby5544worker1 # [ 84.608948] nix-daemon[911]: reaped child process 1376, status = succeeded5545worker1 # [ 84.750548] nix-daemon[911]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5546worker1 # [ 84.769561] nix-daemon[911]: reaped child process 1394, status = succeeded5547lb # [ 84.766994] niks3-server[1039]: 2026/09/20 12:20:55 INFO Received uploads request method=POST path=/api/pending_closures5548worker1 # [ 84.787915] nix-grpc-daemon[1351]: time=2026-09-20T12:20:55.669Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5549worker1 # [ 84.792546] nix-grpc-daemon[1351]: time=2026-09-20T12:20:55.671Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5550lb # [ 84.794270] niks3-server[1039]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5551lb # [ 84.812795] niks3-server[1039]: 2026/09/20 12:20:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5552lb # [ 84.819841] niks3-server[1039]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5553worker1 # [ 84.840397] nix-grpc-daemon[1351]: time=2026-09-20T12:20:55.721Z level=INFO msg="Uploading 1 narinfos"5554lb # [ 84.824630] niks3-server[1039]: 2026/09/20 12:20:55 INFO Signed narinfos id=28 count=15555lb # [ 84.841544] niks3-server[1039]: 2026/09/20 12:20:55 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5556lb # [ 84.848859] niks3-server[1039]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5557lb # [ 84.852182] niks3-server[1039]: 2026/09/20 12:20:55 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5558lb # [ 84.856521] niks3-server[1039]: 2026/09/20 12:20:55 INFO Completed upload id=285559worker1 # [ 84.872360] nix-grpc-daemon[1351]: time=2026-09-20T12:20:55.754Z level=INFO msg="Upload complete. (289ms)"5560worker1 # [ 84.879525] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37506 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15561worker1 # [ 84.892191] nix-daemon[911]: reaped child process 1374, status = succeeded5562worker1 # [ 84.895057] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:55Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05563worker1 # [ 84.897440] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:55Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15564worker1 # [ 84.941364] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375065565worker1 # [ 84.950707] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375065566worker1 # [ 84.961647] nix-grpc-daemon[1351]: time=2026-09-20T12:20:55.843Z level=INFO msg="All 1 paths already cached"5567client # copying 1 paths to 'lb:50051'...5568client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5569worker1 # [ 84.967044] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375065570worker1 # [ 84.973087] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5571worker1 # [ 85.122049] nix-daemon[911]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5572worker1 # [ 85.136837] nix-daemon[911]: reaped child process 1403, status = succeeded5573worker1 # [ 85.300642] nix-daemon[911]: accepted connection from pid 1405, user nix-grpc-daemon (trusted)5574worker1 # [ 85.313564] nix-daemon[911]: reaped child process 1408, status = succeeded5575lb # [ 85.313090] niks3-server[1039]: 2026/09/20 12:20:56 INFO Received uploads request method=POST path=/api/pending_closures5576worker1 # [ 85.334788] nix-grpc-daemon[1351]: time=2026-09-20T12:20:56.216Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5577worker1 # [ 85.336168] nix-grpc-daemon[1351]: time=2026-09-20T12:20:56.216Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5578lb # [ 85.328242] niks3-server[1039]: 2026/09/20 12:20:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5579lb # [ 85.330312] niks3-server[1039]: 2026/09/20 12:20:56 INFO Signed narinfos id=29 count=15580worker1 # [ 85.345968] nix-grpc-daemon[1351]: time=2026-09-20T12:20:56.227Z level=INFO msg="Uploading 1 narinfos"5581lb # [ 85.334668] niks3-server[1039]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5582lb # [ 85.336963] niks3-server[1039]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5583lb # [ 85.342217] niks3-server[1039]: 2026/09/20 12:20:56 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5584lb # [ 85.345363] niks3-server[1039]: 2026/09/20 12:20:56 INFO Completed upload id=295585worker1 # [ 85.362462] nix-grpc-daemon[1351]: time=2026-09-20T12:20:56.244Z level=INFO msg="Upload complete. (383ms)"5586lb # [ 85.347442] niks3-server[1039]: 2026/09/20 12:20:56 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5587worker1 # [ 85.364873] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37506 duration_s=0 paths=1 nar_bytes_in=6405588worker1 # [ 85.367597] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375065589worker1 # [ 85.375692] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5590worker1 # [ 85.377915] nix-daemon[911]: reaped child process 1398, status = succeeded5591worker1 # [ 85.383525] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5592client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5593client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5594client # farm-b> LOG-farm-b-on-standby5595worker1 # [ 86.800473] nix-daemon[911]: reaped child process 1412, status = succeeded5596worker1 # [ 86.928625] nix-daemon[911]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)5597worker1 # [ 86.940461] nix-daemon[911]: reaped child process 1430, status = succeeded5598lb # [ 86.936776] niks3-server[1039]: 2026/09/20 12:20:57 INFO Received uploads request method=POST path=/api/pending_closures5599worker1 # [ 86.958481] nix-grpc-daemon[1351]: time=2026-09-20T12:20:57.840Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5600worker1 # [ 86.961462] nix-grpc-daemon[1351]: time=2026-09-20T12:20:57.842Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5601lb # [ 86.963138] niks3-server[1039]: 2026/09/20 12:20:57 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5602lb # [ 86.967117] niks3-server[1039]: 2026/09/20 12:20:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5603worker1 # [ 86.987551] nix-grpc-daemon[1351]: time=2026-09-20T12:20:57.869Z level=INFO msg="Uploading 1 narinfos"5604lb # [ 86.971782] niks3-server[1039]: 2026/09/20 12:20:57 INFO Signed narinfos id=30 count=15605lb # [ 86.974366] niks3-server[1039]: 2026/09/20 12:20:57 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5606lb # [ 86.979091] niks3-server[1039]: 2026/09/20 12:20:57 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5607lb # [ 86.987715] niks3-server[1039]: 2026/09/20 12:20:57 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5608lb # [ 86.990751] niks3-server[1039]: 2026/09/20 12:20:57 INFO Completed upload id=305609worker1 # [ 87.006560] nix-grpc-daemon[1351]: time=2026-09-20T12:20:57.888Z level=INFO msg="Upload complete. (212ms)"5610lb # [ 86.992441] niks3-server[1039]: 2026/09/20 12:20:57 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5611worker1 # [ 87.010398] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37506 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15612worker1 # [ 87.017321] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:57Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05613worker1 # [ 87.019665] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:57Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15614worker1 # [ 87.024719] nix-daemon[911]: reaped child process 1410, status = succeeded5615worker1 # [ 87.061775] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375065616worker1 # [ 87.068913] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375065617worker1 # [ 87.076283] nix-grpc-daemon[1351]: time=2026-09-20T12:20:57.958Z level=INFO msg="All 2 paths already cached"5618client # copying 1 paths to 'lb:50051'...5619worker1 # [ 87.081857] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375065620client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5621worker1 # [ 87.088445] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5622worker1 # [ 87.233074] nix-daemon[911]: accepted connection from pid 1436, user nix-grpc-daemon (trusted)5623worker1 # [ 87.245904] nix-daemon[911]: reaped child process 1439, status = succeeded5624worker1 # [ 87.405567] nix-daemon[911]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5625worker1 # [ 87.417729] nix-daemon[911]: reaped child process 1444, status = succeeded5626lb # [ 87.417929] niks3-server[1039]: 2026/09/20 12:20:58 INFO Received uploads request method=POST path=/api/pending_closures5627worker1 # [ 87.440423] nix-grpc-daemon[1351]: time=2026-09-20T12:20:58.322Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5628worker1 # [ 87.441995] nix-grpc-daemon[1351]: time=2026-09-20T12:20:58.322Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5629lb # [ 87.434239] niks3-server[1039]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5630lb # [ 87.437268] niks3-server[1039]: 2026/09/20 12:20:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5631lb # [ 87.439896] niks3-server[1039]: 2026/09/20 12:20:58 INFO Signed narinfos id=31 count=15632worker1 # [ 87.455840] nix-grpc-daemon[1351]: time=2026-09-20T12:20:58.337Z level=INFO msg="Uploading 1 narinfos"5633lb # [ 87.443714] niks3-server[1039]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5634lb # [ 87.447774] niks3-server[1039]: 2026/09/20 12:20:58 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5635lb # [ 87.452215] niks3-server[1039]: 2026/09/20 12:20:58 INFO Completed upload id=315636worker1 # [ 87.467894] nix-grpc-daemon[1351]: time=2026-09-20T12:20:58.349Z level=INFO msg="Upload complete. (374ms)"5637lb # [ 87.453424] niks3-server[1039]: 2026/09/20 12:20:58 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5638worker1 # [ 87.470156] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37506 duration_s=0 paths=1 nar_bytes_in=7685639worker1 # [ 87.473218] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375065640worker1 # [ 87.479952] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5641worker1 # [ 87.482346] nix-daemon[911]: reaped child process 1434, status = succeeded5642worker1 # [ 87.488099] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5643client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5644client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5645worker1 # [ 88.501658] nix-daemon[911]: reaped child process 1448, status = succeeded5646client # farm-top> LOG-farm-top-on-standby5647worker1 # [ 88.627306] nix-daemon[911]: accepted connection from pid 1463, user nix-grpc-daemon (trusted)5648worker1 # [ 88.640434] nix-daemon[911]: reaped child process 1466, status = succeeded5649lb # [ 88.637612] niks3-server[1039]: 2026/09/20 12:20:59 INFO Received uploads request method=POST path=/api/pending_closures5650worker1 # [ 88.658285] nix-grpc-daemon[1351]: time=2026-09-20T12:20:59.540Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5651worker1 # [ 88.659820] nix-grpc-daemon[1351]: time=2026-09-20T12:20:59.541Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5652lb # [ 88.660368] niks3-server[1039]: 2026/09/20 12:20:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5653lb # [ 88.664475] niks3-server[1039]: 2026/09/20 12:20:59 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5654lb # [ 88.666852] niks3-server[1039]: 2026/09/20 12:20:59 INFO Signed narinfos id=32 count=15655worker1 # [ 88.683982] nix-grpc-daemon[1351]: time=2026-09-20T12:20:59.565Z level=INFO msg="Uploading 1 narinfos"5656lb # [ 88.670525] niks3-server[1039]: 2026/09/20 12:20:59 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5657lb # [ 88.673697] niks3-server[1039]: 2026/09/20 12:20:59 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5658lb # [ 88.678613] niks3-server[1039]: 2026/09/20 12:20:59 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5659lb # [ 88.682235] niks3-server[1039]: 2026/09/20 12:20:59 INFO Completed upload id=325660worker1 # [ 88.699199] nix-grpc-daemon[1351]: time=2026-09-20T12:20:59.581Z level=INFO msg="Upload complete. (203ms)"5661lb # [ 88.683861] niks3-server[1039]: 2026/09/20 12:20:59 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5662worker1 # [ 88.703245] nix-grpc-daemon[1278]: ts=2026-09-20T12:20:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37506 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15663worker1 # [ 88.709319] nix-daemon[911]: reaped child process 1446, status = succeeded5664client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.70 seconds)5665client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5666client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5667worker2 # [ 89.013884] systemd[1]: Listening on Nix worker-protocol over gRPC.5668worker2 # [ 89.022381] systemd[1]: Starting scheduler-token.service...5669worker2 # [ 89.077088] systemd[1]: scheduler-token.service: Deactivated successfully.5670worker2 # [ 89.078756] systemd[1]: Finished scheduler-token.service.5671worker2 # [ 89.080500] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 57ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5672worker2 # [ 89.085543] systemd[1]: Starting Nix worker-protocol over gRPC...5673worker2 # [ 89.231617] nix-grpc-daemon[1252]: ts=2026-09-20T12:20:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5674worker2 # [ 89.233468] nix-grpc-daemon[1252]: ts=2026-09-20T12:20:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5675worker2 # [ 89.237161] nix-grpc-daemon[1252]: ts=2026-09-20T12:20:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15676worker2 # [ 89.243993] nix-grpc-daemon[1252]: ts=2026-09-20T12:20:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5677worker2 # [ 89.245618] systemd[1]: Started Nix worker-protocol over gRPC.5678lb: must succeed: curl -sf localhost:9901/clusters5679worker2 # [ 89.260488] nix-grpc-daemon[1252]: ts=2026-09-20T12:20:59Z level=info event=scheduler_connected addr=lb:500515680worker1 # [ 89.272541] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=info event=oidc_jwks provider=mock keys=15681worker1 # [ 89.275315] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:374985682worker1 # [ 89.277588] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05683lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5684worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5685worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5686lb: must succeed: curl -sf localhost:9901/clusters5687lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5688lb: must succeed: curl -sf localhost:9901/clusters5689lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5690worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5691worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5692(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.20 seconds)5693subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5694client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5695client # [ 89.567298] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5696client # Running as unit: rs.service; invocation ID: de48d55674c94fceadd0116c7189afce5697client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.04 seconds)5698client # [ 89.746699] nix[1176]: don't know how to build these paths:5699client # [ 89.748515] nix[1176]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5700worker1 # [ 89.731407] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375065701worker1 # [ 89.742577] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05702worker1 # [ 89.747555] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15703worker1 # [ 89.786264] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375065704worker1 # [ 89.794549] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375065705client # [ 89.817266] nix[1176]: copying 1 paths to 'lb:50051'...5706worker1 # [ 89.803297] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375065707client # [ 89.819202] nix[1176]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5708worker1 # [ 89.813862] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5709worker1 # [ 89.978031] nix-daemon[911]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)5710worker1 # [ 89.990415] nix-daemon[911]: reaped child process 1482, status = succeeded5711worker1 # [ 90.147423] nix-daemon[911]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)5712worker1 # [ 90.159591] nix-daemon[911]: reaped child process 1487, status = succeeded5713lb # [ 90.159590] niks3-server[1039]: 2026/09/20 12:21:01 INFO Received uploads request method=POST path=/api/pending_closures5714worker1 # [ 90.181220] nix-grpc-daemon[1351]: time=2026-09-20T12:21:01.062Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5715worker1 # [ 90.182953] nix-grpc-daemon[1351]: time=2026-09-20T12:21:01.064Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5716lb # [ 90.180830] niks3-server[1039]: 2026/09/20 12:21:01 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5717lb # [ 90.188804] niks3-server[1039]: 2026/09/20 12:21:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5718lb # [ 90.193079] niks3-server[1039]: 2026/09/20 12:21:01 INFO Signed narinfos id=33 count=15719worker1 # [ 90.210872] nix-grpc-daemon[1351]: time=2026-09-20T12:21:01.092Z level=INFO msg="Uploading 1 narinfos"5720lb # [ 90.198425] niks3-server[1039]: 2026/09/20 12:21:01 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5721lb # [ 90.208537] niks3-server[1039]: 2026/09/20 12:21:01 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5722lb # [ 90.213261] niks3-server[1039]: 2026/09/20 12:21:01 INFO Completed upload id=335723worker1 # [ 90.230934] nix-grpc-daemon[1351]: time=2026-09-20T12:21:01.112Z level=INFO msg="Upload complete. (403ms)"5724lb # [ 90.216258] niks3-server[1039]: 2026/09/20 12:21:01 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5725worker1 # [ 90.232390] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37506 duration_s=0 paths=1 nar_bytes_in=4165726worker1 # [ 90.238376] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375065727worker1 # [ 90.242924] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5728worker1 # [ 90.245255] nix-daemon[911]: reaped child process 1477, status = succeeded5729client # [ 90.264571] nix[1176]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5730worker1 # [ 90.250452] nix-daemon[911]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5731lb: must succeed: curl -sf localhost:9901/clusters5732lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5733lb: must succeed: curl -sf localhost:9901/clusters5734lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5735lb: must succeed: curl -sf localhost:9901/clusters5736lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5737lb: must succeed: curl -sf localhost:9901/clusters5738lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5739worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5740worker2 # [ 92.378889] systemd-journald[287]: Received client request to rotate journal, rotating.5741worker2 # [ 92.398674] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e39ea3b7ead54565ad48a5f1b2a8c2e0.5742worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5743worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5744worker1 # [ 92.316918] systemd[1]: Stopping Nix worker-protocol over gRPC...5745worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5746??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5747 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735748client: must succeed: sleep 5.05749worker1 # [ 93.226348] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:04Z level=info event=draining5750client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5751worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5752worker1 # [ 97.671763] nix-daemon[911]: reaped child process 1491, status = succeeded5753worker1 # [ 97.795647] nix-daemon[911]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5754worker1 # [ 97.807089] nix-daemon[911]: reaped child process 1534, status = succeeded5755lb # [ 97.804651] niks3-server[1039]: 2026/09/20 12:21:08 INFO Received uploads request method=POST path=/api/pending_closures5756worker1 # [ 97.830641] nix-grpc-daemon[1351]: time=2026-09-20T12:21:08.712Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5757worker1 # [ 97.833241] nix-grpc-daemon[1351]: time=2026-09-20T12:21:08.714Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5758lb # [ 97.852686] niks3-server[1039]: 2026/09/20 12:21:08 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5759lb # [ 97.865353] niks3-server[1039]: 2026/09/20 12:21:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5760worker1 # [ 97.887075] nix-grpc-daemon[1351]: time=2026-09-20T12:21:08.768Z level=INFO msg="Uploading 1 narinfos"5761lb # [ 97.871113] niks3-server[1039]: 2026/09/20 12:21:08 INFO Signed narinfos id=34 count=15762lb # [ 97.881205] niks3-server[1039]: 2026/09/20 12:21:08 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5763lb # [ 97.889470] niks3-server[1039]: 2026/09/20 12:21:08 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5764worker1 # [ 97.910877] nix-grpc-daemon[1351]: time=2026-09-20T12:21:08.792Z level=INFO msg="Upload complete. (247ms)"5765lb # [ 97.895914] niks3-server[1039]: 2026/09/20 12:21:08 INFO Completed upload id=345766lb # [ 97.897830] niks3-server[1039]: 2026/09/20 12:21:08 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5767lb # [ 97.901141] niks3-server[1039]: 2026/09/20 12:21:08 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5768worker1 # [ 97.914216] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37506 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15769worker1 # [ 97.926254] nix-daemon[911]: reaped child process 1489, status = succeeded5770client # [ 97.975613] systemd[1]: rs.service: Deactivated successfully.5771client # [ 97.979439] systemd[1]: rs.service: Consumed 120ms CPU time over 8.410s wall clock time, 39.1M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5772lb # [ 98.036829] envoy[919]: conn peer=[2001:db8:1::1]:45524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9564 tx=3843 ms=74655773lb # [ 98.041738] envoy[919]: conn peer=[2001:db8:1::1]:58124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=1321 ms=82025774worker1 # [ 98.229086] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=signal_exit5775worker1 # [ 98.451148] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=scheduler_yield5776lb # [ 98.437762] niks3-server[1039]: 2026/09/20 12:21:09 INFO lead: released remote=192.168.1.3:599585777worker2 # [ 98.452617] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:08Z level=info event=scheduler_restarting addr=lb:500515778worker1 # [ 98.459508] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:374985779worker1 # [ 98.463859] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=worker_gone addr=192.168.1.4:500515780worker1 # [ 98.467628] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=scheduler_restarting addr=lb:500515781worker1 # [ 98.470656] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:374985782worker1 # [ 98.474571] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=worker_gone addr=192.168.1.3:500515783worker2 # [ 98.653173] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_connected addr=lb:500515784worker1 # [ 98.664355] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=scheduler_connected addr=lb:500515785worker2 # [ 98.659212] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5786worker1 # [ 98.666120] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5787worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.10 seconds)5788client: waiting for success: ! systemctl is-active rs5789client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5790client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5791client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5792client: must fail: journalctl -u rs | grep -q 'reconnecting'5793client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5794lb: must succeed: curl -sf localhost:9901/clusters5795worker2 # [ 98.858558] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_connected addr=lb:500515796worker2 # [ 98.860364] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5797worker1 # [ 98.866661] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=scheduler_connected addr=lb:500515798worker1 # [ 98.868710] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5799lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5800worker2 # [ 99.061588] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_connected addr=lb:500515801worker1 # [ 99.069932] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=scheduler_connected addr=lb:500515802worker2 # [ 99.065610] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5803worker1 # [ 99.071266] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5804lb # [ 99.241214] niks3-server[1039]: 2026/09/20 12:21:10 INFO lead: acquired remote=192.168.1.4:339565805worker2 # [ 99.253437] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_take_over5806worker2 # [ 99.265912] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_connected addr=lb:500515807worker1 # [ 99.270465] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:10Z level=info event=scheduler_connected addr=lb:500515808worker2 # [ 99.267555] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5809worker1 # [ 99.272384] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5810worker1 # [ 99.454801] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:10Z level=info event=scheduler_restarting5811worker2 # [ 99.468793] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_connected addr=lb:500515812worker2 # [ 99.470575] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5813worker1 # [ 99.485424] nix-grpc-daemon[1278]: ts=2026-09-20T12:21:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5814worker1 # [ 99.490442] nix-daemon[911]: reaped child process 1342, status = succeeded5815worker1 # [ 99.536177] nix-grpc-daemon[1278]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5816worker1 # [ 99.538095] nix-grpc-daemon[1278]: E0000 00:00:1789906870.418080 1321 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5817worker1 # [ 99.540106] nix-grpc-daemon[1278]: E0000 00:00:1789906870.420085 1321 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5818worker2 # [ 99.671456] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:10Z level=info event=scheduler_connected addr=lb:500515819worker2 # [ 99.673192] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5820worker2 # [ 99.874264] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:10Z level=info event=scheduler_connected addr=lb:500515821worker2 # [ 99.878080] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822lb: must succeed: curl -sf localhost:9901/clusters5823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5824worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5825worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5826worker2 # [ 100.079057] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:10Z level=info event=scheduler_connected addr=lb:500515827worker2 # [ 100.092143] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:10Z level=info event=oidc_jwks provider=mock keys=15828worker2 # [ 100.095907] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:10Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540005829worker2 # [ 100.099907] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:10Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05830lb: must succeed: curl -sf localhost:9901/clusters5831lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5832worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5833worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5834worker1 # [ 101.391200] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5835worker1 # [ 101.393381] systemd[1]: Stopped Nix worker-protocol over gRPC.5836worker1 # [ 101.394414] systemd[1]: nix-grpc-daemon.service: Consumed 1.281s CPU time over 26.340s wall clock time, 33.6M memory peak, 49.9K incoming IP traffic, 92.6K outgoing IP traffic.5837worker1 # [ 101.401986] systemd[1]: Starting Nix worker-protocol over gRPC...5838worker1 # [ 101.524426] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5839worker1 # [ 101.526315] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5840worker1 # [ 101.530585] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15841worker1 # [ 101.537049] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5842worker1 # [ 101.540455] systemd[1]: Started Nix worker-protocol over gRPC.5843worker1 # [ 101.549639] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:12Z level=info event=scheduler_connected addr=lb:500515844worker2 # [ 101.548870] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:11Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:540005845worker2 # [ 101.553630] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05846lb: must succeed: curl -sf localhost:9901/clusters5847lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5848worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5849worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5850(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.69 seconds)5851subtest: scheduler restart mid-build: build finishes, no double build5852client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5853client # [ 102.254834] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5854client # Running as unit: mid.service; invocation ID: 9438f50812b547748a42811d266147465855client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.04 seconds)5856worker1 # [ 102.401727] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:541765857worker1 # [ 102.407142] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)5858client # [ 102.428203] nix[1212]: don't know how to build these paths:5859client # [ 102.429135] nix[1212]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5860worker2 # [ 102.415166] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:12Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05861worker2 # [ 102.418136] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:12Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15862worker2 # [ 102.465108] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540065863worker2 # [ 102.472453] nix-daemon[925]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5864worker2 # [ 102.481069] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)5865worker2 # [ 102.490385] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 2/5)5866worker2 # [ 102.507731] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 3/5)5867worker2 # [ 102.848235] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 603 ms (attempt 4/5)5868worker2 # [ 103.452218] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5869worker2 # [ 103.472306] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540065870client # [ 103.495937] nix[1212]: copying 1 paths to 'lb:50051'...5871worker2 # [ 103.481663] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540065872client # [ 103.499894] nix[1212]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5873worker2 # [ 103.489902] nix-daemon[925]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5874worker2 # [ 103.635148] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:14Z level=info event=niks3_push_started pid=13715875worker2 # [ 103.813701] nix-grpc-daemon[1371]: time=2026-09-20T12:21:14.182Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5876worker2 # [ 103.970099] nix-daemon[925]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)5877worker2 # [ 103.982039] nix-daemon[925]: reaped child process 1387, status = succeeded5878worker2 # [ 104.137401] nix-daemon[925]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)5879worker2 # [ 104.148299] nix-daemon[925]: reaped child process 1392, status = succeeded5880lb # [ 104.148382] niks3-server[1039]: 2026/09/20 12:21:15 INFO Received uploads request method=POST path=/api/pending_closures5881worker2 # [ 104.166493] nix-grpc-daemon[1371]: time=2026-09-20T12:21:14.535Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5882worker2 # [ 104.168028] nix-grpc-daemon[1371]: time=2026-09-20T12:21:14.537Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5883lb # [ 104.169412] niks3-server[1039]: 2026/09/20 12:21:15 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5884lb # [ 104.172492] niks3-server[1039]: 2026/09/20 12:21:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5885lb # [ 104.174859] niks3-server[1039]: 2026/09/20 12:21:15 INFO Signed narinfos id=35 count=15886worker2 # [ 104.189433] nix-grpc-daemon[1371]: time=2026-09-20T12:21:14.558Z level=INFO msg="Uploading 1 narinfos"5887lb # [ 104.180393] niks3-server[1039]: 2026/09/20 12:21:15 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5888lb # [ 104.185057] niks3-server[1039]: 2026/09/20 12:21:15 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5889lb # [ 104.190539] niks3-server[1039]: 2026/09/20 12:21:15 INFO Completed upload id=355890worker2 # [ 104.203117] nix-grpc-daemon[1371]: time=2026-09-20T12:21:14.572Z level=INFO msg="Upload complete. (386ms)"5891lb # [ 104.192034] niks3-server[1039]: 2026/09/20 12:21:15 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5892worker2 # [ 104.205719] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54006 duration_s=0 paths=1 nar_bytes_in=4245893worker2 # [ 104.210180] nix-grpc-daemon[1252]: ts=2026-09-20T12:21:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540065894worker2 # [ 104.214832] nix-daemon[925]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5895worker2 # [ 104.220866] nix-daemon[925]: reaped child process 1369, status = succeeded5896client # [ 104.242998] nix[1212]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5897worker2 # [ 104.227070] nix-daemon[925]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5898lb: must succeed: curl -sf localhost:9901/clusters5899lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5900lb: must succeed: curl -sf localhost:9901/clusters5901lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5902worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5903worker2 # [ 105.948150] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1252 (nix-grpc-daemon) on client request.5904worker2 # [ 105.950524] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5905lb # [ 105.946843] niks3-server[1039]: 2026/09/20 12:21:16 INFO lead: released remote=192.168.1.4:339565906client # [ 105.986855] nix[1212]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5907worker1 # [ 105.971401] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:16Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5908worker2 # [ 105.974828] systemd[1]: Starting scheduler-token.service...5909client # [ 105.991520] nix[1212]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5910worker2 # [ 105.985932] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5911worker2 # [ 105.987417] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5912worker2 # [ 105.989207] systemd[1]: nix-grpc-daemon.service: Consumed 445ms CPU time over 16.897s wall clock time, 31M memory peak, 16.8K incoming IP traffic, 29.8K outgoing IP traffic.5913worker2 # [ 106.002216] nix-daemon[925]: reaped child process 1394, status = succeeded5914worker2 # [ 106.003475] nix-daemon[925]: reaped child process 1360, status = succeeded5915worker2 # [ 106.010786] nix-daemon[1396]: unexpected Nix daemon error: error: interrupted by the user5916worker2 # [ 106.016834] nix-daemon[925]: reaped child process 1396, status = failed with exit code 15917worker2 # [ 106.049974] systemd[1]: scheduler-token.service: Deactivated successfully.5918worker2 # [ 106.051680] systemd[1]: Finished scheduler-token.service.5919worker2 # [ 106.053089] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 76ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5920worker2 # [ 106.055899] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5921worker2 # [ 106.060317] systemd[1]: Starting scheduler-token.service...5922worker2 # [ 106.119078] systemd[1]: scheduler-token.service: Deactivated successfully.5923worker2 # [ 106.120408] systemd[1]: Finished scheduler-token.service.5924worker2 # [ 106.121975] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 58ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5925worker2 # [ 106.127694] systemd[1]: Starting Nix worker-protocol over gRPC...5926worker1 # [ 106.171752] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:17Z level=info event=scheduler_connected addr=lb:500515927worker2 # [ 106.279202] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5928worker2 # [ 106.281680] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5929worker2 # [ 106.284962] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15930worker2 # [ 106.291067] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5931worker2 # [ 106.294826] systemd[1]: Started Nix worker-protocol over gRPC.5932lb # [ 106.290208] niks3-server[1039]: 2026/09/20 12:21:17 INFO lead: acquired remote=192.168.1.4:356625933worker2 # [ 106.304703] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=scheduler_take_over5934worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.40 seconds)5935client: must succeed: sleep 5.05936worker1 # [ 106.309278] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5937worker2 # [ 106.311929] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=scheduler_connected addr=lb:500515938worker2 # [ 106.317071] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=oidc_jwks provider=mock keys=15939worker2 # [ 106.318944] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:367785940worker2 # [ 106.320579] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05941worker2 # [ 106.483318] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05942worker2 # [ 106.487956] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15943worker1 # [ 106.510242] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:17Z level=info event=scheduler_connected addr=lb:500515944worker2 # [ 106.508249] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:367785945worker2 # [ 106.514827] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05946worker2 # [ 106.525851] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367805947worker2 # [ 106.536300] nix-daemon[925]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)5948worker2 # [ 106.549876] nix-daemon[925]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)5949client # [ 106.579447] nix[1212]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5950worker2 # [ 106.561960] nix-daemon[925]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)5951lb # [ 108.036864] envoy[919]: conn peer=[2001:db8:1::3]:54138 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5767 tx=1297 ms=242455952lb # [ 108.038589] envoy[919]: conn peer=[2001:db8:1::4]:52664 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7821 tx=590 ms=167105953client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5954client: waiting for success: ! systemctl is-active mid5955worker2 # [ 111.576778] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:21Z level=info event=niks3_push_started pid=15175956worker2 # [ 111.588234] nix-daemon[925]: reaped child process 1493, status = succeeded5957worker2 # [ 111.735477] nix-grpc-daemon[1517]: time=2026-09-20T12:21:22.104Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""5958worker2 # [ 111.880565] nix-daemon[925]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)5959worker2 # [ 111.892402] nix-daemon[925]: reaped child process 1533, status = succeeded5960lb # [ 111.890919] niks3-server[1039]: 2026/09/20 12:21:22 INFO Received uploads request method=POST path=/api/pending_closures5961worker2 # [ 111.911546] nix-grpc-daemon[1517]: time=2026-09-20T12:21:22.280Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5962worker2 # [ 111.913053] nix-grpc-daemon[1517]: time=2026-09-20T12:21:22.282Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5963lb # [ 111.944308] niks3-server[1039]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5964lb # [ 111.948893] niks3-server[1039]: 2026/09/20 12:21:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5965lb # [ 111.951053] niks3-server[1039]: 2026/09/20 12:21:22 INFO Signed narinfos id=36 count=15966worker2 # [ 111.963238] nix-grpc-daemon[1517]: time=2026-09-20T12:21:22.332Z level=INFO msg="Uploading 1 narinfos"5967lb # [ 111.954953] niks3-server[1039]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5968lb # [ 111.960634] niks3-server[1039]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5969lb # [ 111.963932] niks3-server[1039]: 2026/09/20 12:21:22 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5970lb # [ 111.969178] niks3-server[1039]: 2026/09/20 12:21:22 INFO Completed upload id=365971worker2 # [ 111.981590] nix-grpc-daemon[1517]: time=2026-09-20T12:21:22.350Z level=INFO msg="Upload complete. (243ms)"5972lb # [ 111.970563] niks3-server[1039]: 2026/09/20 12:21:22 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5973worker2 # [ 111.983829] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36780 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15974worker2 # [ 111.991839] nix-daemon[925]: reaped child process 1491, status = succeeded5975client # [ 112.027865] systemd[1]: mid.service: Deactivated successfully.5976client # [ 112.029148] systemd[1]: mid.service: Consumed 108ms CPU time over 9.772s wall clock time, 39.3M memory peak, 9.6K incoming IP traffic, 10.7K outgoing IP traffic.5977client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)5978client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5979client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5980(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.44 seconds)5981subtest: interrupting the client stops the build on the worker5982client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5983client # [ 112.693475] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5984client # Running as unit: intr.service; invocation ID: 7cfdef964818463081263a61f058f2f65985client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5986worker2 # [ 112.863807] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:367805987client # [ 112.887943] nix[1248]: don't know how to build these paths:5988client # [ 112.888880] nix[1248]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5989worker2 # [ 112.876146] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:23Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05990worker2 # [ 112.880104] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:23Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15991worker1 # [ 112.931812] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541765992worker1 # [ 112.938438] nix-grpc-daemon[1541]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 1/5)5993worker1 # [ 112.941773] nix-grpc-daemon[1541]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 146 ms (attempt 2/5)5994worker1 # [ 113.090274] nix-grpc-daemon[1541]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 158 ms (attempt 3/5)5995worker1 # [ 113.251123] nix-grpc-daemon[1541]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 751 ms (attempt 4/5)5996worker1 # [ 114.006982] nix-grpc-daemon[1541]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5997worker1 # [ 114.022622] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541765998client # [ 114.048779] nix[1248]: copying 1 paths to 'lb:50051'...5999client # [ 114.053099] nix[1248]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6000worker1 # [ 114.038173] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766001worker1 # [ 114.049497] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6002worker1 # [ 114.068074] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:24Z level=info event=niks3_push_started pid=16336003worker1 # [ 114.259720] nix-grpc-daemon[1633]: time=2026-09-20T12:21:25.141Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker1.key ca=""6004worker1 # [ 114.421106] nix-daemon[911]: accepted connection from pid 1646, user nix-grpc-daemon (trusted)6005worker1 # [ 114.434618] nix-daemon[911]: reaped child process 1649, status = succeeded6006worker1 # [ 114.600348] nix-daemon[911]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)6007worker1 # [ 114.614593] nix-daemon[911]: reaped child process 1654, status = succeeded6008lb # [ 114.616783] niks3-server[1039]: 2026/09/20 12:21:25 INFO Received uploads request method=POST path=/api/pending_closures6009worker1 # [ 114.642040] nix-grpc-daemon[1633]: time=2026-09-20T12:21:25.524Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6010worker1 # [ 114.645410] nix-grpc-daemon[1633]: time=2026-09-20T12:21:25.525Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6011lb # [ 114.654424] niks3-server[1039]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6012lb # [ 114.660498] niks3-server[1039]: 2026/09/20 12:21:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6013worker1 # [ 114.682787] nix-grpc-daemon[1633]: time=2026-09-20T12:21:25.564Z level=INFO msg="Uploading 1 narinfos"6014lb # [ 114.666849] niks3-server[1039]: 2026/09/20 12:21:25 INFO Signed narinfos id=37 count=16015lb # [ 114.676461] niks3-server[1039]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6016lb # [ 114.684503] niks3-server[1039]: 2026/09/20 12:21:25 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6017lb # [ 114.688914] niks3-server[1039]: 2026/09/20 12:21:25 INFO Completed upload id=376018worker1 # [ 114.704722] nix-grpc-daemon[1633]: time=2026-09-20T12:21:25.586Z level=INFO msg="Upload complete. (440ms)"6019lb # [ 114.690641] niks3-server[1039]: 2026/09/20 12:21:25 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6020worker1 # [ 114.707148] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=4246021worker1 # [ 114.710674] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766022worker1 # [ 114.716590] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6023worker1 # [ 114.722060] nix-daemon[911]: reaped child process 1631, status = succeeded6024worker1 # [ 114.727334] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6025client # [ 114.741243] nix[1248]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6026client: must succeed: systemctl kill -s INT intr6027client # [ 116.152243] systemd[1]: intr.service: Sent signal SIGINT to main process 1248 (nix) on client request.6028client # [ 116.154597] nix[1248]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6029worker1 # [ 116.140380] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:27Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6030client # [ 116.158055] nix[1248]: error: interrupted by the user6031client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6032client # [ 116.183121] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6033client # [ 116.184616] systemd[1]: intr.service: Failed with result 'exit-code'.6034client # [ 116.186596] systemd[1]: intr.service: Consumed 99ms CPU time over 3.492s wall clock time, 39.3M memory peak, 7.6K incoming IP traffic, 8.7K outgoing IP traffic.6035worker1 # [ 116.345663] nix-daemon[911]: reaped child process 1656, status = succeeded6036worker1 # [ 116.349052] nix-daemon[1658]: unexpected Nix daemon error: error: interrupted by the user6037worker1 # [ 116.353413] nix-daemon[911]: reaped child process 1658, status = failed with exit code 16038(finished: subtest: interrupting the client stops the build on the worker, in 4.71 seconds)6039subtest: a deploy mid-build drains: the build finishes, the new generation takes over6040client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6041client # [ 117.424250] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6042client # Running as unit: sw.service; invocation ID: fa12c38514cb489e83a8343af20757796043client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6044worker1 # [ 117.602276] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:541766045client # [ 117.621614] nix[1269]: don't know how to build these paths:6046client # [ 117.622919] nix[1269]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6047worker2 # [ 117.615206] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:27Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06048worker2 # [ 117.619229] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:27Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16049worker2 # [ 117.657089] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367806050worker2 # [ 117.668171] nix-grpc-daemon[1438]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)6051worker2 # [ 117.683885] nix-grpc-daemon[1438]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5)6052worker2 # [ 117.841557] nix-grpc-daemon[1438]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 3/5)6053worker2 # [ 117.912692] nix-grpc-daemon[1438]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 775 ms (attempt 4/5)6054lb # [ 118.037341] envoy[919]: conn peer=[2001:db8:1::1]:45126 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3848 tx=1661 ms=95966055lb # [ 118.041208] envoy[919]: conn peer=[2001:db8:1::1]:47104 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=755 ms=32866056worker2 # [ 118.690762] nix-grpc-daemon[1438]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6057worker2 # [ 118.703429] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367806058client # [ 118.731353] nix[1269]: copying 1 paths to 'lb:50051'...6059client # [ 118.734735] nix[1269]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6060worker2 # [ 118.717105] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367806061worker2 # [ 118.726716] nix-daemon[925]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)6062worker2 # [ 118.888056] nix-daemon[925]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6063worker2 # [ 118.900666] nix-daemon[925]: reaped child process 1579, status = succeeded6064worker2 # [ 119.053982] nix-daemon[925]: accepted connection from pid 1581, user nix-grpc-daemon (trusted)6065worker2 # [ 119.066465] nix-daemon[925]: reaped child process 1584, status = succeeded6066lb # [ 119.068496] niks3-server[1039]: 2026/09/20 12:21:29 INFO Received uploads request method=POST path=/api/pending_closures6067worker2 # [ 119.089481] nix-grpc-daemon[1517]: time=2026-09-20T12:21:29.458Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6068worker2 # [ 119.091551] nix-grpc-daemon[1517]: time=2026-09-20T12:21:29.460Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6069lb # [ 119.095658] niks3-server[1039]: 2026/09/20 12:21:29 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6070lb # [ 119.107131] niks3-server[1039]: 2026/09/20 12:21:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6071lb # [ 119.113483] niks3-server[1039]: 2026/09/20 12:21:30 INFO Signed narinfos id=38 count=16072worker2 # [ 119.129110] nix-grpc-daemon[1517]: time=2026-09-20T12:21:29.498Z level=INFO msg="Uploading 1 narinfos"6073lb # [ 119.120468] niks3-server[1039]: 2026/09/20 12:21:30 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6074lb # [ 119.130212] niks3-server[1039]: 2026/09/20 12:21:30 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6075lb # [ 119.136863] niks3-server[1039]: 2026/09/20 12:21:30 INFO Completed upload id=386076worker2 # [ 119.149269] nix-grpc-daemon[1517]: time=2026-09-20T12:21:29.518Z level=INFO msg="Upload complete. (413ms)"6077lb # [ 119.139305] niks3-server[1039]: 2026/09/20 12:21:30 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6078worker2 # [ 119.152150] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36780 duration_s=0 paths=1 nar_bytes_in=4166079worker2 # [ 119.156894] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367806080worker2 # [ 119.166670] nix-daemon[925]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)6081worker2 # [ 119.170567] nix-daemon[925]: reaped child process 1574, status = succeeded6082worker2 # [ 119.175313] nix-daemon[925]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)6083client # [ 119.192891] nix[1269]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6084worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26085worker2 # Checking switch inhibitors... done6086worker2 # [ 121.578058] nixos[1619]: switching to system configuration /nix/store/jndy2hdcbi69dlgfllfcslk0r7hinys5-nixos-system-worker2-test6087worker2 # activating the configuration...6088worker2 # [ 121.580412] systemd[1]: Stopped target Local File Systems.6089worker2 # [ 121.582171] systemd[1]: Stopped target Remote File Systems.6090worker2 # setting up /etc...6091worker2 # [ 122.239609] systemd[1]: Reload requested from client PID 1619 ('.switch-to-conf') (unit backdoor.service)...6092worker2 # [ 122.241752] systemd[1]: Reloading...6093worker2 # [ 122.851230] systemd[1]: Reloading finished in 606 ms.6094worker2 # restarting sysinit-reactivation.target6095worker2 # [ 122.906468] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6096worker2 # [ 123.042073] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6097worker2 # [ 123.045191] systemd[1]: Reached target Reactivate sysinit units.6098worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6099worker2 # [ 123.049579] systemd[1]: Reloading D-Bus System Message Bus...6100worker2 # [ 123.051475] dbus-broker-launch[578]: Caught SIGHUP, trigger reload.6101worker2 # [ 123.055283] systemd[1]: Reloading Nix worker-protocol over gRPC...6102worker2 # [ 123.082099] dbus-broker-launch[578]: Looking up NSS user entry for 'systemd-timesync'...6103worker2 # [ 123.084466] dbus-broker-launch[578]: NSS returned no entry for 'systemd-timesync'6104worker2 # [ 123.086196] dbus-broker-launch[578]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6105worker2 # [ 123.096202] systemd[1]: Reloaded Nix worker-protocol over gRPC.6106worker2 # [ 123.110588] systemd[1]: Reloaded D-Bus System Message Bus.6107worker2 # [ 123.112579] systemd[1]: Reached target Local File Systems.6108worker2 # [ 123.118731] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6109worker2 # [ 123.125564] systemd[1]: Starting Create SUID/SGID Wrappers...6110worker2 # [ 123.126691] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6111worker2 # [ 123.127938] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676112worker2 # [ 123.130865] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6113worker2 # [ 123.132271] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6114worker2 # [ 123.133521] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6115worker2 # [ 123.136678] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6116worker2 # [ 123.139339] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6117worker2 # [ 123.143676] systemd[1]: Reached target Remote File Systems.6118worker2 # [ 123.299614] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:33Z level=info event=draining6119worker2 # [ 123.530611] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6120worker2 # [ 123.532141] systemd[1]: Finished Create SUID/SGID Wrappers.6121worker2 # [ 123.534115] systemd[1]: Starting linger-users.service...6122worker2 # [ 123.574467] systemd[1]: linger-users.service: Deactivated successfully.6123worker2 # [ 123.576145] systemd[1]: Finished linger-users.service.6124worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6125worker2 # [ 123.830122] nixos[1619]: finished switching to system configuration /nix/store/jndy2hdcbi69dlgfllfcslk0r7hinys5-nixos-system-worker2-test6126worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.87 seconds)6127worker2: must succeed: pgrep -f 'read -t [9]0 x'6128worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6129client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26130client # don't know how to build these paths:6131client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6132worker1 # [ 124.076983] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:541766133worker1 # [ 124.082827] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766134worker2 # [ 124.094908] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:34Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06135worker2 # [ 124.097151] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:34Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16136worker1 # [ 124.145852] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766137worker1 # [ 124.157686] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766138client # copying 1 paths to 'lb:50051'...6139worker1 # [ 124.168975] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766140client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6141worker1 # [ 124.191703] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6142worker1 # [ 124.364024] nix-daemon[911]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6143worker1 # [ 124.376171] nix-daemon[911]: reaped child process 1715, status = succeeded6144worker1 # [ 124.537777] nix-daemon[911]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6145worker1 # [ 124.548514] nix-daemon[911]: reaped child process 1720, status = succeeded6146lb # [ 124.546734] niks3-server[1039]: 2026/09/20 12:21:35 INFO Received uploads request method=POST path=/api/pending_closures6147worker1 # [ 124.569197] nix-grpc-daemon[1633]: time=2026-09-20T12:21:35.450Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6148worker1 # [ 124.570783] nix-grpc-daemon[1633]: time=2026-09-20T12:21:35.452Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6149lb # [ 124.575728] niks3-server[1039]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6150lb # [ 124.587302] niks3-server[1039]: 2026/09/20 12:21:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6151lb # [ 124.593249] niks3-server[1039]: 2026/09/20 12:21:35 INFO Signed narinfos id=39 count=16152worker1 # [ 124.612286] nix-grpc-daemon[1633]: time=2026-09-20T12:21:35.494Z level=INFO msg="Uploading 1 narinfos"6153lb # [ 124.604344] niks3-server[1039]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6154lb # [ 124.614623] niks3-server[1039]: 2026/09/20 12:21:35 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6155lb # [ 124.621077] niks3-server[1039]: 2026/09/20 12:21:35 INFO Completed upload id=396156lb # [ 124.623339] niks3-server[1039]: 2026/09/20 12:21:35 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6157worker1 # [ 124.638237] nix-grpc-daemon[1633]: time=2026-09-20T12:21:35.520Z level=INFO msg="Upload complete. (432ms)"6158worker1 # [ 124.643265] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=5286159worker1 # [ 124.647890] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766160worker1 # [ 124.656716] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6161worker1 # [ 124.664134] nix-daemon[911]: reaped child process 1710, status = succeeded6162worker1 # [ 124.668892] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6163client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6164client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6165client # farm-a> LOG-farm-a-during-drain6166worker1 # [ 125.885112] nix-daemon[911]: reaped child process 1724, status = succeeded6167worker1 # [ 126.021092] nix-daemon[911]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6168worker1 # [ 126.032235] nix-daemon[911]: reaped child process 1742, status = succeeded6169lb # [ 126.031634] niks3-server[1039]: 2026/09/20 12:21:36 INFO Received uploads request method=POST path=/api/pending_closures6170worker1 # [ 126.051741] nix-grpc-daemon[1633]: time=2026-09-20T12:21:36.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6171worker1 # [ 126.054144] nix-grpc-daemon[1633]: time=2026-09-20T12:21:36.935Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6172lb # [ 126.051492] niks3-server[1039]: 2026/09/20 12:21:36 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6173lb # [ 126.057443] niks3-server[1039]: 2026/09/20 12:21:36 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6174lb # [ 126.060277] niks3-server[1039]: 2026/09/20 12:21:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6175lb # [ 126.062340] niks3-server[1039]: 2026/09/20 12:21:36 INFO Signed narinfos id=40 count=16176worker1 # [ 126.079912] nix-grpc-daemon[1633]: time=2026-09-20T12:21:36.961Z level=INFO msg="Uploading 1 narinfos"6177lb # [ 126.066296] niks3-server[1039]: 2026/09/20 12:21:36 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6178lb # [ 126.071172] niks3-server[1039]: 2026/09/20 12:21:36 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6179lb # [ 126.074612] niks3-server[1039]: 2026/09/20 12:21:36 INFO Completed upload id=406180worker1 # [ 126.090215] nix-grpc-daemon[1633]: time=2026-09-20T12:21:36.972Z level=INFO msg="Upload complete. (214ms)"6181lb # [ 126.076444] niks3-server[1039]: 2026/09/20 12:21:36 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6182worker1 # [ 126.094129] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16183worker1 # [ 126.100908] nix-daemon[911]: reaped child process 1722, status = succeeded6184worker2 # [ 126.095968] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:36Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06185worker2 # [ 126.099244] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:36Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16186worker1 # [ 126.144535] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766187worker1 # [ 126.155676] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766188worker1 # [ 126.167709] nix-grpc-daemon[1633]: time=2026-09-20T12:21:37.049Z level=INFO msg="All 1 paths already cached"6189client # copying 1 paths to 'lb:50051'...6190client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6191worker1 # [ 126.173855] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766192worker1 # [ 126.182123] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6193worker1 # [ 126.343472] nix-daemon[911]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6194worker1 # [ 126.356566] nix-daemon[911]: reaped child process 1749, status = succeeded6195worker1 # [ 126.508894] nix-daemon[911]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)6196worker1 # [ 126.520904] nix-daemon[911]: reaped child process 1754, status = succeeded6197lb # [ 126.519425] niks3-server[1039]: 2026/09/20 12:21:37 INFO Received uploads request method=POST path=/api/pending_closures6198worker1 # [ 126.542961] nix-grpc-daemon[1633]: time=2026-09-20T12:21:37.424Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6199worker1 # [ 126.544429] nix-grpc-daemon[1633]: time=2026-09-20T12:21:37.424Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6200lb # [ 126.546506] niks3-server[1039]: 2026/09/20 12:21:37 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6201lb # [ 126.555875] niks3-server[1039]: 2026/09/20 12:21:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6202lb # [ 126.560963] niks3-server[1039]: 2026/09/20 12:21:37 INFO Signed narinfos id=41 count=16203worker1 # [ 126.580101] nix-grpc-daemon[1633]: time=2026-09-20T12:21:37.461Z level=INFO msg="Uploading 1 narinfos"6204lb # [ 126.566669] niks3-server[1039]: 2026/09/20 12:21:37 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6205lb # [ 126.577940] niks3-server[1039]: 2026/09/20 12:21:37 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6206lb # [ 126.584925] niks3-server[1039]: 2026/09/20 12:21:37 INFO Completed upload id=416207worker1 # [ 126.602797] nix-grpc-daemon[1633]: time=2026-09-20T12:21:37.484Z level=INFO msg="Upload complete. (411ms)"6208lb # [ 126.587922] niks3-server[1039]: 2026/09/20 12:21:37 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6209worker1 # [ 126.605382] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=6406210worker1 # [ 126.610444] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766211worker1 # [ 126.618887] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6212worker1 # [ 126.624286] nix-daemon[911]: reaped child process 1744, status = succeeded6213worker1 # [ 126.629264] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6214client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6215client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6216client # farm-b> LOG-farm-b-during-drain6217worker1 # [ 127.641433] nix-daemon[911]: reaped child process 1758, status = succeeded6218worker1 # [ 127.781727] nix-daemon[911]: accepted connection from pid 1773, user nix-grpc-daemon (trusted)6219worker1 # [ 127.792649] nix-daemon[911]: reaped child process 1776, status = succeeded6220lb # [ 127.793757] niks3-server[1039]: 2026/09/20 12:21:38 INFO Received uploads request method=POST path=/api/pending_closures6221worker1 # [ 127.820075] nix-grpc-daemon[1633]: time=2026-09-20T12:21:38.701Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6222worker1 # [ 127.822070] nix-grpc-daemon[1633]: time=2026-09-20T12:21:38.703Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6223lb # [ 127.828243] niks3-server[1039]: 2026/09/20 12:21:38 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6224lb # [ 127.838524] niks3-server[1039]: 2026/09/20 12:21:38 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6225lb # [ 127.850779] niks3-server[1039]: 2026/09/20 12:21:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6226worker1 # [ 127.872918] nix-grpc-daemon[1633]: time=2026-09-20T12:21:38.754Z level=INFO msg="Uploading 1 narinfos"6227lb # [ 127.857107] niks3-server[1039]: 2026/09/20 12:21:38 INFO Signed narinfos id=42 count=16228lb # [ 127.860696] niks3-server[1039]: 2026/09/20 12:21:38 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6229lb # [ 127.872108] niks3-server[1039]: 2026/09/20 12:21:38 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6230lb # [ 127.875187] niks3-server[1039]: 2026/09/20 12:21:38 INFO Completed upload id=426231worker1 # [ 127.890958] nix-grpc-daemon[1633]: time=2026-09-20T12:21:38.772Z level=INFO msg="Upload complete. (255ms)"6232lb # [ 127.876691] niks3-server[1039]: 2026/09/20 12:21:38 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6233worker1 # [ 127.893352] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16234worker1 # [ 127.900214] nix-daemon[911]: reaped child process 1756, status = succeeded6235worker2 # [ 127.897443] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:38Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06236worker2 # [ 127.900349] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:38Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16237worker1 # [ 127.946915] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766238worker1 # [ 127.958036] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766239worker1 # [ 127.966820] nix-grpc-daemon[1633]: time=2026-09-20T12:21:38.848Z level=INFO msg="All 2 paths already cached"6240client # copying 1 paths to 'lb:50051'...6241worker1 # [ 127.971259] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766242client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6243worker1 # [ 127.979807] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6244worker1 # [ 128.122197] nix-daemon[911]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6245worker1 # [ 128.134723] nix-daemon[911]: reaped child process 1783, status = succeeded6246worker1 # [ 128.273406] nix-daemon[911]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6247worker1 # [ 128.285254] nix-daemon[911]: reaped child process 1788, status = succeeded6248lb # [ 128.284794] niks3-server[1039]: 2026/09/20 12:21:39 INFO Received uploads request method=POST path=/api/pending_closures6249worker1 # [ 128.309736] nix-grpc-daemon[1633]: time=2026-09-20T12:21:39.191Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6250worker1 # [ 128.311171] nix-grpc-daemon[1633]: time=2026-09-20T12:21:39.191Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6251lb # [ 128.311598] niks3-server[1039]: 2026/09/20 12:21:39 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6252lb # [ 128.324769] niks3-server[1039]: 2026/09/20 12:21:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6253worker1 # [ 128.345642] nix-grpc-daemon[1633]: time=2026-09-20T12:21:39.227Z level=INFO msg="Uploading 1 narinfos"6254lb # [ 128.331141] niks3-server[1039]: 2026/09/20 12:21:39 INFO Signed narinfos id=43 count=16255lb # [ 128.337411] niks3-server[1039]: 2026/09/20 12:21:39 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6256lb # [ 128.350108] niks3-server[1039]: 2026/09/20 12:21:39 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6257lb # [ 128.355118] niks3-server[1039]: 2026/09/20 12:21:39 INFO Completed upload id=436258worker1 # [ 128.370849] nix-grpc-daemon[1633]: time=2026-09-20T12:21:39.252Z level=INFO msg="Upload complete. (384ms)"6259lb # [ 128.358241] niks3-server[1039]: 2026/09/20 12:21:39 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6260worker1 # [ 128.374501] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=7686261worker1 # [ 128.380142] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766262worker1 # [ 128.391074] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6263worker1 # [ 128.394511] nix-daemon[911]: reaped child process 1778, status = succeeded6264worker1 # [ 128.399514] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6265client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6266client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6267worker1 # [ 129.412627] nix-daemon[911]: reaped child process 1792, status = succeeded6268client # farm-top> LOG-farm-top-during-drain6269worker1 # [ 129.523210] nix-daemon[911]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)6270worker1 # [ 129.535715] nix-daemon[911]: reaped child process 1810, status = succeeded6271lb # [ 129.531930] niks3-server[1039]: 2026/09/20 12:21:40 INFO Received uploads request method=POST path=/api/pending_closures6272worker1 # [ 129.555856] nix-grpc-daemon[1633]: time=2026-09-20T12:21:40.437Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6273worker1 # [ 129.557532] nix-grpc-daemon[1633]: time=2026-09-20T12:21:40.439Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6274lb # [ 129.569421] niks3-server[1039]: 2026/09/20 12:21:40 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6275lb # [ 129.575975] niks3-server[1039]: 2026/09/20 12:21:40 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6276lb # [ 129.589894] niks3-server[1039]: 2026/09/20 12:21:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6277lb # [ 129.595113] niks3-server[1039]: 2026/09/20 12:21:40 INFO Signed narinfos id=44 count=16278worker1 # [ 129.613994] nix-grpc-daemon[1633]: time=2026-09-20T12:21:40.495Z level=INFO msg="Uploading 1 narinfos"6279lb # [ 129.601221] niks3-server[1039]: 2026/09/20 12:21:40 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6280lb # [ 129.611680] niks3-server[1039]: 2026/09/20 12:21:40 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6281lb # [ 129.617669] niks3-server[1039]: 2026/09/20 12:21:40 INFO Completed upload id=446282worker1 # [ 129.633357] nix-grpc-daemon[1633]: time=2026-09-20T12:21:40.515Z level=INFO msg="Upload complete. (227ms)"6283lb # [ 129.619277] niks3-server[1039]: 2026/09/20 12:21:40 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6284worker1 # [ 129.635878] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16285worker1 # [ 129.642657] nix-daemon[911]: reaped child process 1790, status = succeeded6286client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.80 seconds)6287client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6288client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.21 seconds)6289worker2: must succeed: pkill -f 'read -t [9]0 x'6290worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6291client: waiting for success: ! systemctl is-active sw6292worker2 # [ 129.999695] nix-daemon[925]: reaped child process 1588, status = succeeded6293worker2 # [ 130.140097] nix-daemon[925]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6294worker2 # [ 130.151876] nix-daemon[925]: reaped child process 1768, status = succeeded6295lb # [ 130.156140] niks3-server[1039]: 2026/09/20 12:21:41 INFO Received uploads request method=POST path=/api/pending_closures6296worker2 # [ 130.176318] nix-grpc-daemon[1517]: time=2026-09-20T12:21:40.545Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6297worker2 # [ 130.178492] nix-grpc-daemon[1517]: time=2026-09-20T12:21:40.547Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6298lb # [ 130.185339] niks3-server[1039]: 2026/09/20 12:21:41 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6299lb # [ 130.201337] niks3-server[1039]: 2026/09/20 12:21:41 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6300lb # [ 130.209297] niks3-server[1039]: 2026/09/20 12:21:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6301lb # [ 130.215218] niks3-server[1039]: 2026/09/20 12:21:41 INFO Signed narinfos id=45 count=16302worker2 # [ 130.230641] nix-grpc-daemon[1517]: time=2026-09-20T12:21:40.599Z level=INFO msg="Uploading 1 narinfos"6303lb # [ 130.220176] niks3-server[1039]: 2026/09/20 12:21:41 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6304lb # [ 130.229950] niks3-server[1039]: 2026/09/20 12:21:41 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6305lb # [ 130.233704] niks3-server[1039]: 2026/09/20 12:21:41 INFO Completed upload id=456306worker2 # [ 130.246248] nix-grpc-daemon[1517]: time=2026-09-20T12:21:40.615Z level=INFO msg="Upload complete. (252ms)"6307lb # [ 130.236040] niks3-server[1039]: 2026/09/20 12:21:41 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6308worker2 # [ 130.252535] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36780 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16309worker2 # [ 130.263876] nix-daemon[925]: reaped child process 1586, status = succeeded6310client # [ 130.308646] systemd[1]: sw.service: Deactivated successfully.6311client # [ 130.313402] systemd[1]: sw.service: Consumed 114ms CPU time over 12.886s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.6312worker2 # [ 130.302721] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:40Z level=info event=signal_exit6313worker2 # [ 130.522286] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:40Z level=info event=scheduler_yield6314lb # [ 130.512342] niks3-server[1039]: 2026/09/20 12:21:41 INFO lead: released remote=192.168.1.4:356626315worker1 # [ 130.533227] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:41Z level=info event=scheduler_restarting addr=lb:500516316worker2 # [ 130.530194] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:40Z level=info event=scheduler_restarting addr=lb:500516317worker2 # [ 130.534461] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:367786318worker2 # [ 130.537929] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:40Z level=info event=worker_gone addr=192.168.1.3:500516319worker2 # [ 130.541819] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:367786320worker2 # [ 130.545313] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:40Z level=info event=worker_gone addr=192.168.1.4:500516321worker2 # [ 130.731569] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500516322worker1 # [ 130.738211] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500516323worker2 # [ 130.737268] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6324worker1 # [ 130.742393] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6325worker2 # [ 130.938761] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500516326worker1 # [ 130.943292] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500516327worker1 # [ 130.947569] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6328worker2 # [ 130.945532] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6329client: (finished: waiting for success: ! systemctl is-active sw, in 1.08 seconds)6330client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6331client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6332worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6333worker2 # [ 131.129969] systemd[1]: Starting scheduler-token.service...6334worker2 # [ 131.146232] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500516335worker1 # [ 131.148609] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_connected addr=lb:500516336worker2 # [ 131.148150] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6337worker1 # [ 131.152602] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6338worker2 # [ 131.197871] systemd[1]: scheduler-token.service: Deactivated successfully.6339worker2 # [ 131.199171] systemd[1]: Finished scheduler-token.service.6340worker2 # [ 131.200528] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 70ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6341worker2 # [ 131.349813] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_connected addr=lb:500516342worker1 # [ 131.353637] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_connected addr=lb:500516343worker2 # [ 131.353722] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6344worker1 # [ 131.357489] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6345worker2 # [ 131.527306] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=scheduler_restarting6346worker1 # [ 131.546735] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_take_over6347lb # [ 131.531401] niks3-server[1039]: 2026/09/20 12:21:42 INFO lead: acquired remote=192.168.1.3:600426348worker2 # [ 131.549155] nix-daemon[925]: reaped child process 1489, status = succeeded6349worker1 # [ 131.558108] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_connected addr=lb:500516350worker1 # [ 131.562138] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6351worker2 # [ 131.563711] nix-grpc-daemon[1438]: ts=2026-09-20T12:21:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6352worker1 # [ 131.764182] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_connected addr=lb:500516353worker1 # [ 131.766647] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6354worker2 # [ 131.953790] nix-grpc-daemon[1438]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6355worker2 # [ 131.957366] nix-grpc-daemon[1438]: E0000 00:00:1789906902.322889 1481 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6356worker1 # [ 131.964666] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_connected addr=lb:500516357worker1 # [ 131.966515] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6358worker2 # [ 131.962285] nix-grpc-daemon[1438]: E0000 00:00:1789906902.323089 1481 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6359worker1 # [ 132.167207] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=scheduler_connected addr=lb:500516360worker1 # [ 132.169361] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6361worker2 # [ 132.328919] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6362worker2 # [ 132.331165] systemd[1]: nix-grpc-daemon.service: Consumed 624ms CPU time over 26.347s wall clock time, 31.1M memory peak, 25.8K incoming IP traffic, 36.5K outgoing IP traffic.6363worker2 # [ 132.340668] systemd[1]: Starting Nix worker-protocol over gRPC...6364worker1 # [ 132.370311] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=scheduler_connected addr=lb:500516365worker1 # [ 132.374300] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6366worker2 # [ 132.496900] nix-grpc-daemon[1785]: ts=2026-09-20T12:21:42Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6367worker2 # [ 132.498650] nix-grpc-daemon[1785]: ts=2026-09-20T12:21:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6368worker2 # [ 132.502474] nix-grpc-daemon[1785]: ts=2026-09-20T12:21:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16369worker2 # [ 132.509062] nix-grpc-daemon[1785]: ts=2026-09-20T12:21:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6370worker2 # [ 132.511726] systemd[1]: Started Nix worker-protocol over gRPC.6371worker2 # [ 132.523937] nix-grpc-daemon[1785]: ts=2026-09-20T12:21:42Z level=info event=scheduler_connected addr=lb:500516372worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.47 seconds)6373lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6374lb: must succeed: curl -sf localhost:9901/clusters6375worker1 # [ 132.542187] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=oidc_jwks provider=mock keys=16376worker1 # [ 132.545475] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:452306377worker1 # [ 132.550504] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06378lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6379worker1 # [ 132.574433] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=scheduler_connected addr=lb:500516380worker1 # [ 132.576145] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:452306381worker1 # [ 132.577584] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06382lb: must succeed: curl -sf localhost:9901/clusters6383lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6384lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6385lb: must succeed: curl -sf localhost:9901/clusters6386lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6387worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6388worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6389(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.38 seconds)6390subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6391client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16392worker1 # [ 133.956356] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:541766393worker2 # [ 133.962933] nix-grpc-daemon[1785]: ts=2026-09-20T12:21:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559606394worker2 # [ 133.975808] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6395worker2 # [ 133.993465] nix-grpc-daemon[1785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 1/5)6396worker2 # [ 134.086825] nix-grpc-daemon[1785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 108 ms (attempt 2/5)6397worker2 # [ 134.201390] nix-grpc-daemon[1785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 177 ms (attempt 3/5)6398worker2 # [ 134.378864] nix-grpc-daemon[1785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 4/5)6399worker2 # [ 134.537772] nix-grpc-daemon[1785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6400worker1 # [ 134.588844] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06401worker1 # [ 134.594301] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16402worker1 # [ 134.598546] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06403worker1 # [ 134.602376] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06404worker1 # [ 134.633299] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766405worker1 # [ 134.645376] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766406worker1 # [ 134.656963] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766407worker1 # [ 134.665925] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6408worker1 # [ 134.843334] nix-daemon[911]: accepted connection from pid 1819, user nix-grpc-daemon (trusted)6409worker1 # [ 134.855300] nix-daemon[911]: reaped child process 1822, status = succeeded6410worker1 # [ 135.023426] nix-daemon[911]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6411worker1 # [ 135.033894] nix-daemon[911]: reaped child process 1827, status = succeeded6412lb # [ 135.032439] niks3-server[1039]: 2026/09/20 12:21:45 INFO Received uploads request method=POST path=/api/pending_closures6413worker1 # [ 135.057651] nix-grpc-daemon[1633]: time=2026-09-20T12:21:45.939Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6414worker1 # [ 135.059476] nix-grpc-daemon[1633]: time=2026-09-20T12:21:45.941Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6415lb # [ 135.070220] niks3-server[1039]: 2026/09/20 12:21:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6416lb # [ 135.072985] niks3-server[1039]: 2026/09/20 12:21:45 INFO Signed narinfos id=46 count=16417worker1 # [ 135.090309] nix-grpc-daemon[1633]: time=2026-09-20T12:21:45.972Z level=INFO msg="Uploading 1 narinfos"6418lb # [ 135.086142] niks3-server[1039]: 2026/09/20 12:21:45 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6419lb # [ 135.087714] niks3-server[1039]: 2026/09/20 12:21:45 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6420lb # [ 135.093393] niks3-server[1039]: 2026/09/20 12:21:45 INFO Completed upload id=466421worker1 # [ 135.108729] nix-grpc-daemon[1633]: time=2026-09-20T12:21:45.990Z level=INFO msg="Upload complete. (430ms)"6422worker1 # [ 135.110983] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=5046423worker1 # [ 135.114148] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766424lb # [ 135.100390] niks3-server[1039]: 2026/09/20 12:21:45 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6425worker1 # [ 135.116307] nix-grpc-daemon[1633]: time=2026-09-20T12:21:45.998Z level=WARN msg="Failed to register uploaded object" key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo error="server returned 404: object is not pending upload\n"6426worker1 # [ 135.122963] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6427worker1 # [ 135.127237] nix-daemon[911]: reaped child process 1817, status = succeeded6428worker1 # [ 135.131807] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6429worker1 # [ 135.943955] nix-daemon[911]: reaped child process 1831, status = succeeded6430worker1 # [ 136.094304] nix-daemon[911]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6431worker1 # [ 136.104938] nix-daemon[911]: reaped child process 1849, status = succeeded6432lb # [ 136.105864] niks3-server[1039]: 2026/09/20 12:21:46 INFO Received uploads request method=POST path=/api/pending_closures6433worker1 # [ 136.134172] nix-grpc-daemon[1633]: time=2026-09-20T12:21:47.015Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6434worker1 # [ 136.136613] nix-grpc-daemon[1633]: time=2026-09-20T12:21:47.017Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6435lb # [ 136.147996] niks3-server[1039]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6436lb # [ 136.155171] niks3-server[1039]: 2026/09/20 12:21:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6437worker1 # [ 136.174504] nix-grpc-daemon[1633]: time=2026-09-20T12:21:47.056Z level=INFO msg="Uploading 1 narinfos"6438lb # [ 136.158773] niks3-server[1039]: 2026/09/20 12:21:47 INFO Signed narinfos id=47 count=16439lb # [ 136.163249] niks3-server[1039]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6440lb # [ 136.166022] niks3-server[1039]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6441lb # [ 136.170146] niks3-server[1039]: 2026/09/20 12:21:47 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6442lb # [ 136.174521] niks3-server[1039]: 2026/09/20 12:21:47 INFO Completed upload id=476443worker1 # [ 136.190224] nix-grpc-daemon[1633]: time=2026-09-20T12:21:47.072Z level=INFO msg="Upload complete. (252ms)"6444lb # [ 136.176333] niks3-server[1039]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6445worker1 # [ 136.192925] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16446worker1 # [ 136.198598] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:47Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16447worker1 # [ 136.201378] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766448worker1 # [ 136.206078] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:47Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06449worker1 # [ 136.210232] nix-daemon[911]: reaped child process 1829, status = succeeded6450worker1 # [ 136.212371] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766451worker1 # [ 136.218244] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766452worker1 # [ 136.223496] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6453worker1 # [ 136.380978] nix-daemon[911]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6454worker1 # [ 136.393638] nix-daemon[911]: reaped child process 1858, status = succeeded6455worker1 # [ 136.542122] nix-daemon[911]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)6456worker1 # [ 136.555129] nix-daemon[911]: reaped child process 1863, status = succeeded6457lb # [ 136.549783] niks3-server[1039]: 2026/09/20 12:21:47 INFO Received uploads request method=POST path=/api/pending_closures6458worker1 # [ 136.573708] nix-grpc-daemon[1633]: time=2026-09-20T12:21:47.455Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6459worker1 # [ 136.575238] nix-grpc-daemon[1633]: time=2026-09-20T12:21:47.457Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6460lb # [ 136.569133] niks3-server[1039]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6461lb # [ 136.577220] niks3-server[1039]: 2026/09/20 12:21:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6462worker1 # [ 136.596968] nix-grpc-daemon[1633]: time=2026-09-20T12:21:47.479Z level=INFO msg="Uploading 1 narinfos"6463lb # [ 136.581553] niks3-server[1039]: 2026/09/20 12:21:47 INFO Signed narinfos id=48 count=16464lb # [ 136.587075] niks3-server[1039]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6465lb # [ 136.595913] niks3-server[1039]: 2026/09/20 12:21:47 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6466lb # [ 136.600891] niks3-server[1039]: 2026/09/20 12:21:47 INFO Completed upload id=486467worker1 # [ 136.616623] nix-grpc-daemon[1633]: time=2026-09-20T12:21:47.498Z level=INFO msg="Upload complete. (387ms)"6468lb # [ 136.603339] niks3-server[1039]: 2026/09/20 12:21:47 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6469worker1 # [ 136.618717] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=5046470worker1 # [ 136.621387] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766471worker1 # [ 136.627631] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6472worker1 # [ 136.630928] nix-daemon[911]: reaped child process 1853, status = succeeded6473worker1 # [ 136.635564] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6474worker1 # [ 137.450770] nix-daemon[911]: reaped child process 1867, status = succeeded6475worker1 # [ 137.571615] nix-daemon[911]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6476worker1 # [ 137.584332] nix-daemon[911]: reaped child process 1885, status = succeeded6477lb # [ 137.577819] niks3-server[1039]: 2026/09/20 12:21:48 INFO Received uploads request method=POST path=/api/pending_closures6478worker1 # [ 137.597730] nix-grpc-daemon[1633]: time=2026-09-20T12:21:48.479Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6479worker1 # [ 137.599179] nix-grpc-daemon[1633]: time=2026-09-20T12:21:48.481Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6480lb # [ 137.601377] niks3-server[1039]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6481lb # [ 137.606623] niks3-server[1039]: 2026/09/20 12:21:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6482lb # [ 137.608903] niks3-server[1039]: 2026/09/20 12:21:48 INFO Signed narinfos id=49 count=16483lb # [ 137.610307] niks3-server[1039]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6484worker1 # [ 137.626185] nix-grpc-daemon[1633]: time=2026-09-20T12:21:48.508Z level=INFO msg="Uploading 1 narinfos"6485lb # [ 137.613671] niks3-server[1039]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6486lb # [ 137.617677] niks3-server[1039]: 2026/09/20 12:21:48 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6487lb # [ 137.621366] niks3-server[1039]: 2026/09/20 12:21:48 INFO Completed upload id=496488worker1 # [ 137.636904] nix-grpc-daemon[1633]: time=2026-09-20T12:21:48.518Z level=INFO msg="Upload complete. (195ms)"6489lb # [ 137.622890] niks3-server[1039]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6490worker1 # [ 137.639266] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16491worker1 # [ 137.643761] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:48Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16492worker1 # [ 137.646232] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766493worker1 # [ 137.649682] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:48Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06494worker1 # [ 137.654932] nix-daemon[911]: reaped child process 1865, status = succeeded6495worker1 # [ 137.686065] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766496worker1 # [ 137.693788] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766497worker1 # [ 137.698675] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6498worker1 # [ 137.837673] nix-daemon[911]: accepted connection from pid 1891, user nix-grpc-daemon (trusted)6499worker1 # [ 137.849801] nix-daemon[911]: reaped child process 1894, status = succeeded6500worker1 # [ 137.991148] nix-daemon[911]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)6501worker1 # [ 138.002959] nix-daemon[911]: reaped child process 1899, status = succeeded6502lb # [ 137.997958] niks3-server[1039]: 2026/09/20 12:21:48 INFO Received uploads request method=POST path=/api/pending_closures6503worker1 # [ 138.021719] nix-grpc-daemon[1633]: time=2026-09-20T12:21:48.903Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6504worker1 # [ 138.023120] nix-grpc-daemon[1633]: time=2026-09-20T12:21:48.903Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6505lb # [ 138.021914] niks3-server[1039]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6506lb # [ 138.033973] niks3-server[1039]: 2026/09/20 12:21:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6507lb # [ 138.039557] envoy[919]: conn peer=[2001:db8:1::1]:34630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9555 tx=3832 ms=55866508worker1 # [ 138.060278] nix-grpc-daemon[1633]: time=2026-09-20T12:21:48.942Z level=INFO msg="Uploading 1 narinfos"6509lb # [ 138.044644] envoy[919]: conn peer=[2001:db8:1::1]:47108 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2936 tx=1376 ms=126686510lb # [ 138.048391] envoy[919]: conn peer=[2001:db8:1::4]:35664 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4876 tx=568 ms=252476511lb # [ 138.051979] niks3-server[1039]: 2026/09/20 12:21:48 INFO Signed narinfos id=50 count=16512lb # [ 138.056285] niks3-server[1039]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6513lb # [ 138.065468] niks3-server[1039]: 2026/09/20 12:21:48 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6514lb # [ 138.070813] niks3-server[1039]: 2026/09/20 12:21:48 INFO Completed upload id=506515worker1 # [ 138.086214] nix-grpc-daemon[1633]: time=2026-09-20T12:21:48.968Z level=INFO msg="Upload complete. (382ms)"6516lb # [ 138.074742] niks3-server[1039]: 2026/09/20 12:21:48 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6517worker1 # [ 138.090232] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=5046518worker1 # [ 138.096318] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766519worker1 # [ 138.106106] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6520worker1 # [ 138.110474] nix-daemon[911]: reaped child process 1889, status = succeeded6521worker1 # [ 138.115393] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6522worker1 # [ 139.140224] nix-daemon[911]: reaped child process 1903, status = succeeded6523worker1 # [ 139.288719] nix-daemon[911]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)6524worker1 # [ 139.301421] nix-daemon[911]: reaped child process 1921, status = succeeded6525lb # [ 139.293815] niks3-server[1039]: 2026/09/20 12:21:50 INFO Received uploads request method=POST path=/api/pending_closures6526worker1 # [ 139.315360] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.197Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6527worker1 # [ 139.317143] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.198Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6528lb # [ 139.314942] niks3-server[1039]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6529lb # [ 139.317251] niks3-server[1039]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6530lb # [ 139.321939] niks3-server[1039]: 2026/09/20 12:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6531lb # [ 139.324192] niks3-server[1039]: 2026/09/20 12:21:50 INFO Signed narinfos id=51 count=16532worker1 # [ 139.339904] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.221Z level=INFO msg="Uploading 1 narinfos"6533lb # [ 139.326459] niks3-server[1039]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6534lb # [ 139.332388] niks3-server[1039]: 2026/09/20 12:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6535lb # [ 139.335314] niks3-server[1039]: 2026/09/20 12:21:50 INFO Completed upload id=516536worker1 # [ 139.352217] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.234Z level=INFO msg="Upload complete. (230ms)"6537lb # [ 139.336766] niks3-server[1039]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6538worker1 # [ 139.356660] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16539worker1 # [ 139.360134] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:50Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16540worker1 # [ 139.362292] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766541worker1 # [ 139.364607] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:50Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06542worker1 # [ 139.370778] nix-daemon[911]: reaped child process 1901, status = succeeded6543worker1 # [ 139.372134] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766544worker1 # [ 139.377378] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.259Z level=INFO msg="All 1 paths already cached"6545worker1 # [ 139.380891] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766546worker1 # [ 139.384882] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6547worker1 # [ 139.551848] nix-daemon[911]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6548worker1 # [ 139.565720] nix-daemon[911]: reaped child process 1930, status = succeeded6549worker1 # [ 139.716256] nix-daemon[911]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6550worker1 # [ 139.728993] nix-daemon[911]: reaped child process 1935, status = succeeded6551lb # [ 139.728083] niks3-server[1039]: 2026/09/20 12:21:50 INFO Received uploads request method=POST path=/api/pending_closures6552worker1 # [ 139.751550] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.633Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6553worker1 # [ 139.753440] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.635Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6554lb # [ 139.752797] niks3-server[1039]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6555lb # [ 139.764684] niks3-server[1039]: 2026/09/20 12:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6556lb # [ 139.771774] niks3-server[1039]: 2026/09/20 12:21:50 INFO Signed narinfos id=52 count=16557worker1 # [ 139.790492] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.672Z level=INFO msg="Uploading 1 narinfos"6558lb # [ 139.779574] niks3-server[1039]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6559lb # [ 139.788558] niks3-server[1039]: 2026/09/20 12:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6560lb # [ 139.800887] niks3-server[1039]: 2026/09/20 12:21:50 INFO Completed upload id=526561worker1 # [ 139.816600] nix-grpc-daemon[1633]: time=2026-09-20T12:21:50.698Z level=INFO msg="Upload complete. (425ms)"6562worker1 # [ 139.821277] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=6246563lb # [ 139.809312] niks3-server[1039]: 2026/09/20 12:21:50 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6564worker1 # [ 139.826271] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766565worker1 # [ 139.836065] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6566worker1 # [ 139.839895] nix-daemon[911]: reaped child process 1925, status = succeeded6567worker1 # [ 139.845088] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6568worker1 # [ 141.071885] nix-daemon[911]: reaped child process 1939, status = succeeded6569worker1 # [ 141.227614] nix-daemon[911]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)6570worker1 # [ 141.240171] nix-daemon[911]: reaped child process 1957, status = succeeded6571lb # [ 141.234877] niks3-server[1039]: 2026/09/20 12:21:52 INFO Received uploads request method=POST path=/api/pending_closures6572worker1 # [ 141.259327] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.141Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6573worker1 # [ 141.262059] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.143Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6574lb # [ 141.278231] niks3-server[1039]: 2026/09/20 12:21:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6575worker1 # [ 141.304307] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.186Z level=INFO msg="Uploading 1 narinfos"6576lb # [ 141.290124] niks3-server[1039]: 2026/09/20 12:21:52 INFO Signed narinfos id=53 count=16577lb # [ 141.296784] niks3-server[1039]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6578lb # [ 141.303854] niks3-server[1039]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6579lb # [ 141.307582] niks3-server[1039]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6580lb # [ 141.313199] niks3-server[1039]: 2026/09/20 12:21:52 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6581lb # [ 141.317044] niks3-server[1039]: 2026/09/20 12:21:52 INFO Completed upload id=536582lb # [ 141.318940] niks3-server[1039]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6583worker1 # [ 141.334188] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.216Z level=INFO msg="Upload complete. (280ms)"6584worker1 # [ 141.345877] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16585worker1 # [ 141.352284] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:52Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16586worker1 # [ 141.357254] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766587worker1 # [ 141.360904] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:52Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06588worker1 # [ 141.368229] nix-daemon[911]: reaped child process 1937, status = succeeded6589worker1 # [ 141.369780] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766590worker1 # [ 141.375633] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.257Z level=INFO msg="All 1 paths already cached"6591worker1 # [ 141.379583] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766592worker1 # [ 141.383527] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6593worker1 # [ 141.522866] nix-daemon[911]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6594worker1 # [ 141.536073] nix-daemon[911]: reaped child process 1966, status = succeeded6595worker1 # [ 141.663558] nix-daemon[911]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)6596worker1 # [ 141.675707] nix-daemon[911]: reaped child process 1971, status = succeeded6597lb # [ 141.670437] niks3-server[1039]: 2026/09/20 12:21:52 INFO Received uploads request method=POST path=/api/pending_closures6598worker1 # [ 141.693789] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.575Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6599worker1 # [ 141.695251] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.577Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6600lb # [ 141.699843] niks3-server[1039]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6601lb # [ 141.708620] niks3-server[1039]: 2026/09/20 12:21:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6602lb # [ 141.715356] niks3-server[1039]: 2026/09/20 12:21:52 INFO Signed narinfos id=54 count=16603worker1 # [ 141.734219] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.616Z level=INFO msg="Uploading 1 narinfos"6604lb # [ 141.721523] niks3-server[1039]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6605lb # [ 141.730666] niks3-server[1039]: 2026/09/20 12:21:52 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6606lb # [ 141.738728] niks3-server[1039]: 2026/09/20 12:21:52 INFO Completed upload id=546607worker1 # [ 141.754611] nix-grpc-daemon[1633]: time=2026-09-20T12:21:52.636Z level=INFO msg="Upload complete. (365ms)"6608lb # [ 141.741349] niks3-server[1039]: 2026/09/20 12:21:52 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6609worker1 # [ 141.759135] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=6246610worker1 # [ 141.764457] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766611worker1 # [ 141.774907] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6612worker1 # [ 141.781675] nix-daemon[911]: reaped child process 1961, status = succeeded6613worker1 # [ 141.785333] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6614worker1 # [ 142.814895] nix-daemon[911]: reaped child process 1975, status = succeeded6615worker1 # [ 142.935173] nix-daemon[911]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6616worker1 # [ 142.947463] nix-daemon[911]: reaped child process 1993, status = succeeded6617lb # [ 142.944908] niks3-server[1039]: 2026/09/20 12:21:53 INFO Received uploads request method=POST path=/api/pending_closures6618worker1 # [ 142.965165] nix-grpc-daemon[1633]: time=2026-09-20T12:21:53.846Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6619worker1 # [ 142.966740] nix-grpc-daemon[1633]: time=2026-09-20T12:21:53.848Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6620lb # [ 142.971609] niks3-server[1039]: 2026/09/20 12:21:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6621lb # [ 142.977433] niks3-server[1039]: 2026/09/20 12:21:53 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6622lb # [ 142.982345] niks3-server[1039]: 2026/09/20 12:21:53 INFO Signed narinfos id=55 count=16623worker1 # [ 143.000522] nix-grpc-daemon[1633]: time=2026-09-20T12:21:53.882Z level=INFO msg="Uploading 1 narinfos"6624lb # [ 142.990893] niks3-server[1039]: 2026/09/20 12:21:53 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6625lb # [ 142.993029] niks3-server[1039]: 2026/09/20 12:21:53 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6626lb # [ 143.004932] niks3-server[1039]: 2026/09/20 12:21:53 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6627lb # [ 143.018673] niks3-server[1039]: 2026/09/20 12:21:53 INFO Completed upload id=556628worker1 # [ 143.034428] nix-grpc-daemon[1633]: time=2026-09-20T12:21:53.916Z level=INFO msg="Upload complete. (239ms)"6629worker1 # [ 143.041978] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16630lb # [ 143.031855] niks3-server[1039]: 2026/09/20 12:21:53 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6631worker1 # [ 143.047965] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:53Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16632worker1 # [ 143.052234] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766633worker1 # [ 143.056147] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:53Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06634worker1 # [ 143.065691] nix-daemon[911]: reaped child process 1973, status = succeeded6635worker1 # [ 143.068262] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766636worker1 # [ 143.086297] nix-grpc-daemon[1633]: time=2026-09-20T12:21:53.968Z level=INFO msg="All 1 paths already cached"6637worker1 # [ 143.090265] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766638worker1 # [ 143.095905] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6639worker1 # [ 143.244951] nix-daemon[911]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)6640worker1 # [ 143.256958] nix-daemon[911]: reaped child process 2002, status = succeeded6641worker1 # [ 143.414322] nix-daemon[911]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)6642worker1 # [ 143.426919] nix-daemon[911]: reaped child process 2007, status = succeeded6643lb # [ 143.421430] niks3-server[1039]: 2026/09/20 12:21:54 INFO Received uploads request method=POST path=/api/pending_closures6644worker1 # [ 143.444378] nix-grpc-daemon[1633]: time=2026-09-20T12:21:54.326Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6645worker1 # [ 143.445703] nix-grpc-daemon[1633]: time=2026-09-20T12:21:54.326Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6646lb # [ 143.444548] niks3-server[1039]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6647lb # [ 143.456734] niks3-server[1039]: 2026/09/20 12:21:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6648lb # [ 143.462078] niks3-server[1039]: 2026/09/20 12:21:54 INFO Signed narinfos id=56 count=16649worker1 # [ 143.480860] nix-grpc-daemon[1633]: time=2026-09-20T12:21:54.362Z level=INFO msg="Uploading 1 narinfos"6650lb # [ 143.468565] niks3-server[1039]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6651lb # [ 143.480080] niks3-server[1039]: 2026/09/20 12:21:54 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6652worker1 # [ 143.502125] nix-grpc-daemon[1633]: time=2026-09-20T12:21:54.384Z level=INFO msg="Upload complete. (400ms)"6653lb # [ 143.486711] niks3-server[1039]: 2026/09/20 12:21:54 INFO Completed upload id=566654lb # [ 143.490216] niks3-server[1039]: 2026/09/20 12:21:54 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6655worker1 # [ 143.505295] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=6246656worker1 # [ 143.510186] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766657worker1 # [ 143.518325] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6658worker1 # [ 143.523478] nix-daemon[911]: reaped child process 1997, status = succeeded6659worker1 # [ 143.528457] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6660worker1 # [ 144.543562] nix-daemon[911]: reaped child process 2011, status = succeeded6661worker1 # [ 144.663490] nix-daemon[911]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6662worker1 # [ 144.675852] nix-daemon[911]: reaped child process 2029, status = succeeded6663lb # [ 144.670780] niks3-server[1039]: 2026/09/20 12:21:55 INFO Received uploads request method=POST path=/api/pending_closures6664worker1 # [ 144.695998] nix-grpc-daemon[1633]: time=2026-09-20T12:21:55.577Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6665worker1 # [ 144.697465] nix-grpc-daemon[1633]: time=2026-09-20T12:21:55.579Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6666lb # [ 144.703523] niks3-server[1039]: 2026/09/20 12:21:55 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6667lb # [ 144.716698] niks3-server[1039]: 2026/09/20 12:21:55 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6668lb # [ 144.729912] niks3-server[1039]: 2026/09/20 12:21:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6669lb # [ 144.734859] niks3-server[1039]: 2026/09/20 12:21:55 INFO Signed narinfos id=57 count=16670worker1 # [ 144.753890] nix-grpc-daemon[1633]: time=2026-09-20T12:21:55.635Z level=INFO msg="Uploading 1 narinfos"6671lb # [ 144.739273] niks3-server[1039]: 2026/09/20 12:21:55 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6672lb # [ 144.749761] niks3-server[1039]: 2026/09/20 12:21:55 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6673lb # [ 144.753510] niks3-server[1039]: 2026/09/20 12:21:55 INFO Completed upload id=576674lb # [ 144.755126] niks3-server[1039]: 2026/09/20 12:21:55 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6675worker1 # [ 144.770495] nix-grpc-daemon[1633]: time=2026-09-20T12:21:55.652Z level=INFO msg="Upload complete. (235ms)"6676worker1 # [ 144.777762] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16677worker1 # [ 144.787203] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:55Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16678worker1 # [ 144.792229] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766679worker1 # [ 144.795292] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:55Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06680worker1 # [ 144.805369] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766681worker1 # [ 144.807657] nix-daemon[911]: reaped child process 2009, status = succeeded6682worker1 # [ 144.813307] nix-grpc-daemon[1633]: time=2026-09-20T12:21:55.695Z level=INFO msg="All 2 paths already cached"6683worker1 # [ 144.817711] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766684worker1 # [ 144.821567] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6685worker1 # [ 144.960224] nix-daemon[911]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)6686worker1 # [ 144.972976] nix-daemon[911]: reaped child process 2038, status = succeeded6687worker1 # [ 145.115075] nix-daemon[911]: accepted connection from pid 2040, user nix-grpc-daemon (trusted)6688worker1 # [ 145.128908] nix-daemon[911]: reaped child process 2043, status = succeeded6689lb # [ 145.120806] niks3-server[1039]: 2026/09/20 12:21:56 INFO Received uploads request method=POST path=/api/pending_closures6690worker1 # [ 145.140712] nix-grpc-daemon[1633]: time=2026-09-20T12:21:56.022Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6691worker1 # [ 145.142090] nix-grpc-daemon[1633]: time=2026-09-20T12:21:56.022Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6692lb # [ 145.132896] niks3-server[1039]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6693lb # [ 145.136184] niks3-server[1039]: 2026/09/20 12:21:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6694lb # [ 145.139236] niks3-server[1039]: 2026/09/20 12:21:56 INFO Signed narinfos id=58 count=16695worker1 # [ 145.155161] nix-grpc-daemon[1633]: time=2026-09-20T12:21:56.037Z level=INFO msg="Uploading 1 narinfos"6696lb # [ 145.142909] niks3-server[1039]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6697lb # [ 145.146238] niks3-server[1039]: 2026/09/20 12:21:56 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6698lb # [ 145.150071] niks3-server[1039]: 2026/09/20 12:21:56 INFO Completed upload id=586699worker1 # [ 145.165746] nix-grpc-daemon[1633]: time=2026-09-20T12:21:56.047Z level=INFO msg="Upload complete. (338ms)"6700lb # [ 145.151524] niks3-server[1039]: 2026/09/20 12:21:56 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6701worker1 # [ 145.167728] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=7526702worker1 # [ 145.170796] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766703worker1 # [ 145.176834] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6704worker1 # [ 145.180052] nix-daemon[911]: reaped child process 2033, status = succeeded6705worker1 # [ 145.184883] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6706worker1 # [ 146.410831] nix-daemon[911]: reaped child process 2047, status = succeeded6707worker1 # [ 146.565093] nix-daemon[911]: accepted connection from pid 2062, user nix-grpc-daemon (trusted)6708worker1 # [ 146.576570] nix-daemon[911]: reaped child process 2065, status = succeeded6709lb # [ 146.572105] niks3-server[1039]: 2026/09/20 12:21:57 INFO Received uploads request method=POST path=/api/pending_closures6710worker1 # [ 146.597473] nix-grpc-daemon[1633]: time=2026-09-20T12:21:57.479Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6711worker1 # [ 146.599918] nix-grpc-daemon[1633]: time=2026-09-20T12:21:57.481Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6712lb # [ 146.604188] niks3-server[1039]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6713lb # [ 146.616443] niks3-server[1039]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6714lb # [ 146.631304] niks3-server[1039]: 2026/09/20 12:21:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6715lb # [ 146.635735] niks3-server[1039]: 2026/09/20 12:21:57 INFO Signed narinfos id=59 count=16716worker1 # [ 146.651728] nix-grpc-daemon[1633]: time=2026-09-20T12:21:57.533Z level=INFO msg="Uploading 1 narinfos"6717lb # [ 146.641434] niks3-server[1039]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6718lb # [ 146.649250] niks3-server[1039]: 2026/09/20 12:21:57 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6719lb # [ 146.653081] niks3-server[1039]: 2026/09/20 12:21:57 INFO Completed upload id=596720lb # [ 146.654640] niks3-server[1039]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6721worker1 # [ 146.668880] nix-grpc-daemon[1633]: time=2026-09-20T12:21:57.550Z level=INFO msg="Upload complete. (274ms)"6722worker1 # [ 146.677588] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16723worker1 # [ 146.684333] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:57Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16724worker1 # [ 146.693408] nix-daemon[911]: reaped child process 2045, status = succeeded6725worker1 # [ 146.718172] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766726worker1 # [ 146.728640] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766727worker1 # [ 146.739707] nix-grpc-daemon[1633]: time=2026-09-20T12:21:57.621Z level=INFO msg="All 2 paths already cached"6728worker1 # [ 146.746592] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766729worker1 # [ 146.756142] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6730worker1 # [ 146.905957] nix-daemon[911]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6731worker1 # [ 146.918315] nix-daemon[911]: reaped child process 2072, status = succeeded6732worker1 # [ 147.048733] nix-daemon[911]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6733worker1 # [ 147.061711] nix-daemon[911]: reaped child process 2077, status = succeeded6734lb # [ 147.058497] niks3-server[1039]: 2026/09/20 12:21:57 INFO Received uploads request method=POST path=/api/pending_closures6735worker1 # [ 147.081841] nix-grpc-daemon[1633]: time=2026-09-20T12:21:57.963Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6736worker1 # [ 147.083206] nix-grpc-daemon[1633]: time=2026-09-20T12:21:57.963Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6737lb # [ 147.086204] niks3-server[1039]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6738lb # [ 147.093705] niks3-server[1039]: 2026/09/20 12:21:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6739worker1 # [ 147.114036] nix-grpc-daemon[1633]: time=2026-09-20T12:21:57.995Z level=INFO msg="Uploading 1 narinfos"6740lb # [ 147.098625] niks3-server[1039]: 2026/09/20 12:21:57 INFO Signed narinfos id=60 count=16741lb # [ 147.105575] niks3-server[1039]: 2026/09/20 12:21:57 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6742lb # [ 147.116225] niks3-server[1039]: 2026/09/20 12:21:58 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6743lb # [ 147.122557] niks3-server[1039]: 2026/09/20 12:21:58 INFO Completed upload id=606744worker1 # [ 147.140662] nix-grpc-daemon[1633]: time=2026-09-20T12:21:58.022Z level=INFO msg="Upload complete. (374ms)"6745worker1 # [ 147.142681] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=7526746lb # [ 147.128096] niks3-server[1039]: 2026/09/20 12:21:58 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6747worker1 # [ 147.148716] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766748worker1 # [ 147.157558] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6749worker1 # [ 147.160988] nix-daemon[911]: reaped child process 2067, status = succeeded6750worker1 # [ 147.166317] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6751worker1 # [ 148.389575] nix-daemon[911]: reaped child process 2081, status = succeeded6752worker1 # [ 148.535279] nix-daemon[911]: accepted connection from pid 2096, user nix-grpc-daemon (trusted)6753worker1 # [ 148.547467] nix-daemon[911]: reaped child process 2099, status = succeeded6754lb # [ 148.545785] niks3-server[1039]: 2026/09/20 12:21:59 INFO Received uploads request method=POST path=/api/pending_closures6755worker1 # [ 148.566191] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.447Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6756worker1 # [ 148.567785] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.449Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6757lb # [ 148.567506] niks3-server[1039]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6758lb # [ 148.571763] niks3-server[1039]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6759lb # [ 148.575877] niks3-server[1039]: 2026/09/20 12:21:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6760lb # [ 148.578295] niks3-server[1039]: 2026/09/20 12:21:59 INFO Signed narinfos id=61 count=16761worker1 # [ 148.594199] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.476Z level=INFO msg="Uploading 1 narinfos"6762lb # [ 148.581660] niks3-server[1039]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6763lb # [ 148.586061] niks3-server[1039]: 2026/09/20 12:21:59 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6764lb # [ 148.590177] niks3-server[1039]: 2026/09/20 12:21:59 INFO Completed upload id=616765worker1 # [ 148.605683] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.487Z level=INFO msg="Upload complete. (230ms)"6766lb # [ 148.592584] niks3-server[1039]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6767worker1 # [ 148.609629] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16768worker1 # [ 148.612445] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:59Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16769worker1 # [ 148.614660] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766770worker1 # [ 148.620284] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766771worker1 # [ 148.623124] nix-daemon[911]: reaped child process 2079, status = succeeded6772worker1 # [ 148.626676] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.508Z level=INFO msg="All 2 paths already cached"6773worker1 # [ 148.629965] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:541766774worker1 # [ 148.633755] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6775worker1 # [ 148.777551] nix-daemon[911]: accepted connection from pid 2103, user nix-grpc-daemon (trusted)6776worker1 # [ 148.790760] nix-daemon[911]: reaped child process 2106, status = succeeded6777worker1 # [ 148.958441] nix-daemon[911]: accepted connection from pid 2108, user nix-grpc-daemon (trusted)6778worker1 # [ 148.969595] nix-daemon[911]: reaped child process 2111, status = succeeded6779lb # [ 148.970251] niks3-server[1039]: 2026/09/20 12:21:59 INFO Received uploads request method=POST path=/api/pending_closures6780worker1 # [ 148.993568] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.875Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6781worker1 # [ 148.995182] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.877Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6782lb # [ 149.001182] niks3-server[1039]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6783lb # [ 149.008677] niks3-server[1039]: 2026/09/20 12:21:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6784worker1 # [ 149.030117] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.912Z level=INFO msg="Uploading 1 narinfos"6785lb # [ 149.014953] niks3-server[1039]: 2026/09/20 12:21:59 INFO Signed narinfos id=62 count=16786lb # [ 149.022648] niks3-server[1039]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6787lb # [ 149.033845] niks3-server[1039]: 2026/09/20 12:21:59 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6788worker1 # [ 149.056474] nix-grpc-daemon[1633]: time=2026-09-20T12:21:59.938Z level=INFO msg="Upload complete. (417ms)"6789lb # [ 149.041100] niks3-server[1039]: 2026/09/20 12:21:59 INFO Completed upload id=626790lb # [ 149.043672] niks3-server[1039]: 2026/09/20 12:21:59 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6791worker1 # [ 149.058960] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=0 paths=1 nar_bytes_in=7526792worker1 # [ 149.064294] nix-grpc-daemon[1541]: ts=2026-09-20T12:21:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:541766793worker1 # [ 149.074509] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6794worker1 # [ 149.079399] nix-daemon[911]: reaped child process 2101, status = succeeded6795worker1 # [ 149.084890] nix-daemon[911]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6796worker1 # [ 150.299066] nix-daemon[911]: reaped child process 2115, status = succeeded6797worker1 # [ 150.442501] nix-daemon[911]: accepted connection from pid 2130, user nix-grpc-daemon (trusted)6798worker1 # [ 150.455636] nix-daemon[911]: reaped child process 2133, status = succeeded6799lb # [ 150.449068] niks3-server[1039]: 2026/09/20 12:22:01 INFO Received uploads request method=POST path=/api/pending_closures6800worker1 # [ 150.473466] nix-grpc-daemon[1633]: time=2026-09-20T12:22:01.355Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6801worker1 # [ 150.474961] nix-grpc-daemon[1633]: time=2026-09-20T12:22:01.356Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6802lb # [ 150.486769] niks3-server[1039]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6803lb # [ 150.494698] niks3-server[1039]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6804lb # [ 150.504831] niks3-server[1039]: 2026/09/20 12:22:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6805lb # [ 150.510769] niks3-server[1039]: 2026/09/20 12:22:01 INFO Signed narinfos id=63 count=16806worker1 # [ 150.529561] nix-grpc-daemon[1633]: time=2026-09-20T12:22:01.411Z level=INFO msg="Uploading 1 narinfos"6807lb # [ 150.513734] niks3-server[1039]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6808lb # [ 150.524775] niks3-server[1039]: 2026/09/20 12:22:01 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6809lb # [ 150.529574] niks3-server[1039]: 2026/09/20 12:22:01 INFO Completed upload id=636810worker1 # [ 150.545429] nix-grpc-daemon[1633]: time=2026-09-20T12:22:01.427Z level=INFO msg="Upload complete. (253ms)"6811lb # [ 150.531923] niks3-server[1039]: 2026/09/20 12:22:01 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6812worker1 # [ 150.553242] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54176 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16813worker1 # [ 150.566545] nix-daemon[911]: reaped child process 2113, status = succeeded6814client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.86 seconds)6815worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6816worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6817client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16818worker1 # [ 150.830163] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:541766819worker1 # [ 150.834752] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:541766820worker1 # [ 150.845727] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:01Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06821lb # [ 158.039309] envoy[919]: conn peer=[2001:db8:1::1]:45672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27085 tx=10656 ms=166186822client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.22 seconds)6823(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.14 seconds)6824subtest: low disk drains a worker and builds go to the other6825worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6826worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.98 seconds)6827worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6828worker1 # [ 182.571187] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:33Z level=info event=unhealthy reason=min_free6829worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6830client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26831worker1 # [ 183.167838] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:541766832client # don't know how to build these paths:6833client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6834worker2 # [ 183.175078] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559606835worker1 # [ 183.202261] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:34Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06836worker1 # [ 183.206379] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:34Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16837worker2 # [ 183.252241] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559606838worker2 # [ 183.265381] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559606839client # copying 1 paths to 'lb:50051'...6840client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6841worker2 # [ 183.277814] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559606842worker2 # [ 183.290458] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6843worker2 # [ 183.309939] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:33Z level=info event=niks3_push_started pid=18416844worker2 # [ 183.503231] nix-grpc-daemon[1841]: time=2026-09-20T12:22:33.871Z level=INFO msg="Configuring client TLS" cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.pem key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/worker2.key ca=""6845worker2 # [ 183.663824] nix-daemon[925]: accepted connection from pid 1854, user nix-grpc-daemon (trusted)6846worker2 # [ 183.677105] nix-daemon[925]: reaped child process 1857, status = succeeded6847worker2 # [ 183.828814] nix-daemon[925]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)6848worker2 # [ 183.841606] nix-daemon[925]: reaped child process 1862, status = succeeded6849lb # [ 183.844677] niks3-server[1039]: 2026/09/20 12:22:34 INFO Received uploads request method=POST path=/api/pending_closures6850worker2 # [ 183.866992] nix-grpc-daemon[1841]: time=2026-09-20T12:22:34.236Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6851worker2 # [ 183.868542] nix-grpc-daemon[1841]: time=2026-09-20T12:22:34.237Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6852lb # [ 183.882567] niks3-server[1039]: 2026/09/20 12:22:34 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6853lb # [ 183.894871] niks3-server[1039]: 2026/09/20 12:22:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6854lb # [ 183.899827] niks3-server[1039]: 2026/09/20 12:22:34 INFO Signed narinfos id=64 count=16855worker2 # [ 183.914794] nix-grpc-daemon[1841]: time=2026-09-20T12:22:34.284Z level=INFO msg="Uploading 1 narinfos"6856lb # [ 183.908654] niks3-server[1039]: 2026/09/20 12:22:34 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6857lb # [ 183.918935] niks3-server[1039]: 2026/09/20 12:22:34 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6858lb # [ 183.926585] niks3-server[1039]: 2026/09/20 12:22:34 INFO Completed upload id=646859worker2 # [ 183.938992] nix-grpc-daemon[1841]: time=2026-09-20T12:22:34.308Z level=INFO msg="Upload complete. (432ms)"6860lb # [ 183.928601] niks3-server[1039]: 2026/09/20 12:22:34 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6861worker2 # [ 183.942904] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55960 duration_s=0 paths=1 nar_bytes_in=5126862worker2 # [ 183.948421] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559606863worker2 # [ 183.959106] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6864worker2 # [ 183.963303] nix-daemon[925]: reaped child process 1839, status = succeeded6865worker2 # [ 183.968404] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6866client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6867client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6868client # farm-a> LOG-farm-a-drain6869worker2 # [ 184.784097] nix-daemon[925]: reaped child process 1866, status = succeeded6870worker2 # [ 184.930424] nix-daemon[925]: accepted connection from pid 1881, user nix-grpc-daemon (trusted)6871worker2 # [ 184.942675] nix-daemon[925]: reaped child process 1884, status = succeeded6872lb # [ 184.946784] niks3-server[1039]: 2026/09/20 12:22:35 INFO Received uploads request method=POST path=/api/pending_closures6873worker2 # [ 184.968883] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.338Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6874worker2 # [ 184.970407] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.339Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6875lb # [ 184.986533] niks3-server[1039]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6876lb # [ 184.992489] niks3-server[1039]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6877lb # [ 185.001901] niks3-server[1039]: 2026/09/20 12:22:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6878worker2 # [ 185.017607] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.386Z level=INFO msg="Uploading 1 narinfos"6879lb # [ 185.005546] niks3-server[1039]: 2026/09/20 12:22:35 INFO Signed narinfos id=65 count=16880lb # [ 185.013782] niks3-server[1039]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6881lb # [ 185.017729] niks3-server[1039]: 2026/09/20 12:22:35 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6882lb # [ 185.020831] niks3-server[1039]: 2026/09/20 12:22:35 INFO Completed upload id=656883worker2 # [ 185.034381] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.403Z level=INFO msg="Upload complete. (259ms)"6884lb # [ 185.023172] niks3-server[1039]: 2026/09/20 12:22:35 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6885worker2 # [ 185.037671] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55960 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16886worker2 # [ 185.047885] nix-daemon[925]: reaped child process 1864, status = succeeded6887worker1 # [ 185.048827] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:35Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06888worker1 # [ 185.053273] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:35Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16889worker2 # [ 185.088959] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559606890worker2 # [ 185.101350] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559606891worker2 # [ 185.115477] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.484Z level=INFO msg="All 1 paths already cached"6892client # copying 1 paths to 'lb:50051'...6893client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6894worker2 # [ 185.123885] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559606895worker2 # [ 185.132922] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6896worker2 # [ 185.311765] nix-daemon[925]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6897worker2 # [ 185.328746] nix-daemon[925]: reaped child process 1891, status = succeeded6898worker2 # [ 185.490239] nix-daemon[925]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)6899worker2 # [ 185.503810] nix-daemon[925]: reaped child process 1896, status = succeeded6900lb # [ 185.505932] niks3-server[1039]: 2026/09/20 12:22:36 INFO Received uploads request method=POST path=/api/pending_closures6901worker2 # [ 185.530090] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.899Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6902worker2 # [ 185.531826] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.899Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6903lb # [ 185.538357] niks3-server[1039]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6904lb # [ 185.548101] niks3-server[1039]: 2026/09/20 12:22:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6905lb # [ 185.552320] niks3-server[1039]: 2026/09/20 12:22:36 INFO Signed narinfos id=66 count=16906worker2 # [ 185.567556] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.936Z level=INFO msg="Uploading 1 narinfos"6907lb # [ 185.561185] niks3-server[1039]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6908lb # [ 185.566864] niks3-server[1039]: 2026/09/20 12:22:36 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6909lb # [ 185.570073] niks3-server[1039]: 2026/09/20 12:22:36 INFO Completed upload id=666910worker2 # [ 185.583689] nix-grpc-daemon[1841]: time=2026-09-20T12:22:35.953Z level=INFO msg="Upload complete. (438ms)"6911lb # [ 185.573126] niks3-server[1039]: 2026/09/20 12:22:36 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6912worker2 # [ 185.586261] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55960 duration_s=0 paths=1 nar_bytes_in=6246913worker2 # [ 185.589609] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559606914worker2 # [ 185.598732] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6915worker2 # [ 185.602786] nix-daemon[925]: reaped child process 1886, status = succeeded6916worker2 # [ 185.608457] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6917client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6918client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6919client # farm-b> LOG-farm-b-drain6920worker2 # [ 186.637615] nix-daemon[925]: reaped child process 1900, status = succeeded6921worker2 # [ 186.801111] nix-daemon[925]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6922worker2 # [ 186.814533] nix-daemon[925]: reaped child process 1918, status = succeeded6923lb # [ 186.817655] niks3-server[1039]: 2026/09/20 12:22:37 INFO Received uploads request method=POST path=/api/pending_closures6924worker2 # [ 187.084176] nix-grpc-daemon[1841]: time=2026-09-20T12:22:37.453Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6925worker2 # [ 187.087127] nix-grpc-daemon[1841]: time=2026-09-20T12:22:37.455Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6926lb # [ 187.097483] niks3-server[1039]: 2026/09/20 12:22:37 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6927lb # [ 187.106872] niks3-server[1039]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6928lb # [ 187.116507] niks3-server[1039]: 2026/09/20 12:22:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6929worker2 # [ 187.135360] nix-grpc-daemon[1841]: time=2026-09-20T12:22:37.504Z level=INFO msg="Uploading 1 narinfos"6930lb # [ 187.122697] niks3-server[1039]: 2026/09/20 12:22:38 INFO Signed narinfos id=67 count=16931lb # [ 187.133810] niks3-server[1039]: 2026/09/20 12:22:38 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6932lb # [ 187.144981] niks3-server[1039]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6933lb # [ 187.150084] niks3-server[1039]: 2026/09/20 12:22:38 INFO Completed upload id=676934worker2 # [ 187.163603] nix-grpc-daemon[1841]: time=2026-09-20T12:22:37.532Z level=INFO msg="Upload complete. (547ms)"6935lb # [ 187.154568] niks3-server[1039]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6936worker2 # [ 187.167224] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55960 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16937worker2 # [ 187.173220] nix-daemon[925]: reaped child process 1898, status = succeeded6938worker1 # [ 187.178746] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:38Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06939worker1 # [ 187.182849] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:38Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16940worker2 # [ 187.218719] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559606941worker2 # [ 187.227397] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559606942worker2 # [ 187.236200] nix-grpc-daemon[1841]: time=2026-09-20T12:22:37.605Z level=INFO msg="All 2 paths already cached"6943client # copying 1 paths to 'lb:50051'...6944client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6945worker2 # [ 187.241880] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559606946worker2 # [ 187.248577] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6947worker2 # [ 187.405776] nix-daemon[925]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6948worker2 # [ 187.420221] nix-daemon[925]: reaped child process 1925, status = succeeded6949worker2 # [ 187.579379] nix-daemon[925]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6950worker2 # [ 187.591487] nix-daemon[925]: reaped child process 1930, status = succeeded6951lb # [ 187.589259] niks3-server[1039]: 2026/09/20 12:22:38 INFO Received uploads request method=POST path=/api/pending_closures6952worker2 # [ 187.606194] nix-grpc-daemon[1841]: time=2026-09-20T12:22:37.975Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6953worker2 # [ 187.607547] nix-grpc-daemon[1841]: time=2026-09-20T12:22:37.975Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6954lb # [ 187.601910] niks3-server[1039]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6955lb # [ 187.606427] niks3-server[1039]: 2026/09/20 12:22:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6956lb # [ 187.608358] niks3-server[1039]: 2026/09/20 12:22:38 INFO Signed narinfos id=68 count=16957worker2 # [ 187.622226] nix-grpc-daemon[1841]: time=2026-09-20T12:22:37.991Z level=INFO msg="Uploading 1 narinfos"6958lb # [ 187.613056] niks3-server[1039]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6959lb # [ 187.618204] niks3-server[1039]: 2026/09/20 12:22:38 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6960lb # [ 187.621399] niks3-server[1039]: 2026/09/20 12:22:38 INFO Completed upload id=686961worker2 # [ 187.633726] nix-grpc-daemon[1841]: time=2026-09-20T12:22:38.003Z level=INFO msg="Upload complete. (378ms)"6962worker2 # [ 187.635594] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55960 duration_s=0 paths=1 nar_bytes_in=7526963lb # [ 187.623905] niks3-server[1039]: 2026/09/20 12:22:38 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6964worker2 # [ 187.638778] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559606965worker2 # [ 187.646200] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6966worker2 # [ 187.647284] nix-daemon[925]: reaped child process 1920, status = succeeded6967client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6968worker2 # [ 187.652449] nix-daemon[925]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6969client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6970lb # [ 188.040185] envoy[919]: conn peer=[2001:db8:1::1]:52688 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=497 ms=300336971client # farm-top> LOG-farm-top-drain6972worker2 # [ 188.668094] nix-daemon[925]: reaped child process 1934, status = succeeded6973worker2 # [ 188.801995] nix-daemon[925]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6974worker2 # [ 188.815169] nix-daemon[925]: reaped child process 1952, status = succeeded6975lb # [ 188.818268] niks3-server[1039]: 2026/09/20 12:22:39 INFO Received uploads request method=POST path=/api/pending_closures6976worker2 # [ 188.839465] nix-grpc-daemon[1841]: time=2026-09-20T12:22:39.208Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6977worker2 # [ 188.841079] nix-grpc-daemon[1841]: time=2026-09-20T12:22:39.210Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6978lb # [ 188.850180] niks3-server[1039]: 2026/09/20 12:22:39 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6979lb # [ 188.865832] niks3-server[1039]: 2026/09/20 12:22:39 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6980lb # [ 188.873906] niks3-server[1039]: 2026/09/20 12:22:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6981lb # [ 188.879239] niks3-server[1039]: 2026/09/20 12:22:39 INFO Signed narinfos id=69 count=16982worker2 # [ 188.894180] nix-grpc-daemon[1841]: time=2026-09-20T12:22:39.263Z level=INFO msg="Uploading 1 narinfos"6983lb # [ 188.884199] niks3-server[1039]: 2026/09/20 12:22:39 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6984lb # [ 188.893492] niks3-server[1039]: 2026/09/20 12:22:39 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6985lb # [ 188.898066] niks3-server[1039]: 2026/09/20 12:22:39 INFO Completed upload id=696986worker2 # [ 188.912335] nix-grpc-daemon[1841]: time=2026-09-20T12:22:39.281Z level=INFO msg="Upload complete. (253ms)"6987lb # [ 188.901096] niks3-server[1039]: 2026/09/20 12:22:39 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6988worker2 # [ 188.917567] nix-grpc-daemon[1785]: ts=2026-09-20T12:22:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55960 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16989worker2 # [ 188.929540] nix-daemon[925]: reaped child process 1932, status = succeeded6990client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ca.pem&client-cert=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.pem&client-key=/nix/store/8vm13krkgkcd458yp3j43c8n27rmhbpz-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.00 seconds)6991client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6992client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.19 seconds)6993worker1: must succeed: rm /nix/.rw-store/fill6994worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.10 seconds)6995worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6996worker1 # [ 189.573767] nix-grpc-daemon[1541]: ts=2026-09-20T12:22:40Z level=info event=healthy reason=min_free6997worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)6998(finished: subtest: low disk drains a worker and builds go to the other, in 9.56 seconds)6999(finished: run the VM test script, in 191.35 seconds)7000test script finished in 191.77s7001cleanup7002kill QemuMachine (pid 50)7003client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7004kill QemuMachine (pid 49)7005lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7006kill QemuMachine (pid 51)7007worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7008kill QemuMachine (pid 52)7009worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7010(finished: cleanup, in 1.09 seconds)7011additionally exposed symbols:7012 client, lb, worker1, worker2,7013 vlan1,7014 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh